Audio-augmented arboreality: wildflowers and language

Hafizur Rahaman*, Michelle Johnston, Erik Champion

*Corresponding author for this work

    Research output: Contribution to journalArticlepeer-review


    Before colonization, there were over 250 languages spoken in Australia. Today only thirteen Indigenous languages are still being taught to children). Language has an important part to play in cultural maintenance and ‘closing the gap’ in terms of First Peoples’ cultural heritage, identity, and sense of belonging. In this work, we aim to develop an engaging and easy way to teach and learn the local Indigenous names of wildflowers using a mobile device. This paper presents the development of a phone application that runs on a local machine, recognizes local wildflowers through its camera, and plays associated sounds and displays associated text in the Noongar language. The prototype mobile application has been developed with MobileNets model on the TensorFlow platform. The dataset is derived from Google searches, while the sound files are generated from label text by running an apple script. UI and interactivity have been developed by using Vuforia and the Unity game engine. Finally, the Android Studio is used to deploy the app. At this point in time, the prototype can only recognize ten local flowers, with 85%∼99% of accuracy. We are working with a larger dataset towards developing the full application.

    Original languageEnglish
    Pages (from-to)22-37
    Number of pages16
    JournalDigital Creativity
    Issue number1
    Publication statusPublished - 2021


    Dive into the research topics of 'Audio-augmented arboreality: wildflowers and language'. Together they form a unique fingerprint.

    Cite this