TourTone

TourTone is an Android audio guide for outdoor adventure activities, designed for blind and partially sighted people. It plays pre-recorded audio messages automatically when the phone enters predefined points along a route (geofencing), so the listener does not need to interact with the phone during the activity. The interface shows a map with the route points, but is operated primarily through audio.

TourTone was developed and evaluated in a field study with blind and partially sighted participants during a dog-sledging tour on Valdresflye, Norway. This repository contains the app, the scripts used to prepare routes and audio, and the route and audio data from the study.

The work is part of the ACCESSTOUR project

Building and running the app

Requirements

• Android Studio Iguana (2023.2.1) or newer, with JDK 17
• An Android phone with Android 9 (API level 28) or newer, Google Play services and GPS
• A Google Maps API key for the Maps SDK for Android

Steps

1. Open the androidapp/ folder in Android Studio.
2. Create a Google Maps API key, as described in https://developers.google.com/maps/documentation/android-sdk/get-api-key, and restrict it to the package name no.sintef.tourtone.
3. Add the key to androidapp/local.properties. This file is created by Android Studio and is not committed to git:

   MAPS_API_KEY=your-key-here
   

   Without a key the app still builds, using the placeholder in local.defaults.properties, but the map will be blank.
4. Build and install the app on the phone with Run in Android Studio, or from the command line:

   cd androidapp
   ./gradlew installDebug
   

Using the app

1. Start the app and choose "Turist" (tourist). The map opens and shows the points of the route.
2. Allow location access, and choose "Allow all the time" for background location. Geofencing does not work without it.
3. Keep the app open. The screen stays on while the map is shown. When the phone enters the radius of a point, the audio message for that point is played. A new message stops the one that is playing.

The app plays English audio by default and Norwegian audio when the phone language is set to Norwegian (Bokmål). This uses Android's resource folders (res/raw/ and res/raw-nb/) and needs no setting in the app.

The app writes a log of geofence events and the phone position every 10 seconds to app_logs.txt in the app's external files folder (Android/data/no.sintef.tourtone/files/). In the study, these logs were used to see which messages each participant received. The log contains location data, so handle it as personal data.

Making a new route

A route is a GPX file with one track point (trkpt) per audio message. The app reads androidapp/app/src/main/assets/route.gpx and uses these fields:

<trkpt lat="61.38692553998385" lon="8.817872230046845">
  <name>Welcome to Valdres Flye</name>
  <desc>Text of the audio message (not used by the app)</desc>
  <extensions><radius>100</radius></extensions>
</trkpt>

• lat and lon give the center of the geofence.
• radius is the geofence radius in meters (default 100).
• name gives the name of the audio file. It is lowercased, spaces are replaced with _, and all characters other than letters, digits and _ are removed. "Welcome to Valdres Flye" becomes welcome_to_valdres_flye, and the app plays res/raw/welcome_to_valdres_flye.mp3. Android resource names may only contain a-z, 0-9 and _, so avoid letters such as æ, ø and å in point names.

To replace the route:

1. Replace androidapp/app/src/main/assets/route.gpx with your GPX file.
2. Put one MP3 per point in androidapp/app/src/main/res/raw/ (default language) and, optionally, the Norwegian versions with the same file names in res/raw-nb/. Other languages can be added with other resource qualifiers, for example res/raw-de/.
3. Rebuild the app.

The route is reloaded from route.gpx every time the map is opened.

Preparing routes and audio (Python)

The scripts in python-route-processing/ were used to prepare the route and audio for the study. They use uv to manage the Python environment:

cd python-route-processing
uv sync

The scripts support the following workflow:

1. Route line. The study route was drawn in Norgeskart (https://norgeskart.no). generate_route_from_norgeskart.py contains the exported coordinates and writes them as a GPX track (norgeskartroute.gpx). The same route is in data/norgeskartroute_elevation_and_time.gpx.
2. Points and texts. Write the points where messages should play into a JSON file with UTM coordinates (zone 33N), a name and a description. tour1.json is an example from an earlier test route at Øyangen. uv run main.py tour1.json converts it into GPX waypoints and a GPX track in routes/.
3. Editing. Adjust the points in a GPX editor. The study route was recreated in GPX Studio (https://gpx.studio/).
4. Track with radius. uv run convert_waypoints_to_track.py routes/tour1waypoints.gpx routes/route.gpx converts the waypoints into a track with a geofence radius per point: 50 m for descriptions under 500 characters, otherwise 100 m.
5. Translation. In the study, the desc texts were translated into Norwegian with OpenAI GPT-4o. Any translation method can be used, but name must stay unchanged, because the audio file names are derived from it.
6. Audio. Copy .env.example to .env and add an ElevenLabs API key. Then run uv run generate_audio_files.py routes/route.gpx --output-dir audio_files. This creates one MP3 per point from its desc text, using the voice and model settings from the study (Eleven Flash v2.5). Existing files are not overwritten.
7. Deploying. Copy the GPX file and the MP3 files into the app, as described in Making a new route.

Known limitations

• Geofencing uses Google Play services and depends on GPS accuracy, typically between 15 and 50 m. Events can be delayed, especially when the app is in the background or the screen is locked.

Authors

The app was developed by Marte Lunde Kvam and Maria Emine Nylund, SINTEF Digital.

The paper is by Jiaxin Li, Maria Emine Nylund, Jasmin Niess, Costas Boletsis, Silje Bechmann Granås and Amela Karahasanovic.

Funding

This work was supported by the Research Council of Norway under grant no. 322655, ACCESSTOUR (https://prosjektbanken.forskningsradet.no/project/FORISS/322655).

License

The code and data in this repository, including the audio files, are licensed under the Apache License 2.0. See LICENSE and NOTICE. The audio was generated with ElevenLabs.