Skip to main content

Sound Instruction

A Sound Instruction describes what audio to play. Its in JSON format for network, file and wasm APIs.

Your language binding likely has a convenient API for creating a Sound Instruction (e.g. SoundBuilder in the Rust API).

The following are the available instructions and their JSON encoding:

Play a file​

A plain string is a path to an audio file, the directory the path is relative to is determined by the context (e.g. directory containing the WASM file for wasm activities, "activities/user/" for WebSocket activities) An absolute path (starting with /) is always relative to the root activities directory regardless of context.

"my_activity/beep.wav"

List (sequential)​

An array plays sounds one after the other.

["my_activity/intro.wav", "my_activity/loop.wav"]

Simultaneous​

Plays multiple sounds at the same time.

{"i": "simultaneous", "sounds": ["my_activity/drums.wav", "my_activity/melody.wav"]}

Repeat​

Repeats a sound. Omit times to loop forever.

{"i": "repeat", "sound": "my_activity/tick.wav", "times": 4}

Volume​

Applies a volume multiplier to a sound. Use 1.0 for original volume, 0.5 for half, 2.0 for double. For dynamic volume changes after playback has started, use a Controller instead.

{"i": "volume", "multiplier": 0.5, "sound": "my_activity/music.wav"}

Speed​

Applies a speed multiplier to a sound. Use 1.0 for original speed, 2.0 for double speed. Pitch adjusts proportionally (faster → higher pitch). For dynamic speed changes after playback has started, use a Controller instead.

{"i": "speed", "multiplier": 2.0, "sound": "my_activity/voice.wav"}

Controller​

Makes a sound controllable at runtime. You can adjust speed, volume, and paused state, stop the sound, and receive notifications when it finishes.

  • id: unique identifier for this controller (integer, 0 ≤ id < 2^53, required)
  • speed: initial speed multiplier (optional, defaults to 1.0)
  • volume: initial volume multiplier (optional, defaults to 1.0)
  • paused: whether the sound starts paused (optional, defaults to false)

Controllers cannot be nested inside a repeat.

{"i": "controller", "sound": "my_activity/music.wav", "id": 1, "speed": 1.0, "volume": 1.0, "paused": false}

Timed Commands​

Executes commands at specific millisecond offsets during playback of a sound. Commands are strings of the form "<MILLIS> <COMMAND>".

  • sound: the sound to play (required)
  • commands_path: path to a file containing commands (optional)
  • commands: inline array of command strings (optional)

If both commands_path and commands are provided, they are merged.

{"i": "timed_commands", "sound": "my_activity/bg.wav", "commands_path": "my_activity/cues.txt", "commands": ["0 START", "5000 STOP"]}

Silence​

Inserts a pause.

{"i": "silence", "millis": 1500}

Speak number​

Speaks a number aloud using the built-in voice.

{"i": "speak_number", "number": 42}

Error sound​

Plays the system error sound (a beep).

{"i": "error_sound"}

Empty sound​

A no-op sound that finishes immediately.

{"i": "empty_sound"}