abrarfahim/moshi-tool-audio
Moshi Tool-Calling — Audio-Grounded Dataset Audio-grounded data teaching Moshi / PersonaPlex to emit tool-call special tokens in its inner monologue when it hears a request — and to stay quiet otherwise (listening/idle frames are trained to PAD). Each row is a code tensor codes[17, T] at 12.5 Hz: rows stream content 0 text monologue PAD while listening/idle, `< 1:9 Moshi audio silence 9:17 user audio the spoken question (edge-tts), Mimi-encoded mask=1 marks… See the full description on the dataset page: https://huggingface.co/datasets/abrarfahim/moshi-tool-audio.
Moshi Tool-Calling — Audio-Grounded Dataset
Audio-grounded data teaching Moshi / PersonaPlex to emit tool-call special tokens in its inner monologue when it hears a request — and to stay quiet otherwise (listening/idle frames are trained to PAD).
Each row is a code tensor codes[17, T] at 12.5 Hz:
mask=1 marks frames the model must produce (PAD suppression + call + reply); the injected <|tool_result|> block is context (mask 0).
Columns
type, query, reply, voice, audio (24 kHz, playable), codes (17×T int), mask (T).
Special tokens
<|tool_call|>=32000, <|tool_end|>=32001, <|tool_result|>=32002, <|tool_result_end|>=32003
Tools
get_time, get_weather <city>.
Composition (2546 examples)
Built by notebooks/01_generate_data.ipynb in moshi-D-gu.