Buiilding/nova-dataset-finetune-2

Nova: Voice-to-Text Companion Dataset

Dataset Description

This dataset contains fine-tuning data for Nova, a real-time voice-to-text companion that can transcribe, translate, and assist with various voice-controlled workflows.

Dataset Summary

• —Total Conversations: 845 conversation objects
• —Total Messages: 4,605 individual messages
• —Languages: English, Vietnamese, and multilingual support
• —Format: Structured conversations with tool calls and responses
• —Use Case: Fine-tuning LLMs for voice-to-text companion applications

Supported Tasks

1. 1.Voice Typing: Transcribe audio and type into detected input fields
2. 2.Live Subtitles: Create real-time subtitle overlays from audio sources
3. 3.App/Web Control: Open applications and websites
4. 4.Input Detection: Automatically detect input fields on screen
5. 5.Multi-language Support: Handle various language pairs for translation

Note: This dataset is designed for fine-tuning models on dialogue generation, text-to-text generation, and language modeling tasks specifically for voice-to-text companion applications.

Tool Calls Supported

• —open_web(url): Open websites
• —open_app(app_name): Open local applications
• —detect_input_boxes(): Detect input fields on screen
• —voice_typing(audio_device_index, source_language, target_language): Start voice typing
• —live_subtitle(audio_device_index, source_language, target_language): Start live subtitles

Data Structure

Each conversation object contains:

• —conversations: Array of message objects with roles (user, assistant, tool, system)
• —Tool calls are formatted with <tool_call_start> and <tool_call_end> tags
• —Tool responses include <tool_response_start> and <tool_response_end> tags
• —Audio device information is provided in <audio_devices_start> and <audio_devices_end> tags

Source

This dataset was created by combining multiple specialized conversation datasets covering various Nova workflow scenarios including:

• —Casual conversations
• —Function call workflows (single and multi-turn)
• —Language-specific workflows
• —Live subtitle workflows
• —Voice typing workflows

Intended Use

This dataset is intended for fine-tuning large language models to:

• —Generate conversational responses in dialogue generation tasks
• —Perform text-to-text generation for voice-to-text companion applications
• —Model language patterns for voice interaction workflows
• —Generate appropriate tool calls based on user requests
• —Handle multi-turn conversations with tool interactions
• —Support multilingual voice interactions
• —Manage audio device selection and workflow orchestration

Limitations

• —Dataset focuses on specific Nova workflow patterns
• —Limited to supported tool calls and conversation structures
• —Audio content is represented as text descriptions, not actual audio files

Citation

If you use this dataset, please cite:

@dataset{nova_voice_to_text_2024,
  title={Nova: Voice-to-Text Companion Dataset},
  description={Fine-tuning dataset for voice-to-text companion AI},
  year={2024}
}