Kanjul/Multimodal-AI-Assistant

🤖 My Multimodal AI Assistant 🎙️💬

This Gradio application serves as an interactive multimodal AI assistant, bringing together speech-to-text, large language model processing, and text-to-speech capabilities. It aims to provide a seamless conversational experience.

✨ Features

• —Speech-to-Text (STT): Transcribes user's spoken audio input into text.
• —Large Language Model (LLM) Interaction: Processes the transcribed text and generates intelligent, context-aware responses.
• —Text-to-Speech (TTS): Converts the LLM's text response back into natural-sounding speech for an auditory reply.
• —Built with Gradio: Provides an easy-to-use web interface for interaction.

🚀 How to Use

1. 1.Launch the App: Once the Space is loaded, you will see the interactive interface.
2. 2.Provide Input:
3. 3.You can speak directly into your microphone (allow browser access if prompted).
4. 4.Alternatively, you can upload an audio file containing your message.
5. 5.Get Response:
6. 6.Your audio will be transcribed and displayed.
7. 7.The AI will process your input and generate a text response.
8. 8.Finally, the AI's response will be converted into speech and played back to you.

🛠️ Technologies Used

• —Gradio: For building the interactive web user interface.
• —gTTS (Google Text-to-Speech): For basic text-to-speech conversion.
• —ElevenLabs: For high-quality, more natural-sounding text-to-speech.
• —Groq API (likely via OpenAI-compatible library): For fast large language model inference and potentially speech-to-text.
• —PyAudio: For handling audio input/output, requiring PortAudio system libraries.

⚙️ Local Development Setup

To run this assistant on your local machine:

1. 1.Clone the Repository:

    git clone [https://huggingface.co/spaces/Kanjul/Multimodal-AI-Assistant](https://huggingface.co/spaces/Kanjul/Multimodal-AI-Assistant)
    cd Multimodal-AI-Assistant

1. 1.Create and Activate a Virtual Environment:

    python -m venv venv
    .\venv\Scripts\activate # On Windows PowerShell
    # source venv/bin/activate # On Linux/macOS

1. 1.Install Dependencies:

    pip install -r requirements.txt

1. 1.Set Environment Variables (API Keys): Create a .env file in the root of your project and add your API keys:

    ELEVEN_API_KEY="your_elevenlabs_api_key_here"
    GROQ_API_KEY="your_groq_api_key_here" # If you use Groq
    # Add any other API keys your app might need

Ensure your Python code accesses these using os.environ.get("YOUR_KEY_NAME").

1. 1.Run the Application:

    python app.py

Your Gradio app will then be accessible in your web browser, typically at http://127.0.0.1:7860.

☁️ Deployment on Hugging Face Spaces

This application is designed for easy deployment on Hugging Face Spaces. The README.md metadata (this file!) and requirements.txt handle the setup. Ensure your API keys are added as Space Secrets in the Hugging Face Space settings, NOT in your code or .env in the repo.

Developed by Kanjul