CoolFace
Apppublic

Agents-MCP-Hackathon/Misty_Climate_Agent

sourceHugging Faceupdated 1y agoView on Hugging Face
3likes
App README

MistyClimate Agent 📈

This is an advanced multi-agent system created using Mistral models, designed to process climate-related documents, analyze images, perform JSON data analysis, and convert text to speech. It provides a comprehensive climate intelligence platform with document processing, image analysis, JSON analysis, and text-to-speech functionalities, all integrated into a user-friendly Gradio interface.

Video Overview

Watch our comprehensive video demonstration to understand the purpose and usage of the MistyClimate Agent:

![Watch the Demo Video](https://youtu.be/b9HGT9l5bcg)

Click the image above or [watch the demo video on YouTube](https://youtu.be/b9HGT9l5bcg) to see:

  • —How to use the climate chat assistant(Used Agent API)
  • —### Agent API [image]
  • —Document processing capabilities(OCR Model)
  • —Image analysis features(Pixtral Model)
  • —JSON data analysis and speech generation(Large Model)
  • —Text-to-speech functionality(Large Model)
  • —Complete workflow demonstrations(Large Model)

📷 Preview Images

Below are preview images showcasing key functionalities and model usage:

Mistral Tokens Usage

[image]

Mistral Agents and OCR Usage

[image]

Key Features

Climate Chat Assistant

  • —Specialized Climate Intelligence: Interact with an AI assistant trained on climate science and sustainability topics
  • —Expert Guidance: Get accurate information on climate change, environmental policies, and sustainability practices
  • —Interactive Interface: User-friendly chat interface with emoji avatars and sample questions

Document Processing

  • —Climate Document Analysis: Extract structured data from climate-related PDFs using OCR capabilities
  • —Multi-format Support: Process various document types including climate reports, analysis papers, and data sheets
  • —Structured Output: Get JSON-formatted results for easy integration

Image Analysis

  • —Visual Data Processing: Analyze image-based documents (PNG, JPG, PDF) to extract text, charts, and tables
  • —Chart Recognition: Specialized analysis of climate charts and graphs
  • —Text Extraction: OCR capabilities for extracting text from images
  • —Table Processing: Extract structured data from tabular images

JSON Analysis & Speech

  • —Climate Data Analysis: Analyze JSON data to extract insights and patterns with focus on climate data
  • —Audio Generation: Convert analysis results into speech using advanced TTS
  • —Statistical Analysis: Perform content, statistical, and structural analysis of climate datasets
  • —Multi-modal Output: Get both text analysis and audio summaries

Text-to-Speech

  • —Natural Voice Synthesis: Convert climate-related text into natural-sounding speech
  • —gTTS Integration: High-quality text-to-speech using Google Text-to-Speech
  • —Audio Export: Generate downloadable audio files

Technical Architecture

MCP Server Integration

The MistyClimate Agent utilizes MCP (Model Context Protocol) servers for enhanced functionality:

  • —Document Agent MCP Server: Handles PDF processing and document analysis
  • —Image Agent MCP Server: Manages image analysis and OCR operations
  • —Server Link: MCP Server Implementation

Multi-Agent System

  • —Document Agent: Specialized in climate document processing
  • —Image Agent: Handles visual data analysis
  • —JSON Analyzer Agent: Processes structured climate data
  • —Speech Agent: Manages text-to-speech conversion
  • —Climate Chat Agent: Provides interactive climate intelligence

Setup and Installation

Prerequisites

  • —Docker (for containerized deployment)
  • —A Mistral API key (obtain from Mistral AI)
  • —Python 3.10+ (for local development)

Quick Start

  1. 1.Clone the Repository:
bash
   git clone <repository-url>
   cd <repository-directory>
  1. 1.Install Dependencies:
bash
   pip install -r requirements.txt
  1. 1.Set Up Mistral API Key:
  2. 2.Obtain your API key from Mistral AI
  3. 3.Input your API key in the Gradio interface
  1. 1.Run with Docker (Recommended):
bash
   # Build the Docker image
   docker build -t mistyclimate-agent .
   
   # Run the container
   docker run -p 7860:7860 mistyclimate-agent
  1. 1.Access the Application:
  2. 2.Open http://localhost:7860 in your browser
  3. 3.Enter your Mistral API key when prompted

Deployment on Hugging Face Spaces

This project is configured for seamless deployment on Hugging Face Spaces:

  1. 1.Fork/clone this repository to your Hugging Face account
  2. 2.Create a new Space with sdk: docker
  3. 3.Push your code to the Space repository
  4. 4.The Space will automatically build and deploy

Usage Guide

1. Climate Chat Assistant

  • —Start by entering your Mistral API key
  • —Use the chat interface to ask climate-related questions
  • —Try sample questions provided in the interface
  • —Get expert guidance on climate science, sustainability, and environmental policies

2. Document Processing

  • —Upload a PDF document (climate reports, research papers)
  • —Select the document type (climate_report, analysis, data)
  • —Click "Process Document" to extract structured data
  • —Review the JSON-formatted output

3. Image Analysis

  • —Upload an image file (PNG, JPG, or PDF)
  • —Choose analysis focus (text extraction, chart analysis, table extraction)
  • —Click "Analyze Image" to process the visual data
  • —Get structured results from the image content

4. JSON Analysis & Speech

  • —Input climate-related JSON data
  • —Select analysis type (statistical, content, structural)
  • —Click "Run Analysis & Generate Speech"
  • —Get both text analysis and audio summary

5. Text-to-Speech

  • —Enter text related to climate topics
  • —Click "Generate Speech" to create audio
  • —Download and play the generated audio file

File Structure

├── agent.py              # Core multi-agent system logic
├── app.py                # Gradio interface and workflow orchestration
├── requirements.txt      # Python dependencies
├── Dockerfile           # Docker configuration
├── README.md           # Project documentation
└── ...             # Preview images and media

Configuration

Environment Variables

  • —MISTRAL_API_KEY: Your Mistral API key (can be set via environment or interface)

Supported File Formats

  • —Documents: PDF
  • —Images: PNG, JPG, JPEG, PDF
  • —Data: JSON format for climate datasets

Performance Metrics

  • —Document Processing: Handles climate PDFs up to 50MB
  • —Image Analysis: Supports images up to 10MB
  • —Response Time: Typically 2-5 seconds per request
  • —Audio Generation: High-quality TTS output

Privacy and Security

  • —API keys are handled securely and not stored
  • —All processing is done in real-time without data persistence
  • —Files are temporarily stored only during processing

API Reference

For detailed API documentation and integration examples, visit our MCP Server Implementation.

Tags

  • —agent-demo-track
  • —climate-intelligence
  • —multi-agent-system
  • —mistral-ai
  • —document-processing
  • —image-analysis
  • —text-to-speech

License

This project is licensed under the MIT License. See the LICENSE file for details.

Acknowledgments

Built with ❤️ by Samudrala Dinesh Naveen Kumar

Making climate intelligence accessible through advanced AI technology