NaomiQ/croissant_chatbot

Croissant Chatbot

This is a Chatbot capable of annotating your dataset in Croissant format.

Features

• —Interactive Chat Interface: Provides a user-friendly interface for interacting with the chatbot.
• —Automatically fetches metadata: If your dataset is from Hugging Face Datasets then the chatbot can retieve metadata and fill in attributes for you
• —Validation and quality checks for attributes: If you enter a value that is considered invalid (e.g. citation not in BibTeX format) or poor quality (e.g. less than 3 keywords), the chatbot will guide you to change the value
• —AI suggestions for attributes: If an attribute value is considered invalid/poor quality or is empty, the chatbot will present AI suggestions for the chatbot based on the information you give it

Demo

This app is deployed on Hugging Face Spaces 🔗 Croissant Chatbot on Hugging Face

This demo is limited in functionality:

• —the API endpoint that is responsible for generating suggestions is a free model with only 200 tokens per day

Large Language Model

This chatbot uses the Mistral 7B Instruct model via OpenRouter to generate metadata suggestions.

• —This model helps by suggesting missing or low-quality metadata attributes.
• —It is free but limited to 200 tokens per day in the demo version.
• —Users can provide their own OpenRouter API key with credits and change the model to the standard version for paid unrestricted access by editing the following method in main/llm.py

def create_llm_response(prompt: str) -> str:
    """
    Use OpenRouter's Mistral 7B Instruct model to generate a response based on the provided prompt.
    Source: https://openrouter.ai/mistralai/mistral-7b-instruct/api 

    Args:
        prompt: The input prompt string for the LLM model.

    Returns:
        The response generated by the LLM model.
    """
    model_propmt = "You are helping a user create metadata for a dataset." + prompt

    api_url = "https://openrouter.ai/api/v1/chat/completions"
    headers = {
        "Authorization": f"Bearer {api_key}",
        "Content-Type": "application/json"
    }
    data = {
        "model": "mistralai/mistral-7b-instruct:free", # can replace this line with the following if free tokens run out: "model": "mistralai/mistral-7b-instruct"
        "messages": [{"role": "user", "content": model_propmt}],
    }
    try:
        response = requests.post(api_url, headers=headers, json=data)
        if response.status_code == 200:
            response_json = response.json()
            if "choices" in response_json and response_json["choices"]:
                return response_json["choices"][0]["message"]["content"]
        else:
            raise Exception(f"An error occurred while trying to use the LLM model.\n {response.status_code}: {response.text}")
    except Exception as e:
        # Handle any exceptions that occur during the request
        return f"Unexpected error occured: {e} \nI'm sorry, I couldn't process your request at the moment. Please try again later."

Installation

1. 1.Clone the repository: https://github.com/NQuartsin/croissantchatbotpublic.git

1. 1.Create a virtual environment

python -m venv venv

source venv/bin/activate

1. 1.Install Dependencies

pip install -r requirements.txt

Environment Variables

To run this project, you will need to add the following environment variables to your .env file

OPENROUTER_API_KEY=<yourapikey>

To create <yourapikey> follow these steps:

1. 1.Visit OpenRouter and sign up
2. 2.Go to your profile > API keys and create a new API key

Run Locally

Activate your virtual environment:

source venv/bin/activate

Run in the terminal:

  python -m main.app

Open the Gradio interface in your browser (usually at http://127.0.0.1:7860).

Acknowledgements

The list of valid licenses was sourced from SPDX License List

The licenses are stored in JSON format in spdx/license-list-data

This list of lisences was used in lisences.json within this repository for the validation of a lisence as a metadata attribute. It was last accessed and downloaded on 23/03/2025. It is located in main/licenses.json and is used in main/validation.py.