CoolFace
Apppublic

Skjerdal/Pretrained_vs_OpenAI_QA

sourceHugging Faceupdated 1y agoView on Hugging Face
0likes
App README

QA Model Comparison: Custom Model vs OpenAI API

This application compares the performance and response time of a custom Question Answering model against OpenAI's API. It allows you to visually see the difference in response quality and speed between a locally hosted model and an API call to a large language model.

Features

  • —Custom QA model inference using a pre-trained Transformer model
  • —OpenAI API integration for comparison
  • —Response time measurement for both models
  • —Side-by-side result comparison
  • —User-friendly Gradio interface

Setup Instructions

1. Clone the repository

bash
git clone <repository-url>
cd <repository-directory>

2. Install dependencies

bash
pip install -r requirements.txt

3. Set up your OpenAI API key

Copy the example environment file and add your OpenAI API key:

bash
cp .env.example .env

Then edit the .env file and replace your_openai_api_key_here with your actual OpenAI API key.

4. Download/prepare your custom model

Make sure your custom QA model is in the tf210_custom_qa_model directory with the following structure:

tf210_custom_qa_model/
├── config.json
├── qa_model_weights
└── tokenizer files

If you don't have a pre-trained model, you can modify the code to use a model from the Hugging Face hub.

5. Run the application

bash
python app.py

The Gradio interface will start and provide a local URL (typically http://127.0.0.1:7860) where you can access the application.

Usage

  1. 1.Enter or paste a context paragraph in the Context box
  2. 2.Enter a question about the context in the Question box
  3. 3.Check the "Use OpenAI API for comparison" box to compare with OpenAI's response
  4. 4.Click Submit to see the results
  5. 5.Compare the answers and response times from both models

Example

The application comes with an example about the SQuAD dataset that you can use to test it immediately.

Custom Model Details

The custom QA model is based on DistilBERT and fine-tuned for question answering tasks. It extracts the exact answer spans from the provided context that best answer the question.

Troubleshooting

Missing Model Files

If you encounter an error like "Error loading model", make sure:

  • —The tf210_custom_qa_model directory exists
  • —All required files (config.json, qamodelweights) are present in the directory
  • —The files have the correct permissions

OpenAI API Issues

If you encounter errors with the OpenAI API:

  • —Verify your API key is correctly set in the .env file
  • —Check your internet connection
  • —Ensure your OpenAI account has available credits
  • —Check if the requested model (gpt-3.5-turbo) is available for your account

Package Version Conflicts

If you encounter errors related to the OpenAI package or TensorFlow:

  • —Try updating to the latest versions with pip install -U openai tensorflow transformers
  • —If using an older version of the OpenAI package, the app has a fallback mechanism to use the legacy API

Limitations

  • —The custom model has a maximum sequence length (default 384 tokens)
  • —Long contexts are processed in chunks with a stride
  • —OpenAI API calls require an internet connection and valid API key
  • —API calls may incur costs depending on your OpenAI usage