whackthejacker/DataHubHub

1---2title: DataHubHub3emoji: ⚡4colorFrom: red5colorTo: indigo6sdk: streamlit7sdk_version: 1.42.28app_file: app.py9pinned: false10license: apache-2.011language: en12---13 14# ML Dataset & Code Generation Manager15 16A comprehensive platform for ML dataset management and code generation with Hugging Face integration.17 18## Features19 20- **Dataset Management**: Upload, explore, and manage machine learning datasets21- **Data Visualization**: Visualize dataset statistics and distributions22- **Code Generation**: Fine-tune models for code generation tasks23- **Code Quality Tools**: Improve code quality with integrated formatters, linters, and type checkers24 25## Technology Stack26 27- **Frontend**: Streamlit28- **Backend**: Python29- **Database**: SQLite (via SQLAlchemy)30- **ML Integration**: Hugging Face Transformers, Datasets31- **Visualization**: Plotly, Matplotlib32 33## Project Structure34 35```36.37├── app.py                     # Main application entry point38├── components/                # UI components39│   ├── code_quality.py        # Code quality tools40│   ├── dataset_preview.py     # Dataset preview component41│   ├── dataset_statistics.py  # Dataset statistics component42│   ├── dataset_uploader.py    # Dataset upload component43│   ├── dataset_validation.py  # Dataset validation component44│   ├── dataset_visualization.py # Dataset visualization component45│   └── fine_tuning/           # Fine-tuning components46│       ├── finetune_ui.py     # Fine-tuning UI47│       └── model_interface.py # Model interface48├── database/                  # Database configuration49│   ├── models.py              # Database models50│   └── operations.py          # Database operations51├── utils/                     # Utility functions52│   ├── dataset_utils.py       # Dataset utilities53│   ├── huggingface_integration.py # Hugging Face integration54│   └── smolagents_integration.py # SmolaAgents integration55└── assets/                    # Static assets56```57 58## Deployment59 60This application is designed to be deployed as a Hugging Face Space.61 62### Hugging Face Space Deployment63 641. Fork this repository652. Create a new Hugging Face Space663. Connect the forked repository to your Space674. The application will be deployed automatically68 69### Local Development70 711. Clone the repository722. Install dependencies:73   ```74   pip install streamlit pandas numpy plotly matplotlib scikit-learn SQLAlchemy huggingface-hub datasets transformers torch75   ```763. Run the application:77   ```78   streamlit run app.py79   ```80 81## Configuration82 83- `.streamlit/config.toml`: Streamlit configuration84- `.streamlit/secrets.toml`: Secrets and API keys85- `huggingface-spacefile`: Hugging Face Space configuration86 87## API Keys88 89To use the Hugging Face integration features, add your Hugging Face API token to `.streamlit/secrets.toml`:90 91```toml92[huggingface]93hf_token = "HF_TOKEN"94```95 96## License97 98This project is licensed under the Apache-2.0 License - see the LICENSE file for details.