QuantFactory/MachineLearningLM-7B-v1-GGUF
5668
1 2---3 4base_model:5- Qwen/Qwen2.5-7B-Instruct6license: apache-2.07pipeline_tag: text-generation8library_name: transformers9datasets:10- MachineLearningLM/machinelearninglm-scm-synthetic-tabularml11tags:12- Tabular Classification13 14---15 16[](https://hf.co/QuantFactory)17 18 19# QuantFactory/MachineLearningLM-7B-v1-GGUF20This is quantized version of [MachineLearningLM/MachineLearningLM-7B-v1](https://huggingface.co/MachineLearningLM/MachineLearningLM-7B-v1) created using llama.cpp21 22# Original Model Card23 24 25# MachineLearningLM26 27This repository contains the model presented in the paper [MachineLearningLM: Scaling Many-shot In-context Learning via Continued Pretraining](https://huggingface.co/papers/2509.06806).28 29## Model Summary30 31Can LLMs learn from 1,000 in-context examples?32 33Introducing **MachineLearningLM** π§ͺπ β a model continuously pretrained on millions of synthetic tabular ML tasks, enabling robust many-shot in-context learning.34 35π **Scales from 8 to 1,024 examples**36 37π β**β~15% improvementβ**β on unseen tabular tasks compared to o3-mini / GPT-5-mini / Qwen-2.5-7B-Instruct38 39π² β**βRandom-Forestβlevel numerical modeling robustnessβ**β40 41π§ β**βMMLU score: 75.4%β**β42 43π Read the paper: https://huggingface.co/papers/2509.0680644 45 GitHub: https://github.com/HaoAreYuDong/MachineLearningLM46 47## Evaluation and Validation48 49We have developed an automated evaluation framework β simply configure the parameters to easily perform validation and evaluation. 50**The code is now open-sourced at our [GitHub repository](https://github.com/HaoAreYuDong/MachineLearningLM).**51 52**Quick Start**53 54```bash55pip install -r requirements.txt56python ./src/evaluation/model_pred/dl_model_pred.py \57 --input_dir ./demo_input.jsonl \58 --output_dir ./demo_output.jsonl \59 --model_name MachineLearningLM/MachineLearningLM-7B-v160```61**Pipeline**62```bash63# modify the evaluate_parameters.sh file64source evaluate_parameters.sh65 66# Option 1 End-to-End Pipeline67./scripts/evaluate_pipeline.sh68 69# Option 2 Parallel Processing70./scripts/multi_process/data_prep.sh71./scripts/multi_process/prompt_gen.sh # For deep learning only72./scripts/multi_process/model_pred.sh73./scripts/multi_process/evaluation.sh74./scripts/multi_process/report.sh75 76# Option3 Sequential Processing77./scripts/single_process/data_prep.sh78./scripts/single_process/prompt_gen.sh # For deep learning only79./scripts/single_process/model_pred.sh80./scripts/single_process/evaluation.sh81./scripts/single_process/report.sh82```83 84For more usage details, please visit our GitHub.85 86**Quants of Checkpoints**87 88https://huggingface.co/mradermacher/MachineLearningLM-7B-v1-GGUF89 90 91## Tabicl Evaluation92 93**This part of the code needs to run in an environment with the tabicl and openpyxl libraries installed.**94 95The evaluation code for tabicl is placed separately in the `./src/evaluation/tabicl_evaluate.py` file. Use `./scripts/tabicl_evaluate.sh` to obtain the evaluation results for tabicl.96 97Use --datasets to specify the datasets to be evaluated, and --sample_sizes to indicate the number of shots. 98 99If multiple datasets need to be evaluated, separate them with spaces. To evaluate all CSV files in the input folder, use **all**.100 101## Prior_data102 103MachineLearningLM uses the code from tabicl to generate prior data.104 105Use `./scripts/generate_data.sh` to generate the prior data. It generates the corresponding .pt and .csv files, and normalizes the feature values in the CSV files to the range of 0β999, as we did in the paper.106 107### Parameter IntroductionοΌrefer to the comments in the file `tabicl\src\tabicl\prior\dataset.py`οΌ108 109**Data Scale & Structure**110 111| Parameter | Type | Description |112| :------------- | :--- | :------------------------------------------------------ |113| `min_features` | int | Minimum number of features per dataset |114| `max_features` | int | Maximum number of features per dataset |115| `max_classes` | int | Maximum number of target classes |116| `min_seq_len` | int | Minimum samples per dataset. Uses `max_seq_len` if None |117| `max_seq_len` | int | Maximum samples per dataset οΌNot IncludeοΌ |118 119**Batch Configuration**120 121| Parameter | Type | Description |122| :--------------------- | :--- | :----------------------------------------------------------- |123| `batch_size` | int | Total number of datasets to generate per batch |124| `batch_size_per_gp` | int | Number of datasets per group (shared characteristics) |125| `batch_size_per_subgp` | int | Number of datasets per subgroup (similar causal structures). Defaults to `batch_size_per_gp` if None |126 127**Sequence Length Control**128 129| Parameter | Type | Description |130| :--------------- | :--- | :----------------------------------------------------------- |131| `log_seq_len` | bool | Sample sequence length from log-uniform distribution if True |132| `seq_len_per_gp` | bool | Sample sequence length per group (enables variable-sized datasets) |133| `replay_small` | bool | Occasionally sample smaller sequences for model robustness |134 135**Train-Test Split**136 137| Parameter | Type | Description |138| :--------------- | :-------- | :----------------------------------------------------------- |139| `min_train_size` | int/float | Start position/ratio for train split (int: absolute, float: fractional) |140| `max_train_size` | int/float | End position/ratio for train split (int: absolute, float: fractional) |141 142**Generation Method**143 144| Parameter | Type | Description |145| :----------- | :--- | :----------------------------------------------------------- |146| `prior_type` | str | Prior type: 'mlp_scm', 'tree_scm', or 'mix_scm' (random selection) |147| `fixed_hp` | dict | Fixed structural configuration parameters |148| `sampled_hp` | dict | Parameters sampled during generation |149 150**Computation Settings**151 152| Parameter | Type | Description |153| :------------------------- | :--- | :------------------------------------------------ |154| `n_jobs` | int | Number of parallel jobs (-1 = use all processors) |155| `num_threads_per_generate` | int | Number of threads per generation job |156| `device` | str | Computation device ('cpu' or 'cuda') |157 158## Train159 160MachineLearningLM uses the LLaMA-Factory framework for training.161 162#### Training Environment Configuration163 164```bash165cd ./third_party/LLaMA-Factory166pip install -e ".[torch,metrics]" --no-build-isolation167pip install wandb168```169 170Use `./scripts/train.sh` for training.171 172## Project Structure173 174```175MachineLearningLM/176βββsrc/177| βββevaluation/178β β βββ data_prep/ # Data preprocessing and chunking utilities179β β βββ prompt_gen/ # Prompt generation for deep learning models180β β βββ model_pred/ # Model inference (ML and DL prediction engines)181β β βββ result_proc/ # 5-layer evaluation architecture and metrics processing182β β βββ zero_summary/ # Result summarization and report generation183β β βββ tabicl_evaluate.py184β βββprior_data185β βββ pt_to_csv.py 186βββ scripts/187β βββ single_process/ # Sequential execution shell scripts188β βββ multi_process/ # Parallel execution shell scripts (with _mp suffix)189β βββ evaluate_parameters.sh # Global parameter configuration190| βββ evaluate_pipeline.sh # automated pipeline191| βββ generate_data.sh192| βββ tabicl_evaluate.sh193| βββ train.sh194βββ datahub_inputs/195β βββ data_demo/ # Demo datasets for testing196β βββ data_raw/ # Raw input datasets197βββ third_party/198β βββ tabicl/ 199β βββ LLaMA-Factory/ 200βββ requirements.txt # Python dependencies for Evaluation Framework201βββ README.md202βββ README_zh.md203βββ THIRD_PARTY_NOTICES.md204βββ LICENSE205```206