minpeter/LoRA-Qwen3-4b-v1-iteration-02-sf-apigen-02
Comparison Table of Test Results by Model (BFCL)
The model learned pretty well. In fact, it is normal because there is no parallel call data in the training data and no irrelevance data.
<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/> <details><summary>See axolotl config</summary>
axolotl version: 0.10.0.dev0
base_model: Qwen/Qwen3-4B
hub_model_id: minpeter/LoRA-Qwen3-4b-v1-iteration-02-sf-apigen-02
plugins:
- axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin
strict: false
datasets:
- path: minpeter/apigen-mt-5k-friendli
data_files:
- train.jsonl
- test.jsonl
type: chat_template
roles_to_train: ["assistant"]
field_messages: messages
message_property_mappings:
role: role
content: content
chat_template: chatml
dataset_prepared_path: last_run_prepared
output_dir: ./output
val_set_size: 0.0
sequence_len: 20000
sample_packing: true
eval_sample_packing: true
pad_to_sequence_len: true
load_in_4bit: true
adapter: qlora
lora_r: 16
lora_alpha: 32
lora_target_modules:
- q_proj
- k_proj
- v_proj
- o_proj
- down_proj
- up_proj
lora_mlp_kernel: true
lora_qkv_kernel: true
lora_o_kernel: true
wandb_project: "axolotl"
wandb_entity: "kasfiekfs-e"
wandb_watch:
wandb_name:
wandb_log_model:
gradient_accumulation_steps: 2
micro_batch_size: 1
num_epochs: 1
optimizer: adamw_torch_4bit
lr_scheduler: cosine
learning_rate: 0.0002
bf16: auto
tf32: true
gradient_checkpointing: offload
gradient_checkpointing_kwargs:
use_reentrant: false
resume_from_checkpoint:
logging_steps: 1
flash_attention: true
warmup_steps: 10
evals_per_epoch: 4
saves_per_epoch: 1
weight_decay: 0.0
special_tokens:
</details><br>
LoRA-Qwen3-4b-v1-iteration-02-sf-apigen-02
This model is a fine-tuned version of Qwen/Qwen3-4B on the minpeter/apigen-mt-5k-friendli dataset.
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- trainbatchsize: 1
- evalbatchsize: 1
- seed: 42
- gradientaccumulationsteps: 2
- totaltrainbatch_size: 2
- optimizer: Use OptimizerNames.ADAMWTORCH4BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lrschedulertype: cosine
- lrschedulerwarmup_steps: 10
- num_epochs: 1.0
Training results
Framework versions
- PEFT 0.15.2
- Transformers 4.51.3
- Pytorch 2.6.0+cu124
- Datasets 3.5.1
- Tokenizers 0.21.1