CoolFace
Modelpublic

build-small-hackathon/job-searcher-qwen3-8B

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
0likes
Model Card

Job Search (Qwen3-8B)

Two LoRA adapters distilled from DeepSeek V4 Pro onto Qwen/Qwen3-8B. Both adapters preserve the teacher's <think> reasoning traces.

SubfolderTaskInput → Output
query_genQuery generationresume → set of LinkedIn search queries
fit_evalFit evaluation(resume, job listing) → 5 × 20-pt dimensions + reasoning

Usage

python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen3-8B"
repo = "emrekuruu/job-search-lora"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, dtype="bfloat16", device_map="auto")

# Load one task adapter
model = PeftModel.from_pretrained(model, repo, subfolder="query_gen")

# Or swap to the other task on the same base
model.load_adapter(repo, subfolder="fit_eval", adapter_name="fit_eval")
model.set_adapter("fit_eval")

Both tasks expect a chat-formatted prompt; see the reference prompts in the source repo.

Training

  • —Base: Qwen/Qwen3-8B, bf16, SDPA attention.
  • —LoRA: r=16, α=16, dropout=0, targets q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj.
  • —Optimizer: AdamW, lr=2e-4, cosine schedule, weight_decay=0.01, warmup 20 steps.
  • —Sequence length: 16,384 (no truncation).
  • —Loss masking: assistant-only via TRL SFTConfig(assistant_only_loss=True).
  • —Selection: best eval_val_loss (early stopping patience=2), evaluated each epoch.
  • —Data: `emrekuruu/job-search-distill` — the query_gen_pairings config for query_gen/, the job_evals config for fit_eval/.
  • —Hardware: A100-40GB on Modal.

Training code: `modal_apps/train.py`.

Intended uses

  • —Resume-aware job-search assistants and ranking systems.
  • —Research baselines for reasoning-trace distillation from large teachers into ≤10B students.

Out-of-scope uses

  • —Production hiring decisions. The 5 fit-evaluation dimensions are author-defined and not a validated rubric. Treat total as an ordinal signal within a single candidate's shortlist, not as cross-candidate ground truth.
  • —High-stakes scoring without human review. Teacher errors propagate through distillation.

License

Apache-2.0, matching the Qwen3-8B base. Teacher labels used during training were generated via the DeepSeek API and are subject to DeepSeek's Open Platform Terms of Service.