Modelpublic

soyuj/llama3.2-1b-bidirectional-mntp-msmarco

sourceHugging Faceupdated 5mo agoView on Hugging Face

0likes8downloads

Model Card

Bidirectional MNTP Adapter For Llama 3.2 1B

This directory contains the LoRA adapter for a bidirectional MNTP-enabled meta-llama/Llama-3.2-1B. We have performed 10,000 pre-training steps on MS MARCO.

It is intended to be loaded in this repository: https://github.com/basnetsoyuj/llama3.2-1b-sae-bidirectional-mntp-msmarco

bash

python bidirectional_mntp.py \
  --adapter-path soyuj/llama3.2-1b-bidirectional-mntp-msmarco \
  --text "hello world"

Files

—adapter_config.json: PEFT adapter config
—adapter_model.safetensors: LoRA adapter weights

Notes

—This adapter does not include the base Meta Llama weights.
—You must have access to meta-llama/Llama-3.2-1B to load it.
—The bidirectional model runtime used to apply this adapter is adapted from HansiZeng/scaling-retriever.
—Published adapter repo: soyuj/llama3.2-1b-bidirectional-mntp-msmarco.