Modelpublic

Remidesbois/f2llm-v2-160m-one-piece-retrieval

sourceHugging Faceupdated 3mo agoView on Hugging Face

0likes84downloads

Model Card

F2LLM-v2-160M One Piece Retrieval

Fine-tuned text embedding model for page retrieval in the One Piece Indexer project.

This model starts from codefuse-ai/F2LLM-v2-160M and was fine-tuned on page descriptions, metadata, OCR bubble text, search logs, and explicit feedback from the project database. It is intended for French semantic search over manga page descriptions and related OCR text.

The repository includes:

—model.safetensors: PyTorch/Sentence Transformers checkpoint.
—onnx/model.onnx: ONNX export for Transformers.js browser inference and local Node inference.
—tokenizer/config files required by Transformers.js.
—training_manifest.json: local training configuration snapshot.

Intended Use

Use the model as a dense retrieval encoder:

—query text uses the F2LLM query instruction prompt: Instruct: Given a question, retrieve passages that can help answer the question.\nQuery:
—document/page text is embedded without the query prompt.
—embeddings are 640 dimensions and should be L2-normalized before cosine/IP search.

Local Evaluation

Evaluation was run on page-level held-out splits, so generated queries for the same page do not leak across train/eval/test.

Split	Queries	Hit@1	Hit@5	Hit@10	MRR@10	NDCG@10	Known bad@10
eval	518	0.670	0.894	0.959	0.761	0.803	0.000
test	485	0.654	0.891	0.944	0.755	0.798	0.002

Baseline on the same splits with the original local F2LLM-v2-160M package:

Split	Queries	Hit@1	Hit@5	Hit@10	MRR@10	NDCG@10	Known bad@10
eval	518	0.552	0.728	0.820	0.632	0.673	0.000
test	485	0.520	0.713	0.800	0.610	0.653	0.002

Historical production-context benchmark rows from earlier real feedback labels are useful context only because they are not the same split:

Variant	Queries	Hit@10	MRR@10	NDCG@10
F2LLM 160M text document	75	0.813	0.533	0.584
current Voyage DB	73	0.932	0.650	0.693
current Gemini DB	75	0.200	0.115	0.121
current dual DB	75	0.747	0.413	0.478

Training Snapshot

—pages: 1060
—training pairs: 4432
—epochs: 1
—batch size: 8
—learning rate: 2e-5
—max sequence length: 512
—loss: MultipleNegativesRankingLoss

Data source mix:

—synthetic page-scene queries
—character/arc/keyword queries
—search feedback positives
—search log top-result positives
—OCR bubble quotes where available

ONNX Export

The ONNX model was exported with Optimum as a feature-extraction graph at sequence length 512. Export verification completed with a maximum absolute PyTorch-vs-ONNX output difference of 0.000141.

Caveats

This model is specialized for the project dataset and is not a general-purpose embedding benchmark result. Real replacement quality should keep being monitored with fresh search_feedback and search_logs labels after deployment.