mariamoracrossitcr/deepseek-llm-7b-base-INBioCR-sp-DAPT

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

deepseek-llm-7b-base-INBioCR-sp-DAPT

This model is a fine-tuned version of deepseek-ai/deepseek-llm-7b-base on the None dataset. It achieves the following results on the evaluation set:

• —Loss: 1.2487

Model description

The adapter adapts DeepSeek LLM 7B Base to biodiversity informatics content related to species from Costa Rica. The training corpus contains species-level textual descriptions generated from INBio's Atta database and organized according to Plinian Core concepts.

This is not a standalone full model. It is a PEFT/LoRA adapter and must be loaded together with the base model.

Intended uses

This adapter is intended for research on biodiversity question answering in Espanish, domain-adaptive pretraining, and uncertainty estimation in large language models.

Limitations

The model should not be used as an authoritative taxonomic or conservation decision system without expert validation. The adapter reflects the content and quality of the training corpus and may contain incomplete, outdated, or uncertain biodiversity information.

Training data

Dataset: mariamoracrossitcr/INBioCR-Species-DAPT

The dataset contains biodiversity text associated with species that live in or visit Costa Rica, including scientific names, common names, Plinian Core concepts, and descriptive answers.

Training procedure

The model was trained using causal language modeling with LoRA.

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 1e-05
• —trainbatchsize: 2
• —evalbatchsize: 2
• —seed: 42
• —gradientaccumulationsteps: 32
• —totaltrainbatch_size: 64
• —optimizer: Use pagedadamw32bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• —lrschedulertype: cosine
• —lrschedulerwarmup_ratio: 0.03
• —num_epochs: 2

Training results

Framework versions

• —PEFT 0.18.0
• —Transformers 4.57.3
• —Pytorch 2.6.0+cu124
• —Datasets 3.6.0
• —Tokenizers 0.22.1