jirack
Datasets
All datasets matching “jirack”JiRack-SmallTalk_4k-DatasetJiRack-SlimOrca_4k-DatasetJiRack-Magpie-Pro-MT-300K_8k-DatasetJiRack-GammaCorpus-Fact-QA-DatasetJiRack-GammaCorpus-Fact-QA-Dataset-2kJiRackBaseDataset
💎 JiRack Base dataset for 1.5B model
Dataset: the dataset formated for JiRack tokenizer . I recommend initializing the model with a 4K context window for initial stability, followed by scaling to 8K context using specialized JiRack 8K datasets. This two-stage approach ensures robust positional encoding before extending the model's long-range dependency.
Time: JiRack 1.5B: High-Efficiency Financial Modeling
We are training a compact 1.5B parameter model on an extensive 11 billion… See the full description on the dataset page: https://huggingface.co/datasets/kgrabko/JiRackBaseDataset.