kgrabko/JiRackBaseDataset

💎 JiRack Base dataset for 1.5B model Dataset: the dataset formated for JiRack tokenizer . I recommend initializing the model with a 4K context window for initial stability, followed by scaling to 8K context using specialized JiRack 8K datasets. This two-stage approach ensures robust positional encoding before extending the model's long-range dependency. Time: JiRack 1.5B: High-Efficiency Financial Modeling We are training a compact 1.5B parameter model on an extensive 11… See the full description on the dataset page: https://huggingface.co/datasets/kgrabko/JiRackBaseDataset.