Modelpublic

paloalma/ECE-TW3-JRGL-V1

sourceHugging Faceapache-2.0updated 2y agoView on Hugging Face

1likes8.5kdownloads

Model Card

ECE-TW3-JRGL-V1

This model has been produced by :

—Louis Garcia, engineering student at French Engineering School ECE
—Matthieu Jollard, engineering student at French Engineering School ECE

Under the supervision of :

—Andre-Louis Rochet, Lecturer at ECE & Co-Founder of TW3 Partners
—Paul Lemaistre, CTO of TW3 Partners

With the contribution of :

—ECE engineering school as sponsor and financial contributor
—RunPod as financial contributor

About ECE

**ECE**, a multi-program, multi-campus, and multi-sector engineering school specializing in digital engineering, trains engineers and technology experts for the 21st century, capable of meeting the challenges of the dual digital and sustainable development revolutions. [French Engineering School ECE](https://www.ece.fr/en/)

Description

ECE-TW3-JRGL-V1 is a merge of the following models using [mergekit](https://github.com/cg123/mergekit):

yaml

slices:
  - sources:
      - model: ShinojiResearch/Senku-70B-Full
        layer_range: [0, 80]
      - model: 152334H/miqu-1-70b-sf
        layer_range: [0, 80]
merge_method: slerp
base_model: 152334H/miqu-1-70b-sf
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: float16

Results

—ECE-TW3-JRGL-v1 scores 83.07 on EQ-Bench V2

@misc{paech2023eqbench, title={EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models}, author={Samuel J. Paech}, year={2023}, eprint={2312.06281}, archivePrefix={arXiv}, primaryClass={cs.CL} } ---