Nuodebot/GenAudit_Mistral_7b_GGUF

Model Details

The 7B Mistral Model from GenAudit served in Q4KS and F16 GGUF format. <br>Merged and Quantised with Unsloth.AI

Model Description

Inspired by this paper: https://genaudit.org/ <br>Original Code here: https://github.com/kukrishna/genaudit

Converted to GGUF format for running it on Ollama/Llama.cpp so as to take advantage of VRAM offloading to RAM (something Huggingface transformers is unable to do for now).

Merged base mistralv0.1instruct with Qlora and quantised to Q4ks gguf format <br>You may find the base 16 bit model here (but further quantisation is advisable as their Qlora module was fine tuned on the 4bit nf4 base llm)

Developed by: Nuode Chen <br>Finetuned from model: Mistral_V0.1_instruct <br>

Model Sources

<!-- Provide the basic links for the model. -->

• —https://github.com/kukrishna/genaudit

Uses

For evaluating the abstractive summaries of LLM given a source article. <br>This tool will be able to extract evidences supporting each sentence in the summary as well as provide edits to correct its factuality (if applicable)

Refer to original paper for more in-depth information. <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->