CoolFace
Modelpublic

rustformers/gpt4all-j-ggml

sourceHugging Faceapache-2.0updated 3y agoView on Hugging Face
11likes76downloads
Model Card

GGML converted version of Nomic AI GPT4All-J-v1.0 models

Description

An Apache-2 licensed chatbot trained over a massive curated corpus of assistant interactions including word problems, multi-turn dialogue, code, poems, songs, and stories.

Converted Models

NameBased onTypeContainerGGML Version
gpt4all-j-f16.binnomic-ai/gpt4all-jF16GGMLV3
gpt4all-j-q4_0.binnomic-ai/gpt4all-jQ4_0GGMLV3
gpt4all-j-q4_0-ggjt.binnomic-ai/gpt4all-jQ4_0GGJTV3
gpt4all-j-q5_1.binnomic-ai/gpt4all-jQ5_1GGMLV3
gpt4all-j-q5_1-ggjt.binnomic-ai/gpt4all-jQ5_1GGJTV3

Usage

Python via llm-rs:

Installation

Via pip: pip install llm-rs

Run inference
python
from llm_rs import AutoModel

#Load the model, define any model you like from the list above as the `model_file`
model = AutoModel.from_pretrained("rustformers/gpt4all-j-ggml",model_file="gpt4all-j-q4_0-ggjt.bin")

#Generate
print(model.generate("The meaning of life is"))

Rust via Rustformers/llm:

Installation
git clone --recurse-submodules https://github.com/rustformers/llm.git
cd llm
cargo build --release
Run inference
cargo run --release -- gptj infer -m path/to/model.bin  -p "Tell me how cool the Rust programming language is:"