gabriellarson/ICONN-1-GGUF

<div align="center" style="line-height: 1;"> <a href="https://huggingface.co/collections/ICONNAI/iconn-1-6851e8a88ed4eb66b4fd0132" target="blank" style="margin: 2px;"> <img alt="ICONN 1 Models" src="https://img.shields.io/badge/📦ICONN1Models-HuggingFace-1CBEEF?style=flat-square&labelColor=2C3E50" style="display: inline-block; vertical-align: middle;" /> </a> <a href="https://huggingface.co/spaces/ICONNAI/ICONN-Mini-Chat" target="blank" style="margin: 2px;"> <img alt="ICONN 1 Chat" src="https://img.shields.io/badge/💬ICONN1Chat-Online-65C7F9?style=flat-square&labelColor=2C3E50" style="display: inline-block; vertical-align: middle;" /> </a> <a href="https://huggingface.co/ICONNAI" target="blank" style="margin: 2px;"> <img alt="ICONN on Hugging Face" src="https://img.shields.io/badge/🤗ICONNonHF-ICONNAI-A4BCF0?style=flat-square&labelColor=2C3E50" style="display: inline-block; vertical-align: middle;" /> </a> <a href="https://opensource.org/license/apache-2-0" target="blank" style="margin: 2px;"> <img alt="License Apache 2.0" src="https://img.shields.io/badge/⚖️License-Apache_2.0-5C63DA?style=flat-square&labelColor=2C3E50" style="display: inline-block; vertical-align: middle;" /> </a> </div>

ICONN 1

We proudly introduce ICONN-1, the most advanced and human-like open-source artificial intelligence model under 100B parameters of its time. Designed to push the boundaries of natural language understanding and generation, ICONN-1 is built on a Mixture-of-Experts (MoE) architecture that enables dynamic routing through specialized expert pathways, allowing for both computational efficiency and enhanced performance.

Developed entirely from scratch, ICONN-1 is based on a customized Mixtral framework and comprises 88 billion parameters, with 22 billion parameters actively utilized per token. This approach allows ICONN-1 to deliver highly nuanced and contextually accurate responses while maintaining the scalability benefits of sparse activation.

ICONN-1 is released in two distinct forms to serve different application needs:

• —ICONN-1 (this version) is optimized for natural, emotionally resonant, and conversational interactions.
• —ICONN-e1 is a specialized variant of the model fine-tuned for advanced reasoning, critical analysis, and complex problem-solving.

Together, these models represent a major leap forward in the evolution of AI systems—demonstrating not only deep reasoning but also a commitment to openness, accessibility, and human-aligned intelligence.

These models were each benchmarked on a collection of 500 questions to compare output to a human for emotion and common sense. Benchmark performance may vary due to the stochastic nature of AI models. ICONN 1 retains the highest human-thinking benchmark score through many tests on different temperatures.

Usage

System Requirements

To run ICONN 1 effectively, ensure you have:

• —4× NVIDIA A100 GPUs or a single NVIDIA B100
• —At least 120 GB of system RAM
• —120–192 GB of GPU VRAM

If your system does not meet these requirements—which may be the case for many users—you can still experience ICONN through alternative options:

• —Use a quantized version of ICONN for lower resource consumption
• —Try the lightweight **ICONN 1 Mini (7B)**

Run the code below to run ICONN 1:

from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
import torch

def run_iconn_chatbot(model_name="ICONNAI/ICONN-1"):

    tokenizer = AutoTokenizer.from_pretrained(model_name)
    model = AutoModelForCausalLM.from_pretrained(model_name)
    
    device = 0 if torch.cuda.is_available() else -1
    
    chat_pipeline = pipeline(
        "text-generation",
        model=model,
        tokenizer=tokenizer,
        device=device,
        max_length=1624,
        do_sample=True,
        top_p=0.9,
        temperature=0.4,
        pad_token_id=tokenizer.eos_token_id
    )
    
    print(f"ICONN chatbot running with model: {model_name}. Type 'exit' to quit.")
    conversation_history = ""
    
    while True:
        user_input = input("You: ")
        if user_input.lower() == "exit":
            print("Goodbye!")
            break
        
        conversation_history += f"User: {user_input}\nBot:"
        
        response = chat_pipeline(conversation_history, max_length=len(tokenizer.encode(conversation_history)) + 100)[0]['generated_text']
        
        bot_reply = response[len(conversation_history):].strip().split("\n")[0]
        
        print(f"Bot: {bot_reply}")
        
        conversation_history += f" {bot_reply}\n"

if __name__ == "__main__":
    run_iconn_chatbot()