likhonsheikh/anthropic-compatible-api

1---2title: Anthropic Compatible API3emoji: 🤖4colorFrom: purple5colorTo: blue6sdk: docker7pinned: false8license: apache-2.09---10 11# Anthropic-Compatible API12 13A **production-ready, self-hosted API** that provides full **Anthropic Messages API compatibility** using the Qwen2.5-Coder-7B model with llama.cpp backend.14 15> **Live Dashboard**: [https://likhonsheikh-anthropic-compatible-api.hf.space](https://likhonsheikh-anthropic-compatible-api.hf.space)16 17## Features18 19| Feature | Description |20|---------|-------------|21| **Full Anthropic API** | Complete Messages API compatibility |22| **OpenAI API** | Dual compatibility with OpenAI Chat API |23| **Streaming (SSE)** | Real-time token streaming |24| **Tool Use** | Function calling / tool use support |25| **Extended Thinking** | `<thinking>` block support for reasoning |26| **Request Queue** | Concurrency control with priority |27| **Prompt Caching** | LRU cache for system prompts |28| **Multi-Model** | Hot-swap between models |29| **Live Dashboard** | Built-in web UI with playground |30| **Logs Viewer** | Real-time API logs |31 32---33 34## Quick Start35 36### 1. Claude Code CLI37 38The easiest way to use this API with Claude Code:39 40```bash41# Set environment variables42export ANTHROPIC_API_KEY="any-key"43export ANTHROPIC_BASE_URL="https://likhonsheikh-anthropic-compatible-api.hf.space/anthropic"44 45# Run Claude Code46claude "Write a Python script that reads a CSV file"47 48# Or with explicit model49claude --model qwen2.5-coder-7b "Explain this code"50```51 52**Persistent Configuration** (add to `~/.bashrc` or `~/.zshrc`):53 54```bash55# Anthropic-Compatible API Configuration56export ANTHROPIC_API_KEY="any-key"57export ANTHROPIC_BASE_URL="https://likhonsheikh-anthropic-compatible-api.hf.space/anthropic"58```59 60### 2. Python SDK61 62```python63import anthropic64 65client = anthropic.Anthropic(66    api_key="any-key",67    base_url="https://likhonsheikh-anthropic-compatible-api.hf.space/anthropic"68)69 70# Basic message71message = client.messages.create(72    model="qwen2.5-coder-7b",73    max_tokens=1024,74    messages=[{"role": "user", "content": "Hello! Write a hello world in Python."}]75)76print(message.content[0].text)77 78# With system prompt79message = client.messages.create(80    model="qwen2.5-coder-7b",81    max_tokens=1024,82    system="You are a helpful coding assistant. Always include comments in your code.",83    messages=[{"role": "user", "content": "Write a function to calculate factorial"}]84)85print(message.content[0].text)86```87 88### 3. Streaming Response89 90```python91import anthropic92 93client = anthropic.Anthropic(94    api_key="any-key",95    base_url="https://likhonsheikh-anthropic-compatible-api.hf.space/anthropic"96)97 98with client.messages.stream(99    model="qwen2.5-coder-7b",100    max_tokens=1024,101    messages=[{"role": "user", "content": "Write a detailed explanation of recursion"}]102) as stream:103    for text in stream.text_stream:104        print(text, end="", flush=True)105```106 107### 4. Tool Use / Function Calling108 109```python110import anthropic111import json112 113client = anthropic.Anthropic(114    api_key="any-key",115    base_url="https://likhonsheikh-anthropic-compatible-api.hf.space/anthropic"116)117 118tools = [119    {120        "name": "get_weather",121        "description": "Get the current weather for a location",122        "input_schema": {123            "type": "object",124            "properties": {125                "location": {"type": "string", "description": "City name"},126                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}127            },128            "required": ["location"]129        }130    }131]132 133message = client.messages.create(134    model="qwen2.5-coder-7b",135    max_tokens=1024,136    tools=tools,137    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}]138)139 140if message.stop_reason == "tool_use":141    for block in message.content:142        if block.type == "tool_use":143            print(f"Tool: {block.name}")144            print(f"Input: {json.dumps(block.input, indent=2)}")145```146 147### 5. Extended Thinking148 149```python150import anthropic151 152client = anthropic.Anthropic(153    api_key="any-key",154    base_url="https://likhonsheikh-anthropic-compatible-api.hf.space/anthropic"155)156 157message = client.messages.create(158    model="qwen2.5-coder-7b",159    max_tokens=2048,160    thinking={"type": "enabled", "budget_tokens": 1024},161    messages=[{"role": "user", "content": "Solve step by step: What is 15% of 240?"}]162)163 164for block in message.content:165    if block.type == "thinking":166        print("=== THINKING ===")167        print(block.thinking)168    elif block.type == "text":169        print("=== ANSWER ===")170        print(block.text)171```172 173### 6. TypeScript/JavaScript174 175```typescript176import Anthropic from '@anthropic-ai/sdk';177 178const client = new Anthropic({179  apiKey: 'any-key',180  baseURL: 'https://likhonsheikh-anthropic-compatible-api.hf.space/anthropic'181});182 183const message = await client.messages.create({184  model: 'qwen2.5-coder-7b',185  max_tokens: 1024,186  messages: [{ role: 'user', content: 'Hello!' }]187});188 189console.log(message.content[0].text);190```191 192### 7. cURL193 194```bash195curl -X POST "https://likhonsheikh-anthropic-compatible-api.hf.space/anthropic/v1/messages" \196  -H "Content-Type: application/json" \197  -H "x-api-key: any-key" \198  -H "anthropic-version: 2023-06-01" \199  -d '{200    "model": "qwen2.5-coder-7b",201    "max_tokens": 256,202    "messages": [{"role": "user", "content": "Hello!"}]203  }'204```205 206### 8. OpenAI SDK (Alternative)207 208```python209from openai import OpenAI210 211client = OpenAI(212    api_key="any-key",213    base_url="https://likhonsheikh-anthropic-compatible-api.hf.space/v1"214)215 216response = client.chat.completions.create(217    model="qwen2.5-coder-7b",218    messages=[{"role": "user", "content": "Hello!"}],219    max_tokens=1024220)221print(response.choices[0].message.content)222```223 224---225 226## API Reference227 228### Endpoints229 230| Method | Endpoint | Description |231|--------|----------|-------------|232| `GET` | `/` | Dashboard with status & playground |233| `GET` | `/health` | Health check with queue/cache stats |234| `GET` | `/logs?lines=100` | View API logs |235| `GET` | `/queue/status` | Request queue statistics |236| `GET` | `/models/status` | Loaded models information |237| `POST` | `/models/{id}/load` | Manually load a model |238| `POST` | `/models/{id}/unload` | Unload a model |239| `GET` | `/anthropic/v1/models` | List models (Anthropic format) |240| `POST` | `/anthropic/v1/messages` | Create message (Anthropic API) |241| `POST` | `/anthropic/v1/messages/count_tokens` | Count tokens |242| `GET` | `/v1/models` | List models (OpenAI format) |243| `POST` | `/v1/chat/completions` | Chat completion (OpenAI API) |244 245### Request Format246 247```json248{249  "model": "qwen2.5-coder-7b",250  "max_tokens": 1024,251  "messages": [{"role": "user", "content": "Hello!"}],252  "system": "You are a helpful assistant.",253  "temperature": 0.7,254  "stream": false,255  "tools": [...],256  "thinking": {"type": "enabled", "budget_tokens": 1024}257}258```259 260### Response Format261 262```json263{264  "id": "msg_abc123",265  "type": "message",266  "role": "assistant",267  "content": [{"type": "text", "text": "Hello!"}],268  "model": "qwen2.5-coder-7b",269  "stop_reason": "end_turn",270  "usage": {"input_tokens": 10, "output_tokens": 25}271}272```273 274---275 276## Model Info277 278| Property | Value |279|----------|-------|280| **Model** | Qwen2.5-Coder-7B-Instruct |281| **Format** | GGUF (Q4_K_M quantization) |282| **Parameters** | 7 Billion |283| **Context Length** | 8,192 tokens |284| **Backend** | llama.cpp |285| **Optimized For** | Code, tool use, agent workflows |286 287---288 289## Troubleshooting290 291| Issue | Solution |292|-------|----------|293| Connection Timeout | Space may be sleeping. First request wakes it (~30s) |294| 503 Queue Full | Too many requests. Retry in a few seconds |295| Slow Response | CPU-based, expect ~10-30 tokens/second |296| Tool Use Issues | Ensure valid JSON schema |297 298---299 300## License301 302Apache 2.0 | Built with llama.cpp + FastAPI by Matrix Agent303