mabemi/onellmWeb
<p align="center"> <img src="https://raw.githubusercontent.com/onellm/onellm-java/main/assets/logo.png" alt="OneLLM Logo" width="180"/> </p>
<h1 align="center">๐ OneLLM</h1>
<p align="center"> <strong>One Interface. Eighteen Providers. Your API Keys.</strong> </p>
<p align="center"> <a href="#-quick-start">Quick Start</a> โข <a href="#-providers">Providers</a> โข <a href="#-rest-api">REST API</a> โข <a href="#-sdk-usage">SDK Usage</a> โข <a href="#-configuration">Configuration</a> </p>
<p align="center"> <img src="https://img.shields.io/badge/Java-17+-blue?style=for-the-badge&logo=openjdk" alt="Java 17+"/> <img src="https://img.shields.io/badge/Spring%20Boot-3.2.0-brightgreen?style=for-the-badge&logo=springboot" alt="Spring Boot 3.2.0"/> <img src="https://img.shields.io/badge/License-MIT-yellow?style=for-the-badge" alt="MIT License"/> <img src="https://img.shields.io/badge/BYOK-Bring%20Your%20Own%20Key-orange?style=for-the-badge" alt="BYOK"/> </p>
OneLLM is a unified Java SDK and REST API that provides a single interface for calling 18 different LLM providers. Users bring their own API keys โ just specify the model name and your API key, and OneLLM automatically routes your request to the right provider.
๐ Bring Your Own Key (BYOK): OneLLM doesn't store or require server-side API keys. Users provide their own API keys in each request, making it perfect for multi-tenant applications.
โจ Features
๐ Quick Start
Prerequisites
- Java 17 or higher
- Maven 3.6+
Run the Server
# Clone and build
git clone https://github.com/onellm/onellm-java.git
cd onellm
# Run (no API keys needed - users provide their own!)
mvn spring-boot:runThe server starts at http://localhost:8080
Make Your First Request
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"apiKey": "YOUR_OPENAI_API_KEY",
"model": "gpt-4",
"messages": [{"role": "user", "content": "Hello!"}]
}'Note: Replace YOUR_OPENAI_API_KEY with your actual API key from the respective provider.๐ Providers
OneLLM supports 18 LLM providers out of the box:
Model Auto-Detection
OneLLM automatically routes to the correct provider based on model name:
"gpt-4" โ OpenAI
"claude-3-opus" โ Anthropic
"gemini-1.5-pro" โ Google
"llama-3-70b" โ Groq
"grok-1" โ xAIYou can also use explicit provider prefixes:
"openai/gpt-4"
"anthropic/claude-3-opus"
"google/gemini-pro"
"gemma/gemma-3-27b-it"
"azure/my-deployment"
"huggingface/meta-llama/Llama-3.3-70B-Instruct"
"hf/mistralai/Mistral-7B-Instruct-v0.3"
"freellm/TinyLlama/TinyLlama-1.1B-Chat-v1.0"
"perplexity/sonar"
"aiml/openai/gpt-4o"๐ REST API
Base URL
http://localhost:8080/apiEndpoints
POST /api/chat/completions
Send a chat completion request with your own API key.
Request Body:
{
"apiKey": "sk-your-api-key-here",
"model": "gpt-4",
"messages": [
{ "role": "system", "content": "You are a helpful assistant." },
{ "role": "user", "content": "Hello, who are you?" }
],
"temperature": 0.7,
"maxTokens": 1000,
"topP": 0.9,
"frequencyPenalty": 0.0,
"presencePenalty": 0.0,
"stop": ["END"],
"stream": false,
"baseUrl": "https://custom-endpoint.com/v1",
"azureResourceName": "my-resource",
"azureDeploymentName": "gpt-4",
"openRouterSiteName": "MyApp",
"openRouterSiteUrl": "https://myapp.com"
}Request Attributes:
Provider-Specific Fields:
Web Search Fields (Real-Time Data):
Response:
{
"id": "chatcmpl-abc123",
"model": "gpt-4-0613",
"content": "Hello! I'm an AI assistant powered by GPT-4...",
"finishReason": "stop",
"provider": "openai",
"latencyMs": 1234,
"usage": {
"promptTokens": 25,
"completionTokens": 45,
"totalTokens": 70
}
}POST /api/chat/completions/stream
Stream responses using Server-Sent Events (SSE).
Request: Same as /chat/completions
Response: SSE stream with events:
event: chunk
data: {"content": "Hello"}
event: chunk
data: {"content": ", I'm"}
event: complete
data: {"id": "...", "model": "gpt-4", "content": "Hello, I'm...", ...}GET /api/providers
List all supported providers.
Response:
{
"providers": ["openai", "anthropic", "google", "azure", "groq", "cerebras", "ollama", "openrouter", "xai", "copilot", "github", "huggingface", "freellm", "llamacpp", "rwkv", "perplexity", "aiml"],
"count": 17
}GET /api/health
Health check endpoint.
Response:
{
"status": "ok",
"service": "OneLLM"
}๐ API Examples
OpenAI (GPT-4)
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"apiKey": "sk-your-openai-key",
"model": "gpt-4",
"messages": [
{"role": "system", "content": "You are a helpful coding assistant."},
{"role": "user", "content": "Write a Python function to reverse a string."}
],
"temperature": 0.5,
"maxTokens": 500
}'Anthropic (Claude)
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"apiKey": "sk-ant-your-anthropic-key",
"model": "claude-3-opus",
"messages": [
{"role": "user", "content": "Explain quantum computing in simple terms."}
],
"maxTokens": 1000
}'Google Gemini
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"apiKey": "AIza-your-google-key",
"model": "gemini-1.5-pro",
"messages": [
{"role": "user", "content": "What is the meaning of life?"}
]
}'Google Gemma (Free via Google API)
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"apiKey": "AIza-your-google-key",
"model": "gemma/gemma-3-27b-it",
"messages": [
{"role": "user", "content": "Explain machine learning simply."}
]
}'Note: Gemma models use the same Google API key as Gemini. Available models:gemma-3-27b-it,gemma-3-12b-it,gemma-3-4b-it,gemma-2-27b-it,gemma-2-9b-it.
Azure OpenAI
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"apiKey": "your-azure-api-key",
"model": "azure/my-gpt4-deployment",
"azureResourceName": "my-azure-resource",
"azureDeploymentName": "my-gpt4-deployment",
"messages": [
{"role": "user", "content": "Hello from Azure!"}
]
}'Groq (Fast Inference)
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"apiKey": "gsk_your-groq-key",
"model": "llama-3-70b",
"messages": [
{"role": "user", "content": "Write a haiku about coding."}
]
}'OpenRouter (100+ Models)
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"apiKey": "sk-or-your-openrouter-key",
"model": "openrouter/anthropic/claude-3-opus",
"openRouterSiteName": "MyApp",
"openRouterSiteUrl": "https://myapp.com",
"messages": [
{"role": "user", "content": "Hello via OpenRouter!"}
]
}'xAI (Grok)
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"apiKey": "xai-your-xai-key",
"model": "grok-1",
"messages": [
{"role": "user", "content": "Tell me a joke."}
]
}'Perplexity (with Web Search)
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"apiKey": "pplx-your-perplexity-key",
"model": "perplexity/sonar",
"messages": [
{"role": "user", "content": "What are the latest AI news?"}
]
}'Note: Perplexity models (sonar,sonar-pro) include built-in web search capabilities for real-time information.
AIML API (Multiple Providers)
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"apiKey": "your-aiml-api-key",
"model": "aiml/openai/gpt-4o",
"messages": [
{"role": "user", "content": "Hello from AIML API!"}
]
}'Note: AIML API provides access to 200+ models including GPT-4o, Gemma, Llama, Mistral, and Qwen. Get your API key at aimlapi.com.
GitHub Models
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"apiKey": "github_pat_your-token",
"model": "github/gpt-4o",
"messages": [
{"role": "user", "content": "Hello from GitHub Models!"}
]
}'Note: Use your GitHub Personal Access Token (PAT) as the API key. GitHub Models provides access to GPT-4o, Llama, Mistral, Phi, and other models.
Ollama (Free Hosted Models) ๐
Ollama is hosted on Hugging Face Spaces - no local installation required!
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "ollama/gemma:2b",
"messages": [
{"role": "user", "content": "Hello from Ollama!"}
]
}'Note: No API key required! Ollama is completely free to use via our hosted endpoint.
Free Ollama Models: | Model | Size | Speed | Quality | Description | |-------|------|-------|---------|-------------| | gemma:270M | 270M | โกโกโก | โญโญโญ | Google's lightweight Gemma model | | gemma:4b | 4B | โกโก | โญโญโญโญ | Google's Gemma model | | mistral:7b | 7B | โกโก | โญโญโญโญ | Mistral AI's powerful model |
Using Local Ollama: To use a local Ollama instance instead, specify a custom baseUrl:
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "ollama/llama2",
"baseUrl": "http://localhost:11434",
"messages": [
{"role": "user", "content": "Hello from local Ollama!"}
]
}'Hugging Face
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"apiKey": "hf_your-huggingface-token",
"model": "huggingface/meta-llama/Llama-3.3-70B-Instruct",
"messages": [
{"role": "user", "content": "Hello from Hugging Face!"}
],
"maxTokens": 500
}'๐ Web Search (Real-Time Data)
Enable real-time web search to give LLMs access to current information:
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"apiKey": "sk-your-openai-key",
"model": "gpt-4o",
"search": true,
"searchResultCount": 3,
"messages": [
{"role": "user", "content": "Who won the 2024 US presidential election?"}
]
}'How it works: When search: true is set, OneLLM: 1. Extracts the user's question 2. Searches the web for relevant, current information 3. Injects the search results as context for the LLM 4. Returns a response with up-to-date dataSearch Parameters: | Parameter | Description | |-----------|-------------| | search | Enable web search (true/false) | | searchResultCount | Number of results (default: 3) | | searchLanguage | Language code (e.g., en) | | searchCountry | Country code (e.g., US) |
Streaming Example
curl -X POST http://localhost:8080/api/chat/completions/stream \
-H "Content-Type: application/json" \
-d '{
"apiKey": "sk-your-openai-key",
"model": "gpt-4",
"messages": [{"role": "user", "content": "Count from 1 to 10 slowly."}]
}'FreeLLM (Free - No API Key!)
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "freellm/TinyLlama/TinyLlama-1.1B-Chat-v1.0",
"messages": [
{"role": "user", "content": "Hello! What can you help me with?"}
],
"maxTokens": 256
}'Note: FreeLLM is completely free - no API key required! It's hosted on Hugging Face Spaces with no rate limiting or billing. Perfect for testing and development.
Free FreeLLM Models (No API Key Required!): | Model | Size | Speed | Quality | Description | |-------|------|-------|---------|-------------| | TinyLlama/TinyLlama-1.1B-Chat-v1.0 | 1.1B | โกโกโก | โญโญ | Fast, lightweight chat model | | Qwen/Qwen2.5-0.5B-Instruct | 0.5B | โกโกโก | โญโญ | Ultra-fast, smallest model | | Qwen/Qwen2.5-1.5B-Instruct | 1.5B | โกโก | โญโญโญ | Balanced speed and quality |
Streaming Example
curl -X POST http://localhost:8080/api/chat/completions/stream \
-H "Content-Type: application/json" \
-d '{
"apiKey": "sk-your-openai-key",
"model": "gpt-4",
"messages": [{"role": "user", "content": "Count from 1 to 10 slowly."}]
}'๐ป JavaScript/TypeScript Client
// Basic request
const response = await fetch('http://localhost:8080/api/chat/completions', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
apiKey: 'sk-your-api-key',
model: 'gpt-4',
messages: [
{ role: 'user', content: 'Hello!' }
]
})
});
const data = await response.json();
console.log(data.content);
// Streaming request
const eventSource = new EventSource('http://localhost:8080/api/chat/completions/stream', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
apiKey: 'sk-your-api-key',
model: 'gpt-4',
messages: [{ role: 'user', content: 'Write a story' }]
})
});
eventSource.addEventListener('chunk', (e) => {
const data = JSON.parse(e.data);
process.stdout.write(data.content);
});
eventSource.addEventListener('complete', (e) => {
console.log('\nDone!');
eventSource.close();
});๐ฆ SDK Usage
Use OneLLM programmatically in your Java application:
Basic Usage
import io.onellm.OneLLM;
import io.onellm.core.*;
// Build the client with your API keys
OneLLM llm = OneLLM.builder()
.openai("sk-your-openai-key")
.anthropic("sk-ant-your-anthropic-key")
.google("AIza-your-google-key")
.build();
// Send a completion request
LLMResponse response = llm.complete(
LLMRequest.builder()
.model("gpt-4")
.system("You are a helpful assistant.")
.user("Explain quantum computing in simple terms.")
.temperature(0.7)
.maxTokens(500)
.build()
);
System.out.println(response.getContent());
System.out.println("Provider: " + response.getProvider());
System.out.println("Latency: " + response.getLatencyMs() + "ms");Streaming
llm.streamComplete(
LLMRequest.builder()
.model("claude-3-opus")
.user("Write a story about a robot learning to cook.")
.build(),
new StreamHandler() {
@Override
public void onChunk(String chunk) {
System.out.print(chunk);
}
@Override
public void onComplete(LLMResponse response) {
System.out.println("\n\nDone! Tokens: " + response.getUsage().getTotalTokens());
}
@Override
public void onError(Throwable error) {
System.err.println("Error: " + error.getMessage());
}
}
);Builder Methods
OneLLM llm = OneLLM.builder()
.openai("sk-...") // OpenAI
.openai("sk-...", "https://custom-url.com") // Custom base URL
.anthropic("sk-ant-...") // Anthropic
.google("AIza...") // Google Gemini & Gemma
.azure("api-key", "resource", "deployment") // Azure OpenAI
.groq("gsk_...") // Groq
.cerebras("cbs-...") // Cerebras
.ollama() // Ollama (localhost)
.ollama("http://custom-host:11434") // Ollama (custom)
.openRouter("or-...") // OpenRouter
.openRouter("or-...", "MySite", "https://...") // OpenRouter with site
.xai("xai-...") // xAI
.copilot("token") // GitHub Copilot
.huggingface("hf_...") // Hugging Face
.huggingface("hf_...", "https://endpoint") // Hugging Face (dedicated endpoint)
.freellm() // FreeLLM (free, no API key!)
.freellm("https://custom-freellm") // FreeLLM (custom host)
.perplexity("pplx-...") // Perplexity AI (web search)
.aiml("aiml-...") // AIML API (200+ models)
.provider(myCustomProvider) // Custom provider
.build();๐ก๏ธ Error Handling
OneLLM provides structured error responses:
{
"error": true,
"message": "API key is required",
"timestamp": "2024-12-12T14:30:00Z",
"type": "validation_error",
"fields": {
"apiKey": "API key is required"
}
}Error Types
๐๏ธ Project Structure
onellm/
โโโ src/main/java/io/onellm/
โ โโโ OneLLM.java # SDK entry point
โ โโโ OneLLMApplication.java # Spring Boot application
โ โโโ config/
โ โ โโโ LLMConfig.java # Spring configuration
โ โโโ controller/
โ โ โโโ ChatController.java # REST API endpoints
โ โโโ service/
โ โ โโโ ProviderFactory.java # Dynamic provider creation
โ โโโ core/
โ โ โโโ LLMProvider.java # Provider interface
โ โ โโโ LLMRequest.java # Request model
โ โ โโโ LLMResponse.java # Response model
โ โ โโโ Message.java # Chat message
โ โ โโโ StreamHandler.java # Streaming callback
โ โ โโโ Usage.java # Token usage
โ โโโ dto/
โ โ โโโ ChatCompletionRequest.java
โ โ โโโ ChatCompletionResponse.java
โ โ โโโ MessageDTO.java
โ โโโ exception/
โ โ โโโ GlobalExceptionHandler.java
โ โ โโโ LLMException.java
โ โ โโโ ModelNotFoundException.java
โ โ โโโ ProviderNotConfiguredException.java
โ โโโ providers/
โ โ โโโ BaseProvider.java
โ โ โโโ OpenAIProvider.java
โ โ โโโ AnthropicProvider.java
โ โ โโโ GoogleProvider.java # Gemini & Gemma models
โ โ โโโ AzureOpenAIProvider.java
โ โ โโโ GroqProvider.java
โ โ โโโ CerebrasProvider.java
โ โ โโโ OllamaProvider.java
โ โ โโโ OpenRouterProvider.java
โ โ โโโ XAIProvider.java
โ โ โโโ CopilotProvider.java
โ โ โโโ GitHubModelsProvider.java
โ โ โโโ HuggingFaceProvider.java
โ โ โโโ FreeLLMProvider.java
โ โ โโโ LlamaCppProvider.java
โ โ โโโ RWKVProvider.java
โ โ โโโ PerplexityProvider.java
โ โ โโโ AIMLProvider.java
โ โโโ util/
โ โโโ HttpClientWrapper.java
โโโ pom.xml๐ Security Notes
- API keys are never stored on the server
- Each request is processed independently with the provided credentials
- Use HTTPS in production to encrypt API keys in transit
- Consider implementing rate limiting for production deployments
๐ค Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
- Fork the repository
- Create your feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
๐ License
This project is licensed under the MIT License - see the LICENSE file for details.
๐ Acknowledgments
Built with โค๏ธ using:
<p align="center"> Made with โ by the OneLLM Team </p>
