CoolFace
Apppublic

softjapan/gpt-oss-120b-demo

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face
0likes
App README

OpenRouter Demo Playground

This Hugging Face Space showcases how to call the OpenRouter chat-completions API from a Gradio interface. It is built as a lightweight template for teams that want to stream model output, tweak decoding settings on the fly, and inspect thinking traces (<think>...</think>) without wiring up a bespoke frontend.

Highlights

  • —AsyncOpenAI streaming – responses flow token-by-token using the official OpenAI Python client pointed at https://openrouter.ai/api/v1.
  • —Model picker + controls – choose any OpenRouter model (dropdown accepts custom values) and live-adjust max_tokens, temperature, top_p, presence_penalty, and frequency_penalty.
  • —Thinking inspector – <think> sections are captured and rendered in collapsible panels underneath the assistant reply for models that emit hidden reasoning traces.
  • —PWA capable UI – Gradio launches as a progressive web app, so you can pin the playground to your dock or home screen.

Repository Layout

.
├── app.py            # Gradio blocks + OpenRouter integration
├── Dockerfile        # Space runtime definition (installs requirements, runs app.py)
├── requirements.txt  # Dependencies: gradio + openai
└── README.md         # Project documentation

Configuration

Set the following secrets in the Hugging Face Space before launching:

  • —`OPENROUTER_API_KEY` (required) – API key from your OpenRouter dashboard.
  • —`OPENROUTER_API_BASE_URL` (optional) – defaults to https://openrouter.ai/api/v1.
  • —`OPENROUTER_APP_URL` (optional) – sent as the HTTP-Referer header for ranking metrics.
  • —`OPENROUTER_APP_TITLE` (optional) – sent as the X-Title header for ranking metrics.
  • —`OPENROUTER_SYSTEM_PROMPT` (optional) – injected as the first system message in every conversation.
  • —`OPENROUTER_DEFAULT_MODEL` (optional) – pre-selects the dropdown value.
  • —`OPENROUTER_DEFAULT_MAX_TOKENS` (optional) – overrides the initial max_tokens slider value.

All configuration happens at runtime—no code changes required after setting the secrets.

How the App Works

  1. 1.Gradio collects the latest user prompt and conversation history (assistant + user roles).
  2. 2.app.py rebuilds the message list and initialises AsyncOpenAI with the OpenRouter base URL and API key.
  3. 3.The call to client.chat.completions.create(..., stream=True) yields delta chunks; each new token is appended to the response buffer and streamed back to the chat UI.
  4. 4.Whenever <think> blocks are encountered, they are separated and displayed inside collapsible <details> tags so you can inspect or hide the raw reasoning trace.

Deploying to Your Space

  1. 1.Clone this repository and log in with the Hugging Face CLI: huggingface-cli login (token with write scope).
  2. 2.Add your Space as a remote: git remote add hf https://huggingface.co/spaces/softjapan/ollama-demo-playground.
  3. 3.Push the branch: git push hf main (adjust branch name if your Space tracks something else).
  4. 4.Open the Space, watch the Docker build logs complete, and confirm the UI boots at port 7860.
  5. 5.Enter a test prompt to verify the OpenRouter call succeeds with your API key and model selection.

Customisation Ideas

  • —Swap the default model list to match your organisation's preferred OpenRouter endpoints (look for the dropdown in app.py).
  • —Extend the sidebar Markdown with onboarding tips, usage quotas, or guardrail reminders.
  • —Implement lightweight usage logging by tapping into the streaming loop when each chunk arrives.
  • —Surface guardrail toggles (e.g. safety filters) by exposing additional request parameters through Gradio inputs.

Troubleshooting

  • —`OPENROUTER_API_KEY is not configured` – ensure the secret is populated and redeploy the Space.
  • —HTTP 401/403 responses – verify the API key is valid and allowed to access the chosen model.
  • —Streaming stops early – reduce max_tokens or try a different model; some rate limits clamp long outputs.
  • —Thinking blocks missing – the selected model might not emit <think> segments; switch to a thinking-capable model if you need the trace.

Credits & License

The project builds upon an Ollama-based playground by Hadad (UltimaX Intelligence) and is now adapted for OpenRouter. Source files include SPDX headers and are distributed under the Apache-2.0 license. Feedback and collaboration requests are welcome via Twitter @fullstack_se.