CoolFace
Modelpublic

hfendpoints-images/asr-multilingual-whisper

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face
10likes
Model Card

Inference Endpoint - Multilingual Audio Transcription with Whisper models

Deploy OpenAI's Whisper Inference Endpoint to transcribe audio files to text in many languages

Resulting deployment exposes an OpenAI Platform Transcription compatible HTTP endpoint which you can query using the OpenAi Libraries or directly through cURL for instance.

Available Routes

pathdescription
/api/v1/audio/transcriptionsTranscription endpoint to interact with the model
/docsVisual documentation

Getting started

  • —Getting text output from audio file
bash
curl http://localhost:8000/api/v1/audio/transcriptions \
  --request POST \
  --header 'Content-Type: multipart/form-data' \
  -F file=@</path/to/audio/file> \
  -F "response_format": "text"
  • —Getting JSON output from audio file
bash
curl http://localhost:8000/api/v1/audio/transcriptions \
  --request POST \
  --header 'Content-Type: multipart/form-data' \
  -F file=@</path/to/audio/file> \
  -F "response_format": "json"
  • —Getting segmented JSON output from audio file
bash
curl http://localhost:8000/api/v1/audio/transcriptions \
  --request POST \
  --header 'Content-Type: multipart/form-data' \
  -F file=@</path/to/audio/file> \
  -F "response_format": "verbose_json"

Specifications

specvaluedescription
EnginevLLM (v0.8.3)Underlying inference engine leverages vLLM
HardwareGPU (Ada Lovelace)Requires the target endpoint to run over NVIDIA GPUs with at least compute capabilities 8.9 (Ada Lovelace)
Compute data typebfloat16Computations (matmuls, norms, etc.) are done using bfloat16 precision
KV cache data typefloat8 (e4m3)Key-Value cache is stored on the GPU using float8 (float8_e4m3) precision to save space
PyTorch Compile✅Enable the use of torch.compile to further optimize model's execution with more optimizations
CUDA Graphs✅Enable the use of so called "CUDA Graphs" to reduce overhead executing GPU computations