Modelpublic

abnerh/phonoq-2.0-multilingual

sourceHugging Facemitupdated 23d agoView on Hugging Face

2likes43downloads

Model Card

PhonoQ 2.0 Multilingual

Framewise phonological feature recognition for multilingual speech.

This model returns phonological probabilities for manner, vowel height, vowel backness, place, and voicing, plus a hard conditional 22-feature representation per frame. It is a modernized successor to the original PhonoQ system: https://github.com/TAriasVergara/PhonoQ

Usage

bash

pip install torch "transformers>=4.35,<6" soundfile safetensors

python

import soundfile as sf
import torch
from transformers import AutoFeatureExtractor, AutoModel

model_id = "abnerh/phonoq-2.0-multilingual"
audio, sr = sf.read("example.wav")
if audio.ndim > 1:
    audio = audio.mean(axis=1)

processor = AutoFeatureExtractor.from_pretrained(model_id)
inputs = processor(audio, sampling_rate=sr, return_tensors="pt", padding=True)

model = AutoModel.from_pretrained(model_id, trust_remote_code=True)
model.eval()

with torch.no_grad():
    out = model(**inputs)

print(out.features.shape)              # [1, T, 22]
print(out.manner_probabilities.shape)  # [1, T, 9]
print(out.vowel_height.shape)          # [1, T, 3]
print(out.vowel_backness.shape)        # [1, T, 3]
print(out.place_probabilities.shape)   # [1, T, 5]
print(out.voice_probabilities.shape)   # [1, T, 2]

Outputs

—features: hard conditional 22-dimensional features, [B, T, 22]
—manner_probabilities: [B, T, 9]
—vowel_height: [B, T, 3]
—vowel_backness: [B, T, 3]
—place_probabilities: [B, T, 5]
—voice_probabilities: [B, T, 2]
—attention_mask: valid encoder frames, [B, T]
—feature_names: names for the 22 feature dimensions

Feature order:

text

silence, stop, nasal, rhotic, fricative, affricate, approximant, lateral, vowel,
high, mid, low, front, central, back,
labial, alveolar, velar, palatal, postalveolar,
voiceless, voiced

Viewing Probabilities

The following snippet prints only the non-silence region.

python

manner_labels = [
    "silence", "stop", "nasal", "rhotic", "fricative", "affricate",
    "approximant", "lateral", "vowel",
]

manner = out.manner_probabilities[0]
mask = out.attention_mask[0].bool()
manner = manner[mask]

best_manner = manner.argmax(dim=-1)
non_silence = (best_manner != 0).nonzero(as_tuple=True)[0]

if len(non_silence) == 0:
    print("No non-silence frames found.")
else:
    start = int(non_silence[0])
    end = int(non_silence[-1]) + 1

    print(f"Non-silence frame range: {start}-{end - 1}")
    print()

    for frame_idx in range(start, end):
        probs = manner[frame_idx]
        best = int(probs.argmax())
        print(f"{frame_idx:03d}  {manner_labels[best]:10s}  {float(probs[best]):.3f}")

CLI

This repository includes best.ckpt for PhonoQ CLI compatibility:

bash

pip install git+https://github.com/abnerLing/PhonoQ-2.0.git

bash

phonoq predict example.wav \
  --model abnerh/phonoq-2.0-multilingual \
  --outdir outputs \
  --pretty

Notes

This model uses custom Transformers code and must be loaded with trust_remote_code=True.

The multilingual checkpoint is intended for use across English, Spanish, German, and Czech speech.