IBM AI Releases Granite Speech 5.0 2B for Multilingual Transcription and Translation
AIvia MarkTechPost
Quick Read
IBM has released Granite Speech 5.0 2B, a new open speech-language model for transcription and translation. Granite Speech 5.0 2B is a 2B-parameter model from IBM that turns speech into text in 6 languages. It ships under Apache 2.0. IBM also updated 2 existing models: the edge-focused Turbo CTC and the diarization-focused 4.1 2B Plus. For developers, the release covers accurate transcription, fast on-device transcription and speaker-attributed transcripts. TL;DR Size: 2B parameters, dense. LLM backbone uses YaRN extension to 16k positions. IBM recommends audio segments of up to about 2 minutes. Runs on: GPU via Transformers or vLLM, Apple Silicon via mlx-audio, local CPU/Mac via llama.cpp GGUF (Q8_0 recommended). Performance: #2 open-weight model on the Open ASR Leaderboard, 5.68% average English WER. Best: 0.99% WER on LibriSpeech Clean. Worst: 13.79% WER on the leaderboard’s private conversational set. How does Granite Speech 5.0 2B work? The model has 3 parts, per the model card. A speech encoder of 16 Conformer blocks is trained with CTC. It uses block attention over 4-second audio blocks. A 2-layer window q-former then projects and downsamples the acoustic embeddings. Total downsampling is 10x, so the LLM sees embeddings at 10 Hz. The LLM is an intermediate checkpoint of granite-4.0-1b-base. IBM extended it to 16k positions with YaRN and fine-tuned it with LoRA adapters. Training used 390,000 hours of public audio plus synthetic data. The synthetic sets target Japanese ASR, keyword biasing and translation. Training took 40 days on 8 H100 GPUs. What is new versus Granite Speech 4.1 2B? Language ID prompting: name the input language in the prompt to steer transcription. Keyword lists up to 2,000 terms: bias output toward product names, people and acronyms. Punctuation and capitalization on request: capitalization F1 runs from 90.16 to 99.51 across test sets. Robustness: better handling of noise, reverberation and distant microphones. How does Granite Speech 5.0 2B perform on benchmarks? On the Open ASR Leaderboard, IBM’s chart shows a 5.68% average English WER. That average covers 9 public and 2 private test sets, as of October 21, 2026. Only Qwen3-ASR-1.7B scores better, at 5.40%. Note a source conflict. IBM’s blog reports 5.06% instead. That figure is unofficial, dated October 15, and averages public short-form sets only. Per-set results range widely. The model scores 0.99% on LibriSpeech Clean and 2.21% on LibriSpeech Other. It scores 13.79% on the private conversational set. On the noisy FFASR leaderboard, it ranks #4 of 42 at 14.80% average WER. Qwen3-ASR-1.7B scores 13.41% there. Multilingual CommonVoice WER is 2.60% for Portuguese and 3.50% for Spanish. Japanese is the weakest language at 10.74%. For translation, X-to-English CoVoST BLEU reaches 53.6 for Portuguese and 40.1 for German. English-to-Mandarin was not evaluated. Keyword biasing shows the clearest gain. On Earnings-22, keyword F1 rises from 67.2 to 90.1 with correct keyword lists. With 2,000-term lists mixing correct and wrong terms, it still reaches 82.8. window.addEventListener("message",function(e){if(e.data&&e.data.gs5h){var f=document.getElementById("gs5frame");if(f)f.style.height=e.data.gs5h+"px";}}); How does it compare with other open ASR models? *Leaderboard averages as charted in IBM’s model card from the Open ASR and FFASR leaderboards. What changed in Turbo CTC and 4.1 2B Plus? Granite Speech 5.0 Turbo CTC keeps its 470M footprint. It now transcribes 7 languages with native punctuation and capitalization. IBM compiled one Conformer block and reused its graph across the stack. That lifted throughput to 19,870 RTFx, up from about 12,600 at launch. The updated Granite Speech 4.1 2B Plus is a drop-in weight update. Average timestamp shift falls from 210.86 to 98.54. Long-form WER drops from 13.53 to 10.22. Biased-keyword WER falls from 11.79 to 6.48. How can developers deploy it? Granite Speech 5.0 2B runs in Transformers, vLLM, llama.cpp and mlx-audio. For GGUF, IBM recommends Q8_0. Lower-bit quantizations lose accuracy on audio longer than about 1 minute. Key Takeaways #2 open-weight model on the Open ASR Leaderboard at 5.68% WER. 2B parameters, Apache 2.0, runs on GPU, Mac and local CPU. 2,000-term keyword biasing across all supported languages. Turbo CTC reaches 19,870 RTFx and now covers 7 languages. Qwen3-ASR-1.7B still leads on accuracy and noisy audio. Check out the model card, IBM blog and Granite Speech paper. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post IBM AI Releases Granite Speech 5.0 2B for Multilingual Transcription and Translation appeared first on MarkTechPost.
IBM has released Granite Speech 5.0 2B, a new open speech-language model for transcription and translation. Granite Speech 5.0 2B is a 2B-parameter model from IBM that turns speech into text in 6 languages. It ships under Apache 2.0. IBM also updated 2 existing models: the edge-focused Turbo CTC and the diarization-focused 4.1 2B Plus. For developers, the release covers accurate transcription, fast on-device transcription and speaker-attributed transcripts.
TL;DR
Size: 2B parameters, dense. LLM backbone uses YaRN extension to 16k positions. IBM recommends audio segments of up to about 2 minutes.
Runs on: GPU via Transformers or vLLM, Apple Silicon via mlx-audio, local CPU/Mac via llama.cpp GGUF (Q8_0 recommended).
Performance: #2 open-weight model on the Open ASR Leaderboard, 5.68% average English WER.
Best: 0.99% WER on LibriSpeech Clean.
Worst: 13.79% WER on the leaderboard’s private conversational set.
How does Granite Speech 5.0 2B work?
The model has 3 parts, per the model card. A speech encoder of 16 Conformer blocks is trained with CTC. It uses block attention over 4-second audio blocks.
A 2-layer window q-former then projects and downsamples the acoustic embeddings. Total downsampling is 10x, so the LLM sees embeddings at 10 Hz.
The LLM is an intermediate checkpoint of granite-4.0-1b-base. IBM extended it to 16k positions with YaRN and fine-tuned it with LoRA adapters.
Training used 390,000 hours of public audio plus synthetic data. The synthetic sets target Japanese ASR, keyword biasing and translation. Training took 40 days on 8 H100 GPUs.
What is new versus Granite Speech 4.1 2B?
Language ID prompting: name the input language in the prompt to steer transcription.
Keyword lists up to 2,000 terms: bias output toward product names, people and acronyms.
Punctuation and capitalization on request: capitalization F1 runs from 90.16 to 99.51 across test sets.
Robustness: better handling of noise, reverberation and distant microphones.
How does Granite Speech 5.0 2B perform on benchmarks?
On the Open ASR Leaderboard, IBM’s chart shows a 5.68% average English WER. That average covers 9 public and 2 private test sets, as of October 21, 2026. Only Qwen3-ASR-1.7B scores better, at 5.40%.
Note a source conflict. IBM’s blog reports 5.06% instead. That figure is unofficial, dated October 15, and averages public short-form sets only.
Per-set results range widely. The model scores 0.99% on LibriSpeech Clean and 2.21% on LibriSpeech Other. It scores 13.79% on the private conversational set.
On the noisy FFASR leaderboard, it ranks #4 of 42 at 14.80% average WER. Qwen3-ASR-1.7B scores 13.41% there.
Multilingual CommonVoice WER is 2.60% for Portuguese and 3.50% for Spanish. Japanese is the weakest language at 10.74%.
For translation, X-to-English CoVoST BLEU reaches 53.6 for Portuguese and 40.1 for German. English-to-Mandarin was not evaluated.
Keyword biasing shows the clearest gain. On Earnings-22, keyword F1 rises from 67.2 to 90.1 with correct keyword lists. With 2,000-term lists mixing correct and wrong terms, it still reaches 82.8.
*Leaderboard averages as charted in IBM’s model card from the Open ASR and FFASR leaderboards.
What changed in Turbo CTC and 4.1 2B Plus?
Granite Speech 5.0 Turbo CTC keeps its 470M footprint. It now transcribes 7 languages with native punctuation and capitalization. IBM compiled one Conformer block and reused its graph across the stack. That lifted throughput to 19,870 RTFx, up from about 12,600 at launch.
The updated Granite Speech 4.1 2B Plus is a drop-in weight update. Average timestamp shift falls from 210.86 to 98.54. Long-form WER drops from 13.53 to 10.22. Biased-keyword WER falls from 11.79 to 6.48.
How can developers deploy it?
Granite Speech 5.0 2B runs in Transformers, vLLM, llama.cpp and mlx-audio. For GGUF, IBM recommends Q8_0. Lower-bit quantizations lose accuracy on audio longer than about 1 minute.
Key Takeaways
#2 open-weight model on the Open ASR Leaderboard at 5.68% WER.
2B parameters, Apache 2.0, runs on GPU, Mac and local CPU.
2,000-term keyword biasing across all supported languages.
Turbo CTC reaches 19,870 RTFx and now covers 7 languages.
Qwen3-ASR-1.7B still leads on accuracy and noisy audio.
Check out the model card, IBM blog and Granite Speech paper. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
The post IBM AI Releases Granite Speech 5.0 2B for Multilingual Transcription and Translation appeared first on MarkTechPost.
Liquid AI just announced Liquid RIPL, short for Real-time Inference Pipeline. RIPL is a low-latency engine that serves Liquid Foundation Models (LFMs) inside a live request path. It returns complete generated responses end-to-end in under 50 ms. It also claims that latency holds while traffic arrives continuously at peak load. For developers, the message is simple. Swap gradient-boosted trees for a generative model, and keep the latency budget. TL;DR Engine: Serves LFM2.5 models from 230M to 2.6B parameters Runs on: Enterprise deployments on NVIDIA H100, H200, B200 and RTX PRO 6000 GPUs. Benchmarks used a single B200. Performance: 5.6 to 28.4x lower p95 latency than Qwen3.5 on vLLM, per Liquid’s own tests. Best: LFM2.5-1.2B hit 155.8 ms p95 vs 4,431 ms for Qwen3.5-0.8B, a 28.4x gap (256 in, 128 out). Worst: LFM2.5-2.6B gained only 5.6x (449.1 ms vs 2,499 ms). Sub-50 ms fails on longer requests. What problem does Liquid RIPL solve? Most serving stacks target large batches and long agentic jobs. A few seconds per request is acceptable there. Search ranking, fraud checks and payment authorization need answers in milliseconds. Liquid AI argues teams default to classical models for these paths because LLM serving is too slow. RIPL is built to remove that trade-off. How does RIPL work? Liquid AI describes RIPL as optimizing the full path, from request arrival to completed response. In several workloads, Liquid AI plots the same model, LFM2.5-350M, on vLLM and on RIPL. RIPL’s p95 line sits well below the vLLM line in each case. That isolates part of the gain to the engine, not just the model architecture. The LFM2.5 models themselves are hybrids. LFM2.5-1.2B-Instruct has 16 layers: 10 double-gated convolution blocks and 6 GQA blocks. LFM2.5-2.6B has 30 layers: 22 convolution blocks and 8 GQA blocks. RIPL benchmarks: how fast is it vs vLLM? Liquid measured p95 end-to-end latency under open-loop Poisson arrivals on one B200. LFMs ran on RIPL. Baselines, including MiniCPM5 and Granite 4.2, ran on vLLM. Source: Liquid AI blog, Figure 2. Sealf-reported. Liquid AI also mapped 6 real-time workloads against fixed budgets: Adtech query rewriting: 32 in, 32 out, 50 ms budget, LFM2.5-350M. Search intent classification: 128 in, 1 out, 20 ms budget. Search reranking: 1k in, 1 out, 30 ms budget. Payments risk scoring: 512 in, 16 out, 50 ms budget, LFM2.5-1.2B. Credit decisioning: 8k in, 64 out, 150 ms budget. Image alt text: 512×512 image and 64 tokens in, 16 out, 100 ms budget, LFM2.5-VL-450M. window.addEventListener("message",function(e){if(e.data&&e.data.mtpH){var f=document.getElementById("mtp-ripl-frame");if(f&&e.source===f.contentWindow){f.style.height=e.data.mtpH+"px";}}}); <!-- Fallback: upload liquid-ripl-explainer-standalone.html to /wp-content/uploads/ and use --> Competitor Comparison Is RIPL used in production? Yes, according to Liquid AI. Shopify is deploying LFMs across latency-sensitive commerce workloads. In search enhancement, LFMs run with sub-20 ms inference on production traffic. According to Liquid AI’s blog, they use about 50% fewer parameters than alternatives and deliver 2 to 10x faster inference. Key Takeaways RIPL is Liquid AI’s engine for LLMs inside live request paths. Liquid claims 5.6 to 28.4x lower p95 latency than Qwen3.5 on vLLM. Same-model tests (LFM2.5-350M) show RIPL itself beats vLLM. Shopify runs LFMs at sub-20 ms in production search. Enterprise-only, closed, and all figures are self-reported. Check out the RIPL announcement, try the latency calculator, and see the LFM2.5 model cards.. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Liquid AI Launches RIPL, a Real-Time Inference Engine for Sub-50 ms LFMs appeared first on MarkTechPost.
Gradium has open-sourced Phonon, a family of small text-to-speech models built to run on the device. The Gradium Phonon on-device TTS model comes in 31M and 40M sizes. The English Phonon 31m ships under Apache 2.0 and fits in a 64.5 MB file. Developers get natural speech with no cloud call, no GPU and no per-character bill. TL;DR Size: 31M parameters (excluding the audio tokenizer), 64.5 MB q8 GGUF, 24 kHz mono output. Phonon 40m: 40M, 74.6 MB. Runs on: CPUs, phones, Raspberry Pi 5, browsers (WASM, WebGPU) and NVIDIA GPUs. No GPU required. Performance: top English Elo among open models Gradium tested; about 7 ms to first audio on CPU. Best: 1.70% WER on a 500-sentence hard set, versus 3.50% for Kokoro (Gradium’s eval). Worst: 0.51% Seed-TTS LLM WER, behind Kokoro (0.45%) and Magpie (0.46%). What is Gradium Phonon? Phonon is an open-weight TTS family from Gradium, a Paris voice AI company spun out of Kyutai. It generates speech locally on phones, laptops, boards and browsers. The family has three models: Phonon 31m: English only, 37 built-in voices, Apache 2.0. Phonon 40m: 1 model per language (English, French, German, Spanish, Portuguese), CC BY-NC 4.0. Phonon 40m multilingual: all 5 languages in 1 model, 102 voices, CC BY-NC 4.0. How does Phonon work? Phonon uses a causal transformer backbone. Its prefix combines text and speaker embeddings. A SentencePiece tokenizer with a 4,000-token vocabulary reads raw text, with no phoneme conversion. The backbone generates 1 latent vector per 80 ms frame. A flow-matching head turns that vector plus noise into an audio embedding in a single step. A smaller variant of Kyutai’s Mimi decoder then outputs 24 kHz PCM audio. A separate end-of-speech head decides when to stop. Phonon 31m uses 6 transformer layers, width 512 and 8 attention heads. Phonon 40m uses 9 layers at the same width. Both are distilled from a larger Phonon teacher. Training ran on an NVIDIA H100 cluster with NCCL and CUDA Graphs. The work builds on Kyutai’s Continuous Audio Language Models research and Pocket TTS. <iframe id="mtp-phonon-frame" title="Gradium Phonon interactive explainer" srcdoc="<!doctype html> <title>Phonon Explainer</title> #mtp-phonon{--bg:#0d0d0b;--card:#17160f;--line:#2c2a22;--cream:#f2ede3;--mute:#a39e90;--acc:#ff7a3d;--acc2:#ffd2b8;--mtp:#76B900; font-family:Inter,system-ui,-apple-system,Segoe UI,Roboto,sans-serif!important;background:var(--bg)!important;color:var(--cream)!important; max-width:880px;margin:0 auto;border:1px solid var(--line)!important;border-radius:16px;overflow:hidden;position:relative;box-sizing:border-box} #mtp-phonon *{box-sizing:border-box} #mtp-phonon hr,#mtp-phonon p:empty,#mtp-phonon del,#mtp-phonon s{display:none!important} #mtp-phonon .dots{position:absolute;inset:0;background-image:radial-gradient(rgba(242,237,227,.07) 1px,transparent 1.4px);background-size:14px 14px;pointer-events:none} #mtp-phonon .hd{position:relative;padding:22px 24px 14px;border-bottom:1px solid var(--line)!important} #mtp-phonon .kick{font-size:11px;letter-spacing:.14em;text-transform:uppercase;color:var(--acc)!important;font-weight:700} #mtp-phonon h3{margin:6px 0 4px!important;font-size:24px!important;line-height:1.2!important;color:var(--cream)!important;font-weight:800} #mtp-phonon .sub{color:var(--mute)!important;font-size:14px;margin:0} #mtp-phonon .tabs{position:relative;display:flex;gap:6px;padding:12px 24px 0;flex-wrap:wrap} #mtp-phonon .tab{background:transparent!important;border:1px solid var(--line)!important;color:var(--mute)!important;padding:8px 14px;border-radius:999px;font:600 13px Inter,system-ui,sans-serif;cursor:pointer;transition:all .25s} #mtp-phonon .tab:hover{color:var(--cream)!important;border-color:var(--mute)!important} #mtp-phonon .tab.on{background:var(--cream)!important;color:#111!important;border-color:var(--cream)!important} #mtp-phonon .panel{display:none;position:relative;padding:20px 24px 22px;animation:mtpfade .45s ease} #mtp-phonon .panel.on{display:block} @keyframes mtpfade{from{opacity:0;transform:translateY(8px)}to{opacity:1;transform:none}} #mtp-phonon .grid3{display:grid;grid-template-columns:repeat(3,1fr);gap:12px} #mtp-phonon .mc{background:var(--card)!important;border:1px solid var(--line)!important;border-radius:12px;padding:14px;cursor:pointer;transition:transform .25s,border-color .25s} #mtp-phonon .mc:hover,#mtp-phonon .mc.on{transform:translateY(-3px);border-color:var(--acc)!important} #mtp-phonon .mc b{display:block;font-size:16px;color:var(--cream)!important} #mtp-phonon .mc .big{font-size:30px;font-weight:800;color:var(--acc)!important;margin:6px 0 2px} #mtp-phonon .mc small{color:var(--mute)!important;font-size:12px} #mtp-phonon .lic{display:inline-block;margin-top:8px;font-size:11px;font-weight:700;padding:3px 8px;border-radius:6px;border:1px solid var(--line)!important} #mtp-phonon .lic.ok{color:var(--mtp)!important;border-color:var(--mtp)!important} #mtp-phonon .lic.nc{color:var(--acc2)!important} #mtp-phonon .detail{margin-top:12px;background:var(--card)!important;border:1px dashed var(--line)!important;border-radius:12px;padding:12px 14px;font-size:14px;color:var(--cream)!important;min-height:48px} #mtp-phonon .pipe{display:flex;align-items:stretch;gap:6px;margin:4px 0 14px} #mtp-phonon .st{flex:1;background:var(--card)!important;border:1px solid var(--line)!important;border-radius:10px;padding:10px 8px;text-align:center;font-size:12px;color:var(--mute)!important;transition:all .35s;position:relative} #mtp-phonon .st .n{display:block;font-size:10px;letter-spacing:.1em;color:var(--mute)!important} #mtp-phonon .st b{display:block;font-size:13px;color:var(--cream)!important;margin-top:2px} #mtp-phonon .st.on{border-color:var(--acc)!important;background:#231a12!important;transform:translateY(-3px);box-shadow:0 0 0 3px rgba(255,122,61,.15)} #mtp-phonon .arr{align-self:center;color:var(--line)!important;font-size:14px} #mtp-phonon .frames{display:flex;gap:4px;height:46px;align-items:flex-end;margin:6px 0 12px;padding:6px;background:var(--card)!important;border:1px solid var(--line)!important;border-radius:10px;overflow:hidden} #mtp-phonon .fr{flex:1;background:var(--acc)!important;border-radius:2px;opacity:.25;height:20%;transition:height .3s,opacity .3s} #mtp-phonon .fr.live{opacity:1} #mtp-phonon .ctl{display:flex;gap:8px;align-items:center;flex-wrap:wrap} #mtp-phonon .btn{background:var(--acc)!important;color:#111!important;border:0!important;padding:8px 14px;border-radius:8px;font:700 13px Inter,system-ui,sans-serif;cursor:pointer} #mtp-phonon .btn.ghost{background:transparent!important;color:var(--cream)!important;border:1px solid var(--line)!important} #mtp-phonon .cap{color:var(--mute)!important;font-size:12px} #mtp-phonon .seg{display:inline-flex;border:1px solid var(--line)!important;border-radius:8px;overflow:hidden;margin-bottom:12px} #mtp-phonon .seg button{background:transparent!important;border:0!important;color:var(--mute)!important;padding:7px 12px;font:600 12px Inter,system-ui,sans-serif;cursor:pointer} #mtp-phonon .seg button.on{background:var(--cream)!important;color:#111!important} #mtp-phonon .bar{display:grid;grid-template-columns:150px 1fr 64px;align-items:center;gap:10px;margin:7px 0;font-size:13px} #mtp-phonon .bar .lb{color:var(--cream)!important} #mtp-phonon .bar .tr{height:14px;background:var(--card)!important;border-radius:4px;overflow:hidden;border:1px solid var(--line)!important} #mtp-phonon .bar .fl{height:100%;width:0;background:#5b574c!important;transition:width 1s cubic-bezier(.2,.8,.2,1)} #mtp-phonon .bar.me .fl{background:var(--acc)!important} #mtp-phonon .bar .v{text-align:right;font-weight:700;color:var(--cream)!important;font-variant-numeric:tabular-nums} #mtp-phonon .note{font-size:12px;color:var(--mute)!important;margin-top:10px;line-height:1.5} #mtp-phonon .hw{display:grid;grid-template-columns:repeat(3,1fr);gap:10px} #mtp-phonon .hwc{background:var(--card)!important;border:1px solid var(--line)!important;border-radius:12px;padding:12px;cursor:pointer;transition:border-color .25s} #mtp-phonon .hwc.on{border-color:var(--acc)!important} #mtp-phonon .hwc b{font-size:13px;color:var(--cream)!important;display:block} #mtp-phonon .hwc small{color:var(--mute)!important;font-size:11px} #mtp-phonon .gauge{margin-top:14px;display:grid;grid-template-columns:1fr 1fr;gap:12px} #mtp-phonon .gc{background:var(--card)!important;border:1px solid var(--line)!important;border-radius:12px;padding:14px} #mtp-phonon .gc .k{font-size:11px;letter-spacing:.1em;text-transform:uppercase;color:var(--mute)!important} #mtp-phonon .gc .val{font-size:34px;font-weight:800;color:var(--acc)!important;font-variant-numeric:tabular-nums} #mtp-phonon .gc .src{font-size:11px;color:var(--mute)!important} #mtp-phonon .to{display:grid;grid-template-columns:1fr 1fr;gap:12px} #mtp-phonon .tc{background:var(--card)!important;border:1px solid var(--line)!important;border-radius:12px;padding:14px} #mtp-phonon .tc h4{margin:0 0 8px!important;font-size:14px!important;color:var(--cream)!important} #mtp-phonon .tc ul{margin:0;padding-left:18px} #mtp-phonon .tc li{font-size:13px;color:var(--cream)!important;margin:5px 0;line-height:1.45} #mtp-phonon .tc.p h4{color:var(--mtp)!important} #mtp-phonon .tc.c h4{color:var(--acc2)!important} #mtp-phonon .ft{position:relative;display:flex;justify-content:space-between;gap:10px;flex-wrap:wrap;padding:12px 24px;border-top:1px solid var(--line)!important;font-size:11px;color:var(--mute)!important} #mtp-phonon .ft .brand{color:var(--mtp)!important;font-weight:700} @media (max-width:640px){ #mtp-phonon h3{font-size:20px!important} #mtp-phonon .grid3{grid-template-columns:1fr} #mtp-phonon .hw{grid-template-columns:1fr 1fr} #mtp-phonon .gauge,#mtp-phonon .to{grid-template-columns:1fr} #mtp-phonon .pipe{flex-direction:column} #mtp-phonon .arr{transform:rotate(90deg)} #mtp-phonon .bar{grid-template-columns:110px 1fr 56px;font-size:12px} #mtp-phonon .hd,#mtp-phonon .panel,#mtp-phonon .ft{padding-left:16px;padding-right:16px} #mtp-phonon .tabs{padding-left:16px;padding-right:16px} } <div id="mtp-phonon"> <div class="dots"></div> <div class="hd"> <div class="kick">Gradium · Open-weight TTS</div> <h3>Phonon: speech on the device, in 64.5 MB</h3> <p class="sub">31M and 40M parameter text-to-speech models. No cloud call, no GPU required.</p> </div> <div class="tabs" role="tablist"> <button class="tab on" data-t="fam">Model family</button> <button class="tab" data-t="how">How it works</button> <button class="tab" data-t="bench">Benchmarks</button> <button class="tab" data-t="speed">Speed by device</button> <button class="tab" data-t="trade">Trade-offs</button> </div> <div class="panel on" id="p-fam"> <div class="grid3"> <div class="mc on" data-d="English only, 37 built-in voices (25 + 12 character voices). Weights file model.q8.gguf, 64.5 MB. Apache 2.0, so commercial use is allowed."> <b>Phonon 31m</b><div class="big">31M</div><small>English · 64.5 MB q8</small><br><span class="lic ok">Apache 2.0</span></div> <div class="mc" data-d="1 model per language: English, French, German, Spanish, Portuguese. 9 transformer layers. Best expressivity and accuracy in the family. Commercial use needs a deal with Gradium."> <b>Phonon 40m</b><div class="big">40M</div><small>1 model per language · 74.6 MB</small><br><span class="lic nc">CC BY-NC 4.0</span></div> <div class="mc" data-d="All 5 languages in a single set of weights, with 102 built-in voices. Shares the 40m architecture. Commercial use needs a deal with Gradium."> <b>40m multilingual</b><div class="big">5</div><small>languages in 1 model · 74.6 MB</small><br><span class="lic nc">CC BY-NC 4.0</span></div> </div> <div class="detail" id="fam-d">English only, 37 built-in voices (25 + 12 character voices). Weights file model.q8.gguf, 64.5 MB. Apache 2.0, so commercial use is allowed.</div> </div> <div class="panel" id="p-how"> <div class="pipe"> <div class="st on"><span class="n">STEP 1</span><b>Text + voice</b></div><span class="arr"></span> <div class="st"><span class="n">STEP 2</span><b>SentencePiece</b></div><span class="arr"></span> <div class="st"><span class="n">STEP 3</span><b>Causal transformer</b></div><span class="arr"></span> <div class="st"><span class="n">STEP 4</span><b>Flow-matching head</b></div><span class="arr"></span> <div class="st"><span class="n">STEP 5</span><b>Mimi decoder</b></div> </div> <div class="detail" id="how-d"></div> <div class="frames" id="frames"></div> <div class="ctl"> <button class="btn ghost" id="prev"> Back</button> <button class="btn" id="next">Next step </button> <button class="btn ghost" id="play">Stream frames</button> <span class="cap">Each bar is 1 latent frame = 80 ms of 24 kHz audio</span> </div> </div> <div class="panel" id="p-bench"> <div class="seg" id="bseg"> <button class="on" data-b="wer">Hard-set WER (lower is better)</button> <button data-b="elo">English Elo (higher is better)</button> <button data-b="seed">Seed-TTS WER</button> </div> <div id="bars"></div> <div class="note" id="bnote"></div> </div> <div class="panel" id="p-speed"> <div class="hw" id="hw"></div> <div class="gauge"> <div class="gc"><div class="k">Time to first audio</div><div class="val" id="g1">6.9 ms</div><div class="src" id="g1s"></div></div> <div class="gc"><div class="k">Speed vs real time</div><div class="val" id="g2">29.9x</div><div class="src" id="g2s"></div></div> </div> <div class="note">Gradium blog rows are Phonon 31m; partner and press-release rows do not name the variant. Higher "x real time" means more seconds of audio per second of compute. Measured by Gradium and launch partners.</div> </div> <div class="panel" id="p-trade"> <div class="to"> <div class="tc p"><h4>Where Phonon 31m leads</h4><ul> <li>1.70% hard-set WER vs 3.50% for Kokoro</li> <li>6.9 ms to first audio on a MacBook Air M5 CPU</li> <li>3.7x real time on a Raspberry Pi 5</li> <li>Apache 2.0 weights, tokenizer and voices</li></ul></div> <div class="tc c"><h4>Where it falls short</h4><ul> <li>0.51% Seed-TTS WER, behind Kokoro at 0.45%</li> <li>English only at 31M; 40m models are CC BY-NC</li> <li>Prosody resets between sentences on long text</li> <li>All benchmarks are run by Gradium</li></ul></div> </div> </div> <div class="ft"><span>Sources: gradium.ai/blog, huggingface.co/gradium · self-reported numbers</span><span class="brand">Built by Marktechpost</span></div> </div> (function(){ var R=document.getElementById('mtp-phonon'); function send(){try{parent.postMessage({mtpFrameHeight:R.offsetHeight+40,id:'mtp-phonon'},'*')}catch(e){}} R.querySelectorAll('.tab').forEach(function(t){t.addEventListener('click',function(){ R.querySelectorAll('.tab').forEach(function(x){x.classList.remove('on')});t.classList.add('on'); R.querySelectorAll('.panel').forEach(function(p){p.classList.remove('on')}); R.querySelector('#p-'+t.dataset.t).classList.add('on'); if(t.dataset.t==='bench')drawBars(cur); setTimeout(send,60);});}); R.querySelectorAll('.mc').forEach(function(c){c.addEventListener('click',function(){ R.querySelectorAll('.mc').forEach(function(x){x.classList.remove('on')});c.classList.add('on'); document.getElementById('fam-d').textContent=c.dataset.d;setTimeout(send,30);});}); var steps=['Input text plus a speaker embedding. The voice can be built-in, cloned or designed in Gradium Studio, then run locally.', 'Raw text becomes tokens from a 4,000-token SentencePiece vocabulary. No phoneme conversion, so new languages are easier to add.', 'A causal transformer (6 layers, width 512, 8 heads in 31m) generates 1 latent vector per 80 ms frame, autoregressively.', 'A flow-matching head turns each latent plus noise into a continuous audio embedding in a single step. An end-of-speech head decides when to stop.', 'A smaller variant of Kyutai's Mimi decoder converts embeddings into 24 kHz PCM audio, streamed while generation continues.']; var si=0,sts=R.querySelectorAll('.st'); function setStep(i){si=(i+5)%5;sts.forEach(function(s,k){s.classList.toggle('on',k===si)});document.getElementById('how-d').textContent=steps[si];} setStep(0); document.getElementById('next').onclick=function(){setStep(si+1)}; document.getElementById('prev').onclick=function(){setStep(si-1)}; var F=document.getElementById('frames');for(var i=0;i<24;i++){var d=document.createElement('div');d.className='fr';F.appendChild(d);} var tm=null;document.getElementById('play').onclick=function(){ var fr=F.children,k=0;clearInterval(tm);setStep(2); for(var j=0;j=fr.length){clearInterval(tm);setStep(4);return;} fr[k].classList.add('live');fr[k].style.height=(30+Math.round(Math.abs(Math.sin(k*1.3))*65))+'%'; if(k===8)setStep(3);k++;},110);}; var B={wer:{max:9,u:'%',rows:[['Phonon 31m',1.70,1],['Phonon 40m multi',2.11,1],['Phonon 40m',2.22,1],['Kokoro',3.50],['Kitten TTS',7.40],['NeuTTS Nano',8.71]], n:'500 hard English sentences (phone numbers, emails, postal codes). Magpie scored 27.51%, off this scale. The Hugging Face card lists 1.5% for Phonon 31m; the blog lists 1.70%.'}, elo:{max:2150,min:1900,u:'',rows:[['Phonon 40m',2117,1],['Phonon 31m',2098,1],['Phonon 40m multi',2097,1],['Pocket TTS 100m',2056],['Kokoro',2043],['NeuTTS',2034],['Magpie',1939]], n:'Blind pairwise listening tests, US English: 468 raters, 8,290 votes. Bars start at 1900 Elo.'}, seed:{max:0.7,u:'%',rows:[['Phonon 40m',0.45,1],['Kokoro',0.45],['Magpie',0.46],['Phonon 40m multi',0.48,1],['Phonon 31m',0.51,1],['NeuTTS',0.62]], n:'Seed-TTS English evaluation sentences, LLM-assisted WER. Here Phonon 31m trails Kokoro and Magpie.'}}; var cur='wer'; function drawBars(k){cur=k;var b=B[k],h='';b.rows.forEach(function(r){h+='<div class="bar'+(r[2]?' me':'')+'"><span class="lb">'+r[0]+'</span><div class="tr"><div class="fl" data-w="'+(((r[1]-(b.min||0))/(b.max-(b.min||0)))*100).toFixed(1)+'"></div></div><span class="v">'+(k==='elo'?r[1]:r[1].toFixed(2)+b.u)+'</span></div>'}); document.getElementById('bars').innerHTML=h;document.getElementById('bnote').textContent=b.n; setTimeout(function(){R.querySelectorAll('#bars .fl').forEach(function(f){f.style.width=f.dataset.w+'%'})},40);setTimeout(send,80);} R.querySelectorAll('#bseg button').forEach(function(x){x.onclick=function(){R.querySelectorAll('#bseg button').forEach(function(y){y.classList.remove('on')});x.classList.add('on');drawBars(x.dataset.b);}}); drawBars('wer'); var H=[['MacBook Air M5 CPU','4 threads, no KleidiAI','6.9 ms','29.9x','Gradium blog'], ['M5 + KleidiAI + Accelerate','4 threads','5.4 ms','46.7x','Gradium blog'], ['Raspberry Pi 5','CPU','38 ms','3.7x','Gradium blog'], ['Snapdragon 8 Elite Gen 5','Oryon CPU','7.5 ms','24x','Gradium with Qualcomm'], ['Arm CPU + KleidiAI','Arm-measured','28 ms','15x','Measured by Arm'], ['NVIDIA L40S','CUDA backend','5 ms','Not disclosed','Gradium press release']]; var hw=document.getElementById('hw'),hh='';H.forEach(function(r,i){hh+='<div class="hwc'+(i?'':' on')+'" data-i="'+i+'"><b>'+r[0]+'</b><small>'+r[1]+'</small></div>'});hw.innerHTML=hh; function count(el,txt){var m=txt.match(/[d.]+/);if(!m){el.textContent=txt;return;}var end=parseFloat(m[0]),dec=(m[0].split('.')[1]||'').length,suf=txt.slice(m.index+m[0].length),t0=null; function f(ts){if(!t0)t0=ts;var p=Math.min((ts-t0)/600,1);el.textContent=(end*p).toFixed(dec)+suf;if(p<1)requestAnimationFrame(f);}requestAnimationFrame(f);} function pick(i){var r=H[i];R.querySelectorAll('.hwc').forEach(function(c){c.classList.toggle('on',+c.dataset.i===i)}); count(document.getElementById('g1'),r[2]);count(document.getElementById('g2'),r[3]); document.getElementById('g1s').textContent=r[4];document.getElementById('g2s').textContent=r[4];} R.querySelectorAll('.hwc').forEach(function(c){c.onclick=function(){pick(+c.dataset.i)}}); pick(0); window.addEventListener('load',send);window.addEventListener('resize',send);setTimeout(send,300); })(); " style="width:100%;height:600px;border:0;display:block;background:transparent;" loading="lazy" scrolling="no"> window.addEventListener('message',function(e){if(e.data&&e.data.id==='mtp-phonon'&&e.data.mtpFrameHeight){var f=document.getElementById('mtp-phonon-frame');if(f)f.style.height=e.data.mtpFrameHeight+'px';}}); How does Phonon perform on benchmarks? Gradium ran blind pairwise listening tests with 1,189 raters and 17,270 votes. In US English, Phonon 40m scored 2117 Elo and Phonon 31m scored 2098. Kokoro scored 2043, NeuTTS 2034 and Magpie 1939. Accuracy was tested on a public hard-sentence set. It holds 500 sentences per language, full of phone numbers, emails and postal codes. Phonon 31m scored 1.70% WER against 3.50% for Kokoro and 8.71% for NeuTTS Nano. Built-in text normalization drives much of that gap. On the easier Seed-TTS set, the gap closes. Phonon 31m scored 0.51% LLM WER, while Kokoro scored 0.45%. How fast is Phonon on a CPU? On a MacBook Air M5 CPU with 4 threads, Phonon 31m reaches first audio in 6.9 ms. It runs 29.9x faster than real time. Kokoro takes 517 ms, because it does not stream. With Arm KleidiAI and Apple Accelerate, Phonon 31m reaches 46.7x real time. On a Raspberry Pi 5, Phonon 31m starts in 38 ms and runs 3.7x faster than real time. Gradium says Kokoro needs 7 seconds to start there. On Snapdragon 8 Elite Gen 5, Phonon starts in 7.5 ms at 24x real time. On an NVIDIA L40S, CUDA reaches first speech in 5 ms. An AMD EPYC 7413 CPU takes 43 ms, per the press release. Comparison: Phonon vs Kokoro, NeuTTS Nano and Kitten TTS Benchmark rows come from Gradium’s blog and model card. Size, language and license rows come from each model’s own card. How can developers deploy Phonon? The runtime is MIT or Apache-2.0 licensed. It covers Rust, Python (pip install ptts), Swift, C, C++ and the browser via phonon-tts. It also ships an OpenAI-compatible server and a WebSocket server. Phonon loads in Transformers, MLX-Audio, Ollama and LM Studio. Voice cloning and voice design run in Gradium Studio, then the voice runs locally. Key Takeaways Phonon 31m is a 31M, 64.5 MB English TTS model under Apache 2.0. It reports 1.70% WER on hard text, against 3.50% for Kokoro. First audio arrives in 6.9 ms on a MacBook Air M5 CPU. It runs 3.7x faster than real time on a Raspberry Pi 5. Multilingual 40m models exist, but under a non-commercial license. Check out the technical blog, model card, GitHub runtime and hard-sentence dataset. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Gradium Open-Sources Phonon, a 31M-Parameter On-Device TTS Model Under Apache 2.0 appeared first on MarkTechPost.
Cohere AI has released Rerank 5, a new reranker family in Pro and Fast tiers. Cohere Rerank 5 is a retrieval model that reorders search results before they reach an LLM. It adds 2 developer features: Context Focus, which returns relevant passages inside each document, and calibrated relevance scores. For RAG and agent builders, the message is simple. Send fewer, better tokens to the LLM. TL;DR Size: Parameter count not disclosed. 32K token context window. Two tiers: rerank-v5.0-pro and rerank-v5.0-fast. Runs on: Hosted API, Model Vault, Microsoft Foundry, Amazon SageMaker, or private VPC and on-prem. Cohere benchmarks speed on a single NVIDIA A100. No open weights or quantized builds. Performance: Pro ranks first on 39 of 46 composite datasets in Cohere’s own evaluation. Best: Pro scores 82.2 on BRIGHT (RCP-nDCG@10), 5.8 points above Voyage AI Rerank 3. Worst: Pro median latency is 11.7 s per 96-document request, versus 4.1 s for Fast. What is new in Cohere Rerank 5? Rerank 5 ships as two models. Pro targets ranking quality on reasoning-heavy queries and long, domain-specific documents. Fast targets low latency and high throughput for agents running many searches. Both models support over 100 languages and a 32K context window. They also rerank semi-structured data such as JSON, tables and YAML records. Upgrading needs no re-indexing. Developers change the model parameter and keep their existing keyword, vector or hybrid search. How does Context Focus work? Context Focus is an optional feature, off by default. A single Rerank call returns document scores plus scored character spans. Each span carries start and end offsets into the original text. Developers set it through the relevant_spans field, with a threshold and max_spans value. Cohere tested it on 30 QA datasets. Removing half the retrieved text kept 96% of answer quality. Removing three quarters kept 87%. Cohere says teams can push to 10x less context at some quality cost. In a search-agent test across 17 datasets, BM25 alone scored 46% accuracy. Adding Rerank 5 Pro raised it to 49%. Context Focus at the default 0.1 threshold cut tokens per task by 19% and reached 57%. At 0.4, tokens fell 34% and accuracy hit 58%. Versus BM25 alone, the full stack used 35% fewer tokens. On 6 deep-search benchmarks, tokens fell 41% and accuracy rose from 39% to 52%. What do calibrated relevance scores change? Rerank 5 scores read as probabilities. A score of 0.7 means a 70% chance the document is useful. Cohere team measured expected calibration error (ECE) against blind human graders and NIST TREC-DL assessors. Rerank 5 scores about 0.06 on both. Voyage AI Rerank 3 scores 0.234 and 0.307. Rerank 4 Pro scores 0.384 and 0.309. In practice, dev teams can filter by threshold instead of a fixed top-k. An agent can also detect when nothing relevant came back. How does Rerank 5 perform on benchmarks? All scores below are self-reported by Cohere using RCP-nDCG@10, a Cohere-developed variant of nDCG. Cohere ran the competitor models itself. Pro leads all four main benchmarks. It scores 92.2 on ViDoRe V3, 90.4 on BEIR, 82.2 on BRIGHT and 87.3 on nanoMTEB. BRIGHT shows the largest jump: 12.4 points over Rerank 4 Pro. Fast beats Voyage AI Rerank 3 Lite, Qwen3-Reranker-8B and Rerank 4 Pro on all 4. It ties Voyage AI Rerank 3 on ViDoRe V3 at 90.6. It trails it on BEIR, 88.1 versus 88.6. On parsed PDFs, averaged over 47 datasets, Pro scores 89.9 and Fast 88.5. Voyage AI Rerank 3 scores 88.4. Pro ranks first on 14 of 19 non-English languages in Cohere’s internal suite. (function(){var f=document.getElementById("mtp-rr5-frame");window.addEventListener("message",function(e){if(e.data&&e.data.mtpFrame==="mtp-rr5"&&e.data.h){f.style.height=e.data.h+"px";}});})(); How does Rerank 5 compare with competing rerankers? *Scores are RCP-nDCG@10 or ECE from Cohere’s Rerank 5 post, measured by Cohere. Not independently verified. How can developers deploy Rerank 5? Rerank 5 is generally available on the Cohere API and Model Vault. It also runs on Microsoft Foundry (Pro, Fast) and Amazon SageMaker (Pro, Fast). It is also available to all North users. API pricing is $0.05 per 1M tokens for Pro and $0.02 for Fast. Model Vault lists $4.00 per instance-hour for both tiers on its Medium tier. Integrations include LangChain, Haystack, Elasticsearch and Weaviate, per Cohere’s integration docs. Key Takeaways Rerank 5 ships in Pro ($0.05) and Fast ($0.02) per 1M tokens, both with 32K context. Pro scores 82.2 on BRIGHT, 5.8 points above Voyage AI Rerank 3. Context Focus returns scored passages in 1 call, cutting agent tokens 35% to 41%. Calibrated scores reach about 0.06 ECE, so 1 threshold works across domains. All benchmarks are self-reported on Cohere’s RCP-nDCG@10 metric. Check out the official announcement, the Rerank 5 changelog and the full results sheet. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Cohere Releases Rerank 5 With Passage-Level Context Focus and Calibrated Scores appeared first on MarkTechPost.
