Breaking
Advertisement

AI

AI

Prime Intellect Releases Prime Mega MoE: Trainable Fused MoE Kernel, 3.11x Faster Forward Pass

Prime Intellect has released Prime Mega MoE, a training extension of DeepSeek’s fused Mega MoE kernel. DeepSeek’s kernel only ran the forward pass. Prime Intellect added a fast, overlapping backward pass. That makes the fused kernel usable for training, not just inference. In its prime-rl trainer, full training steps on Qwen3 MoE models run up to 34% faster. Against TorchTitan, the gain reaches 55%. For teams training mixture-of-experts (MoE) models, that is a direct cut in GPU hours. TL;DR Size: A GPU kernel, not a model. Tested on Qwen3-30B-A3B and Qwen3-235B-A22B (128 experts, top-8 routing). Runs on: NVIDIA Hopper (blockwise FP8) and Blackwell (MXFP8) GPUs, BF16 on both. Published numbers use 8×H200 nodes, 32 to 128 GPUs. Performance: 32% to 34% faster full training steps than the prime-rl baseline. 48% to 55% faster than TorchTitan. Best: 3.11x faster forward pass on a Qwen3-30B-A3B MoE layer (12.0 ms to 3.9 ms). Worst: 1.24x backward pass on a Qwen3-235B-A22B MoE layer (30.5 ms to 24.6 ms). What is Prime Mega MoE? Prime Mega MoE is a fused, overlapping MoE kernel for training from Prime Intellect. It builds on the Mega MoE kernel in DeepSeek’s open-source DeepGEMM library. Why is MoE training communication-bound? In an MoE model, each token goes to a small subset of experts. Those experts sit on different GPUs. So every layer sends tokens out, runs expert math, and sends results back. This happens in the forward pass and again in the backward pass. In a standard setup, these are separate operations. The GPU sorts tokens and waits on the transport layer. Then it computes, hands results back, and waits again. Those waits are idle GPU time. How does the fused kernel work? DeepSeek’s Mega MoE, added to DeepGEMM in April 2026, replaces that chain with one kernel call. It fuses expert-parallel dispatch, the 2 expert linear layers, SwiGLU, and combine. While one token group is in transit, the kernel computes on a group that already arrived. It also skips intermediate tensors by using GPU-only scratch buffers. But it only shipped a forward implementation. Prime Intellect wrote the missing backward kernel, also overlapping. Supported types are BF16, MXFP8 (Blackwell only) and blockwise FP8 (Hopper only). The prime-rl config also exposes an fp8_bwd mode: BF16 forward, FP8 backward. (function(){var f=document.getElementById("mtp-pmm-frame");window.addEventListener("message",function(e){if(e.data&&e.data.mtpEmbed==="pmm"&&e.data.height&&f&&e.source===f.contentWindow){f.style.height=e.data.height+"px";}});})(); How fast is Prime Mega MoE? All numbers below are self-reported by Prime Intellect. Training steps use 8×H200 nodes, FP8, 16,384-token sequences and full activation checkpointing. Each result is the mean of steps 6 to 20. Full training step, seconds (lower is better): Against TorchTitan, throughput rose from 5,337 to 7,886 tokens/s/GPU on the 30B model. On the 235B model, it rose from 1,094 to 1,697. The prime-rl baseline uses torch all-to-all plus DeepGEMM FP8 experts. Single MoE layer, milliseconds (1 node, 8×H200, EP=8, 128 experts, top-8, random routing): The pattern is clear. Forward gains are large, and backward gains are smaller. The headline “up to 3.11x” refers to the forward pass only. How does it compare with other fused MoE options? TorchTitan’s 41% gain comes from a different setup (B200, 671B model, BF16 baseline). It is not directly comparable. How do developers use it? Prime Mega MoE is a dispatch backend in prime-rl. Set type = "mega_moe" under [trainer.model.moe.dispatch]. Choose precision from bf16, mxfp8, fp8 or fp8_bwd. The training kernels ship in a prebuilt DeepGEMM wheel from prime-kernels. The stock DeepGEMM build lacks them. Key Takeaways Prime Mega MoE adds a fused, overlapping backward pass to DeepSeek’s Mega MoE. Full training steps run 32% to 34% faster than the prime-rl baseline. Against TorchTitan, steps run 48% to 55% faster on Qwen3 MoE models. Forward gains reach 3.11x, but backward gains drop to 1.24x on large layers. It ships in prime-rl as the mega_moe dispatch backend. Check out the Prime Mega MoE blog post, the announcement on X, and the prime-rl repo. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Prime Intellect Releases Prime Mega MoE: Trainable Fused MoE Kernel, 3.11x Faster Forward Pass appeared first on MarkTechPost.

AI

Paris-based Mistral releases “Le Chonk,” a strong coding model that still trails China overall

Mistral has released the weights for Large 4, aka "Le Chonk," a trillion-parameter model trained entirely in Europe. Mistral says it beats its open rivals at coding, and independent testing shows it's fast and fairly cheap. It still can't match China's top models, scoring 26 points to the 32 posted by DeepSeek V4.1 Flash and GLM-5.3. The article Paris-based Mistral releases "Le Chonk," a strong coding model that still trails China overall appeared first on The Decoder.

Advertisement
AI

Google’s AI search problem finally shows up in the earnings report, some investors believe

Google's stock dropped as much as ten percent after its quarterly earnings. Cloud revenue more than doubled to $30.7 billion, but search revenue grew just 12 percent, slower than expected. Google acknowledged that its fast-growing AI mode brings in less revenue than traditional search, and it is now expanding its AI mode ad program internationally. The article Google's AI search problem finally shows up in the earnings report, some investors believe appeared first on The Decoder.

Advertisement
AI

Infini-AI-Lab Releases Hermit, an Open-Source Harness Adapter Framework for AI Agents

Infini-AI-Lab has released Hermit, an open-source harness adapter framework for AI agents. Hermit is a framework from Infini-AI-Lab that trains an external ‘harness adapter’ while the agent’s model and harness stay frozen. The first release, v0.1.0, shipped on October 25, 2026. CMU postdoc Haizhong Zheng announced it on X on October 27. For developers, the pitch is simple: adapt Codex or Claude Code agents to new tasks without fine-tuning or forking the harness. TL;DR Footprint: A Python framework. No parameters of its own. Apache-2.0, pip install hermit-harness. Runs on: The offline Quick Start needs no GPU, model account or dataset. Real runs use any API backend or a local vLLM, SGLang or Ollama server. Performance: Reported gains over no-adapter baselines on OfficeQA, CL-Bench and ServeLearnBench, all with GLM 5.3 Flash. Best: MWM adapter hits 0.795 accuracy on CL-Bench database exploration, vs 0.580 for the in-context baseline. Worst: The prompt-only adapter scores 0.470 on the same venue, below that 0.580 baseline. Bottom line: Best: one training loop for 5 harness evolution methods. Worst: most results are single runs, self-reported. What is a harness adapter? A harness adapter is a trainable external harness. It works like a plugin with training support built in. A model adapter, such as LoRA, adapts a model. A harness adapter adapts an existing agent instead. The adapter can hold instructions, skills, memory, tools, executable code and agent coordination logic. It can even be a whole repository. Its specification says which parts are learnable and how updates get checked. The base model and harness never change. The team gives 3 reasons for this design. Many harnesses are closed-source. Editing open ones can break carefully tuned component interactions. And a learned adapter can move between compatible models and harnesses. How does Hermit train an adapter? Hermit uses an RL-style training loop. A Trainer runs a batch of tasks through the acting agent with the current adapter. A judge scores the outputs. An Evolver reads the trajectories and feedback, then proposes a new adapter. The Trainer checks it against the template and commits it as the next version. Every adapter version lives in a git tree. You can inspect, roll back or ship any version. Learner programs can run sandboxed, which requires Linux with bubblewrap. Five evolution methods ship today: GEPA, SkillOpt, MetaWorld Model (MWM), Harness² and Continual Harness. ACE and ReHarness are listed as coming soon. You can also write your own Evolver. <iframe id="mtp-hermit-frame" title="Hermit interactive explainer" style="width:100%;height:600px;border:0;display:block;overflow:hidden;" scrolling="no" loading="lazy" srcdoc="<!doctype html> html,body{margin:0!important;padding:0!important;background:transparent!important} #mtp-hermit-x{--bg:#1B1D22;--card:#23262d;--line:#33363f;--ink:#F1F0EA;--mut:#9a9ea8;--cu:#C6743C;--cu2:#E08E52;--bl:#8ab6e4;--grn:#76B900; max-width:880px;margin:0 auto;background:var(--bg)!important;color:var(--ink)!important;border:1px solid var(--line)!important;border-radius:16px;font-family:Inter,system-ui,-apple-system,Segoe UI,Roboto,sans-serif;overflow:hidden;box-sizing:border-box} #mtp-hermit-x *{box-sizing:border-box} #mtp-hermit-x hr,#mtp-hermit-x p:empty,#mtp-hermit-x del,#mtp-hermit-x s{display:none!important} #mtp-hermit-x .hd{padding:22px 26px 14px;border-bottom:1px solid var(--line)!important;background:linear-gradient(135deg,#1B1D22 0%,#2a2017 100%)!important} #mtp-hermit-x .eb{font-size:11px;letter-spacing:.14em;text-transform:uppercase;color:var(--cu2)!important;font-weight:700} #mtp-hermit-x h3{margin:6px 0 4px!important;font-size:22px!important;line-height:1.25!important;color:var(--ink)!important;font-weight:800!important} #mtp-hermit-x .sub{margin:0!important;color:var(--mut)!important;font-size:14px} #mtp-hermit-x .tabs{display:flex;gap:6px;padding:12px 18px;border-bottom:1px solid var(--line)!important;flex-wrap:wrap} #mtp-hermit-x .tab{background:transparent!important;color:var(--mut)!important;border:1px solid var(--line)!important;border-radius:999px;padding:7px 14px;font-size:13px;font-weight:600;cursor:pointer;font-family:inherit;transition:all .2s} #mtp-hermit-x .tab:hover{color:var(--ink)!important;border-color:var(--cu)!important} #mtp-hermit-x .tab.on{background:var(--cu)!important;color:#1B1D22!important;border-color:var(--cu)!important} #mtp-hermit-x .pn{display:none;padding:22px 26px;animation:mhx-in .35s ease} #mtp-hermit-x .pn.on{display:block} @keyframes mhx-in{from{opacity:0;transform:translateY(6px)}to{opacity:1;transform:none}} #mtp-hermit-x .lead{color:var(--ink)!important;font-size:15px;line-height:1.55;margin:0 0 16px!important} #mtp-hermit-x .lead b{color:var(--cu2)!important} /* Adapter diagram */ #mtp-hermit-x .dg{display:grid;grid-template-columns:1fr 56px 1fr;align-items:center;gap:0;margin:4px 0 14px} #mtp-hermit-x .box{background:var(--card)!important;border:1px solid var(--line)!important;border-radius:12px;padding:14px} #mtp-hermit-x .box h4{margin:0 0 8px!important;font-size:13px!important;letter-spacing:.06em;text-transform:uppercase;color:var(--mut)!important} #mtp-hermit-x .frz{position:relative} #mtp-hermit-x .frz:after{content:"FROZEN";position:absolute;top:10px;right:10px;font-size:10px;font-weight:800;letter-spacing:.1em;color:var(--bl)!important;border:1px solid var(--bl)!important;border-radius:6px;padding:2px 6px} #mtp-hermit-x .chip{display:inline-block;margin:3px 4px 3px 0;padding:5px 9px;border-radius:8px;font-size:12px;font-weight:600;background:#2c2f37!important;color:var(--ink)!important;border:1px solid var(--line)!important;cursor:pointer;transition:all .2s} #mtp-hermit-x .ad .chip{border-color:#6b4423!important} #mtp-hermit-x .chip:hover,#mtp-hermit-x .chip.sel{background:var(--cu)!important;color:#1B1D22!important;border-color:var(--cu)!important} #mtp-hermit-x .ad{border-color:var(--cu)!important;box-shadow:0 0 0 0 rgba(198,116,60,.5);animation:mhx-glow 2.4s infinite} @keyframes mhx-glow{0%{box-shadow:0 0 0 0 rgba(198,116,60,.45)}70%{box-shadow:0 0 0 12px rgba(198,116,60,0)}100%{box-shadow:0 0 0 0 rgba(198,116,60,0)}} #mtp-hermit-x .plug{height:4px!important;background:linear-gradient(90deg,var(--bl),var(--cu))!important;border-radius:4px;position:relative;margin:0 6px} #mtp-hermit-x .plug:before{content:"";position:absolute;top:-4px;left:0;width:12px;height:12px;border-radius:50%;background:var(--cu2)!important;animation:mhx-dot 1.8s linear infinite} @keyframes mhx-dot{from{left:0}to{left:calc(100% - 12px)}} #mtp-hermit-x .info{min-height:44px;background:#16181c!important;border:1px dashed var(--line)!important;border-radius:10px;padding:10px 12px;font-size:13.5px;color:var(--ink)!important;line-height:1.5} /* Loop */ #mtp-hermit-x .loop{display:grid;grid-template-columns:repeat(5,1fr);gap:8px;margin-bottom:14px} #mtp-hermit-x .st{background:var(--card)!important;border:1px solid var(--line)!important;border-radius:12px;padding:12px 10px;text-align:center;transition:all .3s;opacity:.55} #mtp-hermit-x .st .n{font-size:11px;font-weight:800;color:var(--mut)!important} #mtp-hermit-x .st .t{font-size:13.5px;font-weight:700;margin-top:4px;color:var(--ink)!important} #mtp-hermit-x .st.on{opacity:1;border-color:var(--cu)!important;transform:translateY(-3px);box-shadow:0 6px 18px rgba(198,116,60,.25)} #mtp-hermit-x .st.on .n{color:var(--cu2)!important} #mtp-hermit-x .ctl{display:flex;gap:8px;align-items:center;margin-top:12px;flex-wrap:wrap} #mtp-hermit-x .btn{background:var(--cu)!important;color:#1B1D22!important;border:0!important;border-radius:8px;padding:8px 14px;font-weight:700;font-size:13px;cursor:pointer;font-family:inherit} #mtp-hermit-x .btn.gh{background:transparent!important;color:var(--ink)!important;border:1px solid var(--line)!important} #mtp-hermit-x .ver{margin-left:auto;font-size:13px;color:var(--mut)!important} #mtp-hermit-x .ver b{color:var(--cu2)!important;font-size:16px} /* Bars */ #mtp-hermit-x .seg{display:flex;gap:6px;margin-bottom:14px;flex-wrap:wrap} #mtp-hermit-x .bars{display:flex;flex-direction:column;gap:9px} #mtp-hermit-x .row{display:grid;grid-template-columns:170px 1fr 56px;align-items:center;gap:10px;font-size:13px} #mtp-hermit-x .row .lb{color:var(--ink)!important} #mtp-hermit-x .row .tr{height:16px!important;background:#2a2d34!important;border-radius:6px;overflow:hidden} #mtp-hermit-x .row .fl{height:100%!important;width:0;border-radius:6px;background:#5c616d!important;transition:width 1s cubic-bezier(.2,.8,.2,1)} #mtp-hermit-x .row.hl .fl{background:linear-gradient(90deg,var(--cu),var(--cu2))!important} #mtp-hermit-x .row.hl .lb{font-weight:700;color:var(--cu2)!important} #mtp-hermit-x .row .v{font-weight:700;text-align:right;font-variant-numeric:tabular-nums;color:var(--ink)!important} #mtp-hermit-x .note{font-size:12px;color:var(--mut)!important;margin-top:12px;line-height:1.5} /* Deploy */ #mtp-hermit-x .term{background:#0f1013!important;border:1px solid var(--line)!important;border-radius:10px;padding:14px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace!important;font-size:12.5px;color:#d7d4c8!important;line-height:1.7;overflow-x:auto;white-space:pre} #mtp-hermit-x .term .p{color:var(--cu2)!important} #mtp-hermit-x .term .c{color:#6B6F7A!important} #mtp-hermit-x .hosts{display:flex;gap:8px;flex-wrap:wrap;margin:12px 0} #mtp-hermit-x .ft{padding:12px 26px;border-top:1px solid var(--line)!important;font-size:12px;color:var(--mut)!important;display:flex;justify-content:space-between;flex-wrap:wrap;gap:6px} #mtp-hermit-x .ft .mtp{color:#76B900!important;font-weight:700} #mtp-hermit-x a{color:var(--cu2)!important} @media (max-width:640px){ #mtp-hermit-x .hd,#mtp-hermit-x .pn{padding:18px 16px} #mtp-hermit-x h3{font-size:19px!important} #mtp-hermit-x .dg{grid-template-columns:1fr;gap:10px} #mtp-hermit-x .plug{height:24px!important;width:4px!important;margin:0 auto;background:linear-gradient(180deg,var(--bl),var(--cu))!important} #mtp-hermit-x .plug:before{display:none} #mtp-hermit-x .loop{grid-template-columns:1fr 1fr} #mtp-hermit-x .row{grid-template-columns:110px 1fr 48px;font-size:12px} #mtp-hermit-x .ft{padding:12px 16px} } <div id="mtp-hermit-x"> <div class="hd"> <div class="eb">Infini-AI-Lab · Hermit v0.1.0 · Apache-2.0</div> <h3>How Hermit trains a harness adapter</h3> <p class="sub">The adapter learns from task feedback. The model and harness stay frozen.</p> </div> <div class="tabs" role="tablist"> <button class="tab on" data-p="0">1 · The adapter</button> <button class="tab" data-p="1">2 · Training loop</button> <button class="tab" data-p="2">3 · Results</button> <button class="tab" data-p="3">4 · Deploy</button> </div> <div class="pn on" data-i="0"> <p class="lead">A <b>harness adapter</b> is a trainable external harness, like a plugin with training built in. Click any part to see what it does.</p> <div class="dg"> <div class="box frz"><h4>Your agent</h4> <span class="chip" data-d="The base model. Hermit never updates its weights.">Model</span> <span class="chip" data-d="The existing harness (for example Codex or Claude Code). Hermit never edits it, so closed-source harnesses work too.">Harness</span> </div> <div class="plug" aria-hidden="true"></div> <div class="box ad"><h4>Harness adapter</h4> <span class="chip" data-d="Instruction files such as ADAPTER.md or AGENTS.md that the agent reads as context.">Instructions</span> <span class="chip" data-d="Reusable skills, including executable helper scripts.">Skills</span> <span class="chip" data-d="Notes and stored knowledge the agent carries across tasks.">Memory</span> <span class="chip" data-d="Extra tools, such as the wm tool MWM exposes for its world model.">Tools</span> <span class="chip" data-d="Executable code. An adapter can be an entire repository.">Code</span> <span class="chip" data-d="Coordination logic for several agents, as in the Lean adapter that links 5 Codex agents.">Coordination</span> </div> </div> <div class="info" id="mhx-info">Every adapter version is stored in a git tree, so you can inspect, roll back or ship any one.</div> </div> <div class="pn" data-i="1"> <p class="lead">Hermit runs an <b>RL-style loop</b>, but the thing being updated is the adapter, not the weights.</p> <div class="loop"> <div class="st"><div class="n">STEP 1</div><div class="t">Run tasks</div></div> <div class="st"><div class="n">STEP 2</div><div class="t">Judge scores</div></div> <div class="st"><div class="n">STEP 3</div><div class="t">Evolver proposes</div></div> <div class="st"><div class="n">STEP 4</div><div class="t">Check vs template</div></div> <div class="st"><div class="n">STEP 5</div><div class="t">Commit version</div></div> </div> <div class="info" id="mhx-step">Press Play to step through one training iteration.</div> <div class="ctl"> <button class="btn" id="mhx-play"> Play</button> <button class="btn gh" id="mhx-next">Next step</button> <button class="btn gh" id="mhx-reset">Reset</button> <span class="ver">Adapter version <b id="mhx-v">V0</b></span> </div> <p class="note">Built-in Evolvers: GEPA, SkillOpt, MetaWorld Model (MWM), Harness², Continual Harness. ACE and ReHarness are listed as coming soon.</p> </div> <div class="pn" data-i="2"> <p class="lead">Self-reported results, all with <b>GLM 5.3 Flash</b> as the acting model. Pick a benchmark.</p> <div class="seg"> <button class="tab on" data-b="0">OfficeQA</button> <button class="tab" data-b="1">CL-Bench database</button> <button class="tab" data-b="2">ServeLearnBench (H)</button> </div> <div class="bars" id="mhx-bars"></div> <p class="note" id="mhx-bnote"></p> </div> <div class="pn" data-i="3"> <p class="lead">Train once, then <b>deploy</b> the adapter into an agent workspace or export it as a native plugin.</p> <div class="term"><span class="p">$</span> pip install hermit-harness <span class="p">$</span> hermit train --config run.yaml <span class="p">$</span> hermit eval --config run.yaml --adapter final <span class="p">$</span> hermit deploy --adapter DIR --workspace WS --host codex <span class="p">$</span> hermit convert --adapter DIR --output OUT --target claude-code <span class="c"># other convert targets: codex, pi, hermes</span></div> <div class="hosts"> <span class="chip" data-d="Deploy target and plugin export target.">Codex</span> <span class="chip" data-d="Deploy target and plugin export target.">Claude Code</span> <span class="chip" data-d="Deploy target and plugin export target.">Pi</span> <span class="chip" data-d="Plugin export target.">Hermes</span> </div> <div class="info" id="mhx-dinfo">Backends by reference: Fireworks, OpenAI, Anthropic, DeepSeek, vLLM, SGLang, Ollama. The offline Quick Start needs no GPU, model account or dataset.</div> </div> <div class="ft"><span>Source: github.com/Infini-AI-Lab/hermit · self-reported</span><span class="mtp">© Marktechpost</span></div> </div> (function(){ var R=document.getElementById('mtp-hermit-x');if(!R)return; function ping(){try{var h=R.offsetHeight+40;parent.postMessage({mtpEmbedHeight:h,height:h,frameHeight:h},'*');}catch(e){}} var tabs=R.querySelectorAll('.tabs .tab'),pns=R.querySelectorAll('.pn'); tabs.forEach(function(t){t.addEventListener('click',function(){var p=+t.getAttribute('data-p');tabs.forEach(function(x){x.classList.remove('on')});t.classList.add('on');pns.forEach(function(x){x.classList.toggle('on',+x.getAttribute('data-i')===p)});if(p===2)drawBars(cur);setTimeout(ping,60);});}); function wire(sel,out){R.querySelectorAll(sel).forEach(function(c){c.addEventListener('click',function(){R.querySelectorAll(sel).forEach(function(x){x.classList.remove('sel')});c.classList.add('sel');document.getElementById(out).textContent=c.textContent+': '+c.getAttribute('data-d');ping();});});} wire('.dg .chip','mhx-info');wire('.hosts .chip','mhx-dinfo'); var steps=R.querySelectorAll('.st'),si=-1,ver=0,timer=null; var msg=['The Trainer runs a batch of tasks through the acting agent with the current adapter.','Your judge scores each output and writes feedback.','An Evolver reads trajectories and feedback, then proposes a new adapter.','The Trainer checks the proposal against the adapter template.','The checked update is committed as the next adapter version.']; function go(i){si=i;steps.forEach(function(s,k){s.classList.toggle('on',k===i)});document.getElementById('mhx-step').textContent=msg[i];if(i===4){ver++;document.getElementById('mhx-v').textContent='V'+ver;}} function next(){go((si+1)%5)} document.getElementById('mhx-next').onclick=function(){stop();next()}; document.getElementById('mhx-play').onclick=function(){if(timer){stop();return}this.textContent='❚❚ Pause';next();timer=setInterval(next,1400)}; function stop(){clearInterval(timer);timer=null;document.getElementById('mhx-play').textContent=' Play'} document.getElementById('mhx-reset').onclick=function(){stop();si=-1;ver=0;steps.forEach(function(s){s.classList.remove('on')});document.getElementById('mhx-v').textContent='V0';document.getElementById('mhx-step').textContent='Press Play to step through one training iteration.'}; var B=[ {max:1,fmt:3,rows:[['No adapter',.541],['Initial skill',.576],['GEPA',.580],['SkillOpt',.600],['Continual Harness',.634,1],['Harness²',.651,1]],n:'Accuracy on 172 test questions. GEPA and SkillOpt are means of 4 trainings, tested frozen. Continual Harness and Harness² learn while streaming the test set once. Standard error about 3.7 points.'}, {max:1,fmt:3,rows:[['Stateless baseline',.400],['Prompt-only',.470],['ICL baseline',.580],['MWM',.795,1]],n:'Accuracy on a 40-question stream over an undocumented, drifting SQLite database. Mean of 5 stream orders.'}, {max:100,fmt:1,rows:[['Blind',4.5],['Prompt-only',18.8],['SkillOpt (leaderboard)',22.0],['CH (leaderboard)',28.7],['MWM',45.3,1],['Oracle (policy given)',89.2]],n:'retail_l3: 624 serving requests, 537 held-out tests. H = score on tests that need the hidden policy. One run each.'}]; var cur=0; function drawBars(k){cur=k;var d=B[k],el=document.getElementById('mhx-bars');el.innerHTML='';d.rows.forEach(function(r){var row=document.createElement('div');row.className='row'+(r[2]?' hl':'');row.innerHTML='<span class="lb">'+r[0]+'</span><span class="tr"><span class="fl" style="display:block"></span></span><span class="v">'+r[1].toFixed(d.fmt)+'</span>';el.appendChild(row);var f=row.querySelector('.fl');setTimeout(function(){f.style.width=(r[1]/d.max*100)+'%'},40);});document.getElementById('mhx-bnote').textContent=d.n;setTimeout(ping,60);} R.querySelectorAll('.seg .tab').forEach(function(b){b.addEventListener('click',function(){R.querySelectorAll('.seg .tab').forEach(function(x){x.classList.remove('on')});b.classList.add('on');drawBars(+b.getAttribute('data-b'));});}); drawBars(0); window.addEventListener('load',ping);window.addEventListener('resize',ping);setTimeout(ping,300); })(); "> (function(){var f=document.getElementById("mtp-hermit-frame");window.addEventListener("message",function(e){if(!f||e.source!==f.contentWindow)return;var h=e.data&&e.data.mtpEmbedHeight;if(h&&h>100)f.style.height=h+"px";});})(); What are the benchmark results? All runs below use GLM 5.3 Flash as the acting model. The numbers come from the repo’s example READMEs and are self-reported. OfficeQA (recipe) asks 172 test questions about U.S. Treasury Bulletins, scored by exact match: GEPA and SkillOpt train on 50 questions and are tested frozen. Continual Harness and Harness² instead learn while streaming the test set once. The team reports a +7.6 point gain for Harness² over its reference run, with a 95% interval of +1.7 to +13.4. CL-Bench (database, poker) tests stateful environments from the Continual Learning Bench paper. On database exploration, MWM reaches 0.795 accuracy. The benchmark’s in-context baseline scores 0.580. On exploitable poker, MWM wins 3.08 big blinds per hand, vs 2.51 for in-context learning. ServeLearnBench (recipe) streams 624 retail support requests under a hidden, shifting policy. MWM scores 45.3 on hidden-policy tests. The best learning method on the leaderboard, CH, scores 28.7. An oracle given the policy scores 89.2. How does Hermit compare with other agent optimizers? How do you install and deploy it? Install from PyPI with pip install hermit-harness, or clone the repo for examples. The CLI covers check, train, eval, deploy and convert. Models are referenced as /. Supported backends include Fireworks, OpenAI, Anthropic, DeepSeek, vLLM, SGLang and Ollama. hermit deploy writes a workspace for Codex, Claude Code or Pi. hermit convert exports a trained adapter as a native plugin for Codex, Claude Code, Pi or Hermes. A finished Lean collaboration adapter coordinates 5 Codex agents proving one theorem together. Key Takeaways Hermit trains an external adapter; model and harness stay frozen. 5 methods ship: GEPA, SkillOpt, MWM, Harness², Continual Harness. MWM reaches 0.795 on CL-Bench database vs 0.580 in-context. Adapters deploy to Codex, Claude Code and Pi, or export as plugins. Results are self-reported, mostly single runs, with GLM 5.3 Flash. Check out the GitHub repo, project page and docs. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Infini-AI-Lab Releases Hermit, an Open-Source Harness Adapter Framework for AI Agents appeared first on MarkTechPost.

AI

Block Unveils Dessa, a Full-Duplex Speech Model Trained Only on Speech

Block’s AI research lab has released Dessa, a full-duplex speech model trained only on human conversation. Block announced it on October 23, 2026 as a research preview. Dessa is a speech-to-speech model from Block that listens and talks at the same time. Block says it used no text and no pretrained models. That is a different design from the text-first stacks behind most voice agents today. TL;DR Runs on: A MacBook Pro M4 Max or better, locally, faster than real time. Performance: Demo-only so far. Live recordings show backchannels, mid-sentence laughs and natural interruptions. Best: Trained on 1 NVIDIA GB200 NVL72 rack (72 GPUs). Moshi used 1,016 H100 GPUs. Worst: 0 published benchmark scores. No weights, paper, API or price. What is Block’s Dessa? Dessa is a true speech-to-speech model: audio in, audio out. According to Block’s project page, everything it knows came from human speech. It is full-duplex, so it listens while it talks. Block says it learned from real conversation when to cut in. It also learned when to backchannel with short cues like “yeah” or “got it”. Block’s demo clips show Dessa laughing mid-sentence and getting louder when asked to speak up. One clip shows it talking with OpenAI’s GPT-Live-1. Block says nobody steered any clip. How does speech-only training work? Most voice systems chain 3 parts: speech recognition, a text model, then speech synthesis. Block team argues that chain throws away emotion, sarcasm, laughter and sighs. Dessa hears and produces those signals directly. Block has not published the architecture. Here is what it has disclosed: Training data: human conversation audio only. No text and no pretrained models. Compute: 1 NVIDIA GB200 NVL72, which links 72 Blackwell GPUs and 36 Grace CPUs. Data efficiency: Block says Dessa learned language from hundreds of times fewer words than LLMs need. Inference: faster than real time, on a MacBook Pro M4 Max or better. The lab also explains why this is hard. Speech never repeats exactly. A spoken word takes thousands of times more bytes than the same word in text. Kyutai’s Moshi, the best-known open full-duplex model, takes another route. Moshi starts from a text language model backbone and predicts time-aligned text tokens before audio tokens. Its model card lists 7 million hours of pretraining audio and 1,016 H100 GPUs. (function(){var f=document.getElementById('mtp-dessa-frame'); window.addEventListener('message',function(e){if(e.source!==f.contentWindow)return;var d=e.data;if(d&&d.mtpDessaH){f.style.height=d.mtpDessaH+'px';}});})(); How does Dessa compare with Moshi, PersonaPlex and GPT-Live-1? Key Takeaways Dessa is full-duplex and speech-to-speech, trained only on human conversation audio. Block trained it on 1 GB200 NVL72 rack; Moshi used 1,016 H100s. It runs locally on a MacBook Pro M4 Max, faster than real time. Block published 0 benchmarks, no parameter count and no weights. Access is a waitlist. Check out the Dessa project page. Compare it with the Moshi paper, the PersonaPlex model card and the GPT-Live-1 docs. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Block Unveils Dessa, a Full-Duplex Speech Model Trained Only on Speech appeared first on MarkTechPost.

AI

AssemblyAI Ships AssemblyAI-TTS-1: A $15 per 1M Character Text-to-Speech API for Voice Agents

AssemblyAI has released AssemblyAI-TTS-1, its first text-to-speech model, now generally available. The AssemblyAI-TTS-1 text-to-speech API targets voice agents running on live phone calls. In that setting, latency and pronunciation decide whether a caller stays on the line. Pricing is $15 per 1M characters, with no annual contract and no concurrency tier to buy. For developers already using AssemblyAI speech-to-text, it closes the voice loop with one vendor. TL;DR Size: It is a hosted API model. Runs on: AssemblyAI’s cloud only, in the US (us-east-1) or EU (eu-north-1). No open weights or on-prem option. Any client with HTTP or WebSocket works. Performance: Ties ElevenLabs v4 on entity accuracy in AssemblyAI’s tests, at under 1/5 of its price. Best: 47 ms median time to first byte across 6 models in AssemblyAI’s API test (self-reported). Worst: 159 ms median streaming latency on Cekura’s independent benchmark, 8th of 19 timed models. Bottom line: Best: $15 per 1M characters with unlimited concurrency. Worst: no SSML, speed control, pronunciation dictionaries or emotion tags yet. What is AssemblyAI-TTS-1? AssemblyAI-TTS-1 is a streaming text-to-speech model from AssemblyAI that turns agent replies into speech for live calls. It was trained on millions of hours of audio, including real conversational data. Most TTS models represent audio as discrete tokens. AssemblyAI says TTS-1 operates on continuous audio representations instead. The AssemblyAI claims this improves realism and cuts time to first spoken audio. The launch includes a library of 4,000+ voices and instant voice cloning. A clone needs a short recording and a confirmation of the speaker’s permission. The clone in the launch post used 24 seconds of audio. Six languages are supported at launch: English, Spanish, French, German, Italian and Portuguese. How do developers integrate AssemblyAI-TTS-1? There are 3 interfaces: an HTTP API, a WebSocket API and an OpenAI-compatible route. The WebSocket session keeps one connection per conversation and accepts a cancel message for barge-in. Output is mono PCM, mu-law or A-law, at 8 kHz to 48 kHz. 24 kHz PCM is the native format; telephony stacks can request 8 kHz mu-law directly. Word timestamps are included. AssemblyAI also published LiveKit and Pipecat reference agents. How does AssemblyAI-TTS-1 perform on benchmarks? AssemblyAI tested TTS-1 against 5 models on 4 English sets, using 50 matched cloned voices. All figures below are self-reported, from an eval dated October 28, 2026. Entity accuracy (580 hard targets such as phone numbers and codes): 93.7%, vs ElevenLabs v4 93.6%, Cartesia Sonic 3.6 82.5%, ElevenLabs Flash v2.5 68.0%. Word accuracy: 99.0% on Seed-TTS and LibriSpeech sentences. Predicted MOS: 3.85, behind Inworld TTS-2 Flash at 3.88 and ahead of Sonic 3.6 at 3.82. Median time to first byte: 47 ms. Flash v2.5 took 119 ms, Sonic 3.6 143 ms, Inworld TTS-2 Flash 244 ms. Other: 85.5 TTSDS prosody score and 0.64 speaker similarity for cloned audio. Independent results are more mixed. On Cekura’s benchmark of 27 models, TTS-1 scored 81.5% reading accuracy and 4.50 naturalness. Its median streaming latency, timed from the last word sent, was 159 ms. Inworld TTS-2 Flash posted 45 ms and 93.2% accuracy there, at the same $15 price. AssemblyAI also cites a 1.7% word error rate on Coval’s leaderboard. It reports a top-10 debut in Artificial Analysis’ controlled-voice arena. <iframe id="mtp-aai-tts1-frame" title="AssemblyAI-TTS-1 interactive explainer" loading="lazy" style="width:100%;height:600px;border:0;display:block;background:transparent;" scrolling="no" srcdoc="<!doctype html><title>AssemblyAI-TTS-1 Explainer</title>html,body{margin:0;padding:0;background:transparent} #mtp-aai-tts1{--cob:#3923c7;--cob3:#887bdd;--cob2:#b0a7e9;--ink:#1d1b16;--ink2:#2a2721;--line:#4a4945;--cream:#fdfcf8;--muted:#a5a4a2;--blush:#e39389;--mtp:#76B900; max-width:880px!important;margin:0 auto!important;background:var(--ink)!important;color:var(--cream)!important;border:1px solid #3a3730!important;border-radius:16px!important;font-family:Inter,system-ui,-apple-system,Segoe UI,Roboto,sans-serif!important;overflow:hidden;box-sizing:border-box;line-height:1.5} #mtp-aai-tts1 *{box-sizing:border-box} #mtp-aai-tts1 hr,#mtp-aai-tts1 p:empty,#mtp-aai-tts1 del,#mtp-aai-tts1 s{display:none!important} #mtp-aai-tts1 .hd{padding:22px 26px 14px;background:radial-gradient(ellipse at 90% -20%,rgba(57,35,199,.55),transparent 60%)!important} #mtp-aai-tts1 .eyebrow{font:600 11px/1 ui-monospace,Menlo,monospace;letter-spacing:.14em;text-transform:uppercase;color:var(--cob2)!important} #mtp-aai-tts1 h3{margin:8px 0 4px!important;font-size:24px!important;font-weight:700!important;color:var(--cream)!important;border:0!important;padding:0!important} #mtp-aai-tts1 .sub{color:var(--muted)!important;font-size:14px;margin:0!important} #mtp-aai-tts1 .tabs{display:flex;gap:6px;padding:0 26px;border-bottom:1px solid #3a3730!important;overflow-x:auto} #mtp-aai-tts1 .tab{background:transparent!important;border:0!important;border-bottom:2px solid transparent!important;color:var(--muted)!important;font:600 13px Inter,system-ui,sans-serif;padding:12px 10px;cursor:pointer;white-space:nowrap;transition:color .2s} #mtp-aai-tts1 .tab.on{color:var(--cream)!important;border-bottom-color:var(--cob3)!important} #mtp-aai-tts1 .pane{display:none;padding:22px 26px 10px;animation:mtpfade .35s ease} #mtp-aai-tts1 .pane.on{display:block} @keyframes mtpfade{from{opacity:0;transform:translateY(6px)}to{opacity:1;transform:none}} #mtp-aai-tts1 .flow{display:flex;align-items:stretch;gap:8px;margin:4px 0 14px} #mtp-aai-tts1 .node{flex:1;background:var(--ink2)!important;border:1px solid #3a3730!important;border-radius:12px;padding:12px 10px;text-align:center;font-size:12.5px;color:var(--muted)!important;transition:all .35s;position:relative} #mtp-aai-tts1 .node b{display:block;color:var(--cream)!important;font-size:13.5px;margin-bottom:3px} #mtp-aai-tts1 .node.act{border-color:var(--cob3)!important;background:rgba(57,35,199,.28)!important;color:var(--cob2)!important;transform:translateY(-3px);box-shadow:0 8px 24px rgba(57,35,199,.35)} #mtp-aai-tts1 .arrow{align-self:center;color:#4a4945!important;font-size:18px} #mtp-aai-tts1 .wave{display:flex;align-items:center;gap:3px;height:34px;justify-content:center;margin-top:6px} #mtp-aai-tts1 .wave i{display:block;width:4px;border-radius:2px;background:var(--cob3)!important;height:6px;opacity:.35} #mtp-aai-tts1 .playing .wave i{animation:mtpwave 1s ease-in-out infinite;opacity:1} @keyframes mtpwave{0%,100%{height:6px}50%{height:30px}} #mtp-aai-tts1 .detail{background:#24211b!important;border-left:3px solid var(--cob3)!important;border-radius:8px;padding:12px 14px;font-size:14px;min-height:66px;color:var(--cream)!important} #mtp-aai-tts1 .ctrl{display:flex;gap:8px;margin:12px 0 4px;flex-wrap:wrap} #mtp-aai-tts1 .btn{background:var(--cob)!important;color:#fff!important;border:0!important;border-radius:999px;padding:8px 16px;font:600 13px Inter,system-ui,sans-serif;cursor:pointer;transition:background .2s} #mtp-aai-tts1 .btn:hover{background:#614fd2!important} #mtp-aai-tts1 .btn.ghost{background:transparent!important;border:1px solid #4a4945!important;color:var(--cream)!important} #mtp-aai-tts1 .pills{display:flex;gap:6px;flex-wrap:wrap;margin-bottom:14px} #mtp-aai-tts1 .pill{background:transparent!important;border:1px solid #4a4945!important;color:var(--muted)!important;border-radius:999px;padding:6px 12px;font:600 12px Inter,system-ui,sans-serif;cursor:pointer} #mtp-aai-tts1 .pill.on{background:var(--cob)!important;border-color:var(--cob)!important;color:#fff!important} #mtp-aai-tts1 .bars{display:flex;flex-direction:column;gap:9px} #mtp-aai-tts1 .row{display:grid;grid-template-columns:150px 1fr 70px;gap:10px;align-items:center;font-size:13px} #mtp-aai-tts1 .row .nm{color:var(--cream)!important} #mtp-aai-tts1 .track{display:block;height:14px;background:#2a2721!important;border-radius:7px;overflow:hidden} #mtp-aai-tts1 .fill{display:block;height:100%;width:0;border-radius:7px;background:#5a574f!important;transition:width .9s cubic-bezier(.2,.8,.2,1)} #mtp-aai-tts1 .row.me .fill{background:linear-gradient(90deg,#3923c7,#887bdd)!important} #mtp-aai-tts1 .row.me .nm{color:var(--cob2)!important;font-weight:700} #mtp-aai-tts1 .val{text-align:right;font-variant-numeric:tabular-nums;color:var(--cream)!important} #mtp-aai-tts1 .note{font-size:12px;color:var(--muted)!important;margin:12px 0 0!important} #mtp-aai-tts1 .calc label{font-size:13px;color:var(--muted)!important} #mtp-aai-tts1 input[type=range]{width:100%;accent-color:#887bdd;margin:8px 0 4px} #mtp-aai-tts1 .big{font-size:26px;font-weight:700;color:var(--cream)!important} #mtp-aai-tts1 .grid{display:grid;grid-template-columns:1fr 1fr;gap:10px;margin-top:14px;align-items:start} #mtp-aai-tts1 .card{background:var(--ink2)!important;border:1px solid #3a3730!important;border-radius:12px;padding:12px 14px;cursor:pointer;transition:border-color .2s} #mtp-aai-tts1 .card:hover{border-color:var(--cob3)!important} #mtp-aai-tts1 .card h4{margin:0!important;font-size:14px!important;color:var(--cream)!important;display:flex;justify-content:space-between} #mtp-aai-tts1 .card p{margin:8px 0 0!important;font-size:13px;color:var(--muted)!important;display:none} #mtp-aai-tts1 .card.open p{display:block;animation:mtpfade .3s ease} #mtp-aai-tts1 .card .tag{font:600 10px ui-monospace,monospace;color:var(--blush)!important;letter-spacing:.08em} #mtp-aai-tts1 .cost{display:grid;grid-template-columns:1fr auto;gap:6px 12px;font-size:14px;margin-top:10px} #mtp-aai-tts1 .cost .me{color:var(--cob2)!important;font-weight:700} #mtp-aai-tts1 .ft{display:flex;justify-content:space-between;gap:10px;flex-wrap:wrap;padding:14px 26px 18px;font-size:11.5px;color:var(--muted)!important;border-top:1px solid #3a3730!important;margin-top:14px} #mtp-aai-tts1 .ft .mtp{color:var(--mtp)!important;font-weight:700} #mtp-aai-tts1 a{color:var(--cob2)!important} @media (max-width:640px){ #mtp-aai-tts1 .hd,#mtp-aai-tts1 .pane{padding-left:16px;padding-right:16px} #mtp-aai-tts1 .tabs{padding:0 12px} #mtp-aai-tts1 .flow{flex-direction:column} #mtp-aai-tts1 .arrow{transform:rotate(90deg)} #mtp-aai-tts1 .row{grid-template-columns:110px 1fr 58px;font-size:12px} #mtp-aai-tts1 .grid{grid-template-columns:1fr} #mtp-aai-tts1 h3{font-size:20px!important} } <div id="mtp-aai-tts1"> <div class="hd"> <div class="eyebrow">AssemblyAI · Text-to-Speech · Oct 2026</div> <h3>AssemblyAI-TTS-1, explained</h3> <p class="sub">A streaming voice for phone agents: $15 per 1M characters, 4,000+ voices, 6 languages.</p> </div> <div class="tabs" role="tablist"> <button class="tab on" data-t="flow">How it works</button> <button class="tab" data-t="bench">Benchmarks</button> <button class="tab" data-t="cost">Cost calculator</button> <button class="tab" data-t="limits">Limits</button> </div> <div class="pane on" data-p="flow"> <div class="flow" id="aai-flow"> <div class="node" data-i="0"><b>Agent LLM</b>writes the reply</div><div class="arrow">›</div> <div class="node" data-i="1"><b>WebSocket</b>1 session per call</div><div class="arrow">›</div> <div class="node" data-i="2"><b>TTS-1</b>continuous audio</div><div class="arrow">›</div> <div class="node" data-i="3"><b>Audio out</b>8 to 48 kHz<div class="wave"><i></i><i></i><i></i><i></i><i></i><i></i><i></i></div></div><div class="arrow">›</div> <div class="node" data-i="4"><b>Barge-in</b>cancel message</div> </div> <div class="detail" id="aai-detail">Press Play or Next to step through a live call turn.</div> <div class="ctrl"> <button class="btn" id="aai-play"> Play</button> <button class="btn ghost" id="aai-prev">‹ Back</button> <button class="btn ghost" id="aai-next">Next ›</button> </div> </div> <div class="pane" data-p="bench"> <div class="pills" id="aai-pills"> <button class="pill on" data-m="ent">Entity accuracy</button> <button class="pill" data-m="ttfb">Median TTFB (AssemblyAI test)</button> <button class="pill" data-m="cstream">Streaming latency (Cekura)</button> <button class="pill" data-m="cacc">Reading accuracy (Cekura)</button> </div> <div class="bars" id="aai-bars"></div> <p class="note" id="aai-bnote"></p> </div> <div class="pane" data-p="cost"> <div class="calc"> <label for="aai-min">Spoken minutes per month: <b id="aai-minv" style="color:#fdfcf8">10,000</b></label> <div class="cost" id="aai-cost"></div> <p class="note">Assumes 600 characters per minute of phone speech, the figure AssemblyAI cites as typical. List prices per 1M characters: AssemblyAI-TTS-1 $15, Inworld TTS-2 Flash $15 on demand, ElevenLabs Flash v2.5 $40, ElevenLabs v4 $80. Concurrency limits are not priced in.</p> </div> </div> <div class="pane" data-p="limits"> <div class="grid" id="aai-cards"> <div class="card"><h4>No SSML or speed control <span class="tag">LAUNCH GAP</span></h4><p>The WebSocket API has no speed, pitch, emotion or style parameter, and neither API accepts SSML. Punctuation and written fillers like "um" still shape delivery.</p></div> <div class="card"><h4>No pronunciation dictionaries <span class="tag">COMING SOON</span></h4><p>Custom pronunciation dictionaries and emotion tags are announced but not shipped. Cartesia Sonic 3.6 already offers custom pronunciations.</p></div> <div class="card"><h4>6 languages <span class="tag">COVERAGE</span></h4><p>English, Spanish, French, German, Italian and Portuguese. ElevenLabs Flash v2.5 lists 32 and Cartesia Sonic 3.6 lists 44.</p></div> <div class="card"><h4>Latency depends on method <span class="tag">CHECK IT</span></h4><p>About 90 ms advertised, 47 ms median in AssemblyAI's own test, 159 ms median streaming latency on Cekura (timed from the last word sent).</p></div> <div class="card"><h4>Instant cloning only <span class="tag">LAUNCH GAP</span></h4><p>Pro (fine-tuned) voice cloning is not available yet. Instant clones need a short clip and speaker permission.</p></div> <div class="card"><h4>Cloud only <span class="tag">DEPLOYMENT</span></h4><p>US (us-east-1) and EU (eu-north-1) hosts at the same price. No on-prem option is announced.</p></div> </div> <p class="note">Tap a card to expand it.</p> </div> <div class="ft"><span>Sources: AssemblyAI launch post, docs and pricing; Cekura TTS benchmark; vendor pricing pages. Checked Oct 29, 2026.</span><span class="mtp">Built by Marktechpost</span></div> </div> (function(){ var R=document.getElementById('mtp-aai-tts1');if(!R)return; function resize(){try{parent.postMessage({mtpEmbed:'aai-tts1',h:R.offsetHeight+40},'*')}catch(e){}} R.querySelectorAll('.tab').forEach(function(t){t.addEventListener('click',function(){ R.querySelectorAll('.tab').forEach(function(x){x.classList.remove('on')});t.classList.add('on'); R.querySelectorAll('.pane').forEach(function(p){p.classList.toggle('on',p.getAttribute('data-p')===t.getAttribute('data-t'))}); if(t.getAttribute('data-t')==='bench')draw(cur);setTimeout(resize,60);});}); var steps=['Your agent LLM writes a reply. Text can be sent word by word as it is generated.', 'One WebSocket session per conversation. Voice, language, sample rate and encoding are fixed at connect time.', 'TTS-1 models speech on continuous audio representations, not discrete tokens. AssemblyAI says this improves realism and speed.', 'Audio streams back as raw mono PCM, mu-law or A-law. 24 kHz PCM is native; 8 kHz mu-law suits phone lines.', 'If the caller interrupts, a cancel message stops speech. Word timestamps show what was actually said.']; var nodes=R.querySelectorAll('#aai-flow .node'),det=R.querySelector('#aai-detail'),si=-1,timer=null; function show(i){si=(i+nodes.length)%nodes.length;nodes.forEach(function(n,k){n.classList.toggle('act',k===si);n.classList.toggle('playing',k===3&&si>=3)});det.textContent=(si+1)+'/5 · '+steps[si];resize();} R.querySelector('#aai-next').onclick=function(){stop();show(si+1)}; R.querySelector('#aai-prev').onclick=function(){stop();show(si<=0?4:si-1)}; var pb=R.querySelector('#aai-play'); function stop(){if(timer){clearInterval(timer);timer=null;pb.textContent=' Play'}} pb.onclick=function(){if(timer){stop();return}pb.textContent='❚❚ Pause';show(si+1);timer=setInterval(function(){show(si+1)},2200)}; var D={ ent:{u:'%',max:100,hi:true,note:'AssemblyAI pronunciation set: 580 hard targets (phone numbers, dates, URLs, codes). Self-reported, eval dated Oct 28, 2026.', r:[['AssemblyAI-TTS-1',93.7,1],['ElevenLabs v4',93.6],['Cartesia Sonic 3.6',82.5],['ElevenLabs Flash v2.5',68.0]]}, ttfb:{u:' ms',max:650,hi:false,note:'Median time to first byte over each API from one client, one request at a time. Self-reported by AssemblyAI. Lower is better.', r:[['AssemblyAI-TTS-1',47,1],['ElevenLabs Flash v2.5',119],['Cartesia Sonic 3.6',143],['Inworld TTS-2 Flash',244],['Inworld TTS-2',280],['ElevenLabs v4',612]]}, cstream:{u:' ms',max:600,hi:false,note:'Cekura: median from the last word sent to first audible sound, US East, 104 sentences x 3 reads. Independent. Lower is better.', r:[['Inworld TTS-2 Flash',45],['AssemblyAI-TTS-1',159,1],['ElevenLabs Flash v2.5',177],['Cartesia Sonic 3.6',270],['ElevenLabs v4',561]]}, cacc:{u:'%',max:100,hi:true,note:'Cekura: hard parts read correctly, scored by speech-to-text transcribers (an estimate, not human-rated). Independent.', r:[['Inworld TTS-2 Flash',93.2],['AssemblyAI-TTS-1',81.5,1],['ElevenLabs v4',80.4],['Cartesia Sonic 3.6',74.9],['ElevenLabs Flash v2.5',64.4]]}}; var cur='ent',bars=R.querySelector('#aai-bars'),bn=R.querySelector('#aai-bnote'); function draw(m){cur=m;var d=D[m];bars.innerHTML='';d.r.forEach(function(x){ var row=document.createElement('div');row.className='row'+(x[2]?' me':''); row.innerHTML='<span class="nm">'+x[0]+'</span><span class="track"><span class="fill"></span></span><span class="val">'+(d.u==='%'?x[1].toFixed(1):x[1])+d.u+'</span>'; bars.appendChild(row);var f=row.querySelector('.fill');setTimeout(function(){f.style.width=Math.max(2,x[1]/d.max*100)+'%'},40);}); bn.textContent=d.note;setTimeout(resize,60);} R.querySelectorAll('#aai-pills .pill').forEach(function(p){p.addEventListener('click',function(){ R.querySelectorAll('#aai-pills .pill').forEach(function(x){x.classList.remove('on')});p.classList.add('on');draw(p.getAttribute('data-m'))})}); var rng=R.querySelector('#aai-min'),mv=R.querySelector('#aai-minv'),co=R.querySelector('#aai-cost'); var P=[['AssemblyAI-TTS-1',15,1],['Inworld TTS-2 Flash (on demand)',15],['ElevenLabs Flash v2.5',40],['ElevenLabs v4',80]]; function calc(){var m=+rng.value,ch=m*600;mv.textContent=m.toLocaleString('en-US');var h=''; P.forEach(function(p){var c=ch/1e6*p[1];h+='<span class="&apos;+(p[2]?&apos;me&apos;:&apos;&apos;)+&apos;">'+p[0]+'</span><span class="&apos;+(p[2]?&apos;me&apos;:&apos;&apos;)+&apos;">$'+c.toLocaleString('en-US',{maximumFractionDigits:0})+'/mo</span>'}); co.innerHTML=h+'<span style="color:#a5a4a2">Characters per month</span><span style="color:#a5a4a2">'+(ch/1e6).toLocaleString('en-US',{maximumFractionDigits:1})+'M</span>';} rng.addEventListener('input',calc);calc(); R.querySelectorAll('#aai-cards .card').forEach(function(c){c.addEventListener('click',function(){c.classList.toggle('open');setTimeout(resize,40)})}); draw('ent');show(0); window.addEventListener('load',resize);window.addEventListener('resize',resize);setTimeout(resize,300); })(); "> (function(){var f=document.getElementById("mtp-aai-tts1-frame");window.addEventListener("message",function(e){if(e.data&&e.data.mtpEmbed==="aai-tts1"&&e.data.h){f.style.height=e.data.h+"px";}});})(); How does AssemblyAI-TTS-1 compare with ElevenLabs, Cartesia and Inworld? Cartesia’s range divides each plan price by its credits, at 1 credit per character. What does AssemblyAI-TTS-1 cost? Pricing is per input character, including spaces and punctuation, at $15 per 1M. Every voice, cloning, word timestamps and EU processing come at that same rate. There are no per-voice fees, concurrency fees or minimums. AssemblyAI calls it the lowest-priced frontier TTS model for voice agents. However, Inworld TTS-2 Flash also lists $15 per 1M on demand, falling to $7 on paid plans. The sharper difference is concurrency: Inworld’s on-demand tier allows 5 concurrent requests. Key Takeaways AssemblyAI-TTS-1 is AssemblyAI’s first text-to-speech model, built for live voice agents. It costs $15 per 1M characters, with unlimited concurrency and no contract. Self-reported: 93.7% entity accuracy and 47 ms median time to first byte. Cekura measured 159 ms streaming latency and 81.5% reading accuracy. Missing at launch: SSML, speed control, pronunciation dictionaries and emotion tags. Check out the launch post, docs, pricing, and the LiveKit and Pipecat reference agents. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post AssemblyAI Ships AssemblyAI-TTS-1: A $15 per 1M Character Text-to-Speech API for Voice Agents appeared first on MarkTechPost.

Advertisement
AI

NVIDIA Releases Nemotron 3.5 Transcribe: An Open 30B MoE Model for 40-Language Transcription with ~4B Active Parameters

NVIDIA has released Nemotron 3.5 Transcribe, an open-weights speech LLM built for transcription. One model handles 40 languages and recordings up to 1 hour. A text instruction picks the output style for each request. Options include verbatim, written form, speaker-attributed, contextual and translated output. For developers, it can replace separate ASR, diarization and alignment tools. TL;DR Size: 34B total and about 4.3B active parameters; 65,536-token context; audio up to 1 hour. Runs on: 1x H100 80 GB in BF16. FP8 (about 35 GB) and NVFP4 (about 21 GB) checkpoints exist; NVFP4 runs on an RTX 5090. GGUF files target NeMo-Speech.cpp. Performance: Best open-weights model on the English and multilingual Open ASR Leaderboards; top commercial APIs still lead. Best: 4.71% mean WER on the 9 public English Open ASR sets (next open model: 4.94%). Worst: 34.95% cpCER on AliMeeting Mandarin far-field meetings. Bottom line: Best thing: 1 open model covers transcripts, speaker labels, timestamps and translation. Worst thing: offline only, with weak Mandarin meeting accuracy. What is Nemotron 3.5 Transcribe? Nemotron 3.5 Transcribe is a speech-augmented language model (SALM) from NVIDIA that turns audio and instructions into text. An audio encoder feeds embeddings into an LLM decoder, which makes transcription promptable. NVIDIA’s technical research report argues there is no single correct transcript. Numbers, fillers and domain terms depend on who reads the output. NVIDIA positions the model as a transcription specialist, not a general-purpose assistant. How does the architecture work? It has three parts: Parallel Expert Encoder: a trainable 0.6B Transformer ASR encoder runs beside a frozen Nemotron 3 Diarization model. The diarizer tracks up to 8 speakers. Its speaker activity is added to the ASR features every 80 ms. The LLM therefore sees who spoke when, without architecture changes. LLM backbone: Nemotron 3.5 Lightning 30B-A3B, a hybrid Mamba-Transformer MoE. It has 52 layers: 23 Mamba-2, 23 MoE and only 6 attention layers. Each MoE layer routes to 6 of 128 experts plus 1 shared expert. Few attention layers keep the KV cache small for hour-long audio. Add-ons: a shared multi-token prediction (MTP) layer drafts tokens for speculative decoding. A 50M-parameter CTC adapter aligns the finished transcript to audio for word timestamps. The model name says 30B-A4B, but the checkpoint holds about 34B parameters. The extra weight comes from the 1.3B MTP module and the audio components. Training used 1 supervised fine-tuning stage on 256 H100 GPUs for 28,500 steps. A short RL stage followed: 400 GSPO updates with a reward of 1 minus WER. The model saw about 1.72M unique hours of audio, or 6.5M hours with repeats. How does it score on benchmarks? All numbers below are self-reported by NVIDIA in the technical report. English Open ASR (9 public sets): 4.71% mean WER. The next open model, Qwen3-ASR-1.7B, scores 4.94%. Seven commercial systems still lead, topped by OtterAI speech-v3.1 at 3.88%. Multilingual Open ASR (7 languages): 4.62% mean WER, versus 6.49% for Qwen3-ASR-1.7B. ElevenLabs Scribe v2 reaches 3.56%. Long-form ASR: 10.98% mean WER, versus 13.37% for Parakeet-TDT-0.6B-v3. Multi-speaker, full sessions: 15.80% mean cpWER, versus 16.83% for DiCoW v3.3 with DiariZen. NVIDIA says it is the first SALM to beat cascaded systems here. Contextual ASR: an entity list cuts named-entity WER on ContextASR-Bench from 12.21% to 2.20%. <iframe id="mtp-nt35-frame" title="Nemotron 3.5 Transcribe explainer" style="width:100%;max-width:880px;height:600px;border:0;display:block;margin:0 auto;" loading="lazy" srcdoc="<!doctype html><title>Nemotron 3.5 Transcribe Explainer</title>html,body{margin:0;background:#0d0d0d}<div id="mtp-nt35"> #mtp-nt35{--g:#76B900;--g2:#5a8f00;--bg:#0d0d0d;--card:#161616;--line:#2a2a2a;--tx:#ededed;--mu:#9a9a9a;font-family:Inter,-apple-system,Segoe UI,Roboto,sans-serif!important;background:var(--bg)!important;color:var(--tx)!important;max-width:880px;margin:0 auto;border:1px solid var(--line)!important;border-radius:14px;overflow:hidden;box-sizing:border-box} #mtp-nt35 *{box-sizing:border-box} #mtp-nt35 hr,#mtp-nt35 p:empty,#mtp-nt35 del,#mtp-nt35 s{display:none!important} #mtp-nt35 .hd{padding:22px 24px 14px;border-bottom:1px solid var(--line)!important;background:linear-gradient(135deg,#121a05 0%,#0d0d0d 60%)!important} #mtp-nt35 .tag{display:inline-block;font-size:11px;letter-spacing:.12em;text-transform:uppercase;color:var(--g)!important;border:1px solid var(--g)!important;border-radius:999px;padding:3px 10px;margin-bottom:10px} #mtp-nt35 h3{margin:0 0 4px!important;font-size:22px!important;line-height:1.25!important;color:#fff!important;font-weight:800!important} #mtp-nt35 .sub{margin:0!important;color:var(--mu)!important;font-size:14px!important} #mtp-nt35 .tabs{display:flex;gap:4px;padding:10px 12px 0;border-bottom:1px solid var(--line)!important;overflow-x:auto} #mtp-nt35 .tab{background:transparent!important;border:0!important;color:var(--mu)!important;font:600 13px Inter,sans-serif;padding:10px 14px;cursor:pointer;border-bottom:2px solid transparent!important;white-space:nowrap;transition:color .2s} #mtp-nt35 .tab:hover{color:#fff!important} #mtp-nt35 .tab.on{color:var(--g)!important;border-bottom-color:var(--g)!important} #mtp-nt35 .pane{display:none;padding:20px 24px 22px;animation:nt35fade .35s ease} #mtp-nt35 .pane.on{display:block} @keyframes nt35fade{from{opacity:0;transform:translateY(6px)}to{opacity:1;transform:none}} #mtp-nt35 .flow{display:grid;grid-template-columns:repeat(4,1fr);gap:10px;position:relative} #mtp-nt35 .node{background:var(--card)!important;border:1px solid var(--line)!important;border-radius:10px;padding:12px;min-height:96px;transition:all .35s;cursor:pointer;position:relative} #mtp-nt35 .node b{display:block;font-size:13px;color:#fff!important;margin-bottom:4px} #mtp-nt35 .node span{font-size:12px;color:var(--mu)!important;line-height:1.4} #mtp-nt35 .node .n{position:absolute;top:8px;right:10px;font-size:11px;color:var(--mu)!important} #mtp-nt35 .node.on{border-color:var(--g)!important;box-shadow:0 0 0 1px var(--g),0 0 22px rgba(118,185,0,.25);transform:translateY(-2px)} #mtp-nt35 .node.on .n{color:var(--g)!important} #mtp-nt35 .wire{height:4px!important;background:var(--line)!important;border-radius:4px;margin:14px 0 4px;overflow:hidden} #mtp-nt35 .wire i{display:block;height:4px!important;width:0;background:var(--g)!important;transition:width .5s ease} #mtp-nt35 .detail{background:#111!important;border:1px solid var(--line)!important;border-left:3px solid var(--g)!important;border-radius:8px;padding:14px 16px;margin-top:12px;font-size:14px;line-height:1.55;min-height:92px} #mtp-nt35 .detail strong{color:var(--g)!important} #mtp-nt35 .ctrl{display:flex;gap:8px;margin-top:12px;flex-wrap:wrap} #mtp-nt35 .btn{background:#1d1d1d!important;color:#fff!important;border:1px solid var(--line)!important;border-radius:8px;padding:8px 14px;font:600 13px Inter,sans-serif;cursor:pointer;transition:all .2s} #mtp-nt35 .btn:hover{border-color:var(--g)!important} #mtp-nt35 .btn.pri{background:var(--g)!important;color:#0b0b0b!important;border-color:var(--g)!important} #mtp-nt35 .seg{display:flex;gap:6px;flex-wrap:wrap;margin-bottom:14px} #mtp-nt35 .chip{background:#1a1a1a!important;color:var(--mu)!important;border:1px solid var(--line)!important;border-radius:999px;padding:6px 12px;font:600 12px Inter,sans-serif;cursor:pointer;transition:all .2s} #mtp-nt35 .chip.on{color:#0b0b0b!important;background:var(--g)!important;border-color:var(--g)!important} #mtp-nt35 .bar{display:grid;grid-template-columns:170px 1fr 58px;align-items:center;gap:10px;margin:9px 0;font-size:13px} #mtp-nt35 .bar .lb{color:#ddd!important;white-space:nowrap;overflow:hidden;text-overflow:ellipsis} #mtp-nt35 .bar .tr{background:#1c1c1c!important;border-radius:6px;height:22px!important;overflow:hidden} #mtp-nt35 .bar .fl{height:22px!important;width:0;background:#3a3a3a!important;border-radius:6px;transition:width .9s cubic-bezier(.2,.8,.2,1)} #mtp-nt35 .bar.me .fl{background:linear-gradient(90deg,var(--g2),var(--g))!important} #mtp-nt35 .bar.me .lb{color:var(--g)!important;font-weight:700} #mtp-nt35 .bar .v{text-align:right;font-variant-numeric:tabular-nums;color:#fff!important;font-weight:700} #mtp-nt35 .note{font-size:12px;color:var(--mu)!important;margin-top:10px!important;line-height:1.5} #mtp-nt35 .grid2{display:grid;grid-template-columns:1fr 1fr;gap:10px} #mtp-nt35 .card{background:var(--card)!important;border:1px solid var(--line)!important;border-radius:10px;padding:14px;transition:all .25s;cursor:pointer} #mtp-nt35 .card:hover,#mtp-nt35 .card.on{border-color:var(--g)!important} #mtp-nt35 .card h4{margin:0 0 6px!important;font-size:15px!important;color:#fff!important} #mtp-nt35 .card .big{font-size:24px;font-weight:800;color:var(--g)!important} #mtp-nt35 .card p{margin:4px 0 0!important;font-size:13px!important;color:var(--mu)!important;line-height:1.45} #mtp-nt35 .card .more{max-height:0;overflow:hidden;transition:max-height .35s ease;font-size:13px;color:#ccc!important;line-height:1.45} #mtp-nt35 .card.on .more{max-height:120px;margin-top:8px} #mtp-nt35 .out{background:#0a0a0a!important;border:1px solid var(--line)!important;border-radius:8px;padding:14px;margin-top:10px} #mtp-nt35 pre,#mtp-nt35 code{background:#0a0a0a!important;color:#d8f5a8!important;border:0!important;font:13px/1.55 ui-monospace,SFMono-Regular,Menlo,monospace!important;white-space:pre-wrap!important;margin:0!important;padding:0!important} #mtp-nt35 .lbl{font-size:11px;letter-spacing:.1em;text-transform:uppercase;color:var(--mu)!important;margin-bottom:6px} #mtp-nt35 .row{display:flex;gap:10px;align-items:flex-start;padding:10px 0;border-bottom:1px solid #1f1f1f!important;font-size:14px;line-height:1.5} #mtp-nt35 .row:last-child{border-bottom:0!important} #mtp-nt35 .ic{flex:0 0 22px;height:22px;border-radius:6px;display:flex;align-items:center;justify-content:center;font-weight:800;font-size:13px} #mtp-nt35 .ic.up{background:rgba(118,185,0,.15)!important;color:var(--g)!important} #mtp-nt35 .ic.dn{background:rgba(255,120,80,.15)!important;color:#ff8a5c!important} #mtp-nt35 .ft{display:flex;justify-content:space-between;flex-wrap:wrap;gap:6px;padding:12px 24px;border-top:1px solid var(--line)!important;font-size:12px;color:var(--mu)!important} #mtp-nt35 .ft a,#mtp-nt35 .ft .br{color:var(--g)!important;text-decoration:none;font-weight:700} @media(max-width:640px){ #mtp-nt35 .hd{padding:18px 16px 12px} #mtp-nt35 h3{font-size:18px!important} #mtp-nt35 .pane{padding:16px} #mtp-nt35 .flow{grid-template-columns:1fr 1fr} #mtp-nt35 .grid2{grid-template-columns:1fr} #mtp-nt35 .bar{grid-template-columns:110px 1fr 50px;font-size:12px} #mtp-nt35 .ft{padding:12px 16px} } <div class="hd"> <div class="tag">NVIDIA · Open weights · Oct 28, 2026</div> <h3>How Nemotron 3.5 Transcribe works</h3> <p class="sub">34B total, ~4.3B active parameters · 40 languages · audio up to 1 hour</p> </div> <div class="tabs" role="tablist"> <button class="tab on" data-t="p1">How it works</button> <button class="tab" data-t="p2">Benchmarks</button> <button class="tab" data-t="p3">Prompt styles</button> <button class="tab" data-t="p4">Deploy</button> <button class="tab" data-t="p5">Trade-offs</button> </div> <div class="pane on" id="p1"> <div class="flow"> <div class="node on" data-s="0"><span class="n">01</span><b>Audio in</b><span>16 kHz mono, split into 30 s encoder chunks</span></div> <div class="node" data-s="1"><span class="n">02</span><b>Parallel Expert Encoder</b><span>0.6B ASR encoder + frozen diarizer</span></div> <div class="node" data-s="2"><span class="n">03</span><b>Hybrid MoE LLM</b><span>Nemotron 3.5 Lightning 30B-A3B</span></div> <div class="node" data-s="3"><span class="n">04</span><b>Text + timestamps</b><span>CTC adapter aligns words to audio</span></div> </div> <div class="wire"><i id="nt35w"></i></div> <div class="detail" id="nt35d"></div> <div class="ctrl"> <button class="btn" id="nt35prev">← Back</button> <button class="btn pri" id="nt35next">Next step →</button> <button class="btn" id="nt35play"> Auto-play</button> </div> </div> <div class="pane" id="p2"> <div class="seg"> <button class="chip on" data-b="en">English Open ASR</button> <button class="chip" data-b="ml">Multilingual Open ASR</button> <button class="chip" data-b="lf">Long-form</button> </div> <div id="nt35bars"></div> <p class="note" id="nt35bn"></p> </div> <div class="pane" id="p3"> <p class="note" style="margin:0 0 10px!important">1 checkpoint, different instructions. Click a style to see the request pattern and output shape (illustrative, not a model sample).</p> <div class="seg" id="nt35ps"> <button class="chip on" data-p="0">Verbatim</button> <button class="chip" data-p="1">Written form</button> <button class="chip" data-p="2">Context terms</button> <button class="chip" data-p="3">Speakers</button> <button class="chip" data-p="4">Translate</button> </div> <div class="out"><div class="lbl">Instruction</div><pre><code id="nt35pi"></code></pre></div> <div class="out"><div class="lbl">Output shape</div><pre><code id="nt35po"></code></pre></div> </div> <div class="pane" id="p4"> <div class="grid2"> <div class="card on"><h4>BF16</h4><div class="big">67.5 GB</div><p>Reference weights · 1x H100 80 GB</p><div class="more">Ampere, Hopper and Blackwell. NVIDIA estimates about 410 RTFx on 1 H100 with MTP2 (batched English ASR).</div></div> <div class="card"><h4>FP8</h4><div class="big">~35 GB</div><p>Ada Lovelace, Hopper, Blackwell</p><div class="more">Error rate 1.0% higher in relative terms on average. NVIDIA recommends it when non-English accuracy matters most.</div></div> <div class="card"><h4>NVFP4</h4><div class="big">~21 GB</div><p>Blackwell only · runs on RTX 5090</p><div class="more">Error rate 3.6% higher in relative terms on average. Best when memory or throughput is the main constraint.</div></div> <div class="card"><h4>NIM speed</h4><div class="big">2,238x</div><p>Real time on 1 RTX PRO 6000 Blackwell</p><div class="more">NVFP4 + DFlash2 in the Nemotron Transcribe NIM; 30 s clip in 0.13 s. These serving optimizations are not open source yet.</div></div> </div> <p class="note">Runtimes: vLLM 0.32.0 + NeMo Speech plugin, Hugging Face Transformers, SGLang-Omni 0.2.0, GGML (NeMo-Speech.cpp). License: OpenMDW-1.1.</p> </div> <div class="pane" id="p5"> <div class="row"><div class="ic up">+</div><div><b>Best open-weights ASR:</b> 4.71% English and 4.62% multilingual mean WER on the Open ASR Leaderboards.</div></div> <div class="row"><div class="ic up">+</div><div><b>Meetings:</b> 15.80% mean cpWER on full sessions, ahead of DiCoW v3.3 + DiariZen at 16.83%.</div></div> <div class="row"><div class="ic up">+</div><div><b>Context:</b> an entity list cuts named-entity WER on ContextASR-Bench from 12.21% to 2.20%.</div></div> <div class="row"><div class="ic dn">!</div><div><b>Commercial gap:</b> OtterAI speech-v3.1 reaches 3.88% English WER; ElevenLabs Scribe v2 reaches 3.56% multilingual.</div></div> <div class="row"><div class="ic dn">!</div><div><b>Mandarin meetings:</b> 32.91% cpCER on AISHELL-4 and 34.95% on AliMeeting.</div></div> <div class="row"><div class="ic dn">!</div><div><b>Offline only:</b> not a streaming ASR service; release-day vLLM is roughly 2x slower than the NIM.</div></div> </div> <div class="ft"><span>Source: NVIDIA technical report and model card (self-reported), Oct 2026</span><span class="br">© Marktechpost</span></div> (function(){ var R=document.getElementById('mtp-nt35'); function post(){try{parent.postMessage({mtpEmbed:'nt35',h:R.offsetHeight+40},'*')}catch(e){}} R.querySelectorAll('.tab').forEach(function(t){t.addEventListener('click',function(){ R.querySelectorAll('.tab').forEach(function(x){x.classList.remove('on')}); R.querySelectorAll('.pane').forEach(function(x){x.classList.remove('on')}); t.classList.add('on');document.getElementById(t.dataset.t).classList.add('on'); if(t.dataset.t==='p2')drawBars(curB);setTimeout(post,60);setTimeout(post,500);});}); var S=[ '<strong>Audio in.</strong> The model takes 16 kHz single-channel audio, up to 1 hour per request. Encoders process independent 30 s chunks, so encoder compute grows linearly with length. Each second of audio becomes 12.5 LLM input positions.', '<strong>Parallel Expert Encoder.</strong> A trainable 32-layer Transformer ASR encoder (0.6B) runs beside the frozen Nemotron 3 Diarization model (~100M, up to 8 speakers). Speaker activity is added to ASR features every 80 ms, so the LLM sees who spoke when.', '<strong>Hybrid MoE LLM.</strong> Nemotron 3.5 Lightning 30B-A3B: 23 Mamba-2, 23 MoE and 6 attention layers. 128 experts, top-6 routing, 1 shared expert, ~3.5B active. A shared MTP layer drafts tokens for speculative decoding.', '<strong>Text + timestamps.</strong> The LLM writes the transcript in the style the instruction asks for. A 50M-parameter CTC adapter then force-aligns words to audio, giving speaker-attributed word timestamps.']; var cur=0,timer=null,nodes=R.querySelectorAll('.node'),D=document.getElementById('nt35d'),W=document.getElementById('nt35w'); function go(i){cur=(i+4)%4;nodes.forEach(function(n,k){n.classList.toggle('on',k===cur)});D.innerHTML=S[cur];W.style.width=((cur+1)*25)+'%';} nodes.forEach(function(n){n.addEventListener('click',function(){stop();go(+n.dataset.s)})}); document.getElementById('nt35next').onclick=function(){stop();go(cur+1)}; document.getElementById('nt35prev').onclick=function(){stop();go(cur-1)}; var pb=document.getElementById('nt35play'); function stop(){if(timer){clearInterval(timer);timer=null;pb.textContent=' Auto-play'}} pb.onclick=function(){if(timer){stop();return}pb.textContent='❚❚ Pause';timer=setInterval(function(){go(cur+1)},2200)}; go(0); var B={ en:{d:[['Nemotron 3.5 Transcribe',4.71,1],['Qwen3-ASR-1.7B',4.94],['Canary-Qwen-2.5B',5.38],['Parakeet-TDT-0.6B-v3',5.58],['Whisper-large-v3',6.66]],m:8,n:'Mean WER (%) on the 9 public English Open ASR sets, lower is better. Baselines: leaderboard results of Oct 21, 2026, as listed in NVIDIA's report.'}, ml:{d:[['Nemotron 3.5 Transcribe',4.62,1],['Qwen3-ASR-1.7B',6.49],['Voxtral-Small-24B',6.52],['Whisper-large-v3',7.94]],m:9,n:'Mean WER (%) over 7 languages (de, fr, it, es, pt, nl, hi; Hindi uses OIWER), lower is better. Baselines: leaderboard results of Oct 2, 2026.'}, lf:{d:[['Nemotron 3.5 Transcribe',10.98,1],['Parakeet-TDT-0.6B-v3',13.37],['Whisper-large-v3',13.92],['Canary-Qwen-2.5B',14.08]],m:16,n:'Mean WER (%) on the long-form track (Earnings-21, Earnings-22, CORAAL), lower is better.'}}; var curB='en',BB=document.getElementById('nt35bars'),BN=document.getElementById('nt35bn'); function drawBars(k){curB=k;var b=B[k];BB.innerHTML=b.d.map(function(r){return '<div class="bar&apos;+(r[2]?&apos; me&apos;:&apos;&apos;)+&apos;"><div class="lb">'+r[0]+'</div><div class="tr"><div class="fl" data-w="&apos;+(r[1]/b.m*100)+&apos;"></div></div><div class="v">'+r[1].toFixed(2)+'%</div></div>'}).join('');BN.textContent=b.n; setTimeout(function(){BB.querySelectorAll('.fl').forEach(function(f){f.style.width=f.dataset.w+'%'})},40);setTimeout(post,80);} R.querySelectorAll('[data-b]').forEach(function(c){c.addEventListener('click',function(){R.querySelectorAll('[data-b]').forEach(function(x){x.classList.remove('on')});c.classList.add('on');drawBars(c.dataset.b)})}); drawBars('en'); var P=[ ['Audio transcription. Language: as spoken.nWording: verbatim, including fillers and repetitions.nFormatting: punctuation and capitalization.nOutput: transcript only.','So, uh, the the launch moved tonnext Tuesday at three thirty.'], ['Audio transcription. Language: as spoken.nWording: omit fillers and accidental repetitions.nNumbers in their standard written form.','So, the launch moved tonnext Tuesday at 3:30.'], ['Transcribe with punctuation and capitalization.nContext terms: Nemotron, NVFP4, DFlash2.','We tested Nemotron in NVFP4nwith the DFlash2 draft model.'], ['Transcribe this audio with punctuation and capitalization.nPut speaker tags non the transcription.n(+ timestamps enabled in the request)',' Can we ship on Friday?n Yes, if QA signs off.nnwords: [{"word":"Can","start":0.42,"end":0.58,"speaker":"spk:0"}, ...]'], ['Please translate the given speech to German.','Der Start wurde auf nächstennDienstag um 15:30 Uhr verschoben.']]; var PI=document.getElementById('nt35pi'),PO=document.getElementById('nt35po'),tt=null; function showP(i){PI.textContent=P[i][0];var s=P[i][1],k=0;PO.textContent='';if(tt)clearInterval(tt);tt=setInterval(function(){k+=2;PO.textContent=s.slice(0,k);if(k>=s.length){clearInterval(tt);post()}},16);setTimeout(post,60);} R.querySelectorAll('[data-p]').forEach(function(c){c.addEventListener('click',function(){R.querySelectorAll('[data-p]').forEach(function(x){x.classList.remove('on')});c.classList.add('on');showP(+c.dataset.p)})}); showP(0); R.querySelectorAll('.card').forEach(function(c){c.addEventListener('click',function(){R.querySelectorAll('.card').forEach(function(x){if(x!==c)x.classList.remove('on')});c.classList.toggle('on');setTimeout(post,380)})}); window.addEventListener('load',post);window.addEventListener('resize',post);setTimeout(post,300); })(); </div> "> (function(){var f=document.getElementById("mtp-nt35-frame");window.addEventListener("message",function(e){if(e.data&&e.data.mtpEmbed==="nt35"&&e.data.h){f.style.height=e.data.h+"px";}});})(); <!-- Fallback: upload nemotron-transcribe-explainer-standalone.html to /wp-content/uploads/ via FTP and use: --> How do you deploy Nemotron 3.5 Transcribe? Weights ship in BF16, FP8, NVFP4 and GGUF. A DFlash2 draft model is optional. The OpenMDW-1.1 license allows commercial use. Runtimes include vLLM 0.32.0 with the NeMo Speech plugin, Transformers and SGLang-Omni 0.2.0. vLLM exposes an OpenAI-compatible API. NVIDIA model’s main speed comes from its Nemotron Transcribe NIM. With NVFP4 and DFlash2, 1 RTX PRO 6000 Blackwell Server Edition transcribes 2,238 seconds of audio per second. It returns a 30-second clip in 0.13 seconds. There is a catch: those serving optimizations are not open source yet. NVIDIA expects the release-day vLLM path to be roughly 2x slower. On 1 H100 in BF16 with MTP2, NVIDIA estimates about 410 RTFx. For fine-tuning, NVIDIA tested 17 tasks. Each took about 0.5 to 2.3 GPU-hours on a single GB300. How does it compare with other open ASR models? *Competitor scores on the current 9-set leaderboard come from NVIDIA’s report, citing Hugging Face leaderboard results from October 2026. The leaderboard set changed since the competitors published their own cards, so the 2 WER rows are not interchangeable. Key Takeaways One open model handles transcripts, speakers, timestamps and translation across 40 languages. 4.71% English WER is the best open-weights score; commercial APIs still lead. About 4.3B active parameters; BF16 fits 1 H100, NVFP4 fits an RTX 5090. An entity list cuts named-entity WER from 12.21% to 2.20%. Offline only; Mandarin meetings and code-switching remain weak spots. Check out the technical report, model weights, project page and live demo. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post NVIDIA Releases Nemotron 3.5 Transcribe: An Open 30B MoE Model for 40-Language Transcription with ~4B Active Parameters appeared first on MarkTechPost.