Sam Altman believes advanced AI can't be kept locked up forever. Sandboxes and monitoring are only temporary fixes. Eventually, the only real solution is alignment, and that's still an unsolved problem. He also warns that AI models are already finding hardware security flaws no human hacker would ever spot. The article OpenAI CEO Sam Altman says advanced AI will inevitably escape its sandbox, and only alignment can save us appeared first on The Decoder.
Anthropic had Claude generate mathematical proofs that existing code is correct, producing roughly 180 million lines of proof code for 123 open-source libraries. Along the way, the model caught bugs that years of testing had missed. Anthropic's goal: making formal verification with AI as routine as standard testing. The article Anthropic's Claude now writes code and proves it safe automatically appeared first on The Decoder.
OpenAI is closing in on a new funding round of up to $30 billion but has lowered its valuation target to get there. The article OpenAI lowers valuation target, raises up to $30 billion appeared first on The Decoder.
An AI search agent can work in a demo but fail under production workloads when retrieval returns incomplete evidence, retries repeat unsuccessful actions, or execution exceeds time and cost limits. TinyFish, which provides web infrastructure for AI agents, has published two technical guides addressing these challenges. The first defines a production agentic search architecture, while the second shares lessons from building a deep research system. Together, they examine how an autonomous search agent can plan retrieval, evaluate evidence, handle failures, and manage the boundary between model reasoning and deterministic execution. TL;DR TinyFish introduced its agentic search architecture and key engineering lessons from building deep research systems. Built on four web primitives (TinySearch, TinyFetch, TinyBrowser, and TinyAgent), the system operates via a seven-phase control loop around the base model. On OpenBenchmarks, TinyFish achieved a 79.0% completion rate on coding tasks while delivering the lowest median task cost ($0.050) and token footprint, highlighting the importance of rigid control loops, failure-specific retries, and unified reasoning over standalone planners. What is Agentic Search Architecture? Agentic search architecture enables an AI agent to plan retrieval, access sources, evaluate evidence, and adapt its approach as new information becomes available. The model provides reasoning, while the surrounding application manages state, tool execution, evidence handling, runtime policies, and output validation. Together, these components allow an agent to handle incomplete results, recoverable failures, and changing information requirements. TinyFish’s architecture draws foundational concepts from a February 2026 architecture preprint, which decouples reasoning from execution through typed tool interfaces. The paper establishes essential system guardrails, including detailed tracing alongside strict caps on tokens, execution time, overall cost, and tool invocations. How does a production search run work? TinyFish splits a run into 7 phases: Define the task: Turn the goal into an “evidence contract”: required fields, sources, freshness and completion rules. Plan retrieval: Separate independent needs from dependent ones. Dependent queries wait for earlier answers. Pick the first route: Choose a tool based on what the system knows about the evidence. Discover, select, retrieve: Treat search results as candidates until the page content arrives. Evaluate: Label evidence as sufficient, incomplete, conflicting, inaccessible or budget exhausted. Recover or escalate: Revise the query, retry, change sources or change access method. Stop and validate: Return completed, incomplete, failed, cancelled or human review required. Phase 2 addresses a known failure. ChainRAG (ACL 2025) calls it the ‘lost-in-retrieval’ problem, where key entities disappear during multi-hop question decomposition. Phase 6 draws on Search-R1, which demonstrates how models can learn to generate additional search queries during reasoning when earlier retrieval is insufficient. Across 7 QA datasets, its learned multi-turn search beat RAG baselines by 41% (Qwen2.5-7B) and 20% (Qwen2.5-3B). How should a search agent route tools? Routing should follow the information gap, not a fixed ladder. Escalation is not always the fix. Weak results need a better query. Conflicting sources need stronger evidence. A heavier tool fixes neither. Retries should be based on the failure type and remain within defined limits. Transient errors may justify repeating a request, while incomplete or conflicting evidence requires a different approach. Browser retries must also account for actions already completed to avoid unintended consequences, such as submitting the same form twice. <iframe id="mtp-tfas-frame" title="How a production search agent works" loading="lazy" style="width:100%;height:600px;border:0;display:block;background:transparent;" scrolling="no" srcdoc="<!doctype html> <title>Agentic Search Explainer</title> html,body{margin:0!important;padding:0!important;background:transparent!important} #mtp-tfas{--bg:#111;--card:#161616;--line:#2c2c2c;--g:#76B900;--t:#eaeaea;--m:#8f8f8f;--amb:#e0a03a;--red:#e05a4a; max-width:880px;margin:0 auto;background:var(--bg)!important;color:var(--t)!important;border:1px solid var(--line)!important;border-radius:14px; font:15px/1.45 system-ui,-apple-system,"Segoe UI",Roboto,sans-serif;overflow:hidden;box-sizing:border-box} #mtp-tfas *{box-sizing:border-box} #mtp-tfas hr,#mtp-tfas p:empty,#mtp-tfas del,#mtp-tfas s{display:none!important} #mtp-tfas .hd{padding:18px 22px 4px;display:flex;align-items:baseline;gap:12px;flex-wrap:wrap} #mtp-tfas .kick{color:var(--g)!important;font-size:12px;letter-spacing:.12em;text-transform:uppercase;font-weight:700} #mtp-tfas h3{margin:0!important;font-size:20px;color:#fff!important;line-height:1.25;flex-basis:100%} #mtp-tfas .tabs{display:flex;gap:6px;padding:12px 22px 0;border-bottom:1px solid var(--line);overflow-x:auto} #mtp-tfas .tab{background:transparent!important;border:1px solid var(--line)!important;border-bottom:none!important;color:var(--m)!important; padding:9px 14px;border-radius:9px 9px 0 0;cursor:pointer;font:600 13px system-ui,sans-serif;white-space:nowrap} #mtp-tfas .tab.on{background:var(--card)!important;color:var(--g)!important;border-color:var(--g)!important} #mtp-tfas .pane{display:none;padding:14px 22px 16px;background:var(--card)!important} #mtp-tfas .pane.on{display:block;animation:tfIn .35s ease} @keyframes tfIn{from{opacity:0;transform:translateY(6px)}to{opacity:1;transform:none}} #mtp-tfas .chips{display:flex;flex-wrap:wrap;gap:6px;justify-content:center} #mtp-tfas .chip{background:#1d1d1d!important;border:1px solid var(--line)!important;color:var(--t)!important;padding:7px 12px;border-radius:20px;cursor:pointer;font:600 13px system-ui,sans-serif} #mtp-tfas .chip.on{background:var(--g)!important;color:#111!important;border-color:var(--g)!important} #mtp-tfas svg{display:block;width:100%;max-width:700px;height:auto;margin:6px auto 0} #mtp-tfas svg text{font-family:system-ui,-apple-system,"Segoe UI",Roboto,sans-serif} #mtp-tfas .cap{text-align:center;color:var(--m)!important;font-size:13px;margin:4px 0 0;min-height:19px} #mtp-tfas .cap b{color:var(--g)!important} #mtp-tfas .cap a,#mtp-tfas a{color:var(--g)!important} #mtp-tfas .ring{stroke-dasharray:6 10;animation:tfFlow 1.2s linear infinite} @keyframes tfFlow{to{stroke-dashoffset:-16}} #mtp-tfas .wire{stroke-dasharray:7 7;animation:tfFlow .7s linear infinite} #mtp-tfas .ft{display:flex;justify-content:space-between;gap:10px;padding:10px 22px;border-top:1px solid var(--line);font-size:12px;color:var(--m)!important;background:var(--bg)!important;flex-wrap:wrap} #mtp-tfas .ft .brand{color:var(--g)!important;font-weight:700} @media (max-width:640px){ #mtp-tfas .hd{padding:14px 14px 2px}#mtp-tfas .tabs{padding:10px 14px 0}#mtp-tfas .pane{padding:12px 10px 14px} #mtp-tfas h3{font-size:17px}#mtp-tfas .tab{padding:8px 10px;font-size:12px}#mtp-tfas .chip{padding:6px 10px;font-size:12px}#mtp-tfas .ft{padding:10px 14px} } <div id="mtp-tfas"> <div class="hd"><span class="kick">Explainer · Agentic search</span><h3>How a production search agent works</h3></div> <div class="tabs" role="tablist"> <button class="tab on" data-t="p1">The loop</button> <button class="tab" data-t="p2">Tool routing</button> <button class="tab" data-t="p3">Deep research</button> <button class="tab" data-t="p4">Benchmark</button> </div> <div class="pane on" id="p1"> <div class="chips" id="c1"> <button class="chip" data-s="clean">Clean run</button> <button class="chip on" data-s="weak">Weak results</button> <button class="chip" data-s="budget">Budget hit</button> </div> <p class="cap" id="k1"></p> </div> <div class="pane" id="p2"> <div class="chips" id="c2"> <button class="chip on" data-r="0">Source unknown</button> <button class="chip" data-r="1">URL known</button> <button class="chip" data-r="2">Code drives the page</button> <button class="chip" data-r="3">Multi-page goal</button> <button class="chip" data-r="4">Weak or conflicting</button> </div> <p class="cap" id="k2"></p> </div> <div class="pane" id="p3"> <div class="chips" id="c3"> <button class="chip on" data-v="0">v1 Planner</button> <button class="chip" data-v="1">v2 Subagents</button> <button class="chip" data-v="2">v3 Tool calls</button> <button class="chip" data-v="3">Source weight</button> </div> <p class="cap" id="k3"></p> </div> <div class="pane" id="p4"> <div class="chips" id="c4"> <button class="chip on" data-b="sf">Search + fetch</button> <button class="chip" data-b="so">Search only</button> </div> <p class="cap">Median task cost vs grounded completion, 100 tasks, gpt-5.6-sol. Source: <a href=" target=" rel="noopener">OpenBenchmarks</a>, 26 Sept 2026.</p> </div> <div class="ft"><span>Sources: <a href=" target=" rel="noopener">architecture</a> · <a href=" target=" rel="noopener">deep research</a> · <a href=" target=" rel="noopener">OpenBenchmarks</a></span><span class="brand">Built by Marktechpost</span></div> </div> (function(){ var R=document.getElementById('mtp-tfas'),NS=' var G='#76B900',LN='#2c2c2c',BX='#1d1d1d',TX='#eaeaea',MU='#8f8f8f',AM='#e0a03a',RD='#e05a4a'; function $(s){return R.querySelector(s)}function $$(s){return R.querySelectorAll(s)} function el(t,a,p){var e=document.createElementNS(NS,t);for(var k in a)e.setAttribute(k,a[k]);if(p)p.appendChild(e);return e} function tx(p,x,y,s,a){var o={x:x,y:y,fill:TX,'font-size':15,'text-anchor':'middle','dominant-baseline':'middle'};for(var k in a)o[k]=a[k];var t=el('text',o,p);t.textContent=s;return t} function clear(s){while(s.firstChild)s.removeChild(s.firstChild)} function boost(sv,f){if(R.offsetWidth>=560)return;sv.querySelectorAll('text').forEach(function(t){if(t.dataset.b)return;t.dataset.b=1;t.setAttribute('font-size',(+t.getAttribute('font-size')*f).toFixed(1))})} function post(){try{parent.postMessage({mtpTfasH:R.offsetHeight+40},'*')}catch(e){}} var active='p1'; $$('.tab').forEach(function(t){t.onclick=function(){$$('.tab').forEach(function(x){x.classList.remove('on')});$$('.pane').forEach(function(x){x.classList.remove('on')});t.classList.add('on');active=t.dataset.t;$('#'+active).classList.add('on');if(active==='p4')drawScatter(true);setTimeout(post,60)}}); function chips(id,attr,fn){$$('#'+id+' .chip').forEach(function(c){c.onclick=function(){$$('#'+id+' .chip').forEach(function(x){x.classList.remove('on')});c.classList.add('on');fn(c.dataset[attr])}})} function arrowDefs(s,id,col){var d=el('defs',{},s);var m=el('marker',{id:id,viewBox:'0 0 10 10',refX:9,refY:5,markerWidth:7,markerHeight:7,orient:'auto-start-reverse'},d);el('path',{d:'M0 0L10 5L0 10z',fill:col},m)} /* ---------- TAB 1: the loop ---------- */ var s1=$('#s1'),C={x:330,y:180,r:115}; var N={goal:[80,52],plan:[200,52],route:[330,65],retrieve:[445,180],evaluate:[330,295],recover:[215,180],validate:[545,295],partial:[110,295]}; var LB={goal:'Goal',plan:'Plan',route:'Route',retrieve:'Retrieve',evaluate:'Evaluate',recover:'Recover',validate:'Validate',partial:'Partial'}; var ANG={route:-90,retrieve:0,evaluate:90,recover:180}; var box={},packet,bud,budT,center1,center2,termTxt; function build1(){clear(s1);arrowDefs(s1,'a1',LN); el('circle',{cx:C.x,cy:C.y,r:C.r,fill:'none',stroke:'#3a3a3a','stroke-width':2.5,class:'ring'},s1); [['goal','plan'],['plan','route']].forEach(function(p){var a=N[p[0]],b=N[p[1]];el('line',{x1:a[0]+52,y1:a[1],x2:b[0]-54,y2:b[1]>a[1]?b[1]:a[1],stroke:LN,'stroke-width':2,'marker-end':'url(#a1)'},s1)}); el('line',{x1:382,y1:295,x2:491,y2:295,stroke:LN,'stroke-width':2,'marker-end':'url(#a1)'},s1); el('line',{x1:278,y1:295,x2:164,y2:295,stroke:LN,'stroke-width':2,'marker-end':'url(#a1)'},s1); for(var k in N){var g=el('g',{},s1),p=N[k];var r=el('rect',{x:p[0]-52,y:p[1]-19,width:104,height:38,rx:10,fill:BX,stroke:LN,'stroke-width':1.5},g); var t=tx(g,p[0],p[1]+1,LB[k],{'font-size':15,'font-weight':600});box[k]={r:r,t:t}} center1=tx(s1,C.x,C.y-10,'',{'font-size':22,'font-weight':800,fill:'#fff'}); center2=tx(s1,C.x,C.y+18,'',{'font-size':14,fill:MU}); tx(s1,430,24,'Task budget',{'font-size':12,fill:MU,'text-anchor':'start'}); el('rect',{x:430,y:34,width:180,height:10,rx:5,fill:'#222'},s1); bud=el('rect',{x:430,y:34,width:0,height:10,rx:5,fill:G},s1); budT=tx(s1,612,24,'0%',{'font-size':12,fill:TX,'text-anchor':'end'}); tx(s1,330,340,'time · tokens · tool calls · retries · cost',{'font-size':11,fill:'#5c5c5c'}); packet=el('circle',{r:8,fill:G,stroke:'#111','stroke-width':2},s1); el('circle',{r:16,fill:'none',stroke:G,'stroke-opacity':.35,'stroke-width':2,id:'pkh'},s1);boost(s1,1.3)} var SC={ clean:{steps:[['goal'],['plan'],['route'],['retrieve'],['evaluate','Sufficient',G],['validate','Completed',G]],cap:0.42,txt:'Evidence meets the contract on the first pass. <b>Stop and validate.</b>'}, weak:{steps:[['goal'],['plan'],['route'],['retrieve'],['evaluate','Incomplete',AM],['recover','Rewrite query',AM],['route'],['retrieve'],['evaluate','Sufficient',G],['validate','Completed',G]],cap:0.74,txt:'A gap sends the run back around the loop with a <b>changed</b> query.'}, budget:{steps:[['goal'],['plan'],['route'],['retrieve'],['evaluate','Incomplete',AM],['recover','Retry, bounded',AM],['route'],['retrieve'],['evaluate','Budget exhausted',RD],['partial','Partial + review',AM]],cap:1,txt:'At the boundary the agent <b>stops honestly</b>, not with a fluent guess.'}}; function seg(a,b){var pts=[],i; if(ANG[a]!==undefined&&ANG[b]!==undefined){var a0=ANG[a],a1=ANG[b];if(a1<=a0)a1+=360;for(i=0;i<=40;i++){var an=(a0+(a1-a0)*i/40)*Math.PI/180;pts.push([C.x+C.r*Math.cos(an),C.y+C.r*Math.sin(an)])}} else{var p=N[a],q=N[b];for(i=0;i<=30;i++)pts.push([p[0]+(q[0]-p[0])*i/30,p[1]+(q[1]-p[1])*i/30])} return pts} var run1=null; function play1(name){var sc=SC[name];$('#k1').innerHTML=sc.txt; var path=[],marks=[];for(var i=0;i0){var sg=seg(sc.steps[i-1][0],sc.steps[i][0]);sg.shift();path=path.concat(sg)}else path.push(N[sc.steps[0][0]].slice());marks.push(path.length-1)} var L=[0];for(i=1;i100)dt=16; if(wait>0){wait-=dt;if(wait<=0&&ending){reset();return}}else dist=Math.min(tot,dist+dt*speed); var d=dist; var nxt=done+1;if(nxt=L[marks[nxt]]){done=nxt;dist=d=L[marks[nxt]];wait=hold;if(done===marks.length-1){wait=1900;ending=true}var st=sc.steps[done],b=box[st[0]]; b.r.setAttribute('stroke',st[2]||G);b.r.setAttribute('fill',st[2]===RD?'#2a1412':st[2]===AM?'#2a2111':'#1b2410'); center1.textContent=LB[st[0]];center2.textContent=st[1]||'';center2.setAttribute('fill',st[2]||MU); if(st[1]&&(st[0]==='validate'||st[0]==='partial')){b.t.textContent=st[1];b.t.setAttribute('fill',st[2])} } var j=0;while(j<L.length-1&&L[j+1]L[j]?(d-L[j])/(L[j+1]-L[j]):0;f=Math.max(0,Math.min(1,f||0)); var p=j=.99?RD:fr>.6?AM:G);budT.textContent=Math.round(fr*100)+'%'}}} build1();chips('c1','s',play1);play1('weak'); /* ---------- TAB 2: routing ---------- */ var s2=$('#s2'),lanes=[],wire,qmark,needT,rw; var LA=[['Search','find candidate sources','srch'],['Fetch','clean page content','doc'],['Browser','your code drives it','win'],['Web Agent','goal in, JSON out (Mako)','bot']]; function icon(g,k,x,y){var st={fill:'none',stroke:'currentColor','stroke-width':2.2,'stroke-linecap':'round'}; if(k==='srch'){el('circle',Object.assign({cx:x-3,cy:y-3,r:8},st),g);el('line',Object.assign({x1:x+3,y1:y+3,x2:x+9,y2:y+9},st),g)} if(k==='doc'){el('rect',Object.assign({x:x-8,y:y-10,width:16,height:20,rx:2},st),g);[ -4,0,4].forEach(function(o){el('line',Object.assign({x1:x-4,y1:y+o,x2:x+4,y2:y+o},st),g)})} if(k==='win'){el('rect',Object.assign({x:x-11,y:y-9,width:22,height:18,rx:2},st),g);el('line',Object.assign({x1:x-11,y1:y-4,x2:x+11,y2:y-4},st),g)} if(k==='bot'){el('rect',Object.assign({x:x-9,y:y-6,width:18,height:14,rx:4},st),g);el('line',Object.assign({x1:x,y1:y-6,x2:x,y2:y-11},st),g);el('circle',{cx:x-4,cy:y+1,r:1.8,fill:'currentColor'},g);el('circle',{cx:x+4,cy:y+1,r:1.8,fill:'currentColor'},g)}} function build2(){clear(s2);arrowDefs(s2,'a2',G); tx(s2,110,40,'Evidence need',{'font-size':12,fill:MU}); el('circle',{cx:110,cy:150,r:52,fill:'#1b2410',stroke:G,'stroke-width':2},s2); qmark=tx(s2,110,150,'?',{'font-size':40,'font-weight':800,fill:G}); needT=tx(s2,110,222,'',{'font-size':13,fill:TX}); wire=el('path',{fill:'none',stroke:G,'stroke-width':2.5,class:'wire','marker-end':'url(#a2)'},s2); LA.forEach(function(l,i){var y=42+i*70,g=el('g',{style:'color:'+MU},s2); var r=el('rect',{x:380,y:y-24,width:240,height:52,rx:11,fill:BX,stroke:LN,'stroke-width':1.5},g);icon(g,l[2],410,y+2); var t=tx(g,436,y-6,l[0],{'font-size':17,'font-weight':700,'text-anchor':'start'});tx(g,436,y+13,l[1],{'font-size':12,fill:MU,'text-anchor':'start'}); lanes.push({g:g,r:r,t:t,y:y})}); rw=el('g',{opacity:0},s2);el('rect',{x:30,y:262,width:160,height:44,rx:10,fill:'#2a2111',stroke:AM,'stroke-width':1.5},rw); tx(rw,110,284,'Rewrite query',{'font-size':15,'font-weight':700,fill:AM});boost(s2,1.22)} var RT=[['no URL yet',0,'Discover before paying to inspect or operate pages.'],['have a URL',1,'Skip search. It cannot add discovery value here.'],['scripted clicks',2,'Deterministic navigation stays in your code.'],['many pages',3,'Keeps the goal and run state while the path changes.'],['bad evidence',-1,'<b>No escalation.</b> A heavier tool will not fix a weak query.']]; function route(i){i=+i;var r=RT[i];needT.textContent=r[0];$('#k2').innerHTML=r[2]; lanes.forEach(function(l,k){var on=k===r[1];l.r.setAttribute('stroke',on?G:LN);l.r.setAttribute('fill',on?'#1b2410':BX);l.g.setAttribute('style','color:'+(on?G:MU)+';opacity:'+(r[1]=0){var y=lanes[r[1]].y;wire.setAttribute('d','M164 150 C 270 150, 270 '+y+', 374 '+y);wire.setAttribute('opacity',1);rw.setAttribute('opacity',0);qmark.textContent='?'} else{wire.setAttribute('d','M110 204 L 110 256');rw.setAttribute('opacity',1);qmark.textContent='!'} wire.animate&&wire.animate([{opacity:0},{opacity:1}],{duration:400})} build2();chips('c2','r',route);route(0); /* ---------- TAB 3: deep research ---------- */ var s3=$('#s3'),dots=[],spawn=null,stage=0,last3=0; function tagPill(x,y,s,c){var w=s.length*7.4+26;el('rect',{x:x-w/2,y:y-14,width:w,height:28,rx:14,fill:c==G?'#1b2410':c==RD?'#2a1412':'#2a2111',stroke:c,'stroke-width':1.5},s3);tx(s3,x,y+1,s,{'font-size':13,'font-weight':700,fill:c})} function bx(x,y,w,h,label,col,fs){el('rect',{x:x,y:y,width:w,height:h,rx:11,fill:BX,stroke:col||LN,'stroke-width':1.6},s3);if(label)tx(s3,x+w/2,y+h/2,label,{'font-size':fs||15,'font-weight':700})} var weight=0; function v3(v){v=+v;stage=v;clear(s3);dots=[];arrowDefs(s3,'a3',LN); if(v===0){bx(30,50,240,200,'',LN);tx(s3,150,72,'Orchestrator',{'font-size':15,'font-weight':700}); var bars=[['structured state',G],['chat history','#5b8def'],['past reports','#5b8def'],['findings','#5b8def'],['user intent','#5b8def']]; bars.forEach(function(b,i){var y=96+i*30;el('rect',{x:50,y:y,width:200,height:22,rx:5,fill:b[1],'fill-opacity':.18,stroke:b[1],'stroke-opacity':.6},s3);tx(s3,62,y+11,b[0],{'font-size':12,'text-anchor':'start',fill:TX})}); el('line',{x1:340,y1:40,x2:340,y2:260,stroke:RD,'stroke-width':2,'stroke-dasharray':'6 6'},s3);tx(s3,340,30,'boundary',{'font-size':11,fill:RD}); bx(420,110,190,80,'',LN);tx(s3,515,140,'Planner',{'font-size':16,'font-weight':700});tx(s3,515,164,'fresh LLM call',{'font-size':12,fill:MU}); tagPill(515,236,'Context lost',RD); spawn=function(){var i=Math.floor(Math.random()*5),y=107+i*30;dots.push({x:250,y:y,tx:420,ty:150,c:i?'#5b8def':G,die:i?340:null,v:2.4})}; $('#k3').innerHTML='Only structured state crossed. Follow-ups like "focus only on pricing risks" drifted.'} if(v===1){bx(30,110,170,80,'Orchestrator',RD);var ys=[50,150,250]; ys.forEach(function(y,i){bx(430,y-26,180,52,'Subagent '+(i+1),LN,14);el('line',{x1:200,y1:150,x2:430,y2:y,stroke:LN,'stroke-width':1.5},s3)}); tagPill(115,240,'Latency ↑ Cost ↑',RD); spawn=function(){var y=ys[Math.floor(Math.random()*3)],out=Math.random()<.5;dots.push(out?{x:200,y:150,tx:430,ty:y,c:MU,v:4}:{x:430,y:y,tx:200,ty:150,c:AM,v:4})}; $('#k3').innerHTML='1 subagent per query. The orchestrator kept dispatching, waiting and merging.'} if(v===2){bx(30,100,190,100,'',G);tx(s3,125,135,'Orchestrator',{'font-size':16,'font-weight':700});tx(s3,125,160,'plan + decide next',{'font-size':12,fill:G}); ['Search','Fetch','Browse'].forEach(function(n,i){var y=60+i*60;bx(290,y-20,110,40,n,LN,14);el('line',{x1:220,y1:150,x2:290,y2:y,stroke:LN,'stroke-width':1.5},s3);el('line',{x1:400,y1:y,x2:460,y2:120,stroke:LN,'stroke-width':1.5},s3)}); bx(460,70,150,100,'',G);tx(s3,535,106,'Rank + extract',{'font-size':14,'font-weight':700});tx(s3,535,128,'plain code',{'font-size':12,fill:G}); el('path',{d:'M535 170 C 535 250, 125 260, 125 204','fill':'none',stroke:G,'stroke-width':1.5,'stroke-dasharray':'4 5','marker-end':'url(#a3)'},s3); tx(s3,330,262,'ranked sources + extracted claims',{'font-size':12,fill:G}); tagPill(330,20,'Context ↓ Latency ↓ Cost ↓',G); spawn=function(){var i=Math.floor(Math.random()*3),y=60+i*60;dots.push({x:400,y:y,tx:460,ty:120,c:MU,v:2.2,r:6,then:1})}; $('#k3').innerHTML='Deterministic steps became tool calls. The orchestrator only sees the distilled result.'} if(v===3){spawn=null; var w=el('g',{},s3),gl=[['Goal: product sentiment',0],['Goal: unemployment rate',1]]; gl.forEach(function(g,i){var x=i?330:30,on=weight===i,rg=el('g',{style:'cursor:pointer'},w);el('rect',{x:x,y:10,width:280,height:34,rx:17,fill:on?G:BX,stroke:on?G:LN},rg);tx(rg,x+140,28,g[0],{'font-size':13,'font-weight':700,fill:on?'#111':TX});rg.onclick=function(){weight=i;v3(3)}}); tx(s3,170,272,'Social posts',{'font-size':14,'font-weight':700});tx(s3,480,272,'Government report',{'font-size':14,'font-weight':700}); var soc=[[110,110],[160,90],[210,120],[130,170],[185,165],[235,180],[95,215],[150,225],[215,225]]; soc.forEach(function(p){var c=el('circle',{cx:p[0],cy:p[1],r:6,fill:weight===0?G:'#3a3a3a'},s3);c.animate&&c.animate([{r:6},{r:weight===0?20:6}],{duration:700,fill:'forwards',easing:'ease-out'});if(weight===0)c.setAttribute('r',20)}); var gv=el('circle',{cx:480,cy:160,r:weight===1?78:16,fill:weight===1?G:'#3a3a3a','fill-opacity':.9},s3);gv.animate&&gv.animate([{r:16},{r:weight===1?78:16}],{duration:700,easing:'ease-out'}); $('#k3').innerHTML='Same sources, different weight. <b>Rank by the research objective</b>, not one quality score.'} boost(s3,1.25);setTimeout(post,60)} function tick3(ts){if(spawn&&ts-last3>(stage===1?110:260)){spawn();last3=ts} for(var i=dots.length-1;i>=0;i--){var d=dots[i];if(!d.e){d.e=el('circle',{r:d.r||5,fill:d.c,cx:d.x,cy:d.y},s3)} var dx=d.tx-d.x,dy=d.ty-d.y,l=Math.hypot(dx,dy); if(d.die&&d.x>=d.die-2){d.e.setAttribute('fill',RD);d.fade=(d.fade||1)-.06;d.e.setAttribute('opacity',d.fade);if(d.fade<=0){d.e.remove();dots.splice(i,1)}continue} if(l<d.v){if(d.then){d.e.remove();dots.splice(i,1);dots.push({x:535,y:170,tx:125,ty:204,c:G,v:3.2,r:3.5,curve:1});continue}d.e.remove();dots.splice(i,1);continue} d.x+=dx/l*d.v;d.y+=dy/l*d.v;d.e.setAttribute('cx',d.x);d.e.setAttribute('cy',d.y)}} chips('c3','v',v3);v3(0); /* ---------- TAB 4: scatter ---------- */ var s4=$('#s4'),D={sf:[['TinyFish',0.050,79.0],['Exa deep',0.127,83.0],['Perplexity',0.083,77.7],['Firecrawl',0.082,76.0]], so:[['TinyFish',0.035,59.3],['Exa fast',0.103,66.3],['Perplexity',0.059,77.3],['Firecrawl',0.059,70.3]]}; var X0=80,X1=610,Y0=290,Y1=30,pts={},cur='sf'; function sx(v){return X0+(v/0.14)*(X1-X0)}function sy(v){return Y0-((v-50)/40)*(Y0-Y1)} function build4(){clear(s4); for(var c=0;c<=0.14001;c+=0.02){var x=sx(c);el('line',{x1:x,y1:Y1,x2:x,y2:Y0,stroke:'#222'},s4);tx(s4,x,Y0+18,'$'+c.toFixed(2),{'font-size':12,fill:MU})} for(var v=50;v=1)anim4=null}} build4();chips('c4','b',function(b){cur=b;drawScatter(false)});drawScatter(true); function loop(ts){if(active==='p1'&&run1)run1.tick(ts);if(active==='p3')tick3(ts);if(anim4)anim4(ts); if(pts.TinyFish)pts.TinyFish.ring.setAttribute('r',15+3*Math.sin(ts/220));requestAnimationFrame(loop)} requestAnimationFrame(loop); window.addEventListener('load',post);window.addEventListener('resize',post);setTimeout(post,300); })(); "> window.addEventListener("message",function(e){if(e.data&&e.data.mtpTfasH){var f=document.getElementById("mtp-tfas-frame");if(f)f.style.height=e.data.mtpTfasH+"px";}}); What did TinyFish learn building deep research? The deep research agent searches, browses, and generates comprehensive, cited reports. Most engineering challenges centered around context management, system orchestration, and product design. Lesson #1: Removing the separate planner Initial iterations used a standalone planner called fresh each time. Only structured state crossed into it. On follow-up queries like “focus only on pricing risks in Southeast Asia,” the agent drifted back to broad market research. TinyFish subsequently moved planning into the orchestrator’s own unified reasoning loop. Lesson #2: Removing subagents: TinyFish initially assigned a separate subagent to each search query, increasing context-switching overhead, latency, and cost. The team later replaced these subagents with tool calls for repeatable tasks such as ranking search results, classifying sources, and extracting evidence. This reduced the information passed back to the orchestrator, allowing it to focus on research planning and deciding what to investigate next. Lesson #3- Matching researchers: TinyFish initially optimized its research system for source coverage, but users placed greater importance on the quality and relevance of supporting evidence. The team adjusted source ranking to account for the research objective and added controls for prioritizing, excluding, or restricting domains. It also developed an internal evaluation harness using an LLM judge to assess reports for relevance, evidence strength, citation trust, and source diversity. How do cost and latency add up? TinyFish budgets the full task, not single calls: Task latency = planning + search + retrieval + evaluation + retries + interactive steps + queueing. Task cost = tokens + tool calls + page or browser usage + retries + infrastructure. SearchAgent-X (ICLR 2026) shows why. Exact retrieval added overhead, and coarse retrieval forced extra reasoning. Its fixes reached up to 3.4x throughput and 5x lower latency versus tested baselines. How does TinyFish compare on search and fetch? OpenBenchmarks fixes the model (gpt-5.6-sol), 100 tasks and budgets, and varies only the provider. Scores are means of 3 runs. Source: OpenBenchmarks board, updated 26 Sept 2026. Tasks cover enterprise documentation only. OpenBenchmarks also reports search-only results separately. In that configuration, TinyFish achieved 59.3% task completion, compared with 77.3% for Perplexity, 70.3% for Firecrawl, and 66.3% for Exa Fast. These results should not be directly compared with the search-and-fetch scores because the available retrieval capabilities differ. Key Takeaways Control the whole trajectory, not 1 search call. Route by information gap; escalate only when access fails. Keep planning and execution in 1 reasoning loop. Move deterministic steps into tool calls, not subagents. Rank sources by research objective, not one quality score. Read the architecture guide, the deep research lessons and the TinyFish docs. Thanks to the Tinyfish team for the thought leadership / resources for this article. This article is sponsored by TinyFish. The post TinyFish Breaks Down Agentic Search Architecture for Production AI Agents appeared first on MarkTechPost.
Google Cloud has announced Agent Compute Engine, a compute service built specifically for AI agents. Google Agent Compute Engine gives each agent its own hardware-isolated microVM. The sandbox holds zero credentials. The service already powers Gemini as an agent for work tasks. Google unveiled it this week at AGNTCon + MCPCon. Customer preview is targeted for the coming months. The pitch: VM-grade isolation with container-like speed and density. TL;DR Unit: One hardware-isolated microVM per agent session, with its own guest kernel, root access and Hyperdisk. Runs on: Google-managed hosts using Titan silicon and the Titanium IPU. Open-source sibling Agent Substrate runs on any Kubernetes cluster. Performance: Idle agents are checkpointed to zero active resources and resumed from snapshots. Google gives no ACE boot or density numbers. Best: Agent Substrate hit 500 activations/s at under 500 ms suspend/resume across 200,000 agents. Worst: No published boot time, density, pricing or preview date for Agent Compute Engine itself. What is Google Agent Compute Engine? Agent Compute Engine is a Google Cloud service that runs each AI agent inside a dedicated microVM. Per the technical post, it powers Gemini, Antigravity, and internal evaluation and RL pipelines. Google splits every agent into 2 parts. The harness, or ‘brain,’ calls the model. The sandbox, or ‘hands,’ runs code, shell commands and browsers. The harness runs fully outside the sandbox and dispatches explicit commands into it. How does the microVM isolation work? Each session runs in a single-tenant microVM behind a Cloud Hypervisor boundary. Hosts are stateless appliances with a read-only root filesystem enforced by dm-verity. Titanium holds attestation keys outside the guest. The networking design is the unusual part: MicroVMs have no standard network interface, no default gateway and no direct VPC access. All traffic crosses a virtio-vsock boundary to a local proxy, then to an external gateway. Guests get only a session-scoped SPIFFE certificate and a short-lived identity token. Agent Gateway swaps in the real backend credential in flight, after policy checks. Google assumes an attacker controls everything inside the sandbox. A prompt-injected agent finds no secrets to steal. What are the Four Agent Identities? Google defines 4 identities for every agent workload: User identity: authenticates the overall task only. Delegated on-behalf-of identity: a scoped grant stored in a central authorization service, never on the node. Agent Workload Identity: a short-lived SPIFFE identity, the only one used for sandbox traffic. Agent Workforce Identity: an agent’s own account and resources. The workload identity is useless for direct API calls if exfiltrated. It only identifies which sandbox sent a packet. How does it keep idle agents cheap? Google team found agents spend most of their time waiting on models or users. So the sandbox signals when a tool call finishes. The host checkpoints the idle microVM to zero active resources. It resumes on the next call. Density comes from copy-on-write memory sharing. Pre-warmed snapshot images are mapped with DAX, so microVMs on a host share base OS and library pages. Only each agent’s modified pages consume private memory. Harnesses also get cloning, snapshot/forking, rewind and rollback. Storage is split 3 ways: disposable scratch overlays, shared read-only volumes, and persistent user workspaces on Hyperdisk. <iframe id="mtp-ace-frame" title="Agent Compute Engine interactive explainer" loading="lazy" style="width:100%;height:600px;border:0;display:block;background:transparent;" srcdoc="<!doctype html> <title>Agent Compute Engine Explainer</title> #mtp-ace-explainer{--bg:#0e131b;--panel:#151c27;--line:#26303f;--tx:#e8eaed;--mut:#9aa4b2;--blue:#4285F4;--red:#EA4335;--yel:#FBBC04;--grn:#34A853;--mtp:#76B900; max-width:880px;margin:0 auto;background:var(--bg)!important;color:var(--tx)!important;border:1px solid var(--line)!important;border-radius:14px;font-family:Inter,"Google Sans",system-ui,-apple-system,Segoe UI,Roboto,sans-serif;overflow:hidden;box-sizing:border-box} html,body{margin:0;padding:0;background:transparent} #mtp-ace-explainer *{box-sizing:border-box} #mtp-ace-explainer hr,#mtp-ace-explainer p:empty,#mtp-ace-explainer del,#mtp-ace-explainer s{display:none!important} #mtp-ace-explainer p{margin:0} #mtp-ace-explainer .hd{padding:20px 22px 14px;border-bottom:1px solid var(--line)!important;background:linear-gradient(120deg,rgba(66,133,244,.16),rgba(52,168,83,.06) 60%,transparent)!important} #mtp-ace-explainer .kick{display:flex;gap:6px;align-items:center;font-size:11px;letter-spacing:.12em;text-transform:uppercase;color:var(--mut)!important;font-weight:600} #mtp-ace-explainer .dots i{display:inline-block;width:7px;height:7px;border-radius:50%;margin-right:3px} #mtp-ace-explainer h3{margin:8px 0 4px;font-size:21px;line-height:1.25;color:#fff!important;font-weight:700} #mtp-ace-explainer .sub{font-size:13.5px;color:var(--mut)!important;line-height:1.5} #mtp-ace-explainer .tabs{display:flex;gap:4px;padding:10px 12px 0;overflow-x:auto;border-bottom:1px solid var(--line)!important;scrollbar-width:none} #mtp-ace-explainer .tab{flex:0 0 auto;background:transparent!important;border:0!important;border-bottom:2px solid transparent!important;color:var(--mut)!important;padding:9px 12px 10px;font:600 13px/1 inherit;font-family:inherit;cursor:pointer;transition:color .2s} #mtp-ace-explainer .tab:hover{color:#fff!important} #mtp-ace-explainer .tab.on{color:#fff!important;border-bottom-color:var(--blue)!important} #mtp-ace-explainer .pane{display:none;padding:20px 22px 22px;animation:aceIn .35s ease} #mtp-ace-explainer .pane.on{display:block} @keyframes aceIn{from{opacity:0;transform:translateY(6px)}to{opacity:1;transform:none}} #mtp-ace-explainer .note{font-size:12px;color:var(--mut)!important;margin-top:12px;line-height:1.5} /* flow */ #mtp-ace-explainer .flow{position:relative;display:grid;grid-template-columns:repeat(6,1fr);gap:8px;margin:6px 0 16px} #mtp-ace-explainer .node{position:relative;background:var(--panel)!important;border:1px solid var(--line)!important;border-radius:10px;padding:12px 8px;text-align:center;font-size:12px;font-weight:600;line-height:1.3;color:var(--mut)!important;transition:all .35s;min-height:74px;display:flex;flex-direction:column;justify-content:center;gap:6px} #mtp-ace-explainer .node b{font-size:18px;line-height:1} #mtp-ace-explainer .node.done{color:var(--tx)!important;border-color:#3a4658!important} #mtp-ace-explainer .node.cur{color:#fff!important;border-color:var(--blue)!important;box-shadow:0 0 0 3px rgba(66,133,244,.22),0 8px 24px rgba(66,133,244,.18);transform:translateY(-2px)} #mtp-ace-explainer .node .zone{font-size:9.5px;letter-spacing:.08em;text-transform:uppercase;font-weight:700} #mtp-ace-explainer .z-in{color:var(--red)!important}#mtp-ace-explainer .z-out{color:var(--grn)!important} #mtp-ace-explainer .track{position:relative;height:6px;background:var(--line)!important;border-radius:6px;margin:0 0 16px;overflow:hidden} #mtp-ace-explainer .track span{position:absolute;left:0;top:0;bottom:0;background:linear-gradient(90deg,var(--blue),var(--grn))!important;border-radius:6px;transition:width .5s ease} #mtp-ace-explainer .pkt{position:absolute;top:-4px;width:14px;height:14px;border-radius:50%;background:var(--yel)!important;box-shadow:0 0 12px var(--yel);transition:left .5s ease} #mtp-ace-explainer .card{background:var(--panel)!important;border:1px solid var(--line)!important;border-radius:10px;padding:14px 16px} #mtp-ace-explainer .card h4{margin:0 0 6px;font-size:15px;color:#fff!important} #mtp-ace-explainer .card p{font-size:13.5px;line-height:1.55;color:#c9ced6!important} #mtp-ace-explainer .ctl{display:flex;gap:8px;margin-top:14px;flex-wrap:wrap} #mtp-ace-explainer .btn{background:var(--panel)!important;border:1px solid var(--line)!important;color:var(--tx)!important;border-radius:8px;padding:8px 14px;font:600 13px/1 inherit;font-family:inherit;cursor:pointer;transition:all .2s} #mtp-ace-explainer .btn:hover{border-color:var(--blue)!important} #mtp-ace-explainer .btn.pri{background:var(--blue)!important;border-color:var(--blue)!important;color:#fff!important} #mtp-ace-explainer .btn[disabled]{opacity:.4;cursor:default} /* lifecycle */ #mtp-ace-explainer .life{display:grid;grid-template-columns:repeat(4,1fr);gap:10px;margin-bottom:14px} #mtp-ace-explainer .st{background:var(--panel)!important;border:1px solid var(--line)!important;border-radius:10px;padding:12px;transition:all .4s} #mtp-ace-explainer .st .lb{font-size:10px;letter-spacing:.1em;text-transform:uppercase;color:var(--mut)!important;font-weight:700} #mtp-ace-explainer .st .nm{font-size:14px;font-weight:700;margin:4px 0 8px;color:#fff!important} #mtp-ace-explainer .meter{height:8px;background:var(--line)!important;border-radius:6px;overflow:hidden} #mtp-ace-explainer .meter span{display:block;height:100%;border-radius:6px;transition:width .6s ease} #mtp-ace-explainer .st .ds{font-size:12px;color:var(--mut)!important;margin-top:8px;line-height:1.4} #mtp-ace-explainer .st.on{border-color:var(--yel)!important;box-shadow:0 0 0 3px rgba(251,188,4,.18)} #mtp-ace-explainer .mem{display:grid;grid-template-columns:repeat(6,1fr);gap:8px;margin-top:6px} #mtp-ace-explainer .vm{background:var(--panel)!important;border:1px solid var(--line)!important;border-radius:8px;padding:6px;height:120px;display:flex;flex-direction:column-reverse;gap:3px} #mtp-ace-explainer .vm span{display:block;border-radius:4px;transition:all .6s ease} #mtp-ace-explainer .vm .base{background:var(--blue)!important} #mtp-ace-explainer .vm .priv{background:var(--yel)!important} #mtp-ace-explainer .vm small{font-size:10px;color:var(--mut)!important;text-align:center} #mtp-ace-explainer .legend{display:flex;gap:14px;font-size:12px;color:var(--mut)!important;margin:10px 0 0;flex-wrap:wrap} #mtp-ace-explainer .legend i{display:inline-block;width:10px;height:10px;border-radius:3px;margin-right:5px;vertical-align:-1px} #mtp-ace-explainer .shared{margin-top:8px;height:22px;border-radius:6px;background:var(--blue)!important;color:#fff!important;font-size:11px;font-weight:700;display:flex;align-items:center;justify-content:center;transition:opacity .6s;opacity:0} #mtp-ace-explainer .shared.on{opacity:1} /* identities */ #mtp-ace-explainer .ids{display:grid;grid-template-columns:repeat(2,1fr);gap:10px} #mtp-ace-explainer .id{text-align:left;cursor:pointer;background:var(--panel)!important;border:1px solid var(--line)!important;border-radius:10px;padding:14px;color:var(--tx)!important;font-family:inherit;transition:all .25s;position:relative} #mtp-ace-explainer .id:hover{border-color:#41506a!important;transform:translateY(-2px)} #mtp-ace-explainer .id .n{font-size:11px;font-weight:700;color:var(--mut)!important;letter-spacing:.08em} #mtp-ace-explainer .id .t{font-size:15px;font-weight:700;margin:4px 0;color:#fff!important} #mtp-ace-explainer .id .w{font-size:12px;font-weight:600} #mtp-ace-explainer .id .more{max-height:0;overflow:hidden;font-size:13px;line-height:1.5;color:#c9ced6!important;transition:max-height .4s ease,margin .4s} #mtp-ace-explainer .id.open .more{max-height:140px;margin-top:8px} #mtp-ace-explainer .id.key{border-color:var(--grn)!important} #mtp-ace-explainer .badge{position:absolute;top:12px;right:12px;font-size:10px;font-weight:700;background:rgba(52,168,83,.16)!important;color:var(--grn)!important;border-radius:20px;padding:3px 8px} /* numbers */ #mtp-ace-explainer .stats{display:grid;grid-template-columns:repeat(5,1fr);gap:10px;margin-bottom:16px} #mtp-ace-explainer .stat{background:var(--panel)!important;border:1px solid var(--line)!important;border-radius:10px;padding:14px 10px;text-align:center} #mtp-ace-explainer .stat .v{font-size:24px;font-weight:800;color:#fff!important;font-variant-numeric:tabular-nums} #mtp-ace-explainer .stat .k{font-size:11.5px;color:var(--mut)!important;margin-top:4px;line-height:1.35} #mtp-ace-explainer .bars{display:flex;flex-direction:column;gap:12px} #mtp-ace-explainer .bar .row{display:flex;justify-content:space-between;font-size:13px;margin-bottom:5px} #mtp-ace-explainer .bar .row b{color:#fff!important} #mtp-ace-explainer .bar .tr{height:12px;background:var(--line)!important;border-radius:8px;overflow:hidden} #mtp-ace-explainer .bar .tr span{display:block;height:100%;width:0;border-radius:8px;transition:width 1.1s cubic-bezier(.2,.8,.2,1)} /* compare */ #mtp-ace-explainer .tw{overflow-x:auto;border:1px solid var(--line)!important;border-radius:10px} #mtp-ace-explainer table{width:100%;border-collapse:collapse;font-size:12.5px;min-width:640px;background:transparent!important} #mtp-ace-explainer th,#mtp-ace-explainer td{padding:9px 10px;border:0!important;border-bottom:1px solid var(--line)!important;text-align:left;vertical-align:top;color:#c9ced6!important;background:transparent!important} #mtp-ace-explainer th{color:#fff!important;font-weight:700;background:#121925!important} #mtp-ace-explainer td:first-child{color:var(--mut)!important;font-weight:600;white-space:nowrap} #mtp-ace-explainer td.g{color:#fff!important;background:rgba(66,133,244,.08)!important} #mtp-ace-explainer tr:last-child td{border-bottom:0!important} #mtp-ace-explainer .ft{display:flex;justify-content:space-between;gap:10px;flex-wrap:wrap;padding:12px 22px;border-top:1px solid var(--line)!important;font-size:11.5px;color:var(--mut)!important} #mtp-ace-explainer .ft a{color:var(--mut)!important;text-decoration:underline} #mtp-ace-explainer .ft .mtp{color:var(--mtp)!important;font-weight:700} @media (max-width:640px){ #mtp-ace-explainer .hd{padding:16px} #mtp-ace-explainer h3{font-size:18px} #mtp-ace-explainer .pane{padding:16px} #mtp-ace-explainer .flow{grid-template-columns:repeat(3,1fr)} #mtp-ace-explainer .life{grid-template-columns:repeat(2,1fr)} #mtp-ace-explainer .mem{grid-template-columns:repeat(3,1fr)} #mtp-ace-explainer .ids{grid-template-columns:1fr} #mtp-ace-explainer .stats{grid-template-columns:repeat(2,1fr)} #mtp-ace-explainer .ft{padding:12px 16px} } @media (prefers-reduced-motion:reduce){#mtp-ace-explainer *{transition:none!important;animation:none!important}} <div id="mtp-ace-explainer"> <div class="hd"> <div class="kick"><span class="dots"><i style="background:#4285F4"></i><i style="background:#EA4335"></i><i style="background:#FBBC04"></i><i style="background:#34A853"></i></span>Google Cloud · Interactive explainer</div> <h3>Agent Compute Engine: 1 microVM per agent, 0 credentials inside</h3> <p class="sub">Step through how a sandboxed agent reaches your data, sleeps for free, and proves who it is.</p> </div> <div class="tabs" role="tablist"> <button class="tab on" data-t="0">1. Request path</button> <button class="tab" data-t="1">2. Idle economics</button> <button class="tab" data-t="2">3. 4 identities</button> <button class="tab" data-t="3">4. Agent Substrate numbers</button> <button class="tab" data-t="4">5. Compare</button> </div> <div class="pane on" data-p="0"> <div class="flow" id="ace-flow"> <div class="node"><b></b>Harness<span class="zone z-out">outside</span></div> <div class="node"><b></b>Sandbox microVM<span class="zone z-in">untrusted</span></div> <div class="node"><b></b>vsock + local proxy<span class="zone z-out">host</span></div> <div class="node"><b></b>Agent Gateway<span class="zone z-out">off-node</span></div> <div class="node"><b></b>Token swap<span class="zone z-out">off-node</span></div> <div class="node"><b></b>Backend<span class="zone z-out">Workspace / VPC</span></div> </div> <div class="track"><span id="ace-fill"></span><i class="pkt" id="ace-pkt"></i></div> <div class="card"><h4 id="ace-st">Step 1</h4><p id="ace-sd"></p></div> <div class="ctl"> <button class="btn" id="ace-prev"> Back</button> <button class="btn pri" id="ace-next">Next step </button> <button class="btn" id="ace-play"> Auto-play</button> </div> <p class="note">Source: Google Cloud technical deep dive, Oct 22, 2026.</p> </div> <div class="pane" data-p="1"> <div class="life" id="ace-life"> <div class="st"><div class="lb">State 1</div><div class="nm">Running tool call</div><div class="meter"><span style="width:100%;background:#34A853"></span></div><div class="ds">Code, shell or browser executes inside the microVM.</div></div> <div class="st"><div class="lb">State 2</div><div class="nm">"Thinking" signal</div><div class="meter"><span style="width:55%;background:#FBBC04"></span></div><div class="ds">Sandbox reports no commands or background tasks in flight.</div></div> <div class="st"><div class="lb">State 3</div><div class="nm">Checkpointed idle</div><div class="meter"><span style="width:0%;background:#4285F4"></span></div><div class="ds">State preserved. Zero active host resources.</div></div> <div class="st"><div class="lb">State 4</div><div class="nm">Resume</div><div class="meter"><span style="width:100%;background:#34A853"></span></div><div class="ds">Next tool call restores the microVM from its snapshot.</div></div> </div> <div class="card"> <h4>Copy-on-write memory, shared across 6 microVMs</h4> <div class="mem" id="ace-mem"></div> <div class="shared" id="ace-shared">1 shared copy of base OS, libraries and binaries (DAX + CoW)</div> <div class="legend"><span><i style="background:#4285F4"></i>Base image pages</span><span><i style="background:#FBBC04"></i>Private pages the agent modified</span></div> <div class="ctl"><button class="btn pri" id="ace-cow">Turn on CoW sharing</button><button class="btn" id="ace-cycle"> Run idle cycle</button></div> </div> <p class="note">Bar heights are illustrative. Google discloses no per-host density figure for Agent Compute Engine.</p> </div> <div class="pane" data-p="2"> <div class="ids"> <button class="id"><div class="n">ID 1</div><div class="t">User identity</div><div class="w" style="color:#4285F4">Starts the task</div><div class="more">The authenticated user who initiated the request. Used only to authenticate the overall task, never for agentic operations.</div></button> <button class="id"><div class="n">ID 2</div><div class="t">Delegated on-behalf-of</div><div class="w" style="color:#FBBC04">Lives in central auth service</div><div class="more">A cryptographically scoped grant proving the user authorized the harness. These long-lived grants never sit on the compute node.</div></button> <button class="id key"><span class="badge">Only ID in sandbox</span><div class="n">ID 3</div><div class="t">Agent Workload Identity</div><div class="w" style="color:#34A853">Short-lived SPIFFE ID</div><div class="more">Assigned when the microVM spins up. Identifies which sandbox sent a packet. Useless for direct backend calls if exfiltrated.</div></button> <button class="id"><div class="n">ID 4</div><div class="t">Agent Workforce Identity</div><div class="w" style="color:#EA4335">Agent's own account</div><div class="more">Lets an agent act as an assistant with its own identity, account and resources.</div></button> </div> <p class="note">Tap a card to reveal it. Agent Gateway translates ID 3 into the other identities off-node, after verifying host, user, sandbox, session and policy.</p> </div> <div class="pane" data-p="3"> <div class="stats"> <div class="stat"><div class="v" data-n="200000" data-f=",">0</div><div class="k">agents in benchmark</div></div> <div class="stat"><div class="v" data-n="500">0</div><div class="k">activations per second</div></div> <div class="stat"><div class="v" data-n="500" data-p="<" data-s=" ms">0</div><div class="k">suspend / resume latency</div></div> <div class="stat"><div class="v" data-n="10" data-s="x">0</div><div class="k">higher compute density (up to)</div></div> <div class="stat"><div class="v" data-n="1000" data-f="," data-s="+">0</div><div class="k">dormant agents per host</div></div> </div> <div class="bars"> <div class="bar"><div class="row"><span>Published resume latency: Agent Substrate</span><b>< 500 ms</b></div><div class="tr"><span data-w="50" style="background:#4285F4"></span></div></div> <div class="bar"><div class="row"><span>Published resume time: E2B (docs)</span><b>~1 s</b></div><div class="tr"><span data-w="100" style="background:#9aa4b2"></span></div></div> <div class="bar"><div class="row"><span>Agent Compute Engine</span><b>Not disclosed</b></div><div class="tr"><span data-w="0" style="background:#EA4335"></span></div></div> </div> <p class="note">These figures are for open-source Agent Substrate on GKE, not Agent Compute Engine. Self-reported by Google. Lower latency is better.</p> </div> <div class="pane" data-p="4"> <div class="tw"><table> <tr><th></th><th>Agent Compute Engine</th><th>AWS AgentCore Runtime</th><th>Azure dynamic sessions</th><th>E2B</th></tr> <tr><td>Isolation</td><td class="g">Cloud Hypervisor microVM</td><td>Dedicated microVM per session</td><td>Hyper-V sandbox</td><td>Firecracker microVM</td></tr> <tr><td>Idle</td><td class="g">Checkpoint to zero active resources</td><td>CPU billing typically skips I/O wait</td><td>Destroyed after cooldown</td><td>Pause saves files and memory</td></tr> <tr><td>Credentials</td><td class="g">None in sandbox</td><td>Managed by AgentCore Identity</td><td>Not disclosed</td><td>Not disclosed</td></tr> <tr><td>Max session</td><td class="g">Not disclosed</td><td>8 h microVM, 14 days Instances</td><td>Configurable cooldown</td><td>Not compared</td></tr> <tr><td>Status</td><td class="g">Preview in coming months</td><td>Available</td><td>38 listed regions</td><td>Available</td></tr> </table></div> <p class="note">Sources: Google Cloud blog; AWS, Microsoft Learn and E2B docs. Checked Oct 22, 2026.</p> </div> <div class="ft"><span>Sources: <a href=" target=" rel="noopener">Google Cloud deep dive</a> · <a href=" target=" rel="noopener">Agent Substrate</a></span><span class="mtp">Built by Marktechpost</span></div> </div> (function(){ var R=document.getElementById('mtp-ace-explainer'); function post(){try{parent.postMessage({mtpAceHeight:R.offsetHeight+40},'*')}catch(e){}} var tabs=R.querySelectorAll('.tab'),panes=R.querySelectorAll('.pane'); tabs.forEach(function(t){t.addEventListener('click',function(){var i=+t.dataset.t;tabs.forEach(function(x){x.classList.toggle('on',x===t)});panes.forEach(function(p){p.classList.toggle('on',+p.dataset.p===i)});if(i===3)runStats();setTimeout(post,60);setTimeout(post,500)})}); /* flow */ var S=[["Harness dispatches a command","The harness (the brain) runs outside the sandbox. It decides the next tool call and sends an explicit command into the microVM."], ["Sandbox runs untrusted code","The microVM (the hands) executes code, shell or browser actions. It holds no tokens, no API keys and has no metadata server access."], ["Traffic exits over vsock","There is no standard network card. Outbound requests cross a virtio-vsock boundary to a local proxy, which binds Titanium-backed node attestation."], ["Agent Gateway checks the triad","Off-node, Agent Gateway validates user delegation, agent identity and node identity, then applies egress and data loss prevention policy."], ["In-flight token swap","Only now is the real backend credential inserted on the wire, as the request leaves the gateway. The sandbox never sees it."], ["Backend sees a clean identity","Workspace, a private VPC service or an MCP server receives a verifiable identity, auditable to a specific agent acting for a specific user."]]; var nodes=R.querySelectorAll('#ace-flow .node'),fill=R.querySelector('#ace-fill'),pkt=R.querySelector('#ace-pkt'),st=R.querySelector('#ace-st'),sd=R.querySelector('#ace-sd'),prev=R.querySelector('#ace-prev'),next=R.querySelector('#ace-next'),play=R.querySelector('#ace-play'),k=0,timer=null; function draw(){nodes.forEach(function(n,i){n.classList.toggle('cur',i===k);n.classList.toggle('done',i0){k--;draw()}}; play.onclick=function(){if(timer){clearInterval(timer);timer=null;play.textContent=' Auto-play';return}play.textContent='❚❚ Pause';k=0;draw();timer=setInterval(function(){if(k<S.length-1){k++;draw()}else{clearInterval(timer);timer=null;play.textContent=' Auto-play'}},1800)}; draw(); /* memory */ var mem=R.querySelector('#ace-mem'),cow=false,priv=[14,22,10,18,26,12]; for(var i=0;i<6;i++){var v=document.createElement('div');v.className='vm';v.innerHTML='<small>VM '+(i+1)+'</small><span class="base" style="height:70px"></span><span class="priv" style="height:'+priv[i]+'px"></span>';mem.appendChild(v)} var cowb=R.querySelector('#ace-cow'),sh=R.querySelector('#ace-shared'); cowb.onclick=function(){cow=!cow;mem.querySelectorAll('.base').forEach(function(b){b.style.height=cow?'0px':'70px';b.style.opacity=cow?0:1});sh.classList.toggle('on',cow);cowb.textContent=cow?'Turn off CoW sharing':'Turn on CoW sharing'}; var lifes=R.querySelectorAll('#ace-life .st'),lt=null; R.querySelector('#ace-cycle').onclick=function(){if(lt)clearInterval(lt);var j=0;lifes.forEach(function(x){x.classList.remove('on')});lifes[0].classList.add('on');lt=setInterval(function(){lifes[j].classList.remove('on');j++;if(j>=lifes.length){clearInterval(lt);lt=null;return}lifes[j].classList.add('on')},1300)}; /* ids */ R.querySelectorAll('.id').forEach(function(c){c.addEventListener('click',function(){c.classList.toggle('open');setTimeout(post,450)})}); /* stats */ var ran=false; function runStats(){if(ran)return;ran=true;R.querySelectorAll('.stat .v').forEach(function(el){var n=+el.dataset.n,p=el.dataset.p||'',s=el.dataset.s||'',f=el.dataset.f,t0=null;function step(ts){if(!t0)t0=ts;var q=Math.min(1,(ts-t0)/1200),e=1-Math.pow(1-q,3),v=Math.round(n*e);el.textContent=p+(f?v.toLocaleString('en-US'):v)+s;if(q<1)requestAnimationFrame(step)}requestAnimationFrame(step)});setTimeout(function(){R.querySelectorAll('.bar .tr span').forEach(function(b){b.style.width=b.dataset.w+'%'})},80)} window.addEventListener('load',post);window.addEventListener('resize',post);setTimeout(post,300); })(); "> (function(){var f=document.getElementById('mtp-ace-frame');window.addEventListener('message',function(e){if(!f||e.source!==f.contentWindow)return;var d=e.data;if(d&&typeof d.mtpAceHeight==='number'&&d.mtpAceHeight>200){f.style.height=Math.ceil(d.mtpAceHeight)+'px';}});})(); What does Agent Substrate add? Agent Substrate is the open-source, Kubernetes-native counterpart, available on GKE since September 15, 2026. It offers Cloud Hypervisor microVMs or gVisor sandboxes. Google reports these figures: Up to 10x higher compute density than traditional compute. 500 activations per second at under 500 ms suspend/resume, in a 200,000-agent benchmark. Over 1,000 dormant agents per host. It works with harnesses including Claude Code, OpenClaw and Hermes. It is open to all GKE customers for non-production use. Production GA is allowlist-only. Source code is at How does it compare with other agent sandboxes? How is Google expanding agent governance? Google is extending Gemini for business agent governance to Agent Runtime, Cloud Run, Agent Sandboxes, GKE and ACE. The stack includes Agent Gateway, Agent Registry, Agent Identity and OpenTelemetry-based observability. New pieces arrive in stages: Preview soon: Envoy-based enforcement points for multicloud and hybrid setups. Preview soon: governance for local coding agents like Claude Code and Antigravity CLI on managed devices. Preview this quarter: token management, model routing, Service Extensions bindings and Express Link for Cloud Run. Preview this quarter: delegated authorization with Ping Identity, SailPoint and Saviynt. Key Takeaways 1 hardware-isolated microVM per agent, with zero credentials inside. Idle agents checkpoint to zero active resources, then resume from snapshots. 4 separate identities; only a short-lived SPIFFE ID touches the sandbox. Agent Substrate: 500 activations/s, under 500 ms resume, 200,000 agents. No public pricing, boot time or preview date for Agent Compute Engine yet. Check out the launch post, the technical deep dive, and the Agent Substrate announcement. Register interest for the preview here.. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Google Cloud Launches Agent Compute Engine: A MicroVM per AI Agent appeared first on MarkTechPost.
Customers are letting AI agents use their credentials to access accounts at Morgan Stanley, JPMorgan, and other major U.S. banks, sometimes without direct instructions. Morgan Stanley has blocked agents, while other banks are restricting access. Most plan to limit agents to read-only access. The article Major US banks are locking out AI agents that log in with customer credentials appeared first on The Decoder.
With Agent Compute Engine, Google is introducing a runtime environment designed specifically for AI agents, while also expanding the governance capabilities that enable companies to manage agents in cloud, hybrid, and on-premises environments. The article Google's Agent Compute Engine gives every Gemini agent its own hardware-isolated micro virtual machine appeared first on The Decoder.
Liquid AI just announced Liquid RIPL, short for Real-time Inference Pipeline. RIPL is a low-latency engine that serves Liquid Foundation Models (LFMs) inside a live request path. It returns complete generated responses end-to-end in under 50 ms. It also claims that latency holds while traffic arrives continuously at peak load. For developers, the message is simple. Swap gradient-boosted trees for a generative model, and keep the latency budget. TL;DR Engine: Serves LFM2.5 models from 230M to 2.6B parameters Runs on: Enterprise deployments on NVIDIA H100, H200, B200 and RTX PRO 6000 GPUs. Benchmarks used a single B200. Performance: 5.6 to 28.4x lower p95 latency than Qwen3.5 on vLLM, per Liquid’s own tests. Best: LFM2.5-1.2B hit 155.8 ms p95 vs 4,431 ms for Qwen3.5-0.8B, a 28.4x gap (256 in, 128 out). Worst: LFM2.5-2.6B gained only 5.6x (449.1 ms vs 2,499 ms). Sub-50 ms fails on longer requests. What problem does Liquid RIPL solve? Most serving stacks target large batches and long agentic jobs. A few seconds per request is acceptable there. Search ranking, fraud checks and payment authorization need answers in milliseconds. Liquid AI argues teams default to classical models for these paths because LLM serving is too slow. RIPL is built to remove that trade-off. How does RIPL work? Liquid AI describes RIPL as optimizing the full path, from request arrival to completed response. In several workloads, Liquid AI plots the same model, LFM2.5-350M, on vLLM and on RIPL. RIPL’s p95 line sits well below the vLLM line in each case. That isolates part of the gain to the engine, not just the model architecture. The LFM2.5 models themselves are hybrids. LFM2.5-1.2B-Instruct has 16 layers: 10 double-gated convolution blocks and 6 GQA blocks. LFM2.5-2.6B has 30 layers: 22 convolution blocks and 8 GQA blocks. RIPL benchmarks: how fast is it vs vLLM? Liquid measured p95 end-to-end latency under open-loop Poisson arrivals on one B200. LFMs ran on RIPL. Baselines, including MiniCPM5 and Granite 4.2, ran on vLLM. Source: Liquid AI blog, Figure 2. Sealf-reported. Liquid AI also mapped 6 real-time workloads against fixed budgets: Adtech query rewriting: 32 in, 32 out, 50 ms budget, LFM2.5-350M. Search intent classification: 128 in, 1 out, 20 ms budget. Search reranking: 1k in, 1 out, 30 ms budget. Payments risk scoring: 512 in, 16 out, 50 ms budget, LFM2.5-1.2B. Credit decisioning: 8k in, 64 out, 150 ms budget. Image alt text: 512×512 image and 64 tokens in, 16 out, 100 ms budget, LFM2.5-VL-450M. window.addEventListener("message",function(e){if(e.data&&e.data.mtpH){var f=document.getElementById("mtp-ripl-frame");if(f&&e.source===f.contentWindow){f.style.height=e.data.mtpH+"px";}}}); <!-- Fallback: upload liquid-ripl-explainer-standalone.html to /wp-content/uploads/ and use --> Competitor Comparison Is RIPL used in production? Yes, according to Liquid AI. Shopify is deploying LFMs across latency-sensitive commerce workloads. In search enhancement, LFMs run with sub-20 ms inference on production traffic. According to Liquid AI’s blog, they use about 50% fewer parameters than alternatives and deliver 2 to 10x faster inference. Key Takeaways RIPL is Liquid AI’s engine for LLMs inside live request paths. Liquid claims 5.6 to 28.4x lower p95 latency than Qwen3.5 on vLLM. Same-model tests (LFM2.5-350M) show RIPL itself beats vLLM. Shopify runs LFMs at sub-20 ms in production search. Enterprise-only, closed, and all figures are self-reported. Check out the RIPL announcement, try the latency calculator, and see the LFM2.5 model cards.. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Liquid AI Launches RIPL, a Real-Time Inference Engine for Sub-50 ms LFMs appeared first on MarkTechPost.
Gradium has open-sourced Phonon, a family of small text-to-speech models built to run on the device. The Gradium Phonon on-device TTS model comes in 31M and 40M sizes. The English Phonon 31m ships under Apache 2.0 and fits in a 64.5 MB file. Developers get natural speech with no cloud call, no GPU and no per-character bill. TL;DR Size: 31M parameters (excluding the audio tokenizer), 64.5 MB q8 GGUF, 24 kHz mono output. Phonon 40m: 40M, 74.6 MB. Runs on: CPUs, phones, Raspberry Pi 5, browsers (WASM, WebGPU) and NVIDIA GPUs. No GPU required. Performance: top English Elo among open models Gradium tested; about 7 ms to first audio on CPU. Best: 1.70% WER on a 500-sentence hard set, versus 3.50% for Kokoro (Gradium’s eval). Worst: 0.51% Seed-TTS LLM WER, behind Kokoro (0.45%) and Magpie (0.46%). What is Gradium Phonon? Phonon is an open-weight TTS family from Gradium, a Paris voice AI company spun out of Kyutai. It generates speech locally on phones, laptops, boards and browsers. The family has three models: Phonon 31m: English only, 37 built-in voices, Apache 2.0. Phonon 40m: 1 model per language (English, French, German, Spanish, Portuguese), CC BY-NC 4.0. Phonon 40m multilingual: all 5 languages in 1 model, 102 voices, CC BY-NC 4.0. How does Phonon work? Phonon uses a causal transformer backbone. Its prefix combines text and speaker embeddings. A SentencePiece tokenizer with a 4,000-token vocabulary reads raw text, with no phoneme conversion. The backbone generates 1 latent vector per 80 ms frame. A flow-matching head turns that vector plus noise into an audio embedding in a single step. A smaller variant of Kyutai’s Mimi decoder then outputs 24 kHz PCM audio. A separate end-of-speech head decides when to stop. Phonon 31m uses 6 transformer layers, width 512 and 8 attention heads. Phonon 40m uses 9 layers at the same width. Both are distilled from a larger Phonon teacher. Training ran on an NVIDIA H100 cluster with NCCL and CUDA Graphs. The work builds on Kyutai’s Continuous Audio Language Models research and Pocket TTS. <iframe id="mtp-phonon-frame" title="Gradium Phonon interactive explainer" srcdoc="<!doctype html> <title>Phonon Explainer</title> #mtp-phonon{--bg:#0d0d0b;--card:#17160f;--line:#2c2a22;--cream:#f2ede3;--mute:#a39e90;--acc:#ff7a3d;--acc2:#ffd2b8;--mtp:#76B900; font-family:Inter,system-ui,-apple-system,Segoe UI,Roboto,sans-serif!important;background:var(--bg)!important;color:var(--cream)!important; max-width:880px;margin:0 auto;border:1px solid var(--line)!important;border-radius:16px;overflow:hidden;position:relative;box-sizing:border-box} #mtp-phonon *{box-sizing:border-box} #mtp-phonon hr,#mtp-phonon p:empty,#mtp-phonon del,#mtp-phonon s{display:none!important} #mtp-phonon .dots{position:absolute;inset:0;background-image:radial-gradient(rgba(242,237,227,.07) 1px,transparent 1.4px);background-size:14px 14px;pointer-events:none} #mtp-phonon .hd{position:relative;padding:22px 24px 14px;border-bottom:1px solid var(--line)!important} #mtp-phonon .kick{font-size:11px;letter-spacing:.14em;text-transform:uppercase;color:var(--acc)!important;font-weight:700} #mtp-phonon h3{margin:6px 0 4px!important;font-size:24px!important;line-height:1.2!important;color:var(--cream)!important;font-weight:800} #mtp-phonon .sub{color:var(--mute)!important;font-size:14px;margin:0} #mtp-phonon .tabs{position:relative;display:flex;gap:6px;padding:12px 24px 0;flex-wrap:wrap} #mtp-phonon .tab{background:transparent!important;border:1px solid var(--line)!important;color:var(--mute)!important;padding:8px 14px;border-radius:999px;font:600 13px Inter,system-ui,sans-serif;cursor:pointer;transition:all .25s} #mtp-phonon .tab:hover{color:var(--cream)!important;border-color:var(--mute)!important} #mtp-phonon .tab.on{background:var(--cream)!important;color:#111!important;border-color:var(--cream)!important} #mtp-phonon .panel{display:none;position:relative;padding:20px 24px 22px;animation:mtpfade .45s ease} #mtp-phonon .panel.on{display:block} @keyframes mtpfade{from{opacity:0;transform:translateY(8px)}to{opacity:1;transform:none}} #mtp-phonon .grid3{display:grid;grid-template-columns:repeat(3,1fr);gap:12px} #mtp-phonon .mc{background:var(--card)!important;border:1px solid var(--line)!important;border-radius:12px;padding:14px;cursor:pointer;transition:transform .25s,border-color .25s} #mtp-phonon .mc:hover,#mtp-phonon .mc.on{transform:translateY(-3px);border-color:var(--acc)!important} #mtp-phonon .mc b{display:block;font-size:16px;color:var(--cream)!important} #mtp-phonon .mc .big{font-size:30px;font-weight:800;color:var(--acc)!important;margin:6px 0 2px} #mtp-phonon .mc small{color:var(--mute)!important;font-size:12px} #mtp-phonon .lic{display:inline-block;margin-top:8px;font-size:11px;font-weight:700;padding:3px 8px;border-radius:6px;border:1px solid var(--line)!important} #mtp-phonon .lic.ok{color:var(--mtp)!important;border-color:var(--mtp)!important} #mtp-phonon .lic.nc{color:var(--acc2)!important} #mtp-phonon .detail{margin-top:12px;background:var(--card)!important;border:1px dashed var(--line)!important;border-radius:12px;padding:12px 14px;font-size:14px;color:var(--cream)!important;min-height:48px} #mtp-phonon .pipe{display:flex;align-items:stretch;gap:6px;margin:4px 0 14px} #mtp-phonon .st{flex:1;background:var(--card)!important;border:1px solid var(--line)!important;border-radius:10px;padding:10px 8px;text-align:center;font-size:12px;color:var(--mute)!important;transition:all .35s;position:relative} #mtp-phonon .st .n{display:block;font-size:10px;letter-spacing:.1em;color:var(--mute)!important} #mtp-phonon .st b{display:block;font-size:13px;color:var(--cream)!important;margin-top:2px} #mtp-phonon .st.on{border-color:var(--acc)!important;background:#231a12!important;transform:translateY(-3px);box-shadow:0 0 0 3px rgba(255,122,61,.15)} #mtp-phonon .arr{align-self:center;color:var(--line)!important;font-size:14px} #mtp-phonon .frames{display:flex;gap:4px;height:46px;align-items:flex-end;margin:6px 0 12px;padding:6px;background:var(--card)!important;border:1px solid var(--line)!important;border-radius:10px;overflow:hidden} #mtp-phonon .fr{flex:1;background:var(--acc)!important;border-radius:2px;opacity:.25;height:20%;transition:height .3s,opacity .3s} #mtp-phonon .fr.live{opacity:1} #mtp-phonon .ctl{display:flex;gap:8px;align-items:center;flex-wrap:wrap} #mtp-phonon .btn{background:var(--acc)!important;color:#111!important;border:0!important;padding:8px 14px;border-radius:8px;font:700 13px Inter,system-ui,sans-serif;cursor:pointer} #mtp-phonon .btn.ghost{background:transparent!important;color:var(--cream)!important;border:1px solid var(--line)!important} #mtp-phonon .cap{color:var(--mute)!important;font-size:12px} #mtp-phonon .seg{display:inline-flex;border:1px solid var(--line)!important;border-radius:8px;overflow:hidden;margin-bottom:12px} #mtp-phonon .seg button{background:transparent!important;border:0!important;color:var(--mute)!important;padding:7px 12px;font:600 12px Inter,system-ui,sans-serif;cursor:pointer} #mtp-phonon .seg button.on{background:var(--cream)!important;color:#111!important} #mtp-phonon .bar{display:grid;grid-template-columns:150px 1fr 64px;align-items:center;gap:10px;margin:7px 0;font-size:13px} #mtp-phonon .bar .lb{color:var(--cream)!important} #mtp-phonon .bar .tr{height:14px;background:var(--card)!important;border-radius:4px;overflow:hidden;border:1px solid var(--line)!important} #mtp-phonon .bar .fl{height:100%;width:0;background:#5b574c!important;transition:width 1s cubic-bezier(.2,.8,.2,1)} #mtp-phonon .bar.me .fl{background:var(--acc)!important} #mtp-phonon .bar .v{text-align:right;font-weight:700;color:var(--cream)!important;font-variant-numeric:tabular-nums} #mtp-phonon .note{font-size:12px;color:var(--mute)!important;margin-top:10px;line-height:1.5} #mtp-phonon .hw{display:grid;grid-template-columns:repeat(3,1fr);gap:10px} #mtp-phonon .hwc{background:var(--card)!important;border:1px solid var(--line)!important;border-radius:12px;padding:12px;cursor:pointer;transition:border-color .25s} #mtp-phonon .hwc.on{border-color:var(--acc)!important} #mtp-phonon .hwc b{font-size:13px;color:var(--cream)!important;display:block} #mtp-phonon .hwc small{color:var(--mute)!important;font-size:11px} #mtp-phonon .gauge{margin-top:14px;display:grid;grid-template-columns:1fr 1fr;gap:12px} #mtp-phonon .gc{background:var(--card)!important;border:1px solid var(--line)!important;border-radius:12px;padding:14px} #mtp-phonon .gc .k{font-size:11px;letter-spacing:.1em;text-transform:uppercase;color:var(--mute)!important} #mtp-phonon .gc .val{font-size:34px;font-weight:800;color:var(--acc)!important;font-variant-numeric:tabular-nums} #mtp-phonon .gc .src{font-size:11px;color:var(--mute)!important} #mtp-phonon .to{display:grid;grid-template-columns:1fr 1fr;gap:12px} #mtp-phonon .tc{background:var(--card)!important;border:1px solid var(--line)!important;border-radius:12px;padding:14px} #mtp-phonon .tc h4{margin:0 0 8px!important;font-size:14px!important;color:var(--cream)!important} #mtp-phonon .tc ul{margin:0;padding-left:18px} #mtp-phonon .tc li{font-size:13px;color:var(--cream)!important;margin:5px 0;line-height:1.45} #mtp-phonon .tc.p h4{color:var(--mtp)!important} #mtp-phonon .tc.c h4{color:var(--acc2)!important} #mtp-phonon .ft{position:relative;display:flex;justify-content:space-between;gap:10px;flex-wrap:wrap;padding:12px 24px;border-top:1px solid var(--line)!important;font-size:11px;color:var(--mute)!important} #mtp-phonon .ft .brand{color:var(--mtp)!important;font-weight:700} @media (max-width:640px){ #mtp-phonon h3{font-size:20px!important} #mtp-phonon .grid3{grid-template-columns:1fr} #mtp-phonon .hw{grid-template-columns:1fr 1fr} #mtp-phonon .gauge,#mtp-phonon .to{grid-template-columns:1fr} #mtp-phonon .pipe{flex-direction:column} #mtp-phonon .arr{transform:rotate(90deg)} #mtp-phonon .bar{grid-template-columns:110px 1fr 56px;font-size:12px} #mtp-phonon .hd,#mtp-phonon .panel,#mtp-phonon .ft{padding-left:16px;padding-right:16px} #mtp-phonon .tabs{padding-left:16px;padding-right:16px} } <div id="mtp-phonon"> <div class="dots"></div> <div class="hd"> <div class="kick">Gradium · Open-weight TTS</div> <h3>Phonon: speech on the device, in 64.5 MB</h3> <p class="sub">31M and 40M parameter text-to-speech models. No cloud call, no GPU required.</p> </div> <div class="tabs" role="tablist"> <button class="tab on" data-t="fam">Model family</button> <button class="tab" data-t="how">How it works</button> <button class="tab" data-t="bench">Benchmarks</button> <button class="tab" data-t="speed">Speed by device</button> <button class="tab" data-t="trade">Trade-offs</button> </div> <div class="panel on" id="p-fam"> <div class="grid3"> <div class="mc on" data-d="English only, 37 built-in voices (25 + 12 character voices). Weights file model.q8.gguf, 64.5 MB. Apache 2.0, so commercial use is allowed."> <b>Phonon 31m</b><div class="big">31M</div><small>English · 64.5 MB q8</small><br><span class="lic ok">Apache 2.0</span></div> <div class="mc" data-d="1 model per language: English, French, German, Spanish, Portuguese. 9 transformer layers. Best expressivity and accuracy in the family. Commercial use needs a deal with Gradium."> <b>Phonon 40m</b><div class="big">40M</div><small>1 model per language · 74.6 MB</small><br><span class="lic nc">CC BY-NC 4.0</span></div> <div class="mc" data-d="All 5 languages in a single set of weights, with 102 built-in voices. Shares the 40m architecture. Commercial use needs a deal with Gradium."> <b>40m multilingual</b><div class="big">5</div><small>languages in 1 model · 74.6 MB</small><br><span class="lic nc">CC BY-NC 4.0</span></div> </div> <div class="detail" id="fam-d">English only, 37 built-in voices (25 + 12 character voices). Weights file model.q8.gguf, 64.5 MB. Apache 2.0, so commercial use is allowed.</div> </div> <div class="panel" id="p-how"> <div class="pipe"> <div class="st on"><span class="n">STEP 1</span><b>Text + voice</b></div><span class="arr"></span> <div class="st"><span class="n">STEP 2</span><b>SentencePiece</b></div><span class="arr"></span> <div class="st"><span class="n">STEP 3</span><b>Causal transformer</b></div><span class="arr"></span> <div class="st"><span class="n">STEP 4</span><b>Flow-matching head</b></div><span class="arr"></span> <div class="st"><span class="n">STEP 5</span><b>Mimi decoder</b></div> </div> <div class="detail" id="how-d"></div> <div class="frames" id="frames"></div> <div class="ctl"> <button class="btn ghost" id="prev"> Back</button> <button class="btn" id="next">Next step </button> <button class="btn ghost" id="play">Stream frames</button> <span class="cap">Each bar is 1 latent frame = 80 ms of 24 kHz audio</span> </div> </div> <div class="panel" id="p-bench"> <div class="seg" id="bseg"> <button class="on" data-b="wer">Hard-set WER (lower is better)</button> <button data-b="elo">English Elo (higher is better)</button> <button data-b="seed">Seed-TTS WER</button> </div> <div id="bars"></div> <div class="note" id="bnote"></div> </div> <div class="panel" id="p-speed"> <div class="hw" id="hw"></div> <div class="gauge"> <div class="gc"><div class="k">Time to first audio</div><div class="val" id="g1">6.9 ms</div><div class="src" id="g1s"></div></div> <div class="gc"><div class="k">Speed vs real time</div><div class="val" id="g2">29.9x</div><div class="src" id="g2s"></div></div> </div> <div class="note">Gradium blog rows are Phonon 31m; partner and press-release rows do not name the variant. Higher "x real time" means more seconds of audio per second of compute. Measured by Gradium and launch partners.</div> </div> <div class="panel" id="p-trade"> <div class="to"> <div class="tc p"><h4>Where Phonon 31m leads</h4><ul> <li>1.70% hard-set WER vs 3.50% for Kokoro</li> <li>6.9 ms to first audio on a MacBook Air M5 CPU</li> <li>3.7x real time on a Raspberry Pi 5</li> <li>Apache 2.0 weights, tokenizer and voices</li></ul></div> <div class="tc c"><h4>Where it falls short</h4><ul> <li>0.51% Seed-TTS WER, behind Kokoro at 0.45%</li> <li>English only at 31M; 40m models are CC BY-NC</li> <li>Prosody resets between sentences on long text</li> <li>All benchmarks are run by Gradium</li></ul></div> </div> </div> <div class="ft"><span>Sources: gradium.ai/blog, huggingface.co/gradium · self-reported numbers</span><span class="brand">Built by Marktechpost</span></div> </div> (function(){ var R=document.getElementById('mtp-phonon'); function send(){try{parent.postMessage({mtpFrameHeight:R.offsetHeight+40,id:'mtp-phonon'},'*')}catch(e){}} R.querySelectorAll('.tab').forEach(function(t){t.addEventListener('click',function(){ R.querySelectorAll('.tab').forEach(function(x){x.classList.remove('on')});t.classList.add('on'); R.querySelectorAll('.panel').forEach(function(p){p.classList.remove('on')}); R.querySelector('#p-'+t.dataset.t).classList.add('on'); if(t.dataset.t==='bench')drawBars(cur); setTimeout(send,60);});}); R.querySelectorAll('.mc').forEach(function(c){c.addEventListener('click',function(){ R.querySelectorAll('.mc').forEach(function(x){x.classList.remove('on')});c.classList.add('on'); document.getElementById('fam-d').textContent=c.dataset.d;setTimeout(send,30);});}); var steps=['Input text plus a speaker embedding. The voice can be built-in, cloned or designed in Gradium Studio, then run locally.', 'Raw text becomes tokens from a 4,000-token SentencePiece vocabulary. No phoneme conversion, so new languages are easier to add.', 'A causal transformer (6 layers, width 512, 8 heads in 31m) generates 1 latent vector per 80 ms frame, autoregressively.', 'A flow-matching head turns each latent plus noise into a continuous audio embedding in a single step. An end-of-speech head decides when to stop.', 'A smaller variant of Kyutai's Mimi decoder converts embeddings into 24 kHz PCM audio, streamed while generation continues.']; var si=0,sts=R.querySelectorAll('.st'); function setStep(i){si=(i+5)%5;sts.forEach(function(s,k){s.classList.toggle('on',k===si)});document.getElementById('how-d').textContent=steps[si];} setStep(0); document.getElementById('next').onclick=function(){setStep(si+1)}; document.getElementById('prev').onclick=function(){setStep(si-1)}; var F=document.getElementById('frames');for(var i=0;i<24;i++){var d=document.createElement('div');d.className='fr';F.appendChild(d);} var tm=null;document.getElementById('play').onclick=function(){ var fr=F.children,k=0;clearInterval(tm);setStep(2); for(var j=0;j=fr.length){clearInterval(tm);setStep(4);return;} fr[k].classList.add('live');fr[k].style.height=(30+Math.round(Math.abs(Math.sin(k*1.3))*65))+'%'; if(k===8)setStep(3);k++;},110);}; var B={wer:{max:9,u:'%',rows:[['Phonon 31m',1.70,1],['Phonon 40m multi',2.11,1],['Phonon 40m',2.22,1],['Kokoro',3.50],['Kitten TTS',7.40],['NeuTTS Nano',8.71]], n:'500 hard English sentences (phone numbers, emails, postal codes). Magpie scored 27.51%, off this scale. The Hugging Face card lists 1.5% for Phonon 31m; the blog lists 1.70%.'}, elo:{max:2150,min:1900,u:'',rows:[['Phonon 40m',2117,1],['Phonon 31m',2098,1],['Phonon 40m multi',2097,1],['Pocket TTS 100m',2056],['Kokoro',2043],['NeuTTS',2034],['Magpie',1939]], n:'Blind pairwise listening tests, US English: 468 raters, 8,290 votes. Bars start at 1900 Elo.'}, seed:{max:0.7,u:'%',rows:[['Phonon 40m',0.45,1],['Kokoro',0.45],['Magpie',0.46],['Phonon 40m multi',0.48,1],['Phonon 31m',0.51,1],['NeuTTS',0.62]], n:'Seed-TTS English evaluation sentences, LLM-assisted WER. Here Phonon 31m trails Kokoro and Magpie.'}}; var cur='wer'; function drawBars(k){cur=k;var b=B[k],h='';b.rows.forEach(function(r){h+='<div class="bar'+(r[2]?' me':'')+'"><span class="lb">'+r[0]+'</span><div class="tr"><div class="fl" data-w="'+(((r[1]-(b.min||0))/(b.max-(b.min||0)))*100).toFixed(1)+'"></div></div><span class="v">'+(k==='elo'?r[1]:r[1].toFixed(2)+b.u)+'</span></div>'}); document.getElementById('bars').innerHTML=h;document.getElementById('bnote').textContent=b.n; setTimeout(function(){R.querySelectorAll('#bars .fl').forEach(function(f){f.style.width=f.dataset.w+'%'})},40);setTimeout(send,80);} R.querySelectorAll('#bseg button').forEach(function(x){x.onclick=function(){R.querySelectorAll('#bseg button').forEach(function(y){y.classList.remove('on')});x.classList.add('on');drawBars(x.dataset.b);}}); drawBars('wer'); var H=[['MacBook Air M5 CPU','4 threads, no KleidiAI','6.9 ms','29.9x','Gradium blog'], ['M5 + KleidiAI + Accelerate','4 threads','5.4 ms','46.7x','Gradium blog'], ['Raspberry Pi 5','CPU','38 ms','3.7x','Gradium blog'], ['Snapdragon 8 Elite Gen 5','Oryon CPU','7.5 ms','24x','Gradium with Qualcomm'], ['Arm CPU + KleidiAI','Arm-measured','28 ms','15x','Measured by Arm'], ['NVIDIA L40S','CUDA backend','5 ms','Not disclosed','Gradium press release']]; var hw=document.getElementById('hw'),hh='';H.forEach(function(r,i){hh+='<div class="hwc'+(i?'':' on')+'" data-i="'+i+'"><b>'+r[0]+'</b><small>'+r[1]+'</small></div>'});hw.innerHTML=hh; function count(el,txt){var m=txt.match(/[d.]+/);if(!m){el.textContent=txt;return;}var end=parseFloat(m[0]),dec=(m[0].split('.')[1]||'').length,suf=txt.slice(m.index+m[0].length),t0=null; function f(ts){if(!t0)t0=ts;var p=Math.min((ts-t0)/600,1);el.textContent=(end*p).toFixed(dec)+suf;if(p<1)requestAnimationFrame(f);}requestAnimationFrame(f);} function pick(i){var r=H[i];R.querySelectorAll('.hwc').forEach(function(c){c.classList.toggle('on',+c.dataset.i===i)}); count(document.getElementById('g1'),r[2]);count(document.getElementById('g2'),r[3]); document.getElementById('g1s').textContent=r[4];document.getElementById('g2s').textContent=r[4];} R.querySelectorAll('.hwc').forEach(function(c){c.onclick=function(){pick(+c.dataset.i)}}); pick(0); window.addEventListener('load',send);window.addEventListener('resize',send);setTimeout(send,300); })(); " style="width:100%;height:600px;border:0;display:block;background:transparent;" loading="lazy" scrolling="no"> window.addEventListener('message',function(e){if(e.data&&e.data.id==='mtp-phonon'&&e.data.mtpFrameHeight){var f=document.getElementById('mtp-phonon-frame');if(f)f.style.height=e.data.mtpFrameHeight+'px';}}); How does Phonon perform on benchmarks? Gradium ran blind pairwise listening tests with 1,189 raters and 17,270 votes. In US English, Phonon 40m scored 2117 Elo and Phonon 31m scored 2098. Kokoro scored 2043, NeuTTS 2034 and Magpie 1939. Accuracy was tested on a public hard-sentence set. It holds 500 sentences per language, full of phone numbers, emails and postal codes. Phonon 31m scored 1.70% WER against 3.50% for Kokoro and 8.71% for NeuTTS Nano. Built-in text normalization drives much of that gap. On the easier Seed-TTS set, the gap closes. Phonon 31m scored 0.51% LLM WER, while Kokoro scored 0.45%. How fast is Phonon on a CPU? On a MacBook Air M5 CPU with 4 threads, Phonon 31m reaches first audio in 6.9 ms. It runs 29.9x faster than real time. Kokoro takes 517 ms, because it does not stream. With Arm KleidiAI and Apple Accelerate, Phonon 31m reaches 46.7x real time. On a Raspberry Pi 5, Phonon 31m starts in 38 ms and runs 3.7x faster than real time. Gradium says Kokoro needs 7 seconds to start there. On Snapdragon 8 Elite Gen 5, Phonon starts in 7.5 ms at 24x real time. On an NVIDIA L40S, CUDA reaches first speech in 5 ms. An AMD EPYC 7413 CPU takes 43 ms, per the press release. Comparison: Phonon vs Kokoro, NeuTTS Nano and Kitten TTS Benchmark rows come from Gradium’s blog and model card. Size, language and license rows come from each model’s own card. How can developers deploy Phonon? The runtime is MIT or Apache-2.0 licensed. It covers Rust, Python (pip install ptts), Swift, C, C++ and the browser via phonon-tts. It also ships an OpenAI-compatible server and a WebSocket server. Phonon loads in Transformers, MLX-Audio, Ollama and LM Studio. Voice cloning and voice design run in Gradium Studio, then the voice runs locally. Key Takeaways Phonon 31m is a 31M, 64.5 MB English TTS model under Apache 2.0. It reports 1.70% WER on hard text, against 3.50% for Kokoro. First audio arrives in 6.9 ms on a MacBook Air M5 CPU. It runs 3.7x faster than real time on a Raspberry Pi 5. Multilingual 40m models exist, but under a non-commercial license. Check out the technical blog, model card, GitHub runtime and hard-sentence dataset. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Gradium Open-Sources Phonon, a 31M-Parameter On-Device TTS Model Under Apache 2.0 appeared first on MarkTechPost.
