Breaking
Advertisement

AI

AI

Baseten Releases joey-1, a 2B Decision Model at $0.01 per 1M Tokens

Baseten has released joey-1, a 2B model from its in-house research lab, Base Labs. The joey-1 decision model returns a label and a calibrated probability instead of generated text. It targets high-volume routing, classification and guardrail checks. Baseten prices it at $0.01 per million input tokens, with open weights on Hugging Face under MIT. TL;DR Size: 2B parameters, post-trained from Qwen3.5-2B. 16,384-token max sequence (Baseten says 262k on dedicated deployments). Runs on: Baseten Model APIs, or self-hosted on 1 CUDA GPU (VRAM not disclosed). No official quantized builds listed. Performance: Top model under 4B on the Jev Decision Index 0.4 public split, per Baseten. Best: 63.30 on Jev Decision Index 0.4 (public split), ranked #37 of 230 models. Worst: Trails its 9B sibling elephant-1 by 8.94 points on the same split (63.30 vs 72.24). Bottom line: Best: 20 ms P50 decisions for $0.01 per 1M tokens. Worst: less general knowledge than its larger siblings. What is joey-1? joey-1 is a 2B decision model from Baseten’s Base Labs that answers typed questions in 1 forward pass. You send a state (text or images) and 1 or more questions. The model returns a probability for every option of every question. It does not write free-form text that your application must parse. It supports 3 question types: choice: pick 1 named option, such as which team owns a ticket. yes/no (noul in the API): returns the probability of yes. score: ordered levels, lowest first, such as urgency. joey-1 is the third member of a family. Baseten shipped mammoth-1 (27B) and elephant-1 (9B) last week. All 3 share training data and the /v1/systemone API, so switching is a 1-line change. How does joey-1 work? Base Labs post-trained Qwen3.5-2B with a full fine-tune of the language model. The vision encoder stays frozen. Full-attention layers use a block-bidirectional mask. All questions are answered from 1 shared-prefix prefill, so extra questions add little cost. Training details from the model card: Data: 6.21M decision examples. Mix: 58.3% core decision skills, 16.1% safety, 16.1% judging and expert domains, 9.5% public data. Objective: cross-entropy plus a Brier calibration term, with a trained “I don’t know” option. Recipe: 1 epoch, 43,390 steps, peak learning rate 1e-5, bf16. Hardware: 2 nodes of 8 H100 GPUs. Calibration lets you set a confidence threshold. Uncertain answers can escalate to elephant-1, mammoth-1 or a human. (function(){var f=document.getElementById('mtp-joey1-frame'); window.addEventListener('message',function(e){if(e.data&&e.data.type==='mtp-joey1-height'&&e.source===f.contentWindow){f.style.height=e.data.h+'px';}});})(); How fast and cheap is joey-1? Baseten reports a P50 end-to-end latency of 20 ms for an average 2,000-token request. That is about 2.5x faster than elephant-1 (49 ms). The blog’s TL;DR also claims 5x faster than comparable decision models, without naming them. Model APIs charge $0.01 per 1M input tokens; output and cached input are listed at $0.00. elephant-1 costs $0.035 and mammoth-1 costs $0.10. Baseten’s example: 10M support tickets a month at about 500 tokens each is 5B tokens. That is about $50 on joey-1, versus $500 on mammoth-1. How does joey-1 score on benchmarks? All scores below are self-reported on Baseten’s model card. On the Jev Decision Index 0.4 public split, joey-1 scores 63.30. That ranks #37 of 230 models and is the top score under 4B. The Jev Decision Index is a community leaderboard on Hugging Face. On the same split, Cloudflare’s 27B Clef scores 61.71 and Jev 1.14 scores 60.70. joey-1 trails elephant-1 in every area. It beats Jev 1.14 in 4 of 5 areas, including language, tools and retrieval. On Baseten’s replica of the Artificial Analysis Decision Index, joey-1 scores 62.9, versus 65.9 for mammoth-1. joey-1 vs. other small decision models *Jev 0.4 scores for all models come from Baseten’s joey-1 model card. **Latency figures are not comparable. Baseten measures end-to-end on its platform; Cloudflare reports its own Decision Index 0.2.1 run. How do you deploy joey-1? Model APIs: call decision-models/baseten-joey-2b-latest at /v1/systemone. Zero data retention is on by default. Dedicated deployments: Baseten says these unlock 262k context, fine-tuned weights and co-location with LLM workloads. Self-hosted: download from Hugging Face and run setup.sh. It needs 1 CUDA GPU and a PyTorch 2.8 / CUDA 12.8 image. You can fine-tune joey-1 on Baseten Training Jobs using the fine-tuning cookbook. joey-1 is also listed on OpenRouter at $0.01 per 1M input tokens. Key Takeaways joey-1 returns labels plus calibrated probabilities, not text. Baseten reports 20 ms P50 latency at $0.01 per 1M input tokens. 63.30 on Jev Decision Index 0.4 public: top model under 4B. Trails elephant-1 by 8.94 points; best as a first-line router. MIT weights on Hugging Face; self-host on 1 CUDA GPU. Check out the blog post, model card, model library page and fine-tuning cookbook. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Baseten Releases joey-1, a 2B Decision Model at $0.01 per 1M Tokens appeared first on MarkTechPost.

AI

OpenAI doubles usage limits on Pro plans as competition with Anthropic heats up

OpenAI is doubling usage limits across its paid plans, from Pro 100 to Pro 500. A planned cut to the Pro 200 quota was reversed after user pushback. More efficient models like Terra and Sol should push usable capacity even higher. The move comes just ahead of Anthropic's IPO, as both companies fight for B2B dominance in AI. The article OpenAI doubles usage limits on Pro plans as competition with Anthropic heats up appeared first on The Decoder.

Advertisement
AI

Prime Intellect Releases Prime Mega MoE: Trainable Fused MoE Kernel, 3.11x Faster Forward Pass

Prime Intellect has released Prime Mega MoE, a training extension of DeepSeek’s fused Mega MoE kernel. DeepSeek’s kernel only ran the forward pass. Prime Intellect added a fast, overlapping backward pass. That makes the fused kernel usable for training, not just inference. In its prime-rl trainer, full training steps on Qwen3 MoE models run up to 34% faster. Against TorchTitan, the gain reaches 55%. For teams training mixture-of-experts (MoE) models, that is a direct cut in GPU hours. TL;DR Size: A GPU kernel, not a model. Tested on Qwen3-30B-A3B and Qwen3-235B-A22B (128 experts, top-8 routing). Runs on: NVIDIA Hopper (blockwise FP8) and Blackwell (MXFP8) GPUs, BF16 on both. Published numbers use 8×H200 nodes, 32 to 128 GPUs. Performance: 32% to 34% faster full training steps than the prime-rl baseline. 48% to 55% faster than TorchTitan. Best: 3.11x faster forward pass on a Qwen3-30B-A3B MoE layer (12.0 ms to 3.9 ms). Worst: 1.24x backward pass on a Qwen3-235B-A22B MoE layer (30.5 ms to 24.6 ms). What is Prime Mega MoE? Prime Mega MoE is a fused, overlapping MoE kernel for training from Prime Intellect. It builds on the Mega MoE kernel in DeepSeek’s open-source DeepGEMM library. Why is MoE training communication-bound? In an MoE model, each token goes to a small subset of experts. Those experts sit on different GPUs. So every layer sends tokens out, runs expert math, and sends results back. This happens in the forward pass and again in the backward pass. In a standard setup, these are separate operations. The GPU sorts tokens and waits on the transport layer. Then it computes, hands results back, and waits again. Those waits are idle GPU time. How does the fused kernel work? DeepSeek’s Mega MoE, added to DeepGEMM in April 2026, replaces that chain with one kernel call. It fuses expert-parallel dispatch, the 2 expert linear layers, SwiGLU, and combine. While one token group is in transit, the kernel computes on a group that already arrived. It also skips intermediate tensors by using GPU-only scratch buffers. But it only shipped a forward implementation. Prime Intellect wrote the missing backward kernel, also overlapping. Supported types are BF16, MXFP8 (Blackwell only) and blockwise FP8 (Hopper only). The prime-rl config also exposes an fp8_bwd mode: BF16 forward, FP8 backward. (function(){var f=document.getElementById("mtp-pmm-frame");window.addEventListener("message",function(e){if(e.data&&e.data.mtpEmbed==="pmm"&&e.data.height&&f&&e.source===f.contentWindow){f.style.height=e.data.height+"px";}});})(); How fast is Prime Mega MoE? All numbers below are self-reported by Prime Intellect. Training steps use 8×H200 nodes, FP8, 16,384-token sequences and full activation checkpointing. Each result is the mean of steps 6 to 20. Full training step, seconds (lower is better): Against TorchTitan, throughput rose from 5,337 to 7,886 tokens/s/GPU on the 30B model. On the 235B model, it rose from 1,094 to 1,697. The prime-rl baseline uses torch all-to-all plus DeepGEMM FP8 experts. Single MoE layer, milliseconds (1 node, 8×H200, EP=8, 128 experts, top-8, random routing): The pattern is clear. Forward gains are large, and backward gains are smaller. The headline “up to 3.11x” refers to the forward pass only. How does it compare with other fused MoE options? TorchTitan’s 41% gain comes from a different setup (B200, 671B model, BF16 baseline). It is not directly comparable. How do developers use it? Prime Mega MoE is a dispatch backend in prime-rl. Set type = "mega_moe" under [trainer.model.moe.dispatch]. Choose precision from bf16, mxfp8, fp8 or fp8_bwd. The training kernels ship in a prebuilt DeepGEMM wheel from prime-kernels. The stock DeepGEMM build lacks them. Key Takeaways Prime Mega MoE adds a fused, overlapping backward pass to DeepSeek’s Mega MoE. Full training steps run 32% to 34% faster than the prime-rl baseline. Against TorchTitan, steps run 48% to 55% faster on Qwen3 MoE models. Forward gains reach 3.11x, but backward gains drop to 1.24x on large layers. It ships in prime-rl as the mega_moe dispatch backend. Check out the Prime Mega MoE blog post, the announcement on X, and the prime-rl repo. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Prime Intellect Releases Prime Mega MoE: Trainable Fused MoE Kernel, 3.11x Faster Forward Pass appeared first on MarkTechPost.

AI

Paris-based Mistral releases “Le Chonk,” a strong coding model that still trails China overall

Mistral has released the weights for Large 4, aka "Le Chonk," a trillion-parameter model trained entirely in Europe. Mistral says it beats its open rivals at coding, and independent testing shows it's fast and fairly cheap. It still can't match China's top models, scoring 26 points to the 32 posted by DeepSeek V4.1 Flash and GLM-5.3. The article Paris-based Mistral releases "Le Chonk," a strong coding model that still trails China overall appeared first on The Decoder.

Advertisement
AI

Google’s AI search problem finally shows up in the earnings report, some investors believe

Google's stock dropped as much as ten percent after its quarterly earnings. Cloud revenue more than doubled to $30.7 billion, but search revenue grew just 12 percent, slower than expected. Google acknowledged that its fast-growing AI mode brings in less revenue than traditional search, and it is now expanding its AI mode ad program internationally. The article Google's AI search problem finally shows up in the earnings report, some investors believe appeared first on The Decoder.

Advertisement
AI

Infini-AI-Lab Releases Hermit, an Open-Source Harness Adapter Framework for AI Agents

Infini-AI-Lab has released Hermit, an open-source harness adapter framework for AI agents. Hermit is a framework from Infini-AI-Lab that trains an external ‘harness adapter’ while the agent’s model and harness stay frozen. The first release, v0.1.0, shipped on October 25, 2026. CMU postdoc Haizhong Zheng announced it on X on October 27. For developers, the pitch is simple: adapt Codex or Claude Code agents to new tasks without fine-tuning or forking the harness. TL;DR Footprint: A Python framework. No parameters of its own. Apache-2.0, pip install hermit-harness. Runs on: The offline Quick Start needs no GPU, model account or dataset. Real runs use any API backend or a local vLLM, SGLang or Ollama server. Performance: Reported gains over no-adapter baselines on OfficeQA, CL-Bench and ServeLearnBench, all with GLM 5.3 Flash. Best: MWM adapter hits 0.795 accuracy on CL-Bench database exploration, vs 0.580 for the in-context baseline. Worst: The prompt-only adapter scores 0.470 on the same venue, below that 0.580 baseline. Bottom line: Best: one training loop for 5 harness evolution methods. Worst: most results are single runs, self-reported. What is a harness adapter? A harness adapter is a trainable external harness. It works like a plugin with training support built in. A model adapter, such as LoRA, adapts a model. A harness adapter adapts an existing agent instead. The adapter can hold instructions, skills, memory, tools, executable code and agent coordination logic. It can even be a whole repository. Its specification says which parts are learnable and how updates get checked. The base model and harness never change. The team gives 3 reasons for this design. Many harnesses are closed-source. Editing open ones can break carefully tuned component interactions. And a learned adapter can move between compatible models and harnesses. How does Hermit train an adapter? Hermit uses an RL-style training loop. A Trainer runs a batch of tasks through the acting agent with the current adapter. A judge scores the outputs. An Evolver reads the trajectories and feedback, then proposes a new adapter. The Trainer checks it against the template and commits it as the next version. Every adapter version lives in a git tree. You can inspect, roll back or ship any version. Learner programs can run sandboxed, which requires Linux with bubblewrap. Five evolution methods ship today: GEPA, SkillOpt, MetaWorld Model (MWM), Harness² and Continual Harness. ACE and ReHarness are listed as coming soon. You can also write your own Evolver. <iframe id="mtp-hermit-frame" title="Hermit interactive explainer" style="width:100%;height:600px;border:0;display:block;overflow:hidden;" scrolling="no" loading="lazy" srcdoc="<!doctype html> html,body{margin:0!important;padding:0!important;background:transparent!important} #mtp-hermit-x{--bg:#1B1D22;--card:#23262d;--line:#33363f;--ink:#F1F0EA;--mut:#9a9ea8;--cu:#C6743C;--cu2:#E08E52;--bl:#8ab6e4;--grn:#76B900; max-width:880px;margin:0 auto;background:var(--bg)!important;color:var(--ink)!important;border:1px solid var(--line)!important;border-radius:16px;font-family:Inter,system-ui,-apple-system,Segoe UI,Roboto,sans-serif;overflow:hidden;box-sizing:border-box} #mtp-hermit-x *{box-sizing:border-box} #mtp-hermit-x hr,#mtp-hermit-x p:empty,#mtp-hermit-x del,#mtp-hermit-x s{display:none!important} #mtp-hermit-x .hd{padding:22px 26px 14px;border-bottom:1px solid var(--line)!important;background:linear-gradient(135deg,#1B1D22 0%,#2a2017 100%)!important} #mtp-hermit-x .eb{font-size:11px;letter-spacing:.14em;text-transform:uppercase;color:var(--cu2)!important;font-weight:700} #mtp-hermit-x h3{margin:6px 0 4px!important;font-size:22px!important;line-height:1.25!important;color:var(--ink)!important;font-weight:800!important} #mtp-hermit-x .sub{margin:0!important;color:var(--mut)!important;font-size:14px} #mtp-hermit-x .tabs{display:flex;gap:6px;padding:12px 18px;border-bottom:1px solid var(--line)!important;flex-wrap:wrap} #mtp-hermit-x .tab{background:transparent!important;color:var(--mut)!important;border:1px solid var(--line)!important;border-radius:999px;padding:7px 14px;font-size:13px;font-weight:600;cursor:pointer;font-family:inherit;transition:all .2s} #mtp-hermit-x .tab:hover{color:var(--ink)!important;border-color:var(--cu)!important} #mtp-hermit-x .tab.on{background:var(--cu)!important;color:#1B1D22!important;border-color:var(--cu)!important} #mtp-hermit-x .pn{display:none;padding:22px 26px;animation:mhx-in .35s ease} #mtp-hermit-x .pn.on{display:block} @keyframes mhx-in{from{opacity:0;transform:translateY(6px)}to{opacity:1;transform:none}} #mtp-hermit-x .lead{color:var(--ink)!important;font-size:15px;line-height:1.55;margin:0 0 16px!important} #mtp-hermit-x .lead b{color:var(--cu2)!important} /* Adapter diagram */ #mtp-hermit-x .dg{display:grid;grid-template-columns:1fr 56px 1fr;align-items:center;gap:0;margin:4px 0 14px} #mtp-hermit-x .box{background:var(--card)!important;border:1px solid var(--line)!important;border-radius:12px;padding:14px} #mtp-hermit-x .box h4{margin:0 0 8px!important;font-size:13px!important;letter-spacing:.06em;text-transform:uppercase;color:var(--mut)!important} #mtp-hermit-x .frz{position:relative} #mtp-hermit-x .frz:after{content:"FROZEN";position:absolute;top:10px;right:10px;font-size:10px;font-weight:800;letter-spacing:.1em;color:var(--bl)!important;border:1px solid var(--bl)!important;border-radius:6px;padding:2px 6px} #mtp-hermit-x .chip{display:inline-block;margin:3px 4px 3px 0;padding:5px 9px;border-radius:8px;font-size:12px;font-weight:600;background:#2c2f37!important;color:var(--ink)!important;border:1px solid var(--line)!important;cursor:pointer;transition:all .2s} #mtp-hermit-x .ad .chip{border-color:#6b4423!important} #mtp-hermit-x .chip:hover,#mtp-hermit-x .chip.sel{background:var(--cu)!important;color:#1B1D22!important;border-color:var(--cu)!important} #mtp-hermit-x .ad{border-color:var(--cu)!important;box-shadow:0 0 0 0 rgba(198,116,60,.5);animation:mhx-glow 2.4s infinite} @keyframes mhx-glow{0%{box-shadow:0 0 0 0 rgba(198,116,60,.45)}70%{box-shadow:0 0 0 12px rgba(198,116,60,0)}100%{box-shadow:0 0 0 0 rgba(198,116,60,0)}} #mtp-hermit-x .plug{height:4px!important;background:linear-gradient(90deg,var(--bl),var(--cu))!important;border-radius:4px;position:relative;margin:0 6px} #mtp-hermit-x .plug:before{content:"";position:absolute;top:-4px;left:0;width:12px;height:12px;border-radius:50%;background:var(--cu2)!important;animation:mhx-dot 1.8s linear infinite} @keyframes mhx-dot{from{left:0}to{left:calc(100% - 12px)}} #mtp-hermit-x .info{min-height:44px;background:#16181c!important;border:1px dashed var(--line)!important;border-radius:10px;padding:10px 12px;font-size:13.5px;color:var(--ink)!important;line-height:1.5} /* Loop */ #mtp-hermit-x .loop{display:grid;grid-template-columns:repeat(5,1fr);gap:8px;margin-bottom:14px} #mtp-hermit-x .st{background:var(--card)!important;border:1px solid var(--line)!important;border-radius:12px;padding:12px 10px;text-align:center;transition:all .3s;opacity:.55} #mtp-hermit-x .st .n{font-size:11px;font-weight:800;color:var(--mut)!important} #mtp-hermit-x .st .t{font-size:13.5px;font-weight:700;margin-top:4px;color:var(--ink)!important} #mtp-hermit-x .st.on{opacity:1;border-color:var(--cu)!important;transform:translateY(-3px);box-shadow:0 6px 18px rgba(198,116,60,.25)} #mtp-hermit-x .st.on .n{color:var(--cu2)!important} #mtp-hermit-x .ctl{display:flex;gap:8px;align-items:center;margin-top:12px;flex-wrap:wrap} #mtp-hermit-x .btn{background:var(--cu)!important;color:#1B1D22!important;border:0!important;border-radius:8px;padding:8px 14px;font-weight:700;font-size:13px;cursor:pointer;font-family:inherit} #mtp-hermit-x .btn.gh{background:transparent!important;color:var(--ink)!important;border:1px solid var(--line)!important} #mtp-hermit-x .ver{margin-left:auto;font-size:13px;color:var(--mut)!important} #mtp-hermit-x .ver b{color:var(--cu2)!important;font-size:16px} /* Bars */ #mtp-hermit-x .seg{display:flex;gap:6px;margin-bottom:14px;flex-wrap:wrap} #mtp-hermit-x .bars{display:flex;flex-direction:column;gap:9px} #mtp-hermit-x .row{display:grid;grid-template-columns:170px 1fr 56px;align-items:center;gap:10px;font-size:13px} #mtp-hermit-x .row .lb{color:var(--ink)!important} #mtp-hermit-x .row .tr{height:16px!important;background:#2a2d34!important;border-radius:6px;overflow:hidden} #mtp-hermit-x .row .fl{height:100%!important;width:0;border-radius:6px;background:#5c616d!important;transition:width 1s cubic-bezier(.2,.8,.2,1)} #mtp-hermit-x .row.hl .fl{background:linear-gradient(90deg,var(--cu),var(--cu2))!important} #mtp-hermit-x .row.hl .lb{font-weight:700;color:var(--cu2)!important} #mtp-hermit-x .row .v{font-weight:700;text-align:right;font-variant-numeric:tabular-nums;color:var(--ink)!important} #mtp-hermit-x .note{font-size:12px;color:var(--mut)!important;margin-top:12px;line-height:1.5} /* Deploy */ #mtp-hermit-x .term{background:#0f1013!important;border:1px solid var(--line)!important;border-radius:10px;padding:14px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace!important;font-size:12.5px;color:#d7d4c8!important;line-height:1.7;overflow-x:auto;white-space:pre} #mtp-hermit-x .term .p{color:var(--cu2)!important} #mtp-hermit-x .term .c{color:#6B6F7A!important} #mtp-hermit-x .hosts{display:flex;gap:8px;flex-wrap:wrap;margin:12px 0} #mtp-hermit-x .ft{padding:12px 26px;border-top:1px solid var(--line)!important;font-size:12px;color:var(--mut)!important;display:flex;justify-content:space-between;flex-wrap:wrap;gap:6px} #mtp-hermit-x .ft .mtp{color:#76B900!important;font-weight:700} #mtp-hermit-x a{color:var(--cu2)!important} @media (max-width:640px){ #mtp-hermit-x .hd,#mtp-hermit-x .pn{padding:18px 16px} #mtp-hermit-x h3{font-size:19px!important} #mtp-hermit-x .dg{grid-template-columns:1fr;gap:10px} #mtp-hermit-x .plug{height:24px!important;width:4px!important;margin:0 auto;background:linear-gradient(180deg,var(--bl),var(--cu))!important} #mtp-hermit-x .plug:before{display:none} #mtp-hermit-x .loop{grid-template-columns:1fr 1fr} #mtp-hermit-x .row{grid-template-columns:110px 1fr 48px;font-size:12px} #mtp-hermit-x .ft{padding:12px 16px} } <div id="mtp-hermit-x"> <div class="hd"> <div class="eb">Infini-AI-Lab · Hermit v0.1.0 · Apache-2.0</div> <h3>How Hermit trains a harness adapter</h3> <p class="sub">The adapter learns from task feedback. The model and harness stay frozen.</p> </div> <div class="tabs" role="tablist"> <button class="tab on" data-p="0">1 · The adapter</button> <button class="tab" data-p="1">2 · Training loop</button> <button class="tab" data-p="2">3 · Results</button> <button class="tab" data-p="3">4 · Deploy</button> </div> <div class="pn on" data-i="0"> <p class="lead">A <b>harness adapter</b> is a trainable external harness, like a plugin with training built in. Click any part to see what it does.</p> <div class="dg"> <div class="box frz"><h4>Your agent</h4> <span class="chip" data-d="The base model. Hermit never updates its weights.">Model</span> <span class="chip" data-d="The existing harness (for example Codex or Claude Code). Hermit never edits it, so closed-source harnesses work too.">Harness</span> </div> <div class="plug" aria-hidden="true"></div> <div class="box ad"><h4>Harness adapter</h4> <span class="chip" data-d="Instruction files such as ADAPTER.md or AGENTS.md that the agent reads as context.">Instructions</span> <span class="chip" data-d="Reusable skills, including executable helper scripts.">Skills</span> <span class="chip" data-d="Notes and stored knowledge the agent carries across tasks.">Memory</span> <span class="chip" data-d="Extra tools, such as the wm tool MWM exposes for its world model.">Tools</span> <span class="chip" data-d="Executable code. An adapter can be an entire repository.">Code</span> <span class="chip" data-d="Coordination logic for several agents, as in the Lean adapter that links 5 Codex agents.">Coordination</span> </div> </div> <div class="info" id="mhx-info">Every adapter version is stored in a git tree, so you can inspect, roll back or ship any one.</div> </div> <div class="pn" data-i="1"> <p class="lead">Hermit runs an <b>RL-style loop</b>, but the thing being updated is the adapter, not the weights.</p> <div class="loop"> <div class="st"><div class="n">STEP 1</div><div class="t">Run tasks</div></div> <div class="st"><div class="n">STEP 2</div><div class="t">Judge scores</div></div> <div class="st"><div class="n">STEP 3</div><div class="t">Evolver proposes</div></div> <div class="st"><div class="n">STEP 4</div><div class="t">Check vs template</div></div> <div class="st"><div class="n">STEP 5</div><div class="t">Commit version</div></div> </div> <div class="info" id="mhx-step">Press Play to step through one training iteration.</div> <div class="ctl"> <button class="btn" id="mhx-play"> Play</button> <button class="btn gh" id="mhx-next">Next step</button> <button class="btn gh" id="mhx-reset">Reset</button> <span class="ver">Adapter version <b id="mhx-v">V0</b></span> </div> <p class="note">Built-in Evolvers: GEPA, SkillOpt, MetaWorld Model (MWM), Harness², Continual Harness. ACE and ReHarness are listed as coming soon.</p> </div> <div class="pn" data-i="2"> <p class="lead">Self-reported results, all with <b>GLM 5.3 Flash</b> as the acting model. Pick a benchmark.</p> <div class="seg"> <button class="tab on" data-b="0">OfficeQA</button> <button class="tab" data-b="1">CL-Bench database</button> <button class="tab" data-b="2">ServeLearnBench (H)</button> </div> <div class="bars" id="mhx-bars"></div> <p class="note" id="mhx-bnote"></p> </div> <div class="pn" data-i="3"> <p class="lead">Train once, then <b>deploy</b> the adapter into an agent workspace or export it as a native plugin.</p> <div class="term"><span class="p">$</span> pip install hermit-harness <span class="p">$</span> hermit train --config run.yaml <span class="p">$</span> hermit eval --config run.yaml --adapter final <span class="p">$</span> hermit deploy --adapter DIR --workspace WS --host codex <span class="p">$</span> hermit convert --adapter DIR --output OUT --target claude-code <span class="c"># other convert targets: codex, pi, hermes</span></div> <div class="hosts"> <span class="chip" data-d="Deploy target and plugin export target.">Codex</span> <span class="chip" data-d="Deploy target and plugin export target.">Claude Code</span> <span class="chip" data-d="Deploy target and plugin export target.">Pi</span> <span class="chip" data-d="Plugin export target.">Hermes</span> </div> <div class="info" id="mhx-dinfo">Backends by reference: Fireworks, OpenAI, Anthropic, DeepSeek, vLLM, SGLang, Ollama. The offline Quick Start needs no GPU, model account or dataset.</div> </div> <div class="ft"><span>Source: github.com/Infini-AI-Lab/hermit · self-reported</span><span class="mtp">© Marktechpost</span></div> </div> (function(){ var R=document.getElementById('mtp-hermit-x');if(!R)return; function ping(){try{var h=R.offsetHeight+40;parent.postMessage({mtpEmbedHeight:h,height:h,frameHeight:h},'*');}catch(e){}} var tabs=R.querySelectorAll('.tabs .tab'),pns=R.querySelectorAll('.pn'); tabs.forEach(function(t){t.addEventListener('click',function(){var p=+t.getAttribute('data-p');tabs.forEach(function(x){x.classList.remove('on')});t.classList.add('on');pns.forEach(function(x){x.classList.toggle('on',+x.getAttribute('data-i')===p)});if(p===2)drawBars(cur);setTimeout(ping,60);});}); function wire(sel,out){R.querySelectorAll(sel).forEach(function(c){c.addEventListener('click',function(){R.querySelectorAll(sel).forEach(function(x){x.classList.remove('sel')});c.classList.add('sel');document.getElementById(out).textContent=c.textContent+': '+c.getAttribute('data-d');ping();});});} wire('.dg .chip','mhx-info');wire('.hosts .chip','mhx-dinfo'); var steps=R.querySelectorAll('.st'),si=-1,ver=0,timer=null; var msg=['The Trainer runs a batch of tasks through the acting agent with the current adapter.','Your judge scores each output and writes feedback.','An Evolver reads trajectories and feedback, then proposes a new adapter.','The Trainer checks the proposal against the adapter template.','The checked update is committed as the next adapter version.']; function go(i){si=i;steps.forEach(function(s,k){s.classList.toggle('on',k===i)});document.getElementById('mhx-step').textContent=msg[i];if(i===4){ver++;document.getElementById('mhx-v').textContent='V'+ver;}} function next(){go((si+1)%5)} document.getElementById('mhx-next').onclick=function(){stop();next()}; document.getElementById('mhx-play').onclick=function(){if(timer){stop();return}this.textContent='❚❚ Pause';next();timer=setInterval(next,1400)}; function stop(){clearInterval(timer);timer=null;document.getElementById('mhx-play').textContent=' Play'} document.getElementById('mhx-reset').onclick=function(){stop();si=-1;ver=0;steps.forEach(function(s){s.classList.remove('on')});document.getElementById('mhx-v').textContent='V0';document.getElementById('mhx-step').textContent='Press Play to step through one training iteration.'}; var B=[ {max:1,fmt:3,rows:[['No adapter',.541],['Initial skill',.576],['GEPA',.580],['SkillOpt',.600],['Continual Harness',.634,1],['Harness²',.651,1]],n:'Accuracy on 172 test questions. GEPA and SkillOpt are means of 4 trainings, tested frozen. Continual Harness and Harness² learn while streaming the test set once. Standard error about 3.7 points.'}, {max:1,fmt:3,rows:[['Stateless baseline',.400],['Prompt-only',.470],['ICL baseline',.580],['MWM',.795,1]],n:'Accuracy on a 40-question stream over an undocumented, drifting SQLite database. Mean of 5 stream orders.'}, {max:100,fmt:1,rows:[['Blind',4.5],['Prompt-only',18.8],['SkillOpt (leaderboard)',22.0],['CH (leaderboard)',28.7],['MWM',45.3,1],['Oracle (policy given)',89.2]],n:'retail_l3: 624 serving requests, 537 held-out tests. H = score on tests that need the hidden policy. One run each.'}]; var cur=0; function drawBars(k){cur=k;var d=B[k],el=document.getElementById('mhx-bars');el.innerHTML='';d.rows.forEach(function(r){var row=document.createElement('div');row.className='row'+(r[2]?' hl':'');row.innerHTML='<span class="lb">'+r[0]+'</span><span class="tr"><span class="fl" style="display:block"></span></span><span class="v">'+r[1].toFixed(d.fmt)+'</span>';el.appendChild(row);var f=row.querySelector('.fl');setTimeout(function(){f.style.width=(r[1]/d.max*100)+'%'},40);});document.getElementById('mhx-bnote').textContent=d.n;setTimeout(ping,60);} R.querySelectorAll('.seg .tab').forEach(function(b){b.addEventListener('click',function(){R.querySelectorAll('.seg .tab').forEach(function(x){x.classList.remove('on')});b.classList.add('on');drawBars(+b.getAttribute('data-b'));});}); drawBars(0); window.addEventListener('load',ping);window.addEventListener('resize',ping);setTimeout(ping,300); })(); "> (function(){var f=document.getElementById("mtp-hermit-frame");window.addEventListener("message",function(e){if(!f||e.source!==f.contentWindow)return;var h=e.data&&e.data.mtpEmbedHeight;if(h&&h>100)f.style.height=h+"px";});})(); What are the benchmark results? All runs below use GLM 5.3 Flash as the acting model. The numbers come from the repo’s example READMEs and are self-reported. OfficeQA (recipe) asks 172 test questions about U.S. Treasury Bulletins, scored by exact match: GEPA and SkillOpt train on 50 questions and are tested frozen. Continual Harness and Harness² instead learn while streaming the test set once. The team reports a +7.6 point gain for Harness² over its reference run, with a 95% interval of +1.7 to +13.4. CL-Bench (database, poker) tests stateful environments from the Continual Learning Bench paper. On database exploration, MWM reaches 0.795 accuracy. The benchmark’s in-context baseline scores 0.580. On exploitable poker, MWM wins 3.08 big blinds per hand, vs 2.51 for in-context learning. ServeLearnBench (recipe) streams 624 retail support requests under a hidden, shifting policy. MWM scores 45.3 on hidden-policy tests. The best learning method on the leaderboard, CH, scores 28.7. An oracle given the policy scores 89.2. How does Hermit compare with other agent optimizers? How do you install and deploy it? Install from PyPI with pip install hermit-harness, or clone the repo for examples. The CLI covers check, train, eval, deploy and convert. Models are referenced as /. Supported backends include Fireworks, OpenAI, Anthropic, DeepSeek, vLLM, SGLang and Ollama. hermit deploy writes a workspace for Codex, Claude Code or Pi. hermit convert exports a trained adapter as a native plugin for Codex, Claude Code, Pi or Hermes. A finished Lean collaboration adapter coordinates 5 Codex agents proving one theorem together. Key Takeaways Hermit trains an external adapter; model and harness stay frozen. 5 methods ship: GEPA, SkillOpt, MWM, Harness², Continual Harness. MWM reaches 0.795 on CL-Bench database vs 0.580 in-context. Adapters deploy to Codex, Claude Code and Pi, or export as plugins. Results are self-reported, mostly single runs, with GLM 5.3 Flash. Check out the GitHub repo, project page and docs. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Infini-AI-Lab Releases Hermit, an Open-Source Harness Adapter Framework for AI Agents appeared first on MarkTechPost.