A team of researchers from Microsoft Research and UC Berkeley have released TokenEconomizer, an open-source cost controller for coding agents. TokenEconomizer is a plugin that learns from past sessions in a repository to cut waste in later ones. It works with Claude Code, Codex, GitHub Copilot CLI and Pi. It does not touch model weights or agent code. On 277 SWE-Bench tasks, it lowered mean cost by 18.4% to 30.9% across 5 agent-model settings. TL;DR Package: A standard-library Python package and agent plugins, with 3 switchable mechanisms. Runs on: Any machine running Claude Code, Codex, Copilot CLI or Pi with Python 3.10+. No GPU and no API key of its own. Performance: 18.4% to 30.9% lower mean SWE-Bench cost, and cheaper in all 30 repository-setting cells. Best: 30.9% lower cost on Claude Code with Opus 4.8 (33.2% when combined with RTK). Worst: Only 6.4% to 9.3% savings on ReposVul audits; Pass@1 drops 1.0 point on Opus 4.8. Why do coding agents waste tokens? Every agent turn re-bills the system prompt, every loaded tool schema and the full history. The research team profiled 60 SWE-Bench tasks across 6 repositories using Claude Code with Opus 4.8. They found 3 recurring sources of waste: Repeated discovery: once 5 tasks precede the current one, 30% of tool calls repeat an earlier action. Unused tools: 23 tool schemas load every turn. 16 are never called, and 3 cover 98% of calls. Failure loops: 10 failed runs were 17% of tasks but 32% of total spend. They also tested 3 fixed open-source strategies: RTK, Serena and Aider’s RepoMap. None was cheaper everywhere. All 3 cut cost on Django and raised it on SymPy. How does TokenEconomizer work? TokenEconomizer is an external controller that treats the agent as a black box. It keeps a state per repository, built from completed traces. Before each task, it sets 3 policies through the prompt, the tool settings and the event stream. 1. Micro-skills An LLM proposer turns trace digests into 1-line facts, such as the working test command. Each fact carries regex detectors that check later traces without a model call. A fact enters the prompt after 3 confirmations on tasks with non-overlapping edited files. Its need rate must also stay at or above 0.4. One contradiction counts as 5 confirmations, so wrong facts retire quickly. 2. Tool pruning Shell, read, write and edit tools are always kept. Other tools are masked after 3 consecutive unused tasks, or when an LLM judge marks them irrelevant. The full catalog returns every 5 tasks for a re-audit. On Django with Claude Code, the first request on task 2 fell from 43,351 to 16,295 prompt tokens. 3. Failure-loop intervention The controller learns a turn budget: 1.25 times the 90th percentile of successful run lengths. Cheap rule checks flag repeated failures or dense errors. An LLM judge then decides if the agent is stuck, at 0.7 confidence or higher. A stuck agent gets 1 redirect, then a stop if the loop recurs. #mtp-te-embed iframe{width:100%!important;border:0!important;display:block;background:transparent!important}#mtp-te-embed p:empty,#mtp-te-embed hr,#mtp-te-embed del,#mtp-te-embed s{display:none!important} (function(){var f=document.getElementById('mtp-te-frame');window.addEventListener('message',function(e){if(e.data&&e.data.mtpTE&&e.source===f.contentWindow){f.style.height=e.data.h+'px';}});})(); What do the SWE-Bench results show? Source: arXiv 2610.22732, Table 1. Costs include all controller calls. 3 seeds per setting. Pass@1 improved in 4 of 5 settings. Lower reasoning effort saved cost but cut Pass@1 by 1.8 to 10.3 points. RTK alone raised cost in 12 of 30 repository-setting cells, and RepoMap in 15. Savings build over time. Most repositories cost more at first, but all were cheaper by the sixth task. Alone on Claude Code, tool pruning saved 13.2% to 17.7%, micro-skills 11.8% to 14.3%, and the loop guard 5.4% to 7.1%. Results also held outside SWE-Bench. Costs fell 16.0% to 21.2% on Web-Bench and 16.5% to 19.6% on CredData. ReposVul saw only 6.4% to 9.3%, because shorter runs leave less to remove. How does it compare with other token-saving tools? Cost rows: TokenEconomizer paper. Licenses and claims: each project’s repository or docs. How do you install TokenEconomizer? The plugin needs one of the 4 agents and Python 3.10 or newer. On Claude Code, clone the repo, add it as a marketplace, then run claude plugin install tokeneconomizer@tokeneconomizer. Learned data stays local in ~/.tokeneconomizer. Tool pruning and the loop guard are on by default. Micro-skills are off by default, because the proposer adds model calls. Key Takeaways TokenEconomizer cuts SWE-Bench cost 18.4% to 30.9% with an unchanged agent. It was cheaper in all 30 repository-setting cells; RTK and RepoMap were not. Tool pruning is the biggest single lever, at 13.2% to 17.7%. Pass@1 rose in 4 of 5 settings and fell 1.0 point on Opus 4.8. It is Apache 2.0 and installs on Claude Code, Codex, Copilot CLI and Pi. Check out the Paper, GitHub Repo and Project Page. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Meet TokenEconomizer: Microsoft’s Plugin That Cuts Coding Agent Costs by Up to 30.9% appeared first on MarkTechPost.
OpenAI has updated its Model Spec, restructuring its AI behavior guidelines around five core values, including faithfulness to human intent and correctability. GPT models should now draw on conversation history to figure out what users actually want and disclose their own assumptions. Whether the models follow these rules is another matter. The article OpenAI sets new behavioral guidelines for AI models that have already proven hard to control appeared first on The Decoder.
Anthropic, OpenAI, Cloudflare and Google now each ship an AI agent that hunts security bugs in source code. We checked the repos, docs and technical details behind each one. The key finding: all 4 tools converge on one design. One agent finds a bug, a separate agent tries to disprove it, and only survivors get a patch. The differences are in where they run, who can use them and how much proof they demand. Key Takeaways 4 labs, 1 pattern: find, independently disprove, then patch. Claude Security runs inside Claude Code; no extra API key needed. Codex Security’s CLI is Apache-2.0, but scanning still needs OpenAI access. Cloudflare’s skill is MIT, model-agnostic and the most-starred at 47.5K. Google’s Mantis reproduces bugs in a sandbox; Google says isolate it. Why 4 labs shipped the same thing in 2026 The problem every one of them is solving is noise. Google’s security team writes that sloppy AI code scanning produces hallucinated bugs and true-positive rates under 7%. A scanner that is wrong 93% of the time costs reviewers more than it saves. The shared fix is separation of duties. The agent that reports a finding never gets to confirm it. Cloudflare states the rule directly in its README: the verifier is never the agent that found the bug. Anthropic, OpenAI and Google each describe a similar validation step before anything reaches a report. The second driver is offense. Anthropic team states that it built its scanner partly because attackers were already using AI models to find exploitable flaws, per The Hacker News. Google opens its Mantis post by noting that models can now discover and exploit bugs with little human help. 1. Claude Security by Anthropic What it is: A multi-agent vulnerability scanner that runs inside a Claude Code session. It is the local version of Anthropic’s hosted Claude Security product, per the plugin README. How it works: Agents map the architecture, build a threat model, hunt across every component and verify each finding independently. The plugin offers 3 jobs from one menu: scan the codebase, scan changes, or suggest patches, per Claude Code docs. Output: A Markdown report along with a SARIF 2.1.0 file for GitHub code scanning. Each finding carries severity, a CWE category, impact and reproduction steps. Patches are drafted in a scratch copy and land as F.patch files. The plugin never commits, pushes or opens a pull request on its own. Cost and access: It makes no model calls of its own, so there is no separate API key. Scans count against your Claude plan’s usage limits. Copy CodeCopiedUse a different Browser/plugin install claude-security@claude-plugins-official /claude-security Timeline: Anthropic opened a limited preview called Claude Code Security in February 2026. The hosted product reached public beta for Enterprise customers on Claude Opus 4.7. The Claude Code plugin entered beta on July 22, 2026, per Cybersecurity News. In August, the hosted scanner moved to Claude Mythos 5, per Dataconomy. Where it fits: Anthropic positions it as one layer. The security-guidance plugin checks code as Claude writes it, and /security-review does a single pass on a branch. Claude Security is the on-demand deep scan. 2. Codex Security by OpenAI What it is: A CLI and TypeScript SDK, published as @openai/codex-security, for finding, validating and fixing vulnerabilities. How it works: It scans a whole repo, selected paths, or a Git diff. A –mode deep flag runs parallel discovery workers. It validates candidate findings, generates patches and can verify existing fixes. A cs policy command drafts a SECURITY.md that tells future scans what counts as a vulnerability in your project. Output: Threat models, SARIF, JSON or CSV exports, plus publishing to Linear. It can import GitHub code-scanning alerts and suggest owners from Git history. A GitHub Action runs scheduled or pull-request scans; findings are report-only by default. Copy CodeCopiedUse a different Browsernpm install --global @openai/codex-security cs login cs scan Cost and access: The client is Apache-2.0, but the scanner behind it is not. DevOps.com notes the CLI and SDK stay in limited beta for approved customers. The README lists OpenAI, Amazon Bedrock, OpenRouter and Fireworks AI as providers. Requires Node.js 22.13+ and Python 3.10+. 3. Security Audit Skill by Cloudflare What it is: An MIT-licensed skill that turns any capable coding agent into a security auditor. How it works: A 6-phase audit: Reconnaissance: map architecture, trust boundaries and input surfaces into architecture.md and a coverage ledger. Coverage-led hunting: isolated hunters each take ledger units; coverage critics look for gaps. Candidate validation: a fresh verifier tries to disprove every unique candidate. Structured output: confirmed, needs_validation and rejected records go to findings.json, checked against a schema. Independent record verification: new agents re-verify every final source claim. Target-neutral reporting: REPORT.md and detail files are derived from the verified records. Requirements: A coding agent that supports tool use and parallel sub-agents, plus Node.js for the validators. It also wants an OS-enforced sandbox with no external network. Without one, leads stay needs_validation instead of being executed. npx skills add –skill security-audit Then ask the agent: “security audit this codebase.” Output defaults to ~/security-audit-skill//run-, outside your repo. The data behind it: Cloudflare published the skill on June 18, 2026, with its post Build your own vulnerability harness. The roughly 450-line skill seeded an internal harness that now scans 128 repos. Cloudflare reports these lifetime numbers: 20,799 raw candidates from discovery, about 12,057 of which survived validation. 13,841 findings in the shared triage pool across 145 repos, including another harness’s output. 5,442 folded away as duplicates and 1,154 routed out as wrong-repo or low-risk. 7,245 actionable findings sent to engineering teams. Better recon context cut the first-pass rejection rate from 40% to 11%. Cloudflare also uses one model for discovery and a different model for triage, so models check each other. 4. Mantis by Google What it is: An Apache-2.0 set of security-review skills plus an ADK reference harness for AI coding agents. Google team states that it is part of how it finds and fixes bugs internally. How it works: Mantis reads the repo’s history to learn from past security fixes. It builds a hierarchical summary tree, condensing files into directory and root summaries. Google team states that this cut token overhead by over 85%. It then generates a threat model and hypotheses, hunts, deduplicates and critiques findings. What sets it apart: Reproduction and patch verification. Mantis tries to reproduce each bug, from a static exploit sketch up to a unit test or mock server. It patches with an adversarial loop that attacks the fix. It also calibrates severity against a rubric to stop “thousands of criticals.” A mantis-advise skill then feeds what it learned back into code generation. Setup: Clone the repo and point any coding agent at it, or run the ADK harness directly. The harness supports a –focus prompt such as “look for IDOR” and a –max-tokens spend ceiling. Copy CodeCopiedUse a different Browsergit clone cd mantis/reference && ./install.sh ./run.sh path/to/code Side-by-side Beyond the big 4: other AI bug hunters launched in 2026 At least 12 more agent-based vulnerability tools shipped or went public in 2026. Almost all repeat the same find-then-independently-verify loop. They split into free harnesses you run yourself and commercial or internal platforms. Open-source harnesses AppSec Santa also tracks harnesses from Anthropic (defending-code-reference-harness), Trail of Bits, Capital One (vulnhunter) and Cisco (ai-deep-sast). Cisco’s runs fully local on Foundation-Sec-8B, so no code leaves the machine. Commercial platforms and internal systems The pattern holds across vendors: deterministic engines or validators do the proving, models do the reasoning, and a human approves every fix. Which one to run first Already on Claude Code: start with the Claude Security plugin. It installs in 1 line and uses your existing plan. Need CI gating and SARIF in GitHub: Codex Security has the most mature pipeline tooling, if your org has access. Want no vendor lock-in: Cloudflare’s skill runs on any agent with parallel sub-agents and is fully MIT. Have a security team and a sandbox: Mantis gives the deepest proof loop, with reproduction and adversarial patch testing. Cloudflare’s own architecture suggests a practical combination. Run discovery with one tool or model, then validate with a different one. Different models surface different shares of the bugs. The post AI Security Agents Compared: Claude, Codex, Mantis, Cloudflare appeared first on MarkTechPost.
Reddit lost over one million Google search referrals per week in October, CEO Steve Huffman said. The stock dropped more than 25 percent despite strong earnings. Huffman blames Google's AI Overviews and AI Mode for driving zero traffic to external sites. Reddit now tests login barriers and wants to become a direct destination, less reliant on the open web. The shift threatens every content-driven business that depends on search traffic. The article Google's AI Mode doesn't drive traffic "for anybody," Reddit CEO Huffman warns the entire web appeared first on The Decoder.
Epoch AI researchers say the widely available AI model GLM 5.3 turned public security fixes into working exploits before updates reached Firefox users. It autonomously compromised six Firefox releases for about $1,000 in API fees each without triggering safeguards from any provider tested. The article Open-weight model GLM 5.3 built full Firefox exploit chains with zero human input in under 48 hours appeared first on The Decoder.
Samaya AI has released Samaya Sterling, a post-trained model for agentic financial research. Sterling is built on NVIDIA’s Nemotron 3.5 Lightning, a 30B mixture-of-experts model with 3B active parameters. On Samaya’s FrontierFinance benchmark, it scores 58.9%. Claude Fable 5.1 scores 55.9%. Each Sterling question costs $0.61, against $6.89 for Fable 5.1. For developers, the takeaway is a concrete recipe: co-train a small model with its harness, and it can win in a narrow domain. TL;DR Size: 30B total, 3B active (MoE). Base model context: up to 1M tokens. Sterling’s serving context: not disclosed. Runs on: the base model fits a single H100 80GB in BF16 (256K context). Sterling’s serving hardware: not disclosed. Performance: beats every high-quality system on the FrontierFinance leaderboard on quality per dollar (self-reported). Best: 58.9% on FrontierFinance (60.6% in high-effort mode), vs 55.9% for Claude Fable 5.1. Worst: 45.2% on the hardest third of FrontierFinance questions. What is Samaya Sterling? Samaya Sterling is a LoRA post-trained version of Nemotron 3.5 Lightning, tuned to work as a financial research agent. It plans research, calls finance tools, gathers evidence and hands findings to a separate report writer. According to Maithra Raghu’s X announcement, it started as a 2-person effort on agentic orchestration efficiency. How does the harness support a 3B-active model? Samaya team adapted its production finance harness for a small model. Detailed guidance was stripped from the system prompt and tool descriptions. Tools were merged to shorten the prompt. Two helper models manage context. A condenser shortens long tool results. A compactor summarizes the trace as it grows. A research budget caps how long the agent works. After that, the report writer answers from what was found. How was Sterling trained? Training used LoRA on Tinker, Thinking Machines Lab’s training API. It ran in 4 stages: SFT on synthetic traces: Samaya generated 968 synthetic analyst questions. The teacher, Kimi K3, answered each twice, and an LLM judge picked the better answer. The model learned to call 37 tools with valid arguments. SFT on the best of 8 strategies: Kimi K3 answered 1,819 expert-written questions using 8 research strategies. The best-graded answer per question became training data. Best-of-8 met 80% of essential rubric items, against about 66% for the single best strategy. After filtering, 1,157 questions remained. Offline GRPO: GRPO scored all 8 teacher answers against each other. No new generation was needed, so each step was cheap. Online GRPO: The model answered each question 8 times and learned from graded attempts. Truncated importance sampling corrected a known MoE sampler mismatch. How does Sterling score on FrontierFinance? FrontierFinance has 220 public queries scored against expert-written rubrics. The grader uses a 3-model judge panel. Samaya team states that no benchmark question was used in training. Checkpoints were chosen on 100 internal validation questions. Sterling scores 58.9% at $0.61 and 282 s per question. Fable 5.1 scores 55.9% at $6.89 and 268 s. Samaya’s own high-effort system scores 2.9 points lower at $1.81. Every stage added quality: 42.5% (base), 51.4%, 54.8%, 57.5%, then 58.9%. Median tool calls rose from 3 to 17. In high-effort mode, Sterling reaches 60.6% at $1.83 and 372 s. On the hardest third, it rose from 31.8% to 45.2%. The model itself costs $0.04 per question. Tools and helper models make up the rest of the $0.61. (function(){var f=document.getElementById("mtp-sterling-frame");window.addEventListener("message",function(e){if(e.source===f.contentWindow&&e.data&&e.data.type==="mtp-resize"){f.style.height=e.data.mtpHeight+"px";}});})(); How does Sterling compare? FrontierFinance figures for all columns come from the Samaya blog and are self-reported. Key Takeaways Sterling scores 58.9% on FrontierFinance, 3.0 points above Claude Fable 5.1. Cost per question drops 11x: $0.61 vs $6.89. 3B active parameters, trained with 2 SFT and 2 GRPO stages. Harness and model co-training drive the gains. All results are self-reported on Samaya’s own benchmark. Check out the technical blog, the FrontierFinance benchmark, the dataset and the grader repo. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Samaya AI Releases Sterling: A 3B-Active Finance Research Agent Model that Beats Claude Fable 5.1 on FrontierFinance at 11x Lower Cost appeared first on MarkTechPost.
Baseten has released joey-1, a 2B model from its in-house research lab, Base Labs. The joey-1 decision model returns a label and a calibrated probability instead of generated text. It targets high-volume routing, classification and guardrail checks. Baseten prices it at $0.01 per million input tokens, with open weights on Hugging Face under MIT. TL;DR Size: 2B parameters, post-trained from Qwen3.5-2B. 16,384-token max sequence (Baseten says 262k on dedicated deployments). Runs on: Baseten Model APIs, or self-hosted on 1 CUDA GPU (VRAM not disclosed). No official quantized builds listed. Performance: Top model under 4B on the Jev Decision Index 0.4 public split, per Baseten. Best: 63.30 on Jev Decision Index 0.4 (public split), ranked #37 of 230 models. Worst: Trails its 9B sibling elephant-1 by 8.94 points on the same split (63.30 vs 72.24). Bottom line: Best: 20 ms P50 decisions for $0.01 per 1M tokens. Worst: less general knowledge than its larger siblings. What is joey-1? joey-1 is a 2B decision model from Baseten’s Base Labs that answers typed questions in 1 forward pass. You send a state (text or images) and 1 or more questions. The model returns a probability for every option of every question. It does not write free-form text that your application must parse. It supports 3 question types: choice: pick 1 named option, such as which team owns a ticket. yes/no (noul in the API): returns the probability of yes. score: ordered levels, lowest first, such as urgency. joey-1 is the third member of a family. Baseten shipped mammoth-1 (27B) and elephant-1 (9B) last week. All 3 share training data and the /v1/systemone API, so switching is a 1-line change. How does joey-1 work? Base Labs post-trained Qwen3.5-2B with a full fine-tune of the language model. The vision encoder stays frozen. Full-attention layers use a block-bidirectional mask. All questions are answered from 1 shared-prefix prefill, so extra questions add little cost. Training details from the model card: Data: 6.21M decision examples. Mix: 58.3% core decision skills, 16.1% safety, 16.1% judging and expert domains, 9.5% public data. Objective: cross-entropy plus a Brier calibration term, with a trained “I don’t know” option. Recipe: 1 epoch, 43,390 steps, peak learning rate 1e-5, bf16. Hardware: 2 nodes of 8 H100 GPUs. Calibration lets you set a confidence threshold. Uncertain answers can escalate to elephant-1, mammoth-1 or a human. (function(){var f=document.getElementById('mtp-joey1-frame'); window.addEventListener('message',function(e){if(e.data&&e.data.type==='mtp-joey1-height'&&e.source===f.contentWindow){f.style.height=e.data.h+'px';}});})(); How fast and cheap is joey-1? Baseten reports a P50 end-to-end latency of 20 ms for an average 2,000-token request. That is about 2.5x faster than elephant-1 (49 ms). The blog’s TL;DR also claims 5x faster than comparable decision models, without naming them. Model APIs charge $0.01 per 1M input tokens; output and cached input are listed at $0.00. elephant-1 costs $0.035 and mammoth-1 costs $0.10. Baseten’s example: 10M support tickets a month at about 500 tokens each is 5B tokens. That is about $50 on joey-1, versus $500 on mammoth-1. How does joey-1 score on benchmarks? All scores below are self-reported on Baseten’s model card. On the Jev Decision Index 0.4 public split, joey-1 scores 63.30. That ranks #37 of 230 models and is the top score under 4B. The Jev Decision Index is a community leaderboard on Hugging Face. On the same split, Cloudflare’s 27B Clef scores 61.71 and Jev 1.14 scores 60.70. joey-1 trails elephant-1 in every area. It beats Jev 1.14 in 4 of 5 areas, including language, tools and retrieval. On Baseten’s replica of the Artificial Analysis Decision Index, joey-1 scores 62.9, versus 65.9 for mammoth-1. joey-1 vs. other small decision models *Jev 0.4 scores for all models come from Baseten’s joey-1 model card. **Latency figures are not comparable. Baseten measures end-to-end on its platform; Cloudflare reports its own Decision Index 0.2.1 run. How do you deploy joey-1? Model APIs: call decision-models/baseten-joey-2b-latest at /v1/systemone. Zero data retention is on by default. Dedicated deployments: Baseten says these unlock 262k context, fine-tuned weights and co-location with LLM workloads. Self-hosted: download from Hugging Face and run setup.sh. It needs 1 CUDA GPU and a PyTorch 2.8 / CUDA 12.8 image. You can fine-tune joey-1 on Baseten Training Jobs using the fine-tuning cookbook. joey-1 is also listed on OpenRouter at $0.01 per 1M input tokens. Key Takeaways joey-1 returns labels plus calibrated probabilities, not text. Baseten reports 20 ms P50 latency at $0.01 per 1M input tokens. 63.30 on Jev Decision Index 0.4 public: top model under 4B. Trails elephant-1 by 8.94 points; best as a first-line router. MIT weights on Hugging Face; self-host on 1 CUDA GPU. Check out the blog post, model card, model library page and fine-tuning cookbook. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Baseten Releases joey-1, a 2B Decision Model at $0.01 per 1M Tokens appeared first on MarkTechPost.
OpenAI is doubling usage limits across its paid plans, from Pro 100 to Pro 500. A planned cut to the Pro 200 quota was reversed after user pushback. More efficient models like Terra and Sol should push usable capacity even higher. The move comes just ahead of Anthropic's IPO, as both companies fight for B2B dominance in AI. The article OpenAI doubles usage limits on Pro plans as competition with Anthropic heats up appeared first on The Decoder.
Anthropic gave 50 experts an AI agent whose preferences were set by coin toss. The experts followed the agent's view up to 86 percent of the time on judgment calls, and most believed the conclusions were their own. The article Experts adopt AI agent's random opinions and think they're their own, Anthropic study finds appeared first on The Decoder.
