Meta FAIR Releases e-means: An Algorithm Beating k-means in 10x Fewer Epochs
AIvia MarkTechPost
Quick Read
Meta FAIR researchers have released e-means, a drop-in replacement for Lloyd’s k-means algorithm. The e-means clustering algorithm reaches the loss Lloyd’s k-means gets in 25 epochs in 1.4 to 2.6 epochs. It ships in FAISS 1.16.0 under faiss.contrib.e_means. For anyone building vector indexes, quantizers or data-curation pipelines, clustering is often the slowest offline step. e-means targets exactly that step. TL;DR Size: It is an algorithm. Tested from 100K to 10 billion points, up to 16.7M clusters. Runs on: CPU or GPU, with PyTorch tensors or NumPy. Paper GPU runs use H100s. Performance: Lower loss than Lloyd’s k-means in roughly one-tenth the epochs, across 24 dataset configurations (self-reported). Best: 10-45x GPU wall-clock speedup over Flash-KMeans at equal loss. Worst: 16M-centroid runs need 64 GiB of float32 accumulators, so centroids must be sharded across GPUs. What is e-means? e-means is a mini-batch k-means algorithm from Meta FAIR that updates centroids with exponential moving averages. How does e-means work? Every k-means variant keeps a sum S and a count N per cluster. The centroid is S divided by N. Methods differ only in how S and N change. Lloyd recomputes S and N from the full dataset each iteration. MacQueen (1967) keeps adding batch statistics forever, so stale assignments never leave. e-means decays old statistics at rate α: S ← (1−α)S + αs, and the same for N. Older assignments fade, so centroids track recent data. The research team expa Lloyd is a corner case: full batch with α = 1. On DEEP, that corner sits 1.8% above the best loss. The team tested plain gradient descent as an alternative. At its best setting, it ended 1.6% (DEEP) and 0.4% (SIFT) above EMA. The authors suspect EMA wins because centroids always stay inside the hull of their assigned points. What replaces hyperparameter tuning? Two fitted scaling laws set batch size B and initial step α0 from dataset size n, cluster count k and epochs T. On held-out datasets, median regret was 0.07% for the step law. The step then follows a cosine schedule from α0 to 0.1α0. What else changed? Dead-cluster revival: clusters below a size threshold are revived by splitting a large cluster. Finalization: training ends on each cluster’s exact last-epoch mean. Fused GPU kernels: nearest-centroid assignment is fused, so saved epochs become saved seconds. [EMBED: e-means interactive explainer] How fast is e-means in benchmarks? All numbers below are self-reported by the authors. Epochs: e-means beats Lloyd’s 25-epoch loss in 1.4 to 2.6 epochs. GPU: 10-45x wall-clock speedup against Flash-KMeans. CPU: 8-15x speedup against Faiss k-means. VIBE benchmark: all 11 embedding datasets needed 1.6 to 2.4 epochs. Billion scale: DINO10B, 10 billion 1,024-dim embeddings, split into 16.7M clusters in 2.5 epochs. The release thread shows a 1-CPU-thread demo. Lloyd took 4.39 s for 25 epochs. e-means took 0.43 s for 2.2 epochs. One Lloyd step touched 100,000 points (0.176 s). One e-means step touched 512 points (0.001 s). Mini-batch noise also helps escape bad local minima. On a 5-blob toy, Lloyd stayed stuck on 46% of 200 starts. e-means stayed stuck on 0%. Revival matters at large k. On DEEP with 16,384 clusters, proportional splitting left 47 dead clusters (0.3%). With revival off, 4,313 clusters (26.3%) died. How does e-means compare with other k-means options? Key Takeaways e-means swaps Lloyd’s full-batch update for EMA mini-batches. Matches 25 Lloyd epochs in 1.4 to 2.6 epochs. 10-45x faster on GPU, 8-15x on CPU (self-reported). No tuning: scaling laws set batch and step size. Available now in FAISS 1.16.0. Check out the paper, the blog post and the FAISS code. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Meta FAIR Releases e-means: An Algorithm Beating k-means in 10x Fewer Epochs appeared first on MarkTechPost.
Meta FAIR researchers have released e-means, a drop-in replacement for Lloyd’s k-means algorithm. The e-means clustering algorithm reaches the loss Lloyd’s k-means gets in 25 epochs in 1.4 to 2.6 epochs. It ships in FAISS 1.16.0 under faiss.contrib.e_means. For anyone building vector indexes, quantizers or data-curation pipelines, clustering is often the slowest offline step. e-means targets exactly that step.
TL;DR
Size: It is an algorithm. Tested from 100K to 10 billion points, up to 16.7M clusters.
Runs on: CPU or GPU, with PyTorch tensors or NumPy. Paper GPU runs use H100s.
Performance: Lower loss than Lloyd’s k-means in roughly one-tenth the epochs, across 24 dataset configurations (self-reported).
Best: 10-45x GPU wall-clock speedup over Flash-KMeans at equal loss.
Worst: 16M-centroid runs need 64 GiB of float32 accumulators, so centroids must be sharded across GPUs.
What is e-means?
e-means is a mini-batch k-means algorithm from Meta FAIR that updates centroids with exponential moving averages.
How does e-means work?
Every k-means variant keeps a sum S and a count N per cluster. The centroid is S divided by N. Methods differ only in how S and N change.
Lloyd recomputes S and N from the full dataset each iteration.
MacQueen (1967) keeps adding batch statistics forever, so stale assignments never leave.
e-means decays old statistics at rate α: S ← (1−α)S + αs, and the same for N.
Older assignments fade, so centroids track recent data. The research team expa Lloyd is a corner case: full batch with α = 1. On DEEP, that corner sits 1.8% above the best loss.
The team tested plain gradient descent as an alternative. At its best setting, it ended 1.6% (DEEP) and 0.4% (SIFT) above EMA. The authors suspect EMA wins because centroids always stay inside the hull of their assigned points.
What replaces hyperparameter tuning?
Two fitted scaling laws set batch size B and initial step α0 from dataset size n, cluster count k and epochs T. On held-out datasets, median regret was 0.07% for the step law. The step then follows a cosine schedule from α0 to 0.1α0.
What else changed?
Dead-cluster revival: clusters below a size threshold are revived by splitting a large cluster.
Finalization: training ends on each cluster’s exact last-epoch mean.
Fused GPU kernels: nearest-centroid assignment is fused, so saved epochs become saved seconds.
[EMBED: e-means interactive explainer]
How fast is e-means in benchmarks?
All numbers below are self-reported by the authors.
Epochs: e-means beats Lloyd’s 25-epoch loss in 1.4 to 2.6 epochs.
GPU: 10-45x wall-clock speedup against Flash-KMeans.
CPU: 8-15x speedup against Faiss k-means.
VIBE benchmark: all 11 embedding datasets needed 1.6 to 2.4 epochs.
Billion scale: DINO10B, 10 billion 1,024-dim embeddings, split into 16.7M clusters in 2.5 epochs.
The release thread shows a 1-CPU-thread demo. Lloyd took 4.39 s for 25 epochs. e-means took 0.43 s for 2.2 epochs. One Lloyd step touched 100,000 points (0.176 s). One e-means step touched 512 points (0.001 s).
Mini-batch noise also helps escape bad local minima. On a 5-blob toy, Lloyd stayed stuck on 46% of 200 starts. e-means stayed stuck on 0%.
Revival matters at large k. On DEEP with 16,384 clusters, proportional splitting left 47 dead clusters (0.3%). With revival off, 4,313 clusters (26.3%) died.
How does e-means compare with other k-means options?
Key Takeaways
e-means swaps Lloyd’s full-batch update for EMA mini-batches.
Matches 25 Lloyd epochs in 1.4 to 2.6 epochs.
10-45x faster on GPU, 8-15x on CPU (self-reported).
No tuning: scaling laws set batch and step size.
Available now in FAISS 1.16.0.
Check out the paper, the blog post and the FAISS code. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
The post Meta FAIR Releases e-means: An Algorithm Beating k-means in 10x Fewer Epochs appeared first on MarkTechPost.
Fish Audio has released Live 3, a full-duplex voice model built for real-time voice agents. The Fish Audio Live 3 full-duplex voice model keeps listening while it speaks. That lets a caller pause, say “mm-hmm,” or change their mind without breaking the call. For developers, the pitch is simple. Turn-taking moves out of hand-tuned pipeline rules and into the speech model. TL;DR Product: A full-duplex voice model built for real-time voice agents Runs on: Hosted Fish Agent Platform and Fish Audio API; on-premises for enterprises. Performance: Fish Audio shows behavior demos, but publishes no benchmark scores. Best: Can decide to respond in as little as 186 ms for short reactions like “mm-hmm.” What is Fish Audio Live 3? Live 3 is a full-duplex voice model from Fish Audio that listens and speaks at the same time. The core argument is that speed and timing are different problems. An agent can reply fast and still speak at the wrong moment. A pause after “What time do you open?” usually ends a turn. A pause in the middle of a phone number does not. How does Live 3 handle turn-taking? Most voice agents split recognition, reasoning, speech generation and turn-taking across separate pipeline stages. Fish Audio team states that Live 3 moves conversational timing into the speech model. It was trained on real human conversations to learn when to acknowledge, wait, respond or stop. They framed the old approach as piling on “more rules and more thresholds.” Fish Audio instead wants the model to learn timing from data. A separate reasoning layer still holds the agent’s instructions and tools. The Fish Audio team walks through 3 behaviors: Acknowledging without taking over. A caller reads “415… 555… 0132.” Live 3 may say “okay, 415” between chunks, then wait. Telling a backchannel from an interruption. A caller saying “yeah” lets the agent continue. “Wait, actually, I just need Thursday” makes it stop. Waiting through corrections. For “two pizzas… no, three,” Live 3 waits instead of logging the wrong order. How fast is Live 3? Fish Audio states Live 3 can decide to respond in as little as 186 ms. That figure covers short reactions such as “mm-hmm,” “okay” and quick confirmations. The full answer is prepared in parallel. This is a floor for small reactions, not time-to-first-audio for a complete reply. Fish Audio has not published a latency distribution. Third-party tracker OrcaRouter also reports no Artificial Analysis score for Live 3 as of November 11. (function(){window.addEventListener('message',function(e){var d=e.data;if(!d||d.mtpEmbed!=='fl3')return;var f=document.getElementById('mtp-fl3-frame');if(f&&d.height){f.style.height=d.height+'px';}});})(); How do developers deploy Live 3? Live 3 is available on the Fish Agent Platform, through the Fish Audio API, and on-premises for enterprises. The Agents platform is in public beta. Agents expose a REST surface under /v1/agent/*. Browser sessions run through @fishaudio/agent-client or @fishaudio/agent-react. Agents can answer phone numbers, connect over WebRTC, or sit in a website widget. The Live 3 page lists 2,000,000+ library voices, including cloned voices. A pending docs pull request shows Live 3 enabled with voice.engine: "live3". With it on, speech recognition is fixed to fish_audio:transcribe-stream-1. Fish Audio’s current docs say that model also classifies overlaps as interruptions or acknowledgements. How much does Live 3 cost? The Live 3 page lists $0.035 per minute, down 50% from $0.07, until December 31, 2026. It says the plan includes Live 3. The pricing docs say the $0.07 list rate covers STT, TTS, basic LLMs, knowledge retrieval and tool calls. Phone calls add $0.01 per minute. A number costs $1.20 per month. New accounts get a $5 trial balance. How does Live 3 compare with other full-duplex voice models? ElevenLabs scores come from Artificial Analysis, as reported by ElevenLabs. OpenAI and Google figures are self-reported. Key Takeaways Live 3 is Fish Audio’s full-duplex voice model for phone and web agents. It learns turn-taking from real conversations instead of silence thresholds. Short reactions can start in as little as 186 ms. Agents cost $0.035 per minute until December 31, 2026, then $0.07. English only, with no published benchmark scores yet. Check out the launch post, try it at fish.audio/live3, and see the Agents docs. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Fish Audio Releases Live 3, a Full-Duplex Voice Model That Reacts in 186 ms appeared first on MarkTechPost.
OpenAI is opening up ChatGPT as a business platform: through a new Business Hub, companies can claim their profile, add product data, and track how users interact with them. "Business Agents" let eligible US e-commerce providers talk to customers directly inside ChatGPT. The article OpenAI turns ChatGPT into a business directory with AI agents that handle customer questions appeared first on The Decoder.
