Topic

Mixture of Experts (MoE)

All digests tagged Mixture of Experts (MoE)

Suggested route

Topic journey

Move from earlier context through the available material to the newest signals. This route is a reading aid, not a difficulty ranking.

Start here

Thinking Machines Lab drops Inkling & Meta’s Muse Spark 1.1 Jul 17, 2026 · 39:02 This episode analyzes Thinking Machines' Inkling (975B MoE), Meta’s cost-efficient Muse Spark 1.1, and Anthropic’s J-space paper, which reveals internal model processing for enhanced AI safety.

Recent signals

Lecture 118: GSQ-RCO: Better Non-Uniform GGUF Quantization Oct 16, 2026 · 57:13 GSQ and RCO are complementary methods for accurate low-bit LLM quantization. GSQ uses Gumbel-Softmax sampling for stochastic optimization, while RCO uses Riemannian optimization to allocate precision across a model under a fixed size budget. Preferences Over Benchmarks: Model Routing — Archana Kamath & Tyler Gillam, DigitalOcean Aug 22, 2026 · 15:54 Model routing argues that there is no single best LLM. By using a purpose-built Mixture of Experts model, developers can dynamically select the optimal AI model per request based on cost, latency, and task type.

GPU MODE

Lecture 118: GSQ-RCO: Better Non-Uniform GGUF Quantization

GSQ and RCO are complementary methods for accurate low-bit LLM quantization. GSQ uses Gumbel-Softmax sampling for stochastic optimization, while RCO uses Riemannian optimization to allocate precision across a model under a fixed size budget.

Read article

AI Engineer

Preferences Over Benchmarks: Model Routing — Archana Kamath & Tyler Gillam, DigitalOcean

Model routing argues that there is no single best LLM. By using a purpose-built Mixture of Experts model, developers can dynamically select the optimal AI model per request based on cost, latency, and task type.

Read article

IBM Technology

Thinking Machines Lab drops Inkling & Meta’s Muse Spark 1.1

This episode analyzes Thinking Machines' Inkling (975B MoE), Meta’s cost-efficient Muse Spark 1.1, and Anthropic’s J-space paper, which reveals internal model processing for enhanced AI safety.

Read article