This page collects what the sources on blogs to watch have published recently and groups them by subject rather than by publisher. It is rebuilt every hour, so a post usually appears here within an hour of going up. Nothing on this page is required reading.
Grouping by subject is the point. A reader gives you 45 separate streams and leaves you to notice that four companies wrote about the same scheduling problem this week; this page puts those four posts under one heading. Topics come from the filter terms already listed on the blogs-to-watch page, so the vocabulary is the course's.
How a post is filed. Each post is scored by keyword against every topic, using its title and its summary, and it is filed under the topic it scores highest against. A term that names a subject on its own counts for more than one that merely co-occurs with it, and a match in the title counts for three times a match in the summary. Any second topic a post also matches is shown as a label beside it. The method is keyword matching rather than a model, which makes it predictable, cheap, and occasionally wrong.
The page shows the last 90 days, which is 685 posts of the 710 the index holds. Each topic lists its 25 most recent posts and links to the rest. Widen the window with 90 days or 120 days, which is as far back as the index goes.
How an agent loop is built, driven, and kept on task.
You can now run Claude Managed Agents with Chat SDK . Claude Managed Agents handles the agent loop server-side, including the model, tools, session state, and sandboxed web research. That means you can ship a Slack research bot built on…
Learn why standard OCR falls short for KYC compliance and how agentic document extraction delivers the field-level accuracy AML regulations require.
Learn why single-pass extraction fails and how deep extraction uses agentic verification to deliver production-grade accuracy on complex documents.
Agentic document extraction uses AI reasoning and visual grounding to accurately process complex documents without templates. Learn how it works.
Agentic document processing uses AI agents to autonomously handle document workflows end to end. Learn how it works and where to start.
OCR for receipts breaks when layouts vary and rules pile up. Discover how agentic OCR reconstructs line items, totals, and structured data for automation.
Discover how agentic OCR transforms document processing with multimodal reasoning, self-correction loops, and template-free automation.
LlamaIndex is a simple, flexible framework for building knowledge assistants using LLMs connected to your enterprise data.
Discover why mortgage banking document automation fails at workflow handoffs and how AI document extraction improves speed, accuracy, and compliance.
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
Writing effective tools for AI agents—using AI agents
On the the engineering challenges and lessons learned from building Claude's Research system
Discover how Anthropic approaches the development of reliable AI agents. Learn about our research on agent capabilities, safety considerations, and technical framework for building trustworthy AI.
Tunix is Google’s new JAX-native post-training library designed to eliminate TPU idling bottlenecks when training multi-turn, tool-using LLM reasoning agents. It maximizes hardware throughput by combining highly concurrent, asynchronous…
To resolve the scaling bottlenecks and runtime errors caused by monolithic system prompts, engineering teams should treat prompts as build artifacts by modularizing instructions into reusable templates. By running these modular "skill…
Learn how to power DeepSeek Harness with Baseten Model APIs and run open models like Kimi K3, GLM 5.2, and DeepSeek V4 Pro in under 5 minutes.
NVIDIA’s Nemotron 3.5 Lightning, now on Baseten, delivers high-throughput, efficient reasoning for faster and more accurate agentic workflows.
Gemma 4 on Cerebras delivers the fastest multimodal inference—1,500+ TPS for real-time image understanding, agentic workflows, and document AI.
Creative teams produce more assets than ever, but fragmented tools and manual context transfer slow production. This post shows how to build a reusable agent harness with Amazon Quick and fal, connected through the Model Context…
Managed Deep Agents and LLM Gateway hit public beta, plus Deep Agents v0.7, Tuned Evaluators, Bring Your Own Cloud on AWS, and LangSmith Engine upgrades.
AMD shipped ROCm 1.0 in April 2016: an open-source GPU compute stack built around a C++ compiler and a GPU programming language called HIP, aimed at high-performance computing. A decade later, the same platform trains and serves…
Navigation enables a robot to turn perception and motion into purposeful autonomy. Unlike locomotion, which produces stable movement, navigation must be used to...
93 further posts in this topic are not shown. Open the topic on its own to read them.
Agents that read, write, and review code, and the benchmarks that grade them.
Explore Claude's breakthrough performance on SWE-Bench, demonstrating advanced software engineering capabilities and code generation accuracy. Learn about our technical evaluation methods.
We put Poolside’s new Laguna S 2.1 model to the test, tasking it with a repository-scale transformation of the open-source game Hypersomnia.
The AI SDK harness layer now supports Cursor through the official @ai-sdk/harness-cursor adapter. The harness layer lets your application run different coding agents through the same HarnessAgent interface, so you can switch agents…
Managing library updates can be tedious at times. Learn how the GitHub Copilot app can handle this type of repetitive task. The post GitHub Copilot app for Beginners: Automate Dependabot pull request triage appeared first on The GitHub…
Discover how loveholidays uses OpenAI Codex to make software development accessible across the business, helping teams turn ideas into products faster.
Qwen 3.8 Flash from Alibaba is now available on AI Gateway. It takes text and images as input, serves a context window of 1 million tokens, and can return up to 65k tokens in a response. Alibaba recommends it for coding, tool use, and…
Use the Admin plugin for ChatGPT Work and Codex to analyze workspace usage, manage members and permissions, adjust limits, and act on admin requests.
The key skill required to make productive use of coding agents is being able to confidently instruct them on how to make changes and then confidently verify that those changes have been applied in the correct way. Sometimes this…
With a fixed deadline and design resources committed elsewhere, Stampli used Codex and ChatGPT Work to compress weeks of launch production into days.
If you’re juggling multiple Copilot sessions, use the My work pane to track what's in flight, what's done, and what's next. The post GitHub Copilot app for Beginners: Managing your work appeared first on The GitHub Blog .
Asana used OpenAI Codex to replace an outdated testing system in two weeks, completing work expected to take five years for about $12K.
Congrats to the team!
OpenHands joins the NVIDIA-led Open Secure AI Alliance to advance open, inspectable infrastructure for building and running secure AI agents.
Claude Code vs Cursor compared on interface, autonomy, models, context, execution, and cost, plus where OpenHands fits.
LTM has partnered with Cognition to deploy Devin, the AI software engineer, across its global client base and cybersecurity practice serving over 260…
Say you forked the OpenHands app twelve months ago and never merged from upstream. You would now be 2,600 merged PRs behind, including 866 bug fixes you do not have.
Devin, built by Cognition, is an AI software engineer: it plans, writes, tests, and ships code semi-autonomously. With Outposts, Devin can now run its work in Modal sandboxes.
Anhang and Yun have gone deep on everything that keeps software running once it ships, and we are excited to bring their work on automations into Devin.
Someone's subsidizing your coding agent. Aperture shows whether it's you.
Cognition’s entire platform is now FedRAMP Class D (High) In-Process and listed on the FedRAMP Marketplace, giving engineering teams working in federal…
The Devin Security Vulnerability Remediation Program helps organizations clear their vulnerability backlog and set up continuous remediation.
Devin Security Swarm finds vulnerabilities across the codebase, validates exploitability at runtime, and ships remediation PRs.
Introducing Devin Fusion: a hybrid-model harness that keeps frontier-level coding intelligence while cutting costs with sidekick agents and dynamic…
Engineering leaders want to know how much value AI is actually providing. We built a system that measures the number of human-equivalent hours of Devin's…
Introducing the AI Productivity Guarantee for enterprise customers. If Devin delivers less engineering value than you’re paying for, Cognition will fund…
1 further post in this topic is not shown. Open the topic on its own to read them.
Tool interfaces, protocols, memory, and what the model is given to work with.
LlamaIndex is a simple, flexible framework for building knowledge assistants using LLMs connected to your enterprise data.
Claude can now discover, learn, and execute tools dynamically to enable agents that take action in the real world. Here’s how.
Learn how code execution with the Model Context Protocol enables agents to handle more tools while using fewer tokens, reducing context overhead by up to 98.7%.
Discover how Anthropic builds AI agents with practical capabilities through modular skills, enabling them to handle complex real-world tasks more effectively and reliably.
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
Claude Desktop Extensions: One-click MCP server installation for Claude Desktop
A blog post for developers, describing a new method for complex tool-use situations
Explore how Anthropic enhances AI systems through advanced contextual retrieval methods. Learn about our approach to improving information access and relevance in large language models.
Agent Plugins 1.0.0 is a new, vendor-neutral directory specification—backed by Google, Amazon, Microsoft, and others—for packaging Agent Skills and MCP servers into a single portable unit. By standardizing the manifest (plugin.json) and…
The 2026-07-28 Model Context Protocol (MCP) specification replaces legacy stateful constraints with a fully stateless core, enabling cloud-native horizontal scaling, serverless deployments, and standard round-robin load balancing. This…
To prevent context window bloat and reduce token consumption, Genkit Go introduces Agent Skills based on a progressive disclosure architecture. Developers can package specialized instructions, scripts, and references into modular…
Learn how Cerebras redesigned engineering interviews for the AI era, evaluating AI collaboration, engineering judgment, verification, and real-world skills.
Ling 3.0 Flash Fin from Inclusion AI is now available on AI Gateway and free to use through September 25. Ling 3.0 Flash Fin is a finance-focused version of Ling 3.0 Flash . It has a 256K-token context window, produces up to 32K output…
Learn how Natera built an automated voice agent on Amazon Bedrock AgentCore that lets patients book mobile phlebotomy appointments through natural conversation. The post covers the dual-WebSocket bridge, event-driven latency masking,…
Lovable is branching out from AI-powered web app creation and into MCP-powered ‘capabilities’. We talk to CTO Fabian Hedin.
Wiki memory uses an agent to compress raw data into a persistent, file-based knowledge base. How it differs from RAG, real examples, and when to use it.
GLM 5.3 Flash from Z.ai is now available on AI Gateway as zai/glm-5.3-flash . GLM 5.3 Flash is a multimodal coding model with a 1M-token context window. It accepts both text and image inputs and supports function calling, structured…
Protecting capital in today's markets requires immense speed and precision. A financial analyst preparing a deal memo works across licensed market data, internal models, and confidential client files. General-purpose AI lacks the…
Vercel Connect is now generally available on all plans and in v0 . Instead of storing long-lived provider secrets, your code requests short-lived, scoped tokens at runtime. Deployments authenticate with their existing Vercel OIDC…
Learn how to build a customizable, smart-caching knowledge management system on AWS that captures and delivers institutional (tribal) knowledge through a voice-first AI avatar. The accelerator uses Amazon Bedrock Knowledge Bases for…
Anthropic’s best AI model struggles to attract users as cheaper tools thrive A few interesting numbers in this FT story gathered from "people with knowledge of the matter": Anthropic's "annualized revenue" for July is up to $65bn - it…
NVIDIA Holoscan is a platform for building real-time AI applications at the edge, from medical imaging to robotics. HoloHub is its companion repository: a...
the Memory crunch continues - Moore’s Law reversed to 2007 levels
In April 2025, Google released a protocol for agent-to-agent communication. Within three months, Google had donated it to the Linux Foundation...
12 further posts in this topic are not shown. Open the topic on its own to read them.
Containment, permissions, and the attacks that target agents.
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
Learn how Claude Code's new sandboxing feature protects developers with filesystem and network isolation, reducing permission prompts and increasing user safety.
Breaking Claude Code Opus 5 Auto Mode Anthropic are putting a great deal of faith in Claude Code's auto mode for protecting their coding agent users against prompt injection attacks. They recently made that the default and have made…
The Vercel Security Dashboard is now generally available on all plans, giving you one place to see your security posture across every account and project. You can access the Security Dashboard in the UI or run vercel security check in…
Summary Two vulnerabilities affecting Next.js were disclosed in the August 2026 Security Release. Next.js applications hosted on Vercel are protected and require no customer action. Next.js August 2026 vulnerabilities Next.js disclosed…
Vercel Sandbox now runs globally, starting with four regions: iad1 (Washington, D.C.), sfo1 (San Francisco), cle1 (Cleveland), and cdg1 (Paris). iad1 remains the default. Support for all Vercel regions is coming soon. Choose a region…
Welcome to the first Cloud CISO Perspectives for August 2026. Today, Chris Betz explains why the AI era makes it more important than ever to lean into security fundamentals. As with all Cloud CISO Perspectives, the contents of this…
Research: smolmachines / smolvm as a sandbox for untrusted Python & JavaScript I tasked Claude Fable 5 running in Claude Code for web with the following research task: Put https://smolmachines.com through its paces as a fast secure…
My hypothesis is that there is a new opportunity for Extensible Software on the web . LLMs radically lower the cost of authoring extensions, and modern sandbox primitives lower the deployment cost and provide good security boundaries.…
OpenAI launches an initiative to strengthen democratic oversight of AI in national security, supporting government institutions with tools, training, and expertise.
See how the open source projects in Session 4 of the GitHub Secure Open Source Fund combined AI-assisted workflows, maintainer expertise, GitHub security tools, expert guidance, and funding to improve project security. The post What 50…
For organizations deploying AI agents at scale, there’s often a critical divide between structured and unstructured data. While large language models (LLMs) excel at parsing text documents, emails, and PDFs, they can struggle when…
Most SaaS teams shipping agentic features focus on prompt safety and API-layer filtering. But effective AI agent security also depends on...
Compare 8 automated vulnerability remediation tools for 2026 on fixes, prioritization, integrations, and deployment control.
OpenAI accidentally hacked HuggingFace during a security exercise. Learn what they got wrong, and how you can avoid the same mistakes.
Building performant sandbox systems goes way beyond the initial container boot. Here, we unpack what that means, and discuss some tools to help you manage the entire lifecycle.
Together AI has earned ISO 27001:2022 certification, validating our commitment to enterprise-grade security for production AI workloads.
One step further into the power politics of frontier AI systems.
Measuring whether an agent works, and keeping it working.
The most comprehensive document extraction benchmark: 14 systems scored on accuracy, completeness, grounding, and cost across 370 enterprise documents.
Learn what impacts OCR accuracy, how it’s measured, and how to improve performance in real-world document processing workflows.
Discover how OCR for invoices streamlines finance operations, automates data extraction, reduces errors, and speeds up accounts payable with LlamaParse.
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
Demystifying evals for AI agents
Moving live voice agents from demo to production requires rigorous, automated testing to handle the unpredictability of real multi-turn conversations. ADK now provides native live evaluation, allowing developers to test graph-based…
Agent Platform's evaluation service is now generally available, providing developers with a unified engine to measure agent quality consistently across local development experiments and live production traffic. You can evaluate agents…
Stop autoresearch loops from “cheating” by enforcing strict evaluation, isolating experiments, and designing metrics that prevent shortcuts and false gains.
Deep dive into self-improving evaluators in LangSmith, motivated by the rise of LLM-as-a-Judge evaluators plus research on few-shot learning and aligning human preferences.
LangSmith's homepage is now organized into Observability, Evaluation, and Prompt Engineering. Learn why we organized the homepage like this. Plus, see our latest Resource Tags updates.
A recap of Sai Srirampur's POSETTE 2026 talk on storage performance in Postgres, with a local NVMe vs. EBS benchmark and the production setup behind it.
To satisfy the demands of enterprise-grade agentic AI applications, underlying vector databases often struggle to scale effectively as modern use cases can scale to billions of vectors. As a fully managed PostgreSQL-compatible database…
Reinforcement learning (RL) for large language models (LLMs) alternates between two phases: generation (rollout), where the current policy produces responses, and training, where those responses are used to update the policy. In verl,…
XGBoost (Extreme Gradient Boosting) is an open-source library that implements gradient-boosted decision trees, an ensemble method that builds an additive sequence of trees where each new tree is fit to the gradient of the loss left by…
Today, we are releasing OfficeQA Pro V2, a new benchmark designed to evaluate whether...
We ran 900 DeepSWE rollouts on DeepSeek-V4 Flash and GPT-5.6 Luna. Luna leads pass@1 by 14 points; DeepSeek delivers 4.8x the solves per dollar.
A benchmark on how a single analytical query behaves in Redpanda SQL as the dataset and the cluster grow.
We ran 452 DeepSWE rollouts on Kimi K3 and Claude Fable 5. Fable leads pass@1 by 1.4 points; Kimi K3 wins pass@4 and delivers 2.8x the solves per dollar.
We moved low-level I/O execution off application cores to dedicated networking cores using Seastar’s new asymmetric_io_uring backend. Explore the architecture design, trade-offs, and benchmark results.
Mike Shaver joins Tailscale to scale engineering quality.
How vLLM maintains production quality with extensive CI across diverse accelerators, nightly performance benchmark and accuracy evaluation, and a two-week release process.
Reliability numbers are easy to publish. We break down what 99%, 99.9%, and 99.99% uptime actually require, the failure domains each tier has to survive, and the questions to ask any inference provider before you commit.
See how Together AI is improving production GPU clusters with passive health checks, node repair, stronger Slurm reliability, OIDC, and startup scripts.
6 further posts in this topic are not shown. Open the topic on its own to read them.
Serving engines, request paths, and end-to-end latency and throughput.
Access gateways and serving gateways explained, and how they handle identity, tenancy, limits, and metering for teams using and serving AI models.
DeepSeek V4 Pro 0813 is a 1.7T-parameter open frontier model under the MIT license and is available for inference today on Baseten model APIs.
Baseten is proud to power the inference behind You.com's search and answer stack.
At Hot Chips 2026, Cerebras details CS-4 and Nexus, plus the CS-5 and CS-6 roadmap for faster, more efficient frontier AI inference.
Discover how faster AI inference helps cybersecurity teams improve threat detection, AI security, validation, and response without sacrificing speed.
Cerebras and Upstage bring ultra-fast AI inference to Korea, delivering up to 2,000 tokens per second for real-time enterprise AI applications.
Serving automatic speech recognition (ASR) models at scale is costly when each request uses only a fraction of a GPU. Learn how NVIDIA CUDA Multi-Process Service (MPS) with NVIDIA Triton Inference Server on Amazon EC2 GPU instances cuts…
How our new Rust driver load-balances DynamoDB-style requests across a ScyllaDB cluster, and how we extended Latte to measure its performance
Today, the Qwen team open-sourced Qwen3.8-Flash-Next, a multimodal MoE model and an early preview of the Qwen4 architecture. It plays the same role for Qwen4 that Qwen3-Next played for Qwen3.5. The Ga...
Jalapeño is a custom inference chip from OpenAI that delivers faster, more power-efficient AI inference, with higher throughput and lower latency for modern models.
With ScyllaDB, a small engineering team could focus on building their product instead of battling their databases.
Error parsing meta tag attribute “keywords”: No content.
We implement a native sharded weight transfer engine in vLLM utilizing Ray Direct Transport (RDT), achieving weight transfer for the Kimi K2 model in BF16 on 48 8xH100 nodes in 7.53s
IsoExec unifies numerical execution across SkyRL's vLLM and Megatron runtimes, reducing the average rollout-versus-training logprob difference below 1e-6 on Qwen3.5-35B-A3B with 25% overhead.
Nowadays, State-of-the-Art (SOTA) models are getting much bigger and reloading the model service after a crash is very expensive. Therefore, we are introducing the Weight Cache Daemon, a persistent GP...
Research: A shot-scraper-style JSON API on Bun 1.4's new Bun.WebView Today saw the long awaited release of Bun 1.4 , the first stable version since the infamous Rust rewrite a few months ago . Interestingly, the Rust rewrite was…
Distributed Layerwise Offload shards and streams DiT weights across devices, serving a measured 124 GB Cosmos3 model on 64 GB HBM and estimating a path toward 200B+ models.
Sizing the DSpark draft-verification budget from per-request confidence instead of verifying every drafted token, so one configuration holds the throughput/latency frontier from batch size 1 to 256.
Day-0 vLLM support for Qwen3.8-2.4T-A95B: a 2.4-trillion-parameter hybrid MoE model served out of the box, with FP8/BF16 checkpoints plus NVFP4 and MXFP4 quantized weights, and co-developed kernels on NVIDIA and AMD hardware.
Cloudflare for Government achieves FedRAMP Class D (High) Certified status. We also announce our commitment to pursue DoD IL4 authorization. Cloudflare brings world-class security, performance, and developer products to the public sector.
How vLLM serves NVIDIA Nemotron 3.5 Lightning with OpenAI-compatible APIs, speculative decoding, and BF16/NVFP4 checkpoints across NVIDIA GPUs and edge systems.
Decode Context Parallelism (DCP) in vLLM shards KV cache across GPUs by sequence dimension, enabling 3× higher throughput on long-context agentic workloads compared to standard tensor parallelism.
25 further posts in this topic are not shown. Open the topic on its own to read them.
Reusing computed attention state across requests and turns.
How llm-d stitches Gateway, EPP, KV-cache, P/D proxy, and vLLM into one OpenTelemetry trace, and instruments the scheduling decisions that metrics alone cannot explain.
Elastic build machines now consider Turborepo cache hits when deciding whether to use a smaller build machine. A warm-cache build no longer triggers a downgrade. A warm-cache build can use less CPU and memory than the same build with a…
When load balancing or serving topology separates a request from its cached prefix, llm-d can move the KV from a peer instead of recomputing it.
Cost-efficient, responsive GLM-5.2 agentic inference on H200 with llm-d, using KV-cache-aware routing, tiered prefix-cache offloading, and multi-token prediction.
How requests are batched, placed, and split across a fleet.
Learn how AI document classification automates document sorting and routing at scale, and what separates systems that perform on real documents from those that don
Cut parsing costs without losing accuracy. Parse Gateway routes each PDF page to the right tier based on complexity, from free LiteParse to advanced OCR.
Google Cloud API Gateway now offers a model routing feature in Public Preview, allowing developers to dynamically route traffic to models like Gemini, Claude, or OpenAI OSS-GPT without hardcoding endpoints or managing open-source…
Real-time AI agents break traditional request-response load balancing paradigms because they rely on long-lived, stateful bidirectional streams that obscure true server capacity. To solve this, developers must implement…
Python projects can now use routing rules to set response headers or rewrite requests to internal paths, including apps built with FastAPI, Django, and Flask. The Vercel CDN evaluates rules before requests reach your application, so…
Chat SDK's Slack adapter now supports Slack Enterprise Grid . Bots installed org-wide work across every workspace, with correct token resolution, tenant-scoped caches, and event retry deduplication. The adapter now stores org-wide…
We ran 904 DeepSWE rollouts on GLM-5.3 and GPT-5.6 Sol. Sol leads pass@1 by 3.7 points; GLM-5.3 wins pass@4 at half the cost, and a GLM-first cascade hits 85.9%.
We ran 904 DeepSWE rollouts on GLM-5.3 and Claude Fable 5. A tie on pass@1, but GLM-5.3 wins pass@4 and costs 5.4x less: \$3.99 per rollout vs. \$21.63.
Glean CEO Arvind Jain explains why model routing helps control AI costs for organizations, and how human feedback loops at scale improve its routing systems.
We ran 904 DeepSWE rollouts on DeepSeek V4 Pro 0813 and GPT-5.6 Sol. Sol leads pass@1 by 10 points at 35x the cost; Pro wins pass@4, and a Pro-first cascade hits 83.0%.
The llm-d router's default configuration is built on token-aware routing: keep each request on the cache-warm endpoint until a calibrated token-load limit is exceeded, then route by load alone, sustaining 2-3x round-robin throughput…
We ran 904 DeepSWE rollouts on DeepSeek V4 Pro 0813 and Claude Fable 5. Fable leads pass@1 at 90x the cost; Pro wins pass@4, and a Pro-first cascade hits 82.7%.
GPU utilization can read healthy while your queue backs up, and a new replica takes minutes to warm. Here's how to pick autoscaling metrics, tune scale-up/down windows, and budget for cold starts on dedicated inference.
The three-part resource model behind Together AI Dedicated Model Inference—endpoints, deployments, configs—and how capacity-aware routing ties them together.
We ran 904 DeepSWE rollouts on Kimi K3 and GPT-5.6 Sol. Sol leads pass@1; Kimi K3 wins pass@4 at 2.8x the solves per dollar, and routing between them reaches ~85.6%.
ScyllaDB successfully mitigated performance-degrading connection storms by optimizing caching, throttling, and password hashing to achieve a 1000x reduction in tail latency
How we turned to the buddy allocator algorithm for Redpanda’s Cloud Topics to balance batching efficiency against latency and cost.
Recent product updates and news from around the community.
BLIS is a calibrated discrete-event simulator for llm-d control-plane behavior. It helps developers evaluate routing, admission, KV cache, batching, prefill/decode placement, and capacity choices before spending time on cluster validation.
The shapes of the models being served and how tokens come out of them.
Astroparticle physics sits at the exciting intersection of astrophysics and particle physics and stu...
Qwen3.8-Flash-Next Another open weights model from Qwen. This one is "a multimodal MoE model that also serves as an early preview of the architecture used in Qwen4". It's pretty big: 125B tokens, but only 6B active which means it gets a…
NVIDIA Groq 3 LPX is the interactive AI inference accelerator for the NVIDIA Vera Rubin platform. At the core of the platform is NVIDIA Vera Rubin NVL72, the...
A practical guide to speculative decoding in vLLM on AMD GPUs, covering draft-and-verify mechanics, MTP, EAGLE-3, DFlash, DSpark, configuration, tuning, and benchmark results.
Serving a frontier Mixture-of-Experts (MoE) model well is a systems problem, and it gets harder the moment one node is not enough. GLM-5.1 is a good example: it is a large, sparse MoE that users want to run at long context, and it ships…
Speculative Decoding by any other name would distil as sweet
Every day, Meta’s recommendation platforms handle billions of user interactions, generating rich temporal signals that capture individual preferences and intent across products, ads, and content. In our 2024 post on sequence learning…
Speculators and vLLM now support P-EAGLE, DFlash, and DSpark — three parallel drafting algorithms that move beyond sequential token generation to deliver faster, simpler, and more scalable speculative decoding for LLM serving.
Together AI offers day zero access to Inkling, Thinking Machines Lab's multimodal mixture-of-experts model for text, image, and audio reasoning.
Fireworks built a KV-stationary sparse-attention kernel for MiniMax M3 on NVIDIA Blackwell (SM100), reaching ~980 TFLOP/s: 1.9–2.4× a query-stationary baseline and ~1.6× open-source MSA. The post walks through the Q-outer vs KV-outer…
How Modal and Decagon worked together to cut inference latency - and you can too.
Why we're all-in on speculative decoding.
What part of "dtype = 'fp8', num_splits = 0, pack_gqa = True, q_stage = 1, page_size = 1" do you not understand?
The arithmetic itself: kernels, numerics, and compilers.
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
HeyGen ported their 18B+ parameter Avatar IV video generation model to Google Cloud's Trillium (v6e) TPUs via torchax and XLA, utilizing FSDP and Ulysses sequence parallelism across an eight-chip mesh. To achieve a 1.86x speedup for…
As AI coding assistants shift the developer's primary role from writing boilerplate to reviewing and maintaining systems, language choice becomes critical for long-term architectural integrity. Go directly addresses this new paradigm by…
For years, a Python developer who needed a GPU had two realistic choices: Learn NVIDIA CUDA C++ well enough to write an extension, set up a build toolchain, and...
CUDA is the most important software framework in the GPU compute world, and Nvidia is looking at supporting CUDA on RISC-V. Terms and conditions may apply.
DeepSeek-V4-Pro is a 1.6-trillion-parameter Mixture-of-Experts (MoE) model released with both FP8 and FP4 weights. Models at this scale naturally benefit from accelerators such as NVIDIA Blackwell GPU...
Mojo🔥 is now open source The Mojo programming language has been promising an open source release since May 2023 . Last week they shipped their 1.0 and today they have followed through on that original promise, releasing the compiler…
This post introduces a multi-part study of how instruction scheduling can influence the behavior of memory operations within loop iterations in GPU kernels where multiple waves execute in a steady-state lock-stepping manner. In this…
Serving large language models affordably increasingly depends on low-bit quantization, and MXFP4 is one of the most aggressive options — but the smaller models that need it most rely on online rotation to stay accurate, and that…
TL; DR At Meta’s scale, a few milliseconds of latency degradation can have a significant negative impact on ads performance. When a Linux kernel upgrade risked regressing latency across Meta’s ad serving fleet, we turned to sched_ext —…
ParallelKernelBench tests whether LLMs can write fast multi-GPU CUDA kernels across 87 real workloads. The best model solves under a third, but a few generated kernels beat any public implementation.
Silicon, interconnect, and the fabric between them.
Google Cloud has natively integrated TPU support into the vLLM serving engine, allowing developers to elastically scale high-demand embedding pipelines using Google Kubernetes Engine (GKE). To handle massive 15K+ token contexts for…
Google's open-source TPU microbenchmark suite provides developers with granular performance metrics across Network, Compute, HBM, Host Transfer, and Attention components to validate real-world hardware capabilities. By leveraging these…
This second installment explores how Ray’s higher-level libraries—Serve, Data, and Train—abstract the complexities of running AI workloads on Google's TPU slices. Ray Serve uses a simple topology configuration to correctly gang-schedule…
Ray 2.55 introduces official, first-class support for Google Cloud TPUs, enabling developers to run distributed Python workloads on Google's accelerators using the familiar Ray task-and-actor APIs. To handle the strict networking…
AI factories must support increasingly large models and more complex reasoning workloads. To keep up with the insatiable compute demands of AI workloads,...
When an LLM engine process fails, the standard recovery path involves a cold restart. This requires loading weights into HBM from storage, compiling kernels,...
Training and serving frontier AI models depends on fast, reliable networks that move data between GPUs without wasting compute cycles. To meet this challenge at scale, Meta designed MetaRoCE – a clean-sheet RDMA transport protocol…
Machine learning workloads have an insatiable appetite for DRAM capacity. Flash memory is cheaper per gigabyte of capacity than DRAM. Could it offer a way out?
Machine learning applications demand ever more memory capacity and bandwidth. Samsung responded by fabricating their HBM base dies on a logic node, which opens up more opportunities
Use AdaptGrow, a GPU-accelerated matrix factorization algorithm, to turn rolling correlation and tail-dependence matrices into hard clusters, soft factor...
Batch-1 decode keeps getting more important. Xiaomi MiMo, for example, announced MiMo-V2.5-Pro UltraSpeed in June, claiming 1,000 tok/s decode on a one-trillion-parameter MoE model. Batch 1 gives an ...
Uniform Manifold Approximation and Projection (UMAP) is a dimensionality reduction technique widely used for visualization and feature extraction. Applications...
No more two-year compute contracts. Together AI and YC just gave YC startups a faster way to get GPUs.
In just a few short days, AMD will be showing off their brand new MI400 series of Datacenter Accelerators at their Advancing AI event but before that event comes, we thought it would be fun to attempt to scry the tea leaves that are…
By Parth Jain , Rakesh Sukumar , Yingwu Zhao , Renzo Sanchez-Silva & Nathan Fisher A deep dive into the engineering challenges of building a real-time service dependency map at Netflix scale: from streaming architectures and distributed…
Distributed GPU training has become routine across the industry. Teams now train...
The other half of the fleet: pre-training, post-training, and RL loops.
A guide to fine-tuning Qwen3-TTS for high-quality voice cloning, comparing ICL, speaker-embedding-only, and fine-tuning approaches with a full training recipe.
When Pythian rolled out Google Cloud’s Gemini Enterprise across our 500-person company in 27 countries, the goal was simple: use our own company as a proving ground to discover how enterprise AI actually delivers ROI. What we found…
A randomized study of more than 1,000 students examines ChatGPT, critical thinking, originality, and student performance on a real-world university assignment.
The advanced side of supervised fine-tuning data prep. This second post in a two-part series covers evaluating data readiness with learning curves, selecting high-value data subsets, augmenting data with synthetic and distilled…
Data preparation determines the ceiling of any supervised fine-tuning project. This first post in a two-part series covers the foundations of SFT data prep: quality checks, conversational (JSONL) formatting, reasoning and tool-calling…
Amazon SageMaker HyperPod now offers managed Ray support on Amazon EKS. Create and monitor Ray clusters, connect JupyterLab and Code Editor notebooks to live clusters, get out-of-the-box observability, and run resilient distributed…
MTIA 300 is the first of Meta’s family of in-house training and inference accelerators optimized for training ranking and recommendation models. We’re sharing how MTIA 300’s built-in NIC chiplets allow it to meet the communication needs…
Every lab CEO is on X now
A release focused on higher-throughput diffusion rollout, reusable omni adapters, and broader recipe coverage.
Reinforcement learning for large language models combines two very different workloads: rollout generation and model training. During rollout, inference workers run the current policy on a set of pro...
We present Miles v0.1, a full-stack production-ready system for frontier post-training, the successor to our first Miles release. Building upon slime's clean design, Miles optimizes every stage in the...
Teams customize their models to hit their targets for latency, speed, memory, and compute. With the open NVIDIA Nemotron family of models, developers can find...
We Tracked a Shipment of Rare Books. It Ended at an Amazon AI Training Facility Excellent piece of reporting from 404 Media. For a while now there have been stories of book dealers receiving orders for large volumes of books from…
Large-scale training spends a surprising share of its wall-clock time waiting instead of computing. Under Fully Sharded Data Parallel (FSDP), every layer ends in a collective all-gather or reduce-scatter, and every collective is a…
After a few long years of finding time to document my lessons from training open models, my post-training book is done!
Nearly every major AI lab uses Google Cloud infrastructure, including for training of models, inference for agents, and new frontier research. Google Cloud also continues to be the platform of choice for new, high-growth AI startups who…
Together AI partners with Moonshot AI to natively serve Kimi models, starting with the 2.8T parameter Kimi K3, with day zero access and post-training.
A podcast with Florian Brand.
Announcing ROCm support for vime, now running end-to-end on AMD Instinct MI355X GPUs with prebuilt container.
Introducing Co-operative Time-Slicing to eliminate idle accelerators in distributed RL post-training loops.
"Interview" #18
What we've seen helping teams run Reinforcement Learning at scale on Modal. Plus an open-source library to skip the scaffolding.
What the fleet costs to buy, run, and power.
GPT‑5.6 is now available in Kiro, helping developers plan, build, review, and test software with better price-performance.
As demand for AI continues to grow, so does the infrastructure needed to support it.
Musinsa migrated its audience engine from self-hosted ClickHouse to ClickHouse Cloud, cutting storage costs by 86.5% and total cost of ownership by up to 71.4% while simplifying real-time ingestion with ClickPipes.
Meta’s Generative Ads Recommendation Model (GEM), the foundation model behind ads recommendations across Instagram and Facebook, now trains at LLM scale on several thousand of the latest-generation GPUs. This post goes into the details…
Use state-of-the-art, open-source LLMs and image models at blazing fast speed, or fine-tune and deploy your own at no additional cost with Fireworks AI!
Cognition has signed a memorandum of understanding with the U.S. Department of Energy to join the Genesis Mission, a national initiative described as…
Today, we’re launching SWE-1.7, the most capable model we’ve trained so far. It reaches frontier-level intelligence at a much lower cost, advancing the…
Why does AI require a new approach to infrastructure?
We generated 12 landing pages with Kimi K2.7 Code and Claude Fable 5. Kimi cost 94% less and scored within a few points on every page. Here's what actually moved the needle.
Durability, query execution, and the layers under both.
Agents that interact with a traditional OLTP database often create bottlenecks at the storage layer. New deployments...
Your executable is a SQLite database Farid Zakaria describes a neat Linux pattern for creating a SQLite database file that can be directly used as an executable binary. The trick sets the SQLite file format's 4-byte application ID (68…
ClickHouse Managed Postgres uses runtime budgets, cgroup limits, and disk-full session exemptions to keep supporting services from compromising database availability.
Real-time streaming pipelines are the operational backbone of modern enterprises, continuously processing everything from customer support interactions to transaction logs. Traditionally, streaming DAGs are static; once deployed, their…
Qwen 3.8 27B scores 52 on the Artificial Analysis Intelligence Index That's the same score as GPT-5.6 Luna (max), and just one point behind GLM-5.2 (max) and DeepSeek V4 Pro 0813 (max) - that GLM is 753B and that DeepSeek is 1.7T…
Redpanda Operator 26.2 brings GA Stretch Clusters for multi-region replication, Redpanda Connect pipelines as K8s resources, Gateway API support, and safer rolling restarts.
Financial AI agents operating under DORA, the EU AI Act, and GDPR need infrastructure that supports operational resilience, traceability, and reliable transaction processing.
In Ashburn, Virginia, a row of servers draws 40 megawatts from the grid and exhales it as heat.
In today’s retail environment, shoppers expect highly personalized product discovery experiences and conversational assistance that feels genuine, natural, and genuinely helpful. Today, successful product discovery is about…
Discover the strategies, challenges, and trade-offs teams faced in a few real-world migrations to ScyllaDB
When builders ship AI-powered applications, the data layer quietly becomes the hardest part of the stack.
Over the past several years, model capabilities and training dataset sizes have experienced exponential growth. During the past year or so, the time between new-frontier-model releases has gone down from months to weeks. Reliable and…
When I started my PhD at UC Berkeley 16 years ago, my advisor told me: "OLTP databases...
The database you choose at the start of an AI project is the one you'll be living with, or paying to escape, for years.
Stop choosing between fresh data and robust Parquet files. Redpanda SQL bridge queries let you query live streaming topics and historical Iceberg tables together, without the compaction overhead.
Keep Tailscale logs with the rest of your security data.
Agreement, addressing, and moving bytes between machines.
Building autonomous AI agents that mutate production state requires moving beyond soft system prompts to a robust zero-trust architecture. To secure Google Agent Development Kit (ADK) workflows against prompt injections and malicious…
Five Rust-level memory optimizations to the DNS cache layout of Big Pineapple cut per-entry memory by 56%, freeing approximately 100 TB of memory across Cloudflare's fleet.
RFC 9234 lets routers reject route leaks on their own, using BGP Roles and the Only to Customer attribute. We measured who has deployed it, and found two Tier 1 networks unexpectedly stripping OTC.
In the first half of 2026, Cloudflare detected a 519% surge in hyper-volumetric DDos attacks across its network. These attacks were driven heavily by DNS and CLDAP reflection vectors. This report breaks down how major geopolitical…
How and Why Netflix Built a Real-Time Distributed Graph: Part 3 — Querying the graph with gRPC execution API Authors: Nilesh Mishra and Ajit Koti This is the third entry of a multi-part blog series describing how we built a Real-Time…
How ScyllaDB is using per-tablet Raft groups to bring strong consistency to data, without sacrificing the parallelism that makes it fast
How llm-d transfers the KV Cache between prefill and decode workers — NIXL's pluggable backend architecture, the new UCCL backend, head-to-head benchmarks of UCCL/UCX/Mooncake over RDMA and TCP, and preflight tooling for catching…
The unit of deployment and the machinery that starts it.
The idea of orchestrating long-running, stateful logic on top of unreliable, stateless infrastructure isn't new. We've had message queues, job runners, microservice choreographies, and full-blown workflow engines for a long time. What…
In 2024 and 2025, we reassessed remote Spectre attacks on our Workers infrastructure. We share details about the new attack primitives like Spectre gadgets, remote timers, achieving co-location and how new defenses further harden…
Sony LIV consolidated fragmented batch, Elasticsearch, and BigQuery workloads on ClickHouse Cloud, delivering sub-second analytics across billions of daily streaming events.
Introducing Cloudflare Access for Workers. Attach an Access policy directly to a Worker and it applies everywhere that Worker runs — routes, custom domains, workers.dev, and previews — automatically.
How customer needs shaped Oxide's Kubernetes integrations.
SummaryDatabricks' serverless platform launches tens of millions of VMs daily, and...
What’s the best way to recommend products to little-known users? We’ve spent our careers trying to solve this problem for major companies like Spotify and Priceline, and it’s why Sidd founded Malachyte , an AI-powered ecommerce…
Cloudflare is unifying AI Gateway and Workers AI into a single control plane, giving developers observability, billing, and dynamic routing across both managed GPUs and external providers. Learn how unified bindings and model-first…
The production-proven K8s operator is now available with full lifecycle automation, zero-downtime migration, as well as the flexibility and scalability to fit how your team actually works.
Modal’s Function Call data path is now >50ms faster. Our new routing layer is geographically distributed, so you can further reduce your network overhead.
To compare rates for serverless and reserved GPUs, look at your application's peak-to-average ratio.
Compaction can overwhelm poorly sized Kafka clusters, leading to full disks and maxed-out CPUs. Learn how Redpanda's Cloud Topics architecture redesigns compaction to cut redundant work, reduce cloud storage costs, and preserve the…
Border0 ties every connection to a real person, securing databases, Kubernetes, SSH, and more.
A deep dive inside our new ultra-low-latency primitive.
By Alvin Bao , Alex Petrov , Jennifer Lai , Aidan Sherr , and Samartha Chandrashekar As a part of the journey to transition Netflix’s compute infrastructure to be more Kubernetes-native, we have leaned into incorporating components from…
The ScyllaDB Cloud Terraform provider gives you infrastructure-as-code control over your clusters.
Learn how Redpanda's metastore powers Cloud Topics, from offset lookups and whole cluster restore to cross-region read replicas, and why it's built to be a foundational primitive for the future.
Introducing Oracle Kubernetes Engine support, stronger TLS, and a lighter dependency footprint
Seeing what the system did, and what happens after it breaks.
This is a technical report on three bugs that intermittently degraded responses from Claude. Below we explain what happened, why it took time to fix, and what we're changing.
Learn how metrics, logs, and traces work together to catch slow responses, errors, and failed deployments before users do.
After claims that ClickHouse is “winning the observability wars” sparked debate, we reflect on why it has become a leading storage and query engine, where it still falls short, and why winning the database layer isn’t the same as…
Self-hosted speech AI carries an observability trade-off: the numbers that drive capacity planning and cost management stay locked inside the vendor container. Deepgram closes that gap on Amazon SageMaker AI with two capabilities that…
Open Source wins!
OpenAI shares findings from the Hugging Face security incident and the steps we’re taking to strengthen AI model security, monitoring, and alignment.
Amazon OpenSearch Service now supports MCP Apps, which return interactive visualizations alongside your AI agent's text responses. Learn how a single, locally run MCP server lets your agent move from alert to trace to logs to root cause…
Platform and security teams can't govern what they can't see. A 2026 guide to tracing agent reasoning, tool calls, outcomes, and cost at enterprise scale.
What is an incident commander? Learn the IC's role, responsibilities, and how automated triage speeds up incident response.
In our previous blog post, we shared how Databricks uses AI to debug thousands of...
How ClickHouse Cloud collects 50m events per second through OpenTelemetry
An update on the August 17 outage and the steps we're taking to improve reliability. The post The August 17 outage, and the work ahead appeared first on The GitHub Blog .
Shopify unified global-scale observability on ClickHouse, achieving up to 30x faster queries while ingesting 100 million events per second at peak.
What's New with Monitoring in PostgreSQL 19
The ClickHouse Terraform provider now manages ClickStack dashboards, alerts, sources, and webhooks, putting observability config in version control.
ClickHouse Grafana plugin 4.20 brings compact query mode, click-to-filter log investigation, guided variable and annotation editors, and OpenTelemetry dashboards
Cloudflare's Certificate Transparency Monitoring is now generally available. The biggest change: we no longer email you about certificates Cloudflare issued for your domain, so when an alert lands in your inbox, it's worth a look.
ClickStack and Hud now share trace IDs, pairing service-level observability with function-level runtime forensics so coding agents can assess risky changes before they ship, catch regressions right after deploy, and fix them with real…
AI infrastructure spans multiple layers, from compute and networking to storage, orchestration, and applications. When performance degrades, identifying the...
bitdrift joins ClickHouse’s House Mates program with a mobile observability integration, bringing mobile-native telemetry, tracing, and debugging to ClickStack.
Explore the latest ClickHouse Managed Postgres updates, including proactive notifications, richer observability, faster backups, and expanded extension support.
How Mercado Libre rebuilt its observability platform on ClickHouse Cloud, cutting trace query times from over five minutes to about four seconds (a 50x speedup) with up to 89% compression while ingesting 400 million spans per minute.
Hugging Face published a technical timeline of a recent agent intrusion. Modal's platform and isolation were not compromised in this incident.
2 further posts in this topic are not shown. Open the topic on its own to read them.
Posts from the watchlist that no topic above claims.
Learn how mortgage document automation transforms loan processing, from document ingestion and extraction to validation and system integration.
Learn how KYC automation replaces manual verification with scalable, compliant workflows that cut costs, reduce errors, and speed up onboarding.
Learn how unstructured data extraction turns documents, PDFs, and text into structured insights using AI, NLP, and LLMs for scalable data processing.
OCR for tables converts complex document layouts into structured, machine-readable data. Learn how LlamaParse preserves table integrity.
OCR for images helps convert photos, labels, and screenshots into structured text. Compare the top AI OCR tools and learn what makes a reliable image-to-text system.
Learn how OCR for accounts payable automates invoice processing, improves accuracy, reduces costs, and integrates structured data into ERP systems.
OCR that works in a demo often stalls in production. How modern document pipelines handle real corpora, and how to hit 90%+ straight-through rates.
Intelligent OCR turns documents into validated, structured data, not just text. How production pipelines handle parsing, extraction, and confidence.
LlamaIndex is a simple, flexible framework for building knowledge assistants using LLMs connected to your enterprise data.
LLM OCR lowered the error rate but changed what an error looks like. Why fluent output hides silent substitutions, and what has to sit around the model.
Discover why the future of OCR isn
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
Tips and patterns for getting the most out of Claude Code, from configuring your environment to scaling across parallel sessions.
Credentio is a newly released, open-source C++ library from Google that allows developers to integrate high-performance, local-first validation of C2PA Content Credentials into their client and server applications. By processing assets…
Deploying secure, real-time Edge AI on Raspberry Pi is now simplified using LiteRT and lightweight Gemma open models. LiteRT optimizes CPU and GPU performance, delivering fast token speeds for models like Gemma4, enabling real-time…
Conductor has evolved from a Gemini CLI extension into a portable plugin, bringing conversational Spec-Driven Development (SDD) to ecosystems like Antigravity CLI and Claude. Rather than relying on strict command sequences, developers…
Deploy NVIDIA Nemotron 3.5 ASR for low-latency, production-ready speech recognition with 6x higher throughput and multilingual support.
Cerebras CS-4 delivers up to 30x faster AI inference than GPUs, with a modular rack-scale architecture built for hyperscale AI deployment.
Cerebras powers OpenAI’s GPT-5.6 Sol Ultrafast in the OpenAI API, delivering frontier intelligence at real-time speeds for critical AI work.
Build lightning-fast multimodal AI apps with Gemma 4 on Cerebras. Learn image understanding, vision workflows, and high-speed inference for developers.
Discover why AI loops require verification to avoid spiraling. See how Cerebras runs Gemma 4 at 1,500 tokens/sec for fast, autonomous visual loops.
Reasoning boosts AI accuracy, but at a steep cost. Explore test-time compute, agent performance, speed tradeoffs, and when thinking hurts.
Kimi K2.6 on Cerebras matches Gemini 3.5 Flash in intelligence while delivering 5× faster output, lower latency, and open-weight flexibility.
Cerebras powers the world's fastest AI inference on the biggest wafer chip. Cerebras CS-4 delivers up to 30x faster inference than GPUs.
205 further posts in this topic are not shown. Open the topic on its own to read them.
The index covers 45 sources from the watchlist. 37 publish a feed and are read from it; the other 8 publish none, so their index pages are scraped and each new post's own page supplies the title and date its card omits.
Last run finished 28 Aug 2026 at 00:08 UTC. The index holds 710 posts, keeps them for 120 days, and shows 685 posts on this page.
2 sources failed on the last run. Netflix TechBlog (HTTP 429); Replit (HTTP 403)
The last run read 833 posts across every source and added nothing new.
4 sources on the watchlist are not aggregated here, so check them by hand.
A date shown as "first seen" is not a publication date. Some sources publish no date at all, so the page records when the post entered the index instead of guessing.