Disaggregated Inference Added a Latency Tax. More Bandwidth Was Not the Fix.
A push-based connector in vLLM fixes a latency tax in disaggregated inference, where KV transfer setup stalls the decode worker.
Intel Is Selling Inference by Memory, and Its Own Card Holds 160 GB
Intel GPU D-series promotes 480 GB of LPDDR5X for inference. The card Intel itself sells carries 160 GB, and the gap decides the purchase.
The CPU’s Memory Is the Next Line on the AI Power Bill
AMD EPYC Verano is the first EPYC to carry LPDDR5X in SOCAMM2 modules, and server memory is now a rack power decision.
PJM Wants to Review a Ten-Gigawatt AI Campus as One Project
PJM filed to review a ten-gigawatt Ohio AI campus as one integrated project, a new path around the data center interconnection queue.
The AI Buildout Just Met Its Cost of Capital
AI infrastructure spending now runs about five trillion dollars ahead of the AI revenue that pays for it. On Wednesday the bond market priced the gap.
Adani and Jabil Are Building the AI Rack Itself
Adani and Jabil signed a 50:50 MoU for AI rack manufacturing in India, covering racks, networking, power distribution units and liquid cooling.
HBM Cannot Hold the Context Window, and the Memory Industry Just Said So
The KV cache is outgrowing HBM. Marvell, Tensormesh and Inworld each shipped a tier for it this week, and inference economics moves down the stack.
Gemini 4 Argon Is Half Price Until December 1, and the Deadline Is the Point
Gemini 4 Argon is half price on the Gemini API until December 1, then input and output tokens double. What that means for agent budgets.
NVIDIA Run:ai Goes Open Source, and Support Is the Part It Kept
NVIDIA Run:ai open source means platform teams can schedule and share GPUs on Kubernetes without a subscription, but support stays commercial.
Anthropic’s First Data Center in Japan Is a 500MW Bet on Osaka
NTT Data is close to a $6.3bn, 500MW data center for Anthropic in Osaka, the first dedicated site for the lab in Japan.