Inference
Topic archive • 8 matches
AI-generated: summaries written by AI from the linked sources. How we use AI
2026-11-09
Technology
Google expanded its eighth-generation TPU portfolio with TPU 8tx for secure, confidential inference and TPU 8i Max for massive context windows, marking four new TPU systems in a single year. The chips are designed to handle complex agentic workloads with specialized hardware for security, density, and reasoning.
Infrastructure • Hacker News
Permalink
2026-11-13
Technology
South Korea-based DeepX taped out the DX-M2, an edge AI chip designed to run inference of 100B-parameter LLMs on robots and drones in under 5 W. The company is targeting fall 2027 for mass production, according to EE Times.
Hardware • Techmeme
PermalinkEdge0 open-sourced Edge0-145B and Edge0-120B, along with a unified inference framework and deployment tools. The company claims Edge0-145B is the world's first open-source on-device AI model with 145 billion parameters, bringing coding, reasoning, and agent capabilities that rival frontier cloud models to a personal computer.
Models • Hacker News
Permalink
2026-11-14
Technology
Nvidia published a guide for serving GLM-5.3, a quantized 753-billion-parameter mixture-of-experts model from Z.ai with 40 billion parameters activated per token, across two DGX Stations connected via ConnectX-8 RDMA. The setup uses vLLM tensor parallelism to split weights and computation across both GPUs through a single endpoint, reaching 607.8 tokens per second.
Infrastructure • Hacker News
PermalinkA benchmark for wet-lab experiment prediction, described as the strictest of its kind, was released by more than ten top teams including Stanford, Harvard and Cornell. The results indicate that scaling inference and quintupling compute did not improve performance.
AI Research • QbitAI 量子位
Permalink
Investment
Luminal • Series A
Luminal, which provides an inference cloud and develops inference compiler software to test running AI models on different chips, raised a $25M Series A led by Standard Capital. CEO Joe Fioti disclosed the round to Axios Pro.
Permalink