Quant
Topic archive • 2 matches
AI-generated: summaries written by AI from the linked sources. How we use AI
2026-11-11
Technology
Huawei detailed how its Kirin 9050 chip runs a 30-billion-parameter mixture-of-experts model on a phone without an internet connection, activating about 2 billion parameters at a time. The chip's Da Vinci NPU uses low-bit quantization and coordinates memory and compute to run the model locally.
Hardware • TechNode
Permalink
2026-11-14
Technology
Nvidia published a guide for serving GLM-5.3, a quantized 753-billion-parameter mixture-of-experts model from Z.ai with 40 billion parameters activated per token, across two DGX Stations connected via ConnectX-8 RDMA. The setup uses vLLM tensor parallelism to split weights and computation across both GPUs through a single endpoint, reaching 607.8 tokens per second.
Infrastructure • Hacker News
Permalink