---
title: "NVIDIA Shows GLM-5.3 Served Across Two DGX Stations with vLLM: Throughput Peaks at Concurrency 16"
url: https://xcube.enlightcorp.com.tw/en/news/nvidia-glm-5-3-two-dgx-stations-vllm-tensor-parallelism
category: "AI Infrastructure"
source_type: official
published: 2026-11-13
updated: 2026-11-13
lang: en
---

# NVIDIA Shows GLM-5.3 Served Across Two DGX Stations with vLLM: Throughput Peaks at Concurrency 16

Published 2026-11-13 · Updated 2026-11-13 · Official release · Source: [NVIDIA Technical Blog](https://developer.nvidia.com/blog/when-one-gpu-isnt-enough-serving-glm-5-3-across-two-nvidia-dgx-stations-with-vllm)

**Key answer:** On November 13, 2026, NVIDIA showed two DGX Stations linked over ConnectX-8 RDMA serving GLM-5.3 NVFP4 through vLLM tensor parallelism behind a single OpenAI-compatible endpoint. The point: a model too large for one machine can be served across two, but the right concurrency limit has to be measured.

The NVIDIA Technical Blog published a tutorial on November 13, 2026 on serving GLM-5.3 locally with two DGX Stations. The GLM-5.3 NVFP4 model it uses is a quantized version of Z.ai's mixture-of-experts model, with 753 billion parameters and 40 billion activated per token. The checkpoint is about 464 GB, more than the 252 GB of memory on a single GB300 GPU. The post notes that sparse activation reduces computation, but every token can be routed to different experts, so all weights still have to stay in memory.

The approach uses tensor parallelism (TP=2) to split the model's tensors across the two GPUs. The machines are connected over ConnectX-8 RDMA, both compute every request, and one of them answers through a single OpenAI-compatible API. The tutorial uses a vLLM v0.30.0 aarch64 image pinned by version and digest and a model pinned by revision. Each machine needs at least 464 GB of free disk for its copy of the checkpoint.

Among the launch settings, the post singles out two: --max-model-len 98304, the context ceiling covering input plus output, and --gpu-memory-utilization 0.93. It also enables multi-token prediction (drafting four tokens per step, verified by the full model), an FP8 KV cache, and prefix caching.

The performance numbers are NVIDIA's own measurements, taken with AIPerf replaying the SPEED-Bench Coding dataset. Overall output throughput is 107.4 tokens/s at concurrency 1, 386.0 at concurrency 8, and peaks at 607.8 at concurrency 16. At concurrency 32 it falls to 385.0, with mean request latency rising to 453.5 seconds. Mean per-user throughput drops from 115.3 tokens/s at concurrency 1 to 13.0 at concurrency 32.

The post includes a security note: launching with --host 0.0.0.0 and host networking exposes the unauthenticated API on every network interface, including management. It says to firewall the port to intended clients and add authentication and TLS before wider access.

For enterprises evaluating large on-prem models, the tutorial offers three checks that transfer directly. Size memory and disk from the checkpoint, not from activated parameters. Rerun the load test with your own prompts and concurrency to find where throughput starts to fall, and set the gateway's concurrency limit from that. And because the inference server carries no authentication of its own, the gateway in front of it has to handle identity, quotas, and encryption.

## X Cube view

Serving across machines keeps models that do not fit on one system inside your own environment, but the vendor's own numbers also show total throughput and latency degrading past a certain concurrency. When planning on-prem capacity, treat the concurrency limit as a parameter to measure, and have the gateway queue and rate-limit ahead of the inference server.

Tags: vLLM, DGX Station, Tensor Parallelism, On-prem AI, Local Inference, NVIDIA, GB300, GLM-5.3, Z.ai, ConnectX-8, AIPerf
