A vLLM server on an unmapped Vast port starts cleanly, passes its local health
check, and is unreachable from outside. Nothing in the logs says so — the
failure only appears when something tries to route to it.
Background
Vast maps container ports to public ones at instance creation. A port not
declared on the template cannot be added afterwards; the instance has to be
recreated. The external port is assigned from the instance's range and is not
the same number as the internal one:
$ env | grep VAST_TCP_PORT
VAST_TCP_PORT_10100=12711
$ echo "$PUBLIC_IPADDR"
80.188.223.202
So the server is reachable at 80.188.223.202:12711, and 10100 appears
nowhere outside the box.
Three changes
1. Refuse to launch on an unmapped port.
install_set.py should read VAST_TCP_PORT_<port> before writing the
Supervisor unit. If it is absent, fail with a message naming the port rather
than starting a server nobody can reach.
This is the highest-value part. The current failure is silent and looks like
success.
2. Force --host 0.0.0.0 in the generated unit.
The Vast template's own vLLM launches with --host 127.0.0.1:
vllm serve Qwen/Qwen3.5-9B ... --host 127.0.0.1 --port 18000
Bound to loopback, it is unreachable from outside regardless of how the port is
mapped — and it looks identical to a mapping problem from the outside. Worth
setting explicitly rather than inheriting whatever the template does.
3. Report the public address back to llmao.
The box knows PUBLIC_IPADDR and its VAST_TCP_PORT_* map at exactly the
moment it fetches GET /vllm/config. Having install_set.py POST them back
means llmao never has to be told:
POST /vllm/observed
{ "public_ip": "80.188.223.202",
"port_map": { "10100": 12711 },
"name": "gemma4-26b" }
llmao can then build api_base for the route itself. This is an extension of
the observed-state reporting already planned, not a new mechanism.
Also worth doing
Declare ports on the template, not per instance. Every box then gets them
without anyone remembering, and a missing port becomes a template fix rather
than a rebuild.
Note for anything pinning the address
The external port differs per instance, so a CDN backend or hiera value pinned
to IP:port needs updating on every rebuild. Worth asking whether the CDN can
point at a hostname we control instead, so a rebuild becomes a DNS update
rather than a ticket.
Reproducing
# on the box -- is vLLM listening?
ss -tlnp | grep 10100
curl -sS -o /dev/null -w '%{http_code}\n' http://127.0.0.1:10100/v1/models
# from outside -- is the mapping routing?
curl -sS -m 10 -o /dev/null -w '%{http_code}\n' http://$PUBLIC_IPADDR:<external>/v1/models
Local works and external times out → the port was not declared at creation.
Both fail → vLLM is not up, or it is bound to loopback.
A vLLM server on an unmapped Vast port starts cleanly, passes its local health
check, and is unreachable from outside. Nothing in the logs says so — the
failure only appears when something tries to route to it.
Background
Vast maps container ports to public ones at instance creation. A port not
declared on the template cannot be added afterwards; the instance has to be
recreated. The external port is assigned from the instance's range and is not
the same number as the internal one:
So the server is reachable at
80.188.223.202:12711, and10100appearsnowhere outside the box.
Three changes
1. Refuse to launch on an unmapped port.
install_set.pyshould readVAST_TCP_PORT_<port>before writing theSupervisor unit. If it is absent, fail with a message naming the port rather
than starting a server nobody can reach.
This is the highest-value part. The current failure is silent and looks like
success.
2. Force
--host 0.0.0.0in the generated unit.The Vast template's own vLLM launches with
--host 127.0.0.1:Bound to loopback, it is unreachable from outside regardless of how the port is
mapped — and it looks identical to a mapping problem from the outside. Worth
setting explicitly rather than inheriting whatever the template does.
3. Report the public address back to llmao.
The box knows
PUBLIC_IPADDRand itsVAST_TCP_PORT_*map at exactly themoment it fetches
GET /vllm/config. Havinginstall_set.pyPOST them backmeans llmao never has to be told:
llmao can then build
api_basefor the route itself. This is an extension ofthe observed-state reporting already planned, not a new mechanism.
Also worth doing
Declare ports on the template, not per instance. Every box then gets them
without anyone remembering, and a missing port becomes a template fix rather
than a rebuild.
Note for anything pinning the address
The external port differs per instance, so a CDN backend or hiera value pinned
to
IP:portneeds updating on every rebuild. Worth asking whether the CDN canpoint at a hostname we control instead, so a rebuild becomes a DNS update
rather than a ticket.
Reproducing
Local works and external times out → the port was not declared at creation.
Both fail → vLLM is not up, or it is bound to loopback.