Install unison in the worker images so ow ssh works at all - #78
Open
vohonen wants to merge 1 commit into
Open
Conversation
No `ow ssh` command works against any published image:
[ow] unison not found on remote. Please install it in your image.
`cli/common.py` already documents unison as a requirement at both ends and checks
for it on the remote, but no image installs it.
This is not only `--sync`: the check lives in `REMOTE_INIT`, which
`bootstrap_remote` runs in every mode, so plain `ow ssh <command>` dies on it too.
Reproduced with `ow ssh --gpu A100S --count 1 whoami` on a fresh pod.
Added to the two images you can `ow ssh` into: unsloth (the `--image` default)
and vllm. The cluster image is left alone, since nothing syncs into it.
Pinned to the upstream static build rather than apt. Unison refuses to sync
between mismatched versions, and the two sides disagree by default: Homebrew
ships 2.54.0 while Ubuntu 24.04 apt has 2.53.x, so `apt install unison` would
produce images that still cannot talk to a stock macOS client. The static tarball
is 2 MB, checksum-pinned, and `unison -version` at the end of the layer makes a
broken download fail the build rather than the first connection.
Both images already build `--platform linux/amd64` only, so the x86_64 asset is
the only one needed.
Needs a build and push to take effect, since the fix is in the images rather than
in the code.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What's broken
No
ow sshcommand works against any published image:cli/common.pyalready documents unison as a requirement at both ends, and checks for it on theremote, but no image installs it. So this is unreachable for everyone, not just me.
Worth stressing that this is not only
--sync. The check lives inREMOTE_INIT, whichbootstrap_remoteruns in every mode, so plainow ssh <command>dies on it too. I hit it withow ssh --gpu A100S --count 1 whoamion a freshly provisioned pod.What this changes
Dockerfile(unsloth, the--imagedefault) andDockerfile.vllm. Thecluster image is untouched, since nothing syncs into it.
apt install unison.DOCKER_README.md.Why pinned and not apt: unison refuses to sync between mismatched versions, and the two ends
disagree by default. Homebrew ships 2.54.0; Ubuntu 24.04 apt has 2.53.x. So
apt install unisonwould give you images that still can't sync with a stock macOS client, which is the majority of
callers. The static tarball is 2 MB, and
unison -versionat the end of the layer turns a baddownload into a build failure rather than a first-sync failure.
What you need to do
The code fix is inert until the images are rebuilt, so this PR does nothing on its own:
IMAGE_VERSION(stillv0.11). Eitherrebuild
v0.11in place, or bump tov0.12— but if you bump,openweights/images.pyand thepushed images have to land together, or every job points at a tag that doesn't exist yet. Your
release process, your call.
DOCKER_README.md:Verified / not verified
tar --strip-components=1 <dir>/binlands exactly
bin/unisonandbin/unison-fsmonitor. The binary is a statically linked x86-64ELF, so being built on 22.04 is fine on the 24.04 base.
Unrelated, but noticed
DOCKER_README.mdclaims the unsloth image is builtFROM unsloth/unsloth:latestand the vLLM oneFROM vllm/vllm-openai:v0.19.1. Both Dockerfiles are actually onpytorch/pytorch:2.10.0-cuda12.8-cudnn9-runtime. Not touching it here, but it misled me for a bit.