Run a local MLX LLM server on your Mac as a background service — and control it from the menu bar.
mlx_lm.server speaks the OpenAI API, so anything that talks to OpenAI (OpenCode, Continue, Zed,
Aider, your own code) can point at http://127.0.0.1:8080/v1 and get a model running on the Apple
Neural Engine and GPU instead of someone else's datacenter. This repo wraps it in the three pieces
that turn it into something you actually use daily:
| LaunchAgent | The server starts at login, restarts if it crashes, logs to one file. |
| Menu bar app | A cpu icon next to the clock: start, stop, restart, switch model, read the log. |
mlxctl |
The same control from the terminal, for scripts and SSH sessions. |
┌─────────────────────────────────────────┐
│ MLX server: running on port 8080 (pid …)│
├─────────────────────────────────────────┤
│ Start │
│ Stop │
│ Restart │
├─────────────────────────────────────────┤
│ Models (4) ▸ │
│ Manage models… │
│ Copy endpoint │
│ Show log… │
├─────────────────────────────────────────┤
│ ✓ Start server at login │
│ Quit menu bar app │
└─────────────────────────────────────────┘
The icon colour is the server state at a glance:
- grey — stopped (the agent is not loaded)
- orange — process is up but not answering yet; a large model takes a while to load
- green —
/healthreturns 200, ready for requests
- Apple Silicon Mac, macOS 13 or later
mlx-lm—pipx install mlx-lm(orpip install mlx-lm)- Xcode Command Line Tools, for the Swift compiler —
xcode-select --install
git clone https://github.com/aelmen/mlx-server-mac.git
cd mlx-server-mac
./install.shThe installer builds the app, writes both LaunchAgents, installs mlxctl and mlx-pull into
~/.local/bin, and creates ~/.config/mlx-server/config.env if you don't already have one. It
never overwrites an existing config, so re-running it is how you upgrade.
Then pull a model and check that it answers:
mlx-pull mlx-community/Qwen3-8B-4bit
mlxctl status
mlxctl testOne file, ~/.config/mlx-server/config.env, is read by the server script, the CLI and the menu bar
app alike. See config/config.env.example for every key.
MLX_MODEL=mlx-community/Qwen3-8B-4bit # preloaded at startup
MLX_PORT=8080
MLX_MAX_TOKENS=32768
MLX_PROMPT_CACHE_BYTES=24000000000 # 24 GB — lower this on a smaller machine
HF_HUB_OFFLINE=1 # only use models already downloadedManage models… opens a window over the Hugging Face cache:
- Search any term to query the Hugging Face API —
MLX models onlykeeps it to repos taggedmlx. Leave the field empty to list what you already have, with sizes and a disk total. - Download runs
mlx-pullwith a progress bar, and can be cancelled mid-flight. - Delete… removes a model's cache directory after showing you how much it frees. It refuses any path outside the hub cache.
- Load makes a downloaded model the resident one, same as picking it from the Models submenu.
The Models submenu in the menu bar lists only models that mlx-lm can actually load. The server's
/v1/models reports everything in the cache — including diffusion and speech models that 404 when
you request them — so the app checks each snapshot for a config.json and a model*.safetensors
first, and marks the rest Not an LLM in the window.
MLX_MODEL is only the model loaded at startup. The server holds one model at a time but will
swap on demand when a request names a different downloaded model — which is what the Models
submenu does: clicking a model sends a one-token request so it becomes resident. On a 27B model that
swap takes a couple of minutes, and the icon stays orange while it happens.
Run mlxctl restart after editing the config.
mlxctl start | stop | restart # control the agent
mlxctl status # state, health, downloaded models
mlxctl log [n] # tail the log
mlxctl test [model] # one-shot chat completion
mlxctl config # open the config file
mlx-pull <hf-repo> # download a model from Hugging FacePoint any OpenAI-compatible client at the endpoint. For OpenCode, in
~/.config/opencode/opencode.json:
{
"provider": {
"local-mlx": {
"npm": "@ai-sdk/openai-compatible",
"options": { "baseURL": "http://127.0.0.1:8080/v1" },
"models": { "mlx-community/Qwen3-8B-4bit": { "name": "Qwen3 8B (local)" } }
}
}
}Measured on a Mac mini M4 Pro, 64 GB, 4-bit quantised models:
| Model | Generation | Prefill |
|---|---|---|
| 27B dense | ~13 tok/s | ~100 tok/s |
| 35B MoE (3B active) | ~47 tok/s | ~850 tok/s |
Prefill speed is what decides whether a coding agent feels usable: a first turn carrying ~14k tokens
of context costs two or three minutes on the dense model and about fifteen seconds on the MoE one.
If you are driving this from an agent, prefer a mixture-of-experts model and give the prompt cache
room in MLX_PROMPT_CACHE_BYTES.
The icon never turns green. mlxctl log — a large model can take minutes to load, and an
out-of-memory kill shows up there. KeepAlive restarts the process on failure, so a crash loop
looks like an icon flipping grey/orange.
The icon isn't in the menu bar. macOS hides status items when the bar is crowded; widen it by
quitting another app or hiding menu bar items. Confirm the app is running with
pgrep -fl MLXServer.
mlxctl: command not found. ~/.local/bin isn't on your PATH.
Cannot find an appropriate cached snapshot folder for the specified revision. The model was
requested but never downloaded, and HF_HUB_OFFLINE=1 stops the server from fetching it. This is
the common one: a client config (OpenCode, Continue) lists models you haven't pulled yet. Open
Manage models… — anything marked Not downloaded will fail this way — or run
mlx-pull <repo>. The server answers 404 for these, so that is what the log shows.
Requests return 404. Either the model isn't downloaded (above), or the client is calling a path
the server doesn't serve — check that the base URL ends in /v1.
Nothing downloads. HF_HUB_OFFLINE=1 is the default, on purpose. mlx-pull overrides it for
the duration of the download.
./uninstall.shRemoves the agents, the app and the CLI. Your config, log and downloaded models stay where they are; the script prints how to delete those too.
Bug reports and pull requests are welcome — see CONTRIBUTING.md.
MIT — see LICENSE.