diff --git a/README.md b/README.md index 2383b6ef..ebc7f254 100644 --- a/README.md +++ b/README.md @@ -7,11 +7,12 @@ substitution. All of this is done while retaining the convenience of working in an IDE like VS Code. All container traffic is routed through a transparent -[mitmproxy](https://mitmproxy.org/) via WireGuard, capturing HTTP/S, DNS, and -all other TCP/UDP traffic without per-tool proxy configuration. A -straightforward allow/deny list-based engine controls which network requests go -through, and a secret substitution system injects credentials at the proxy level -so the container never sees real values. +[mitmproxy](https://mitmproxy.org/) via WireGuard, without per-tool proxy +configuration. A straightforward allow/deny list-based engine controls which +HTTP/S requests and DNS queries go through; everything else — raw TCP and UDP, +which carries no hostname to match a rule against — is dropped. A secret +substitution system injects credentials at the proxy level so the container +never sees real values. > Sandcat is part of [Visdom](https://virtuslab.com/services/visdom), > VirtusLab's AI-native SDLC platform. diff --git a/cli/lib/agents.bash b/cli/lib/agents.bash index 342ce4f4..35b702e2 100644 --- a/cli/lib/agents.bash +++ b/cli/lib/agents.bash @@ -421,8 +421,9 @@ EOF # Returns mitmproxy --set flags that affect streaming-body handling. # # Cursor's API uses Connect/HTTP-2 streaming for agent calls. Mitmproxy needs -# stream_large_bodies (don't buffer >1MB), connection_strategy=lazy, anticomp, -# and a long read timeout to keep those streams stable. +# stream_large_bodies (don't buffer >1MB), anticomp, and a long read timeout to +# keep those streams stable. connection_strategy=lazy, which these streams also +# need, is on the base command for every agent (see compose-proxy.yml). # # Claude's traffic is plain JSON request/response, so leaving the body # buffered means _substitute_secrets in the addon can run a content-based @@ -436,7 +437,7 @@ sct_agent_mitm_streaming_flags() { local agent=$1 case "$agent" in cursor) - echo "--set stream_large_bodies=1m --set connection_strategy=lazy --set anticomp=true --set timeout_read=300" + echo "--set stream_large_bodies=1m --set anticomp=true --set timeout_read=300" ;; claude|codex|copilot|*) echo "" diff --git a/cli/templates/devcontainer/sandcat/compose-proxy.yml b/cli/templates/devcontainer/sandcat/compose-proxy.yml index 775bfcf6..36ec54af 100644 --- a/cli/templates/devcontainer/sandcat/compose-proxy.yml +++ b/cli/templates/devcontainer/sandcat/compose-proxy.yml @@ -60,7 +60,17 @@ services: # the command) so docker-entrypoint.sh still sees `mitmweb` as $1 and drops # privileges to the mitmproxy user. entrypoint: ["/bin/sh", "-c", "chown -R mitmproxy:mitmproxy /mitmproxy-public && rm -f /home/mitmproxy/.mitmproxy/dns.conf /mitmproxy-public/mitmproxy-ca-cert.pem && (while [ ! -f /home/mitmproxy/.mitmproxy/mitmproxy-ca-cert.pem ]; do sleep 1; done; cp /home/mitmproxy/.mitmproxy/mitmproxy-ca-cert.pem /mitmproxy-public/mitmproxy-ca-cert.pem.tmp && mv /mitmproxy-public/mitmproxy-ca-cert.pem.tmp /mitmproxy-public/mitmproxy-ca-cert.pem) & exec docker-entrypoint.sh \"$@\"", "sh"] - command: mitmweb --mode wireguard --web-host 0.0.0.0 --set web_password=mitmproxy --set http2=__MITM_HTTP2__ __AGENT_MITM_STREAMING_FLAGS__ -s /scripts/__AGENT_MITM_ADDON__ + # connection_strategy=lazy defers the upstream connection until a layer asks + # for it. Without it mitmproxy dials the destination before next_layer runs, + # so a flow the addon is about to deny has already sent a SYN — a signalling + # channel out of the sandbox even though no payload is relayed. + # + # rawtcp=false closes the one raw relay the addon's next_layer hook cannot + # reach: on an HTTP `101 Switching Protocols` with a non-WebSocket upgrade, + # mitmproxy builds a TCPLayer child directly and emits no next_layer hook, + # so a request to an allowed host turns into an unfiltered byte pipe. With + # rawtcp off mitmproxy closes the connection instead. + command: mitmweb --mode wireguard --web-host 0.0.0.0 --set web_password=mitmproxy --set connection_strategy=lazy --set rawtcp=false --set http2=__MITM_HTTP2__ __AGENT_MITM_STREAMING_FLAGS__ -s /scripts/__AGENT_MITM_ADDON__ ports: - "8081" # mitmweb UI; host port assigned dynamically to avoid conflicts volumes: diff --git a/cli/templates/devcontainer/sandcat/scripts/mitmproxy_addon_common.py b/cli/templates/devcontainer/sandcat/scripts/mitmproxy_addon_common.py index b5a81ab9..c9d21dd9 100644 --- a/cli/templates/devcontainer/sandcat/scripts/mitmproxy_addon_common.py +++ b/cli/templates/devcontainer/sandcat/scripts/mitmproxy_addon_common.py @@ -51,6 +51,11 @@ from fnmatch import fnmatch from mitmproxy import ctx, dns, http +from mitmproxy.addons.next_layer import NextLayer +from mitmproxy.proxy import commands as proxy_commands +from mitmproxy.proxy import events as proxy_events +from mitmproxy.proxy import layer as proxy_layer +from mitmproxy.proxy.layers import RawQuicLayer, TCPLayer, UDPLayer _VALID_ENV_NAME = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$") # RFC-1123 label / RFC-2181 length. Trailing dot is stripped before validation @@ -199,6 +204,22 @@ def _pass_cli_session_is_pat(stdout: str) -> bool: return bool(_PAT_SESSION_MARKER.search(stdout or "")) +# The layers mitmproxy falls back to when it cannot parse a flow as HTTP or +# DNS. They copy bytes between client and server verbatim and emit no +# request/dns_request hook, so the allow/deny rules never see them. +RAW_RELAY_LAYERS = (TCPLayer, UDPLayer, RawQuicLayer) + + +class DenyLayer(proxy_layer.Layer): + """Hangs up on the client instead of relaying, without ever dialling the + server. Used in place of a raw relay layer — see ``SandcatAddon.next_layer``. + """ + + def _handle_event(self, event): + if isinstance(event, proxy_events.Start): + yield proxy_commands.CloseConnection(self.context.client) + + class SandcatAddon: """Base sandcat addon: network policy + secret substitution.""" @@ -209,6 +230,9 @@ def __init__(self): self.dns_servers: list[str] = [] # custom upstream DNS for wg-client self.debug_enabled = False # subclasses may flip this in _on_settings_merged self._pass_cli_logged_in = False # True only after a successful pass-cli login + # Our own copy of mitmproxy's layer chooser, so next_layer() below can + # see what mitmproxy would pick and veto it. See next_layer(). + self._layer_chooser = NextLayer() # ------------------------------------------------------------------ load @@ -1057,6 +1081,50 @@ def _substitute_secrets(self, flow: http.HTTPFlow): # -------------------------------------------------------------- handlers + def configure(self, updated): + # Keeps _layer_chooser's compiled host patterns in sync with the + # options mitmproxy's own NextLayer instance reads. + self._layer_chooser.configure(updated) + + def next_layer(self, nextlayer: proxy_layer.NextLayer): + """Deny anything that would be relayed as raw bytes. + + The allow/deny rules key on a hostname, which only HTTP and DNS flows + carry; a raw TCP or UDP flow knows nothing but an address, so there is + nothing to match and no hook to match it in. Left alone, mitmproxy + would relay such a flow straight through and a process in the agent + container could reach any address and port it likes. + + Script addons run before mitmproxy's own NextLayer (ScriptLoader sits + earlier in the default addon chain), and NextLayer skips a decision + another addon already made. So we ask our own chooser first and either + veto what it picked or let it stand — never hand the decision back, + which would let a second, unexamined choice run in its place. + + mitmproxy swallows an addon hook's exception and carries on down the + chain, so anything raised here has to leave the flow denied rather + than half-decided: the chooser runs inside a try, and the assignment + happens before the log line. + """ + try: + self._layer_chooser.next_layer(nextlayer) + # Left as None when the chooser wants more data; the decision is + # retried on the next chunk, so there is nothing to veto yet. + is_raw = isinstance(nextlayer.layer, RAW_RELAY_LAYERS) + except Exception: + logger.exception("Layer choice failed — denying the flow") + is_raw = True + + if not is_raw: + return + + nextlayer.layer = DenyLayer(nextlayer.context) + logger.warning( + "Network deny (not HTTP or DNS): " + f"{nextlayer.context.client.transport_protocol} " + f"{nextlayer.context.server.address}" + ) + def request(self, flow: http.HTTPFlow): method = flow.request.method host = flow.request.pretty_host diff --git a/cli/test/agents/agents.bats b/cli/test/agents/agents.bats index b3f84c3b..80de2c4e 100755 --- a/cli/test/agents/agents.bats +++ b/cli/test/agents/agents.bats @@ -490,9 +490,10 @@ setup() { @test "sct_agent_mitm_streaming_flags: cursor returns streaming flags" { run sct_agent_mitm_streaming_flags cursor assert_output --partial "stream_large_bodies=1m" - assert_output --partial "connection_strategy=lazy" assert_output --partial "anticomp=true" assert_output --partial "timeout_read=300" + # Lives on the base command for every agent instead. + refute_output --partial "connection_strategy" } @test "sct_agent_mitm_streaming_flags: claude returns empty" { diff --git a/cli/test/composefile/composefile.bats b/cli/test/composefile/composefile.bats index a34dacf6..56c32b1a 100644 --- a/cli/test/composefile/composefile.bats +++ b/cli/test/composefile/composefile.bats @@ -509,7 +509,7 @@ YAML cat >"$proxy_compose" <<'YAML' services: mitmproxy: - command: mitmweb --mode wireguard --set http2=true --set stream_large_bodies=1m --set connection_strategy=lazy --set anticomp=true --set timeout_read=300 -s /scripts/mitmproxy_addon_cursor.py + command: mitmweb --mode wireguard --set connection_strategy=lazy --set rawtcp=false --set http2=true --set stream_large_bodies=1m --set anticomp=true --set timeout_read=300 -s /scripts/mitmproxy_addon_cursor.py ports: - "8081" YAML @@ -517,7 +517,7 @@ YAML set_proxy_tui_mode "$proxy_compose" run yq -r '.services.mitmproxy.command' "$proxy_compose" - assert_output "mitmdump --mode wireguard --set http2=true --set stream_large_bodies=1m --set connection_strategy=lazy --set anticomp=true --set timeout_read=300 -s /scripts/mitmproxy_addon_cursor.py" + assert_output "mitmdump --mode wireguard --set connection_strategy=lazy --set rawtcp=false --set http2=true --set stream_large_bodies=1m --set anticomp=true --set timeout_read=300 -s /scripts/mitmproxy_addon_cursor.py" yq -e '.services.mitmproxy.command | contains("/scripts/mitmproxy_addon_cursor.py")' "$proxy_compose" } diff --git a/cli/test/init/extensions.bats b/cli/test/init/extensions.bats index 014c3a2e..92c17b1d 100644 --- a/cli/test/init/extensions.bats +++ b/cli/test/init/extensions.bats @@ -130,8 +130,11 @@ teardown() { run grep 'stream_large_bodies=1m' "$BATS_TEST_TMPDIR/sandcat/compose-proxy.yml" assert_failure + # connection_strategy=lazy is not agent-specific — it is on the base + # command so a denied flow never reaches the destination (see + # compose-proxy.yml). run grep 'connection_strategy=lazy' "$BATS_TEST_TMPDIR/sandcat/compose-proxy.yml" - assert_failure + assert_success run grep 'anticomp=true' "$BATS_TEST_TMPDIR/sandcat/compose-proxy.yml" assert_failure @@ -219,8 +222,11 @@ teardown() { run grep 'stream_large_bodies=1m' "$BATS_TEST_TMPDIR/sandcat/compose-proxy.yml" assert_failure + # connection_strategy=lazy is not agent-specific — it is on the base + # command so a denied flow never reaches the destination (see + # compose-proxy.yml). run grep 'connection_strategy=lazy' "$BATS_TEST_TMPDIR/sandcat/compose-proxy.yml" - assert_failure + assert_success run grep 'anticomp=true' "$BATS_TEST_TMPDIR/sandcat/compose-proxy.yml" assert_failure @@ -255,8 +261,11 @@ teardown() { run grep 'stream_large_bodies=1m' "$BATS_TEST_TMPDIR/sandcat/compose-proxy.yml" assert_failure + # connection_strategy=lazy is not agent-specific — it is on the base + # command so a denied flow never reaches the destination (see + # compose-proxy.yml). run grep 'connection_strategy=lazy' "$BATS_TEST_TMPDIR/sandcat/compose-proxy.yml" - assert_failure + assert_success run grep 'anticomp=true' "$BATS_TEST_TMPDIR/sandcat/compose-proxy.yml" assert_failure diff --git a/cli/test/mitmproxy/test_mitmproxy_addon.py b/cli/test/mitmproxy/test_mitmproxy_addon.py index fd152596..377b9f35 100644 --- a/cli/test/mitmproxy/test_mitmproxy_addon.py +++ b/cli/test/mitmproxy/test_mitmproxy_addon.py @@ -91,13 +91,83 @@ def make(status, body, headers): _http.HTTPFlow = type("HTTPFlow", (), {}) _http.Response = _Response + +# --- proxy layer stubs, for the next_layer hook ---------------------------- +# Names only: the addon compares the layer mitmproxy picked against these +# classes, so identity is all the tests need. +# +# These stubs cannot notice mitmproxy renaming, moving or adding a relay layer +# — the version pin in cli/lib/constants.bash is what holds that end. A bump +# needs mitmproxy's next_layer.py re-read against RAW_RELAY_LAYERS. + + +class _Layer: + def __init__(self, context): + self.context = context + + +class _Start: + pass + + +class _CloseConnection: + def __init__(self, connection): + self.connection = connection + + +class _MitmNextLayer: + """Stand-in for mitmproxy's layer chooser. + + Tests set ``picks`` to the layer class mitmproxy would settle on, or to + None for "needs more data". + """ + + picks = None + + def configure(self, updated): + self.configured = updated + + def next_layer(self, nextlayer): + nextlayer.layer = self.picks(nextlayer.context) if self.picks else None + + +_proxy = types.ModuleType("mitmproxy.proxy") +_proxy_commands = types.ModuleType("mitmproxy.proxy.commands") +_proxy_events = types.ModuleType("mitmproxy.proxy.events") +_proxy_layer = types.ModuleType("mitmproxy.proxy.layer") +_proxy_layers = types.ModuleType("mitmproxy.proxy.layers") +_addons = types.ModuleType("mitmproxy.addons") +_addons_next_layer = types.ModuleType("mitmproxy.addons.next_layer") + +_proxy_commands.CloseConnection = _CloseConnection +_proxy_events.Start = _Start +_proxy_layer.Layer = _Layer +_proxy_layer.NextLayer = type("NextLayer", (), {}) +for _name in ("TCPLayer", "UDPLayer", "RawQuicLayer", "HttpLayer", "DNSLayer"): + setattr(_proxy_layers, _name, type(_name, (_Layer,), {})) +_addons_next_layer.NextLayer = _MitmNextLayer + sys.modules["mitmproxy"] = types.ModuleType("mitmproxy") sys.modules["mitmproxy.ctx"] = _ctx sys.modules["mitmproxy.http"] = _http sys.modules["mitmproxy.dns"] = _dns +sys.modules["mitmproxy.proxy"] = _proxy +sys.modules["mitmproxy.proxy.commands"] = _proxy_commands +sys.modules["mitmproxy.proxy.events"] = _proxy_events +sys.modules["mitmproxy.proxy.layer"] = _proxy_layer +sys.modules["mitmproxy.proxy.layers"] = _proxy_layers +sys.modules["mitmproxy.addons"] = _addons +sys.modules["mitmproxy.addons.next_layer"] = _addons_next_layer sys.modules["mitmproxy"].ctx = _ctx sys.modules["mitmproxy"].http = _http sys.modules["mitmproxy"].dns = _dns +sys.modules["mitmproxy"].proxy = _proxy +sys.modules["mitmproxy"].addons = _addons +_proxy.commands = _proxy_commands +_proxy.events = _proxy_events +_proxy.layer = _proxy_layer +_proxy.layers = _proxy_layers +_addons.next_layer = _addons_next_layer # Allow importing the addon modules from the templates directory. _SCRIPTS_DIR = str( @@ -914,6 +984,77 @@ def test_dns_trailing_dot_allowed(self, addon_cls): assert flow.response is None +# --------------------------------------------------------------------------- +# Raw TCP/UDP flows — denied before they reach the generic relay layers. +# --------------------------------------------------------------------------- + +def _make_next_layer(): + nextlayer = MagicMock() + nextlayer.layer = None + nextlayer.context.client.transport_protocol = "tcp" + nextlayer.context.server.address = ("203.0.113.5", 4444) + return nextlayer + + +@pytest.mark.parametrize("addon_cls", ADDONS) +class TestRawFlowDenial: + @pytest.mark.parametrize("raw_layer", ["TCPLayer", "UDPLayer", "RawQuicLayer"]) + def test_raw_relay_layers_are_replaced(self, addon_cls, raw_layer): + addon = addon_cls() + addon._layer_chooser.picks = getattr(_proxy_layers, raw_layer) + nextlayer = _make_next_layer() + + addon.next_layer(nextlayer) + + assert isinstance(nextlayer.layer, common.DenyLayer) + + @pytest.mark.parametrize("allowed_layer", ["HttpLayer", "DNSLayer"]) + def test_http_and_dns_layers_are_left_alone(self, addon_cls, allowed_layer): + addon = addon_cls() + picked = getattr(_proxy_layers, allowed_layer) + addon._layer_chooser.picks = picked + nextlayer = _make_next_layer() + + addon.next_layer(nextlayer) + + assert isinstance(nextlayer.layer, picked) + + def test_a_chooser_that_raises_denies_the_flow(self, addon_cls): + addon = addon_cls() + addon._layer_chooser.next_layer = MagicMock(side_effect=RuntimeError("boom")) + nextlayer = _make_next_layer() + + addon.next_layer(nextlayer) + + assert isinstance(nextlayer.layer, common.DenyLayer) + + def test_undecided_layer_stays_undecided(self, addon_cls): + addon = addon_cls() + addon._layer_chooser.picks = None + nextlayer = _make_next_layer() + + addon.next_layer(nextlayer) + + assert nextlayer.layer is None + + def test_configure_reaches_the_layer_chooser(self, addon_cls): + addon = addon_cls() + + addon.configure({"tcp_hosts"}) + + assert addon._layer_chooser.configured == {"tcp_hosts"} + + +def test_deny_layer_closes_the_client_without_dialling_the_server(): + context = MagicMock() + + issued = list(common.DenyLayer(context)._handle_event(_Start())) + + assert len(issued) == 1 + assert isinstance(issued[0], _CloseConnection) + assert issued[0].connection is context.client + + # --------------------------------------------------------------------------- # Config loading — exercises the shared load() + write paths. # --------------------------------------------------------------------------- diff --git a/docs/agents/cursor.md b/docs/agents/cursor.md index b8ddb00e..7c050fb3 100644 --- a/docs/agents/cursor.md +++ b/docs/agents/cursor.md @@ -16,8 +16,9 @@ Cursor CLI support is available via `sandcat init --agent cursor`. - **Proxy command defaults tuned for Cursor.** The generated proxy config uses the Cursor addon and keeps mitmproxy HTTP/2 enabled (`http2=true`) (plus streaming-safe mitmproxy - flags such as `stream_large_bodies=1m`, `connection_strategy=lazy`, - `anticomp=true`, and `timeout_read=300`). + flags such as `stream_large_bodies=1m`, `anticomp=true`, and + `timeout_read=300`). `connection_strategy=lazy`, which these streams also + need, is on the base proxy command for every agent. Those streaming-safe flags are **Cursor-only** — they are intentionally omitted on the Claude path (`sct_agent_mitm_streaming_flags`). With diff --git a/docs/configuration/network-rules.md b/docs/configuration/network-rules.md index e29e8ca3..6c18ff6e 100644 --- a/docs/configuration/network-rules.md +++ b/docs/configuration/network-rules.md @@ -92,6 +92,25 @@ A rule like `{"action": "allow", "host": "*", "method": "GET"}` will also allow DNS resolution for any host. Rule ordering matters: a method-specific deny rule will block DNS for that host even if a later rule would allow other methods. +## Non-HTTP traffic + +Rules match on a hostname, and only HTTP/S requests and DNS queries carry one. +Anything else leaving the container — a plain TCP socket, a UDP datagram, a +non-HTTP protocol wrapped in TLS or QUIC, an HTTP request upgraded to a +non-WebSocket protocol — has nothing to match a rule against, so the proxy +closes the connection instead of relaying it. + +What you see in the proxy log depends on which of them it was. UDP and QUIC +give `Network deny (not HTTP or DNS)`; a non-WebSocket upgrade gives `no +protocol is enabled to upgrade to`. A plain TCP connection carrying something +other than HTTP gives nothing at all — the proxy answers it as a malformed HTTP +request and closes. None of the three appear in the mitmweb flow list. + +That includes SSH, which sandcat does not support anyway: `sandcat init` +rewrites GitHub SSH remotes to HTTPS, and no SSH keys reach the container. +Traffic between containers in the same compose project does not go through the +proxy and is unaffected, so a database or other sidecar still works. + ## Examples With the liberal template rules: diff --git a/docs/index.md b/docs/index.md index 550794e4..d5f36b0d 100644 --- a/docs/index.md +++ b/docs/index.md @@ -9,11 +9,12 @@ working in an IDE — both [VS Code](ide/vscode.md) and [JetBrains](ide/jetbrains.md) are supported. All container traffic is routed through a transparent -[mitmproxy](https://mitmproxy.org/) via WireGuard, capturing HTTP/S, DNS, and -all other TCP/UDP traffic without per-tool proxy configuration. A -straightforward allow/deny-list engine controls which network requests go -through, and a secret substitution system injects credentials at the proxy -level so the container never sees real values. +[mitmproxy](https://mitmproxy.org/) via WireGuard, without per-tool proxy +configuration. A straightforward allow/deny-list engine controls which HTTP/S +requests and DNS queries go through; everything else — raw TCP and UDP, which +carries no hostname to match a rule against — is dropped. A secret +substitution system injects credentials at the proxy level so the container +never sees real values. Source code: [github.com/VirtusLab/sandcat](https://github.com/VirtusLab/sandcat).