, 7 min read, ai-agents, sandboxing, systems
A semisolate runs your command for real, capturing filesystem effects in an overlay you can inspect or discard. I built a toy version to find where it breaks.
A coding agent with shell access will eventually run something it shouldn't. Not out of malice, just out of confidence it hasn't earned: a glob that expands wider than anyone meant, an install script that writes somewhere unexpected, a "cleanup" that cleans up the wrong thing.
The usual answer is a container. Let the agent break whatever it wants in there. That does protect your machine, but now the agent is working in a fake copy of your world, without your caches, your credentials, your dotfiles, all the stuff that made the real environment real. Something works in the box and fails outside it, or the other way round.
So I was happy to see that one of this year's OSDI Jay Lepreau Best Paper awards went to a paper about exactly this gap. Three were given out of 136 accepted papers, per Brown's announcement (opens in new tab). "Controlling Opaque-Component Effects with Semisolates and Try (opens in new tab)," by Evangelos Lamprou, Tianyu (Ezri) Zhu and a team mostly from Brown, also won a Distinguished Artifact Award. I read the paper (opens in new tab) and the try README (opens in new tab). Then, since try is Linux-only and I'm on a Mac, I wrote a crude copy-based version to see which parts of the idea actually need the kernel.
One of the paper's first examples is a script an LLM might hand you when asked for something that "cleans up its containing directory":
rm -fr "$(cd "${0%/*}" && echo $PWD)"/*Call it from the wrong place and ${0%/*} expands to the script's own path, cd fails, and rm deletes everything user-writable. You'd catch it in review if you were looking. The problem is that nobody reads every line an agent runs.
With try you prefix the command, and instead of losing your home directory you get this (the transcript is from the paper):
$ try sh delete.sh
/home (deleted)
/etc (deleted)
...
Apply these changes? [y/i/N]Say no and nothing happened. Pick i and you go through the changes one by one and choose which to keep.
The name is "semi-" plus "isolate," and the partial part is the point. A container gives the command a new environment. A semisolate, in the paper's words, provides "the illusion of direct access to its outer environment": the command sees your real files, and its writes land somewhere try controls until you decide what to do with them.
Mechanically it's two Linux features. OverlayFS layers a writable upper directory over your real files: reads fall through to the real file, writes and deletes land in the upper directory (a delete becomes a "whiteout" file). It does this per top-level directory, /bin, /usr, /home and so on, because OverlayFS won't let the upper and lower directories overlap. User and PID namespaces let try do those mounts without root. If you want the order effects happened in, -t traces path-taking calls like open, mkdir and unlink through a Seccomp-BPF filter. When you finally say yes, try walks the upper directory and moves things into place, deleting wherever it finds a whiteout.
On top of that you can stack runs (-L layers a new semisolate over the unapplied effects of an earlier named one), hide paths from the command, and apply subsets.
This isn't mainly an agents paper. It's about "opaque components" in general, meaning anything you run without reading it first: a curl | sh installer, a pre-commit hook, a dependency's install step. But agents keep coming up, and my favourite bit is in the experiences section:
Several students have used try to interpose on agentic workflows, wrapping their session with try and inspecting all produced effects before committing them. Even when the agent performed non-filesystem accesses (e.g., access to webpages, interactions with email and messaging APIs), which often were read-only network effects, try remained useful. It allowed students to mediate all resulting local filesystem effects, critical when the agent performed an incorrect action based on misleading information it fetched from the web.
I think that's the right framing. The agent mistakes I actually worry about aren't attacks. They're the agent acting confidently on something wrong, a stale doc or a misread error message, and the result being a pile of real file changes before anyone looked. A container deals with that by not giving the agent your environment. A semisolate gives it your environment and moves the review to the output. It's git diff for everything the command touched, not just the tracked files in one repo.
The same section has a line that turned out to be the whole point of my experiment: users "often discard effects when the wrapped component wrote outside the expected filesystem subtree, e.g., creating auxiliary files in the home directory." Hold onto that.
On cost, the evaluation includes five shell scripts generated by ChatGPT for tasks "distilled from a survey of LLM-mediated workflows, spanning conventional chat interfaces and autonomous agents such as Codex and OpenClaw." Things like gzipping 10,000 files with find -exec, or sorting a 10-million-line file. Compared with running them bare, try was 1.1x to 1.9x slower, depending on how many effects it had to catch. Compared with Docker set up for full isolation (create a container, copy inputs in, run, tear down, copy outputs out), try was 1.3x to 225.7x faster. The paper calls 225.7x an outlier: that archive script only writes to stdout, so try pays just the cost of creating the semisolate while Docker still pays for setup and teardown. The other four range from 1.3x to 58.2x. I'd read that as "a lot cheaper than a container," not "two hundred times cheaper."
None of this makes try a sandbox, and the README says so: it "should not be used to execute commands that you don't already trust on your system, (i.e. network calls are all allowed)."
-x cuts the command off from the network, and that's the only network option in the paper's flag table. Nothing holds an outgoing request in an upper layer for you to approve later, and I don't see how anything could, because a sent request is sent. The students' agent sessions worked partly because those network effects "often were read-only."So try covers the filesystem half well, gives you a blunt switch for the network, and doesn't try to stage anything else. For an agent that mostly reads files, writes files and runs commands, the filesystem is where the damage tends to land. For code you think is hostile, use a real sandbox.
To find out what the kernel parts are buying, I wrote minitry.py: under a hundred lines of Python doing the dumbest possible version. Copy one project directory to a temp dir, run the command inside the copy, compare the two with filecmp.dircmp, and copy the result back only if told to. --yes and --no stand in for the prompt so the runs are repeatable. The project is two text files, notes.txt ("hello foo world") and data.txt ("another foo line").
First the boring case, npm init -y, applied:
$ python3 minitry.py --yes demo-project -- npm init -y
[minitry] semisolate created: /var/folders/.../minitry-km_gffa2/copy
[minitry] running: npm init -y
[minitry] command exited 0
[minitry] effects captured inside the copy:
+ added: package.json
[minitry] applied to the real project dir.It worked, with one catch I didn't expect. The package.json that landed in my real project says "name": "copy", not "demo-project". npm took the name from the directory it ran in, and that was my temp copy. It's a small thing, but it's the container complaint from the top of this post in miniature: once a command runs somewhere other than your real environment, it can tell, and it acts on that.
Next, a destructive sed -i that deletes every non-empty line (BSD sed syntax, this is macOS), declined:
$ python3 minitry.py --no demo-project -- sed -i '' -e '/./d' notes.txt data.txt
[minitry] semisolate created: /var/folders/.../minitry-vmu32p1a/copy
[minitry] running: sed -i -e /./d notes.txt data.txt
[minitry] command exited 0
[minitry] effects captured inside the copy:
~ modified: data.txt
~ modified: notes.txt
[minitry] discarded. real project dir untouched.Both files really were emptied inside the copy, and the real ones still say "hello foo world" and "another foo line." Inspect-then-discard works fine with a plain copy, as long as the command stays inside the folder you copied.
Now the one where it doesn't. escape.py writes a file into a directory outside the project, standing in for anything that writes to your home directory or a cache:
$ ESCAPE_TARGET=.../outside-sandbox python3 minitry.py --no demo-project -- python3 escape.py
wrote .../outside-sandbox/escaped.txt
[minitry] semisolate created: /var/folders/.../minitry-3va2yqbn/copy
[minitry] running: python3 .../escape.py
[minitry] command exited 0
[minitry] effects captured inside the copy:
(none)"(none)," and then it just exits. It never asks, because as far as it knows there's nothing to ask about. Meanwhile escaped.txt is sitting in outside-sandbox/, written for real1. That's the paper's "auxiliary files in the home directory" case, and my version can't even show it to me, let alone let me say no.
A smarter diff wouldn't fix this. A copy can only ever see the folder you copied. try overlays every top-level directory, so a write to the project and a write to ~/.cache land in the same upper directory, and there's one place to look for everything the command did. That's what the namespaces and OverlayFS are for, and you don't get it from shutil.copytree and good intentions. It's also why I think a staging area like this is a better default for agents than another container, even if my Mac can't run the real one.
wrote line prints before minitry's own lines only because Python buffers minitry's output when it goes to a file or pipe. The write happened during the run, in the child process. ↩
Comments