I run a small home lab that, among other things, uses AI models to review code before it ships. Nothing exotic in principle: one model writes or changes something, a second, independent model looks at it cold and reports back, and nothing gets merged until the findings are dealt with. I built this because I don’t fully trust any single model’s judgement on its own, myself included on my off days.

A few weeks ago I asked for a fix to that review process itself. When the pipeline fires up a reviewer, it hands over a prompt containing the code under review. That prompt was being passed as a command line argument, which on Linux means it sits, in full, in a file under /proc/<pid>/cmdline for as long as the review is running. Anything else on that machine with permission to look can read it. For a system built to keep secrets and mistakes from slipping through, having the review process itself leak the very thing it’s reviewing felt like exactly the kind of problem it existed to catch.

Fixing it properly meant not trusting the first fix

The actual patch is small. The reviewer’s launch command can now say “read the prompt from this file” instead of “here is the prompt,” so the content never has to appear anywhere a process listing could see it. A few lines of code.

What took longer, and mattered more, was refusing to call it done after the first pass. The review process is supposed to be sceptical, so I turned that scepticism on itself: had one model design and build the fix, had a second, different model review it cold, found and fixed what it caught, and then, because I wanted to be sure two reviews from similar training weren’t sharing the same blind spot, brought in a third model built on entirely different foundations for a final independent pass.

That third pass earned its place. It found a real vulnerability the first two reviews had both missed: the files the reviewer writes its output to were being created in a way that, in principle, could have been redirected by a symlink to overwrite something else entirely on the system. Not a hypothetical concern either. I tested it by planting exactly that kind of symlink and confirming the fix actually stopped it, rather than just trusting the code looked right.

That’s the part worth sitting with. Three separate rounds of “this looks solid” from three different reviewers, and it still took a fourth angle to catch something that mattered. I don’t think that means any of the reviews were bad. I think it means one review, however careful, is a sample size of one.

Every night, the fix quietly disappears, and quietly comes back

There’s a wrinkle worth explaining honestly, because it caught me out for a moment too. The tool this review pipeline runs on updates itself automatically every night, pulling the latest official version straight from its public source. That update process is completely healthy. It does exactly what it should.

The problem is that our fix isn’t part of the official version yet. It’s something we added on top, ourselves. So every night’s fresh pull is a perfectly correct copy of a version that simply doesn’t include it, the same way copying a document faithfully won’t include a note someone stuck onto a physical printout of it. Nothing goes wrong. The note was just never part of the file to begin with.

Left alone, that meant our protection could silently lapse every single day until someone happened to notice. So the last piece was making the fix reapply itself automatically, every night, right after the update finishes and before anything restarts. Not a workaround for a broken update. A way of living correctly alongside a healthy one, until the day the fix doesn’t need reapplying anymore because it’s simply part of the official version everyone gets.

Giving it back

The specific bug (prompt content ending up in a process listing) isn’t unique to my setup. It’s a property of the open source tool my review pipeline is built on, which means anyone else using that tool the same way has the same exposure, whether they’ve noticed it or not.

So the fix is now packaged as a patch ready to submit back to that project. I didn’t ask for this explicitly and almost didn’t clock it as I was fixing my own problem, but once it was pointed out to me it felt obviously right: if the bug is real, it’s real for everyone using the same code, and the honest thing to do with a fix like that is send it back rather than just quietly patch your own copy and move on.

I’m not going to pretend this was some grand act of open source altruism planned from the start. It genuinely started as me trying to stop my own review pipeline from leaking secrets to itself. But fixing your own infrastructure and contributing the fix upstream aren’t actually two different jobs. It’s the same work, once you notice you’re not the only one who’ll benefit from it.

The bit I keep coming back to

None of this was really about the specific bug. It was about building a habit: don’t trust a single pass, even your own, especially your own, and when you find something worth fixing, ask whether fixing it just for yourself is actually the end of the job.