A home lab, a local LLM, and letting agents write code

I spend my working days building open-source R tooling for clinical reporting, where data privacy is non-negotiable. That constraint nudged me, at home, toward a question I find genuinely fun: how much can you do with models that never leave your own hardware?

The setup

I have a box at home with a decent GPU running a local LLM. Nothing exotic — the point is that the weights, the prompts, and the data all stay on a machine I control. No tokens leaving the house, no per-call meter ticking.

The first project is turning it into a kind of home assistant: a model that can answer questions, summarize, and eventually wire into the small automations around the flat. It’s early. Most of the joy so far is in the plumbing — getting the model served reliably, keeping latency sane, and learning where a 7B-to-30B class model is genuinely useful versus where it just produces confident nonsense.

The honest takeaway is mixed. Local models have gotten genuinely capable for well-scoped jobs — summarization, drafting, classification, “explain this to me” — and they run offline for the cost of electricity. But the gap between a small local model and a big one with a lot of context is huge: it’s the difference between solving an issue and not even understanding the question. The best of the best is another tier again — large, long-context models that aren’t usually open-source and whose working size is massively beyond anything I can host at home. So local is great for the narrow, private tasks; it doesn’t replace reaching for the big model when the problem is actually hard.

Letting an agent drive

The second experiment is on a spare laptop I had lying around: handing an agentic coding setup the keyboard and watching it work. I let it scaffold, edit, and iterate on real code while I reviewed.

This is the part that genuinely shifted my mental model. Reviewing an agent’s diffs feels less like “using a tool” and more like a fast, tireless junior pair who reads the whole codebase before every change. You still own the judgment — architecture, taste, knowing when the “working” solution is the wrong one — but the mechanical distance between intent and a running prototype has collapsed.

A word on doing this safely. I run these experiments on a clean, dedicated machine — not my daily driver — which mirrors good system hygiene: a known-good baseline you can wipe and rebuild. The sensible next steps are dedicated accounts and emails for the agent, a firewall (or a separate subnet) so it can’t reach the rest of the home network directly, and least-privilege everywhere. Jailbreaks are still too easy; isolation is cheaper than trust.

“Is the token cost cheaper than human labor?”

The obvious question, and the one everyone asks: does it pay off?

Run the back-of-envelope numbers and the framing almost feels wrong. A focused hour of a developer’s time costs more than a lot of tokens. For the kinds of tasks an agent is good at — boilerplate, refactors, test scaffolding, “translate this idea into a first draft” — the marginal cost of compute rounds to noise next to the cost of the same hour done by hand.

But honestly? I’m not sure today’s prices are the real prices. There’s a price war on, and a lot of it looks like running at a deficit to capture users while the investment wave lasts. So the comparison might not hold. A few open questions I keep turning over:

• Who pays in the end? When the subsidies thin out, do we split into power users with budgets for expensive, top-tier tokens, while everyone else drops back to cheaper, smaller models for everyday coding?
• Whose models? Other ecosystems — China’s especially — are pushing hard on cost. In a globalized market, do we just shift providers to chase the best quality-per-cost, and does that quietly reshape who builds with what?
• Quality/cost vs. labor/cost. The trade-off isn’t one ratio, it’s two moving ones, and they don’t move together everywhere.

So “cheaper than a human” is the wrong question, and the honest answer to “is it cheaper?” today is we don’t really know yet. But I don’t think that’s the scary part. Here’s the hopeful version — not a prediction, just the one I’d like to be right:

• The floor rises. The tedious 60% of building software — the parts that were never the fun or valuable bit — gets cheaper and faster. That’s pure upside.
• The ceiling moves too. When prototyping is nearly free, you try more ideas. I’ve built small things at home this year that I simply wouldn’t have started, because the activation energy was too high. More experiments means more of them are good.
• Judgment becomes the scarce resource. Knowing what to build, why, and when something is subtly wrong — that’s still entirely human, and agents make that skill more valuable, not less.

The local-LLM angle reinforces the point: the most useful AI isn’t always the biggest model behind the most expensive API. Often it’s a modest model running on hardware you own, doing a well-scoped job reliably and privately. That’s a future I’m happy to live in — and tinkering toward it on a Sunday is the most fun I’ve had with computers in a while.

More to come as the home assistant grows up. For now: the GPU is warm, the electricity bill is survivable, and the spare laptop is busy writing its own code.