I have a habit of taking automation a little too far.
What started as an experiment with a personal AI agent has gradually worked its way into a surprising amount of my life.
I use Hermes, a self-hosted AI agent, for research and writing. It helps with my social content. It works with parts of my homelab. I’ve been experimenting with connecting it to personal workflows, information and other systems I use every day.
Some of that is practical.
Some of it is me seeing just how far I can push this before I drive my wife crazy.
But there’s a serious reason I keep experimenting with it.
I don’t think the future of personal AI is another chatbot.
I think it’s delegation.
ChatGPT taught millions of us to talk to AI.
Personal agents are going to ask us to trust AI to act.
I ran straight into that distinction recently while working with Hermes on publishing one of my articles.
Researching the article? Easy.
Helping me write it? I do that all the time.
Finding supporting information, organizing files, preparing social posts? Fine.
But publishing something as me?
That felt different.
Now the AI wasn’t helping me do something.
It was doing something on my behalf.
Once you cross that line, a whole new set of questions shows up.
Does it have the right version?
Does it understand what I actually intended?
Is the information it’s relying on current?
Does it have permission to take this particular action?
What happens if two sources disagree?
And maybe most importantly:
What makes it stop?
If an AI gives me a bad answer in a chat window, I can read it and decide what to do.
If an agent sends an email, publishes an article, changes a configuration, moves money, deletes a file or takes some other action on my behalf, the mistake has already left the chat window.
So I’ve found myself asking a different question:
Before my AI agent acts, who checks its work?
Increasingly, I don’t think the answer should always be me.
Personal agents are going to ask us to do something chatbots never did: trust AI to act on our behalf.
This Isn’t Just a Homelab Experiment Anymore
This question is getting real pretty quickly.
Earlier this month, Meta launched Muse, which it explicitly calls a personal AI agent.
That distinction matters.
Muse isn’t positioned as another place to ask questions. Meta says it can send email, book travel, open a browser, fill out forms and continue working after you close the app. It can connect to other services and work on your behalf.
At Meta Connect this week, the company took the idea further.
Muse is coming to Meta’s AI glasses. You’ll be able to talk to your agent throughout the day, have it act on what you’re looking at and let it continue working in the background while you’re still talking to it.
That gets my attention.
What I’ve been tinkering with in my homelab is quickly becoming a consumer product category.
And a lot more people are going to face some version of the same question I’ve been wrestling with:
How much of my digital life am I willing to hand to an AI?
Meta appears to understand the tension.
Muse runs in a dedicated secure virtual machine. More interesting to me, Meta created a separate Sentinel agent that is isolated from Muse at the system level. According to Meta, nothing Muse does reaches the internet unless Sentinel approves it. Sensitive actions such as sending an email or making a purchase can also require user approval. Muse keeps an audit trail of what it has done and what it plans to do.
Think about that architecture for a minute.
Meta didn’t just build a smarter agent.
It built something around the agent that can effectively say:
No. Not yet. Check first.
That changes the conversation.
This isn’t only a model problem.
It’s an architecture problem.
The Problem With One Really Smart Model
Most of us still talk about AI as if there’s one big decision to make:
Which model should I use?
Which one reasons better?
Which one codes better?
Which one has the biggest context window?
Which one should power my agent?
I’ve been doing the same thing with Hermes.
Running your own agent makes this especially obvious because suddenly you’re thinking about models, routing, cost, latency, context and tools instead of simply opening an app and typing.
But the agent doesn’t actually have one job.
Imagine I tell Hermes:
Find the latest version of this article and publish it.
Sounds like one task.
It isn’t.
The agent has to understand what I mean.
Find the available versions.
Determine which one is current.
Evaluate the information it finds.
Resolve conflicts.
Decide whether it has enough information.
Determine whether publishing is appropriate.
Confirm it has permission.
Only then should it publish something.
We tend to collapse all of that into:
Go do this.
Then we ask one very capable model to figure out everything in between.
The more I experiment with agents, the less convinced I am that’s the right architecture.
What If the Agent Had a Second Opinion?
That question sent me down another rabbit hole.
I started experimenting with Jev, a specialized judgment model from TypeSafe AI.
What interested me about Jev wasn’t its ability to have another conversation with me.
It was almost the opposite.
Instead of asking it to research something and explain the answer in three paragraphs, I can give it a situation and ask much narrower questions.
Is there enough information to proceed?
Do these sources agree?
Does this require investigation?
How consequential is the proposed action?
Rather than another essay, I can get structured judgments that software can actually use.
That’s what caught my attention.
Not another model that could give Hermes an answer.
A model that might give Hermes a reason to stop.
So I tested it against a real situation from my workflow.
There was conflicting information about my publishing schedule. One source suggested one thing. Another suggested something else.
A reasoning model could examine the evidence, make its best determination and continue.
Instead, I inserted a judgment step.
The result?
ACTION: INVESTIGATE
It also determined the context was insufficient and the sources conflicted.
Perfect.
Not another answer.
Hesitation.
The call took about 300 milliseconds and cost roughly three-thousandths of a cent.
That matters.
If every decision requires another long, expensive reasoning cycle, we probably aren’t going to check everything.
But if judgment can be fast and inexpensive, asking:
Should I actually do this?
can become a normal part of the agent loop.
As agents become more capable, intelligence isn’t enough. We also need to engineer hesitation.
A Smarter Agent Can Still Have Bad Context
There’s another lesson hiding in that experiment.
The reasoning model didn’t necessarily have to be wrong.
The information was conflicting.
That distinction matters.
Definitions can conflict.
Information gets stale.
Ownership can be unclear.
Two systems can disagree about which version is authoritative.
A model can reason perfectly over the information it has and still produce the wrong outcome.
My own automated research pipeline surfaced a line this week that captures it nicely:
A smarter agent can still make a bad decision from bad context.
Context quality becomes part of the control system, not something we document afterward.
And adding another model doesn’t magically fix it.
Jev won’t always be right either.
Some questions shouldn’t be sent to an AI model in the first place.
Use code for math.
Use deterministic systems for hard rules and date comparisons.
Use reasoning models where reasoning makes sense.
Use specialized models where judgment or classification makes sense.
Use policy to establish boundaries.
Use identity to determine authority.
Bring in a human when uncertainty or consequence crosses the appropriate threshold.
The answer isn’t another all-knowing AI checking the first all-knowing AI.
Maybe we need to stop asking one AI to do every job.
Reason → Judge → Act → Verify
That led me to a simple framework for how I’m starting to think about my own agent:
Reason → Judge → Act → Verify
Reason
What am I asking the agent to accomplish?
What information does it need?
What should happen next?
This is where today’s reasoning models are incredibly powerful.
They can take a messy human request and turn it into something actionable.
Judge
Should the proposed action actually happen?
Do we have enough information?
Are the sources consistent?
How confident are we?
Is the consequence high enough that someone else should be involved?
The judge doesn’t have to be Jev.
It could be another model.
It could be deterministic policy.
It could be several checks working together.
It could be a human.
The important part is simple:
Reasoning and approval don’t have to be the same operation.
Act
Only then does the agent use its tools.
Publish the article.
Send the email.
Turn off the lights.
Update the database.
Change the configuration.
Make the reservation.
This is where personal agents get exciting.
It’s also where they become consequential.
I’m perfectly comfortable letting Hermes interact with some parts of my homelab.
Turning off the wrong light is annoying.
Publishing the wrong article under my name is something else entirely.
Changing production infrastructure is another level again.
The tool call might look similar to the agent.
The consequence isn’t.
Verify
Then comes the piece I think we’re going to talk about much more as agents mature.
Did the intended thing actually happen?
Not:
Did the API call succeed?
But:
Did we get the intended outcome?
The call may have worked exactly as designed.
The action may still have been wrong.
Did the right article publish?
Did the right person receive the email?
Did the configuration actually change?
Can the action be reversed?
Can I reconstruct what happened?
An agent saying “done” isn’t verification.
A successful tool call is not proof of a successful outcome.
Identity Runs Through All Four
There’s another layer across this entire architecture.
Identity.
An agent shouldn’t be able to take an action simply because it concluded the action makes sense.
It also has to answer:
Am I allowed to do this?
Those are different questions.
A judgment system might conclude with extremely high confidence that deleting something would solve a problem.
That doesn’t mean the agent should have permission to delete it.
Reasoning determines what could be done.
Judgment helps determine what should be done.
Identity and policy determine what may be done.
Verification tells us what actually happened.
This connects directly to something I’ve written about before:
Authentication may happen once. Agency happens continuously.
The more access I give Hermes, the more important that distinction becomes.
The agent isn’t simply logging into a system.
It may make hundreds of decisions after access has been granted.
Meta’s Muse illustrates the same issue at consumer scale. Users decide which services Muse can connect to and, in cases such as email, whether it can only read messages or also send them.
Access isn’t just connected or disconnected anymore.
It can be bounded by action.
Trust Shouldn’t Be Binary
This has also changed how I think about autonomy.
The choice isn’t:
AI does everything
or
A human approves everything.
There’s a lot of room between those two extremes.
I think of it as a trust ladder:
Observe → Recommend → Dry Run → Bounded Execution → Verify → Automate → Monitor
Let the agent watch first.
Then let it recommend.
Let it show what it would do without actually doing it.
Give it narrowly bounded authority.
Verify what happens.
As confidence grows, automate more.
Keep monitoring.
That’s basically how we establish trust with people.
We don’t normally give someone unrestricted access to everything on their first day because they interviewed well.
Responsibility expands as judgment is demonstrated.
Why would an AI agent be different?
And the boundaries don’t have to be the same for every task.
I’m comfortable letting Hermes do some things without asking me.
Others still require approval.
Some things it simply shouldn’t have permission to do.
That isn’t a limitation of the agent.
That’s good architecture.
Autonomy is starting to look less like a switch and more like a spectrum.
This Is Why Personal Agents Get Interesting
This is why I’m increasingly convinced personal agents are much more interesting than another generation of chatbots.
The interface may eventually become the least interesting part.
Today I type to Hermes.
Tomorrow I may talk to my agent.
Eventually I may barely interact with many of the applications sitting behind the agent.
I shouldn’t necessarily need to open the right app, navigate to the right screen, find the right button and fill out the right form.
I should be able to say what I want accomplished.
The agent becomes my front door.
My email.
My calendar.
My content.
My files.
My home.
My infrastructure.
Maybe someday my finances.
That’s incredibly powerful.
It’s also why the trust problem gets bigger as the agent gets better.
The more useful Hermes becomes, the more access I want to give it.
The more access I give it, the more consequential its mistakes become.
And suddenly the question isn’t:
How smart is my AI?
It’s:
How much authority am I willing to give it?
The better my personal agent gets, the more I want to give it access. The more access I give it, the more important it becomes to know when it will stop.
Maybe the Future Isn’t One AI
Hermes is my laboratory.
Jev is one experiment in giving it better judgment.
Meta is trying to solve the broader personal-agent problem for a mass-market audience with Muse.
The implementations are very different.
But the question underneath them is remarkably similar:
How do we give an AI enough authority to be genuinely useful without giving it so much authority that a mistake becomes an action?
I don’t know yet whether Jev will become a permanent part of my personal-agent architecture.
And I don’t think Jev itself is really the point.
The idea behind it is.
The more autonomy I give an AI, the less comfortable I am with one model being the thinker, judge, executor and auditor of its own work.
That’s a lot of authority in one place.
Maybe the future isn’t one incredibly smart AI doing everything for us.
Maybe it’s a collection of specialized systems.
One reasons.
Another judges.
Software enforces rules.
Identity establishes authority.
Tools perform actions.
Verification checks the result.
Humans step in when uncertainty or consequence demands it.
That sounds more complicated than a chatbot.
It is.
But we’re not just building chatbots anymore.
We’re building systems that can act on our behalf.
And if I’m going to give an AI the keys to more of my digital life, I don’t just want it to know what to do.
I want it to know when to stop.
Sometimes the most intelligent action my personal agent can take is no action at all.