Justin PenningtonJustin Pennington
All posts

AI Agents Do Not Manage Themselves

Running AI agents inside a seven-person company is not a set-it-and-forget-it proposition. Agents need supervision, review cycles, and someone who decides when to trust the output — the same things any new hire needs, minus the small talk.


There is a version of the AI story where you plug in an agent, point it at your operations, and walk away. I have not lived that version. What I have lived is a seven-person company where agents handle real work inside our products every day, and where the management overhead of those agents is not zero.

The overhead is not large. It is not what we spent on the twenty-three people who are no longer here. But it is real, and if you skip it, the agents become the same thing every other poorly adopted tool becomes: something that runs in the background producing output nobody trusts.

The review is the whole game

I have written before about putting AI on the task board — making agents assignable the same way people are, with tasks, due dates, and review steps. That post is about architecture. This one is about what happens after the architecture is in place.

What happens is reviewing.

When an agent completes a task in IFX Hub — drafting a client summary, assembling data for an invoice, compiling a first pass on a bid section in IFX Bid — the output lands in a review queue. A person reads it, checks it against what they know about the client or the project or the solicitation, and either approves it, edits it, or sends it back. That cycle is the entire value proposition. Without it, you have an agent producing work that might be right and might be subtly wrong, and nobody finding out which until something breaks downstream.

The temptation is to stop reviewing the things the agent gets right consistently. After the twentieth client summary that needed no edits, it feels wasteful to keep reading every one. But the twenty-first is where the agent misses a context change — a project scope shift, a pricing conversation that happened off-system, a relationship nuance that lives in someone's head and not in the data. The review catches that before it reaches the client.

In IFX Hub, agent-completed work shows up in the same review column as human-completed work. The person checking the board in the morning does not have to go looking for agent output — it is already there, mixed in with everything else.

Trust is earned by the task, not by the agent

One thing I got wrong early was thinking about agent trust as binary: either you trust the agent or you do not. In practice, trust is per-task. An agent that reliably drafts internal status updates from structured data might be unreliable at assembling the narrative section of a bid response. The data extraction is deterministic — inputs map cleanly to outputs. The narrative work requires judgment about emphasis and tone. Same agent, same model, different reliability.

Supervision is a spectrum. Some tasks go through light review — a quick scan to confirm the output matches the pattern. Others go through heavy review where the reviewer is essentially rewriting portions. Some tasks stay with people entirely because the cost of a bad output outweighs the time saved.

We keep adjusting that spectrum. A task that started with heavy review migrates to light review after consistent output. A task on light review gets bumped back after a miss. There is no final state. It is ongoing calibration.

What failure actually looks like

At a company of seven people, an agent failure is not an abstract incident report. It is Tuesday morning and the client update that went out had a detail wrong, and the client noticed before we did.

The failures that matter are not the dramatic ones — the obviously wrong output anyone would catch. They are the plausible-looking outputs that contain a subtle error. A summary that describes the project status accurately except for one milestone the agent did not pick up because the status came through a channel it does not monitor. An invoice draft that uses the right line items but the wrong rate because the pricing changed in a conversation last week, not in the system.

These are the same errors a new employee would make. The difference is that a new employee learns from the correction. An agent makes the same class of error next time unless you change the input, the prompt, or the scope. With a person, you coach. With an agent, you adjust the system.

The data underneath has to be right

Every post I have written about clean data and systems of record applies double when agents are doing the work. An agent working from dirty data does not pause and say "this looks off." It processes what it sees and produces output with the same confidence regardless of whether the inputs were right.

This is why consolidation matters more after you start running agents. When a deal, its delivery tasks, and its billing all live in the same place, the agent has the full picture. When they live in three separate systems, the agent either sees one slice or you are building integration logic to assemble the context, and that integration layer becomes another place where data goes stale.

Infraxio has never raised venture capital. The products — Hub, Growth7, IFX Bid, DockOps — were funded from revenue and partnerships. We cannot paper over bad data architecture with headcount. The architecture has to be right, because the agents downstream will surface every seam and every stale field.

The management layer nobody plans for

Most AI tooling is built around capability: what can the agent do? The part that gets less attention is the management layer: what does the operator see, how do they intervene, and how do they know whether the agent is earning its keep?

In our setup, that management layer is the task board itself. Agent work has the same audit trail as human work — who assigned it, when it started, what it produced, who reviewed it, what the reviewer changed. When I want to know whether the agents are actually saving time, I look at the review data. How often does agent output ship as-is versus get edited? Which task types have the highest edit rate? Where are we spending more time reviewing than we would have spent doing the work ourselves?

Those are operations questions, not technology questions. And they determine whether running agents at a small scale is working or just feels like it should be.

Small teams feel the overhead faster

A seven-person company has no room for overhead that does not pay for itself. When we had about thirty people, a poorly performing process could hide — someone would absorb the extra work, and it would not be visible until that person left. At seven, everything is visible. If reviewing agent output takes more time than doing the work manually, someone notices by Wednesday.

That visibility is an advantage. An agent that saves four hours a week on drafting but creates two hours of review is a net gain. An agent that saves one hour but creates three hours of correction is a net loss, and it needs to improve or lose the task.

The businesses I talk to across St. Johns County and Jacksonville and elsewhere that get real results from AI treat this calculus seriously. Not the ones with the most agents deployed. The ones that review the work, watch the edit rate, and adjust scope based on what they actually see.

The takeaway

If you are adding AI agents to your operations, plan for the management layer. Plan for the review cycles. Plan for someone who watches the output and decides, task by task, whether the agent has earned the right to keep doing that work.

Agents do not manage themselves. Neither did the people they replaced. The management just looks different now.