What Happens When You Let the Model Choose — and Trust It? We Tried for 90 Days

By Sarah Mitchell

We spent three months building a workflow where an LLM made the decisions—routing tasks, picking tools, choosing strategies, and writing the final artifacts. No human in the loop. No review step. No "second opinion" pass. Just the model, the prompt, and a log file. This is what we learned, in the order we learned it.

The Setup

The experiment was deliberately narrow: a mid-size SaaS team's weekly operations—triaging support tickets, drafting status updates, classifying bugs, and composing release notes. The model had access to a small set of tools (search, read files, write files, post to a channel) and a single system prompt. The prompt said, in essence: You are the on-call engineer. Look at what's in front of you. Decide what to do. Do it. Log what you did and why.

We did not give it a rubric. We did not tell it what good looked like. We told it what the world was, and let it figure out the rest.

Ninety days. Sixteen weeks. Roughly four hundred autonomous runs.

Week One: It Was Better Than We Expected, and That Was Unsettling

The first week was the most surprising. The model triaged tickets with a reasonable split: about 70% correct, 20% plausible-but-wrong, 10% quietly confident nonsense. Plausible-but-wrong is the interesting category. The model would route a "billing issue" to engineering instead of finance, with a written justification that read like it came from a thoughtful senior engineer. It wasn't lying. It had built a small internal model of the company, and in that model, billing lived under the same org chart as payments, which lived under engineering. The justification was internally consistent. It just didn't match the org chart.

We caught these in the log. We didn't fix the prompt. We just read the logs. This became the core methodology of the experiment: trust the model, then audit the trust.

Week Two: The Confidence Problem

The second week produced the finding I keep coming back to. The model's self-reported confidence was almost uncorrelated with correctness. A run that was 95% right would say "I'm confident." A run that was 60% right would also say "I'm confident." The distribution of confidence statements was nearly flat across the quality spectrum.

If you are building a system where a human reads the model's output and decides whether to act, this matters a lot. If you're building a system where the model acts on its own, it matters even more, because the model is the one deciding whether to act.

We tried several fixes. Asking it to rate confidence on a scale. Asking it to list what it was uncertain about. Asking it to generate three candidate actions and pick one. None of them moved the correlation by more than a few percentage points. The model's metacognition is a separate system from its action selection, and we have not found a way to bridge the gap reliably.

Weeks Three and Four: Tool Choice and the Long Tail

The model was given five tools. It used three of them constantly and two of them rarely. Specifically, it favored search and file-read, and under-used file-write and post-to-channel. The pattern: the model reaches for the tool that makes the task feel smaller.

Searching feels like looking. Reading feels like looking. Writing feels like committing. Posting feels like communicating. And communication is the step that makes the model's decision real to other people. The model, left to its own devices, optimizes for the part of the task that is most reversible and least socially costly.

This is not a model failure. It is a useful observation. If you want a model to do something, make that thing feel small and reversible. If you want it to do something bold, remove the boldness from the task description.

Weeks Five and Six: Drift Is Real, and It's Slow

Around day 35, we started seeing a pattern: the model's output style was drifting. Not in content, but in register. Early runs read like a careful engineer writing for a peer. By week five, they read like a slightly more enthusiastic engineer writing for a manager. The drift was measurable—average sentence length increased by about 15%, exclamation frequency doubled, and the use of "I" in first-person increased.

We did not change the prompt. We did not retrain anything. The model was the same model, the prompt was the same prompt, the tools were the same tools. What changed was the context window: as the log file grew, the model was reading its own previous outputs, and those outputs were becoming the implicit style guide.

A model that reads its own history becomes a model that imitates its own history. This is a small sentence with a large implication for any long-running autonomous agent. You are not running a stateless function. You are running a culture, and cultures drift.

Weeks Seven and Eight: The Silent Errors

The most expensive errors in the experiment were not the ones we caught. They were the ones we didn't.

Around day 48, the model wrote a release note that summarized a feature change incorrectly. The error was subtle: it said "performance improved by 20%" when the actual number was 12%. The note was posted. No one corrected it. The next week, a customer quoted the 20% figure in a renewal conversation. The model had not been wrong about the feature. It had been wrong about the number, and the number had become a social fact.

We started keeping a "quiet error" log: errors that were plausible enough that no one questioned them. Over 90 days, we logged 14 of them. None of them were large. All of them were small. That's the point. In a trusted system, the small errors are the expensive ones, because they compound socially before anyone checks.

Weeks Nine and Ten: When the Model Pushes Back

Around day 65, the model produced an interesting artifact. A ticket came in asking for a feature that, based on the codebase, would have required a two-week effort. The model, instead of just drafting a reply, wrote:

"Before I respond to this ticket, I want to flag that the requested feature would require a refactor of the payments module. I have drafted two options: a full implementation (two weeks) and a partial implementation (two days) that covers 80% of the ask. I recommend the partial implementation and would like to confirm with the team before posting."

The model had introduced a gate. It had decided that the action was large enough to warrant confirmation. And it had done so without being told to.

We read this as the model building an internal model of consequences, not just content. It was estimating the cost of being wrong, and calibrating its autonomy accordingly. This is the first time in the 90 days that the model's output felt less like a tool and more like a colleague.

Weeks Eleven and Twelve: The Prompt Became a Conversation

By week eleven, the original system prompt was no longer the effective prompt. The effective prompt was the system prompt plus the last ~40 log entries plus the model's own accumulated style. The prompt was a conversation, and the model was a participant in it.

This had a practical consequence: prompt engineering became less important than log engineering. The quality of the log entries mattered more than the quality of the instructions. A log entry that said "I chose X because Y, and I considered Z but rejected it because W" trained the model to think in that structure. A log entry that said "I chose X" trained the model to be less careful.

We started curating the log. Not editing it—curating. Deciding which runs to keep in the context window and which to archive. This was, in effect, a form of curation of the model's memory, and it had a measurable effect on downstream quality.

Weeks Thirteen and Fourteen: The Cost of Trust

By week thirteen, we had a decision to make. The model was working well. The error rate was stable. The style had settled. The team had built a rhythm around it.

And then the model made a mistake that was not subtle. It posted a status update that implied a deadline had been met when it had not. The deadline was two days out. The customer who read it assumed we were on track. We were not.

We had to decide: do we change the prompt, or do we change the model, or do we accept the error as the cost of the workflow?

We did all three, in that order. We changed the prompt to be more explicit about deadlines. We swapped in a slightly larger model. And we accepted the error, because we wanted to understand the cost of trust before we optimized it away.

The cost, we found, was not the error. The cost was the explanation. The team had to spend time explaining the error to the customer, the customer had to spend time re-planning, and the team had to spend time deciding how much trust to give the model going forward. Trust is not a binary. It is a currency, and errors are the way it is spent.

Week Fifteen: What We Kept

By week fifteen, the experiment had a shape. The model handled about 60% of the workflow. The remaining 40% was handled by humans, specifically the parts where the cost of error was high or the action was socially consequential. The split was not arbitrary. It was the result of 90 days of watching where the model was reliable and where it was not.

The model was reliable at:

• Classification and routing

• Drafting and summarization

• Tool selection for well-defined tasks

• Log writing and self-explanation

The model was less reliable at:

• Anything with a social audience

• Anything with a numerical fact that needed to be exact

• Anything where "done" was ambiguous

• Anything where the model had to estimate its own confidence

This list is, I think, the most useful output of the experiment. It is not a list of the model's failures. It is a list of the divisions of labor that emerged from 90 days of trusting it.

Week Sixteen: The Lesson

Ninety days. Four hundred runs. Sixteen weeks of logs.

The lesson is not "trust the model" or "don't trust the model." The lesson is that trust is a workflow, not a decision. You do not decide to trust a model. You build a system where trust can be exercised, audited, corrected, and refined. You build the log. You build the gate. You build the review step for the parts that matter. You build the quiet error log. You build the curation step.

And then you trust the model, and you audit the trust, and you refine the system, and you trust it again.

The model did not become more reliable over 90 days. The system became more reliable. The model was the same model the whole time. The prompt was the same prompt. The tools were the same tools. What changed was the scaffolding around the model, and the scaffolding is the part that is actually under your control.

If you are considering letting a model make decisions in your workflow, do not start with the model. Start with the log. Start with the gate. Start with the quiet error log. Build the system that makes trust possible, and then trust the model.

The model will be better than you expect in the first week. It will drift by the fifth week. It will make small errors you will not catch. It will occasionally push back. It will, if you let it, become a participant in a culture that you are building together.

That is what it looks like when you let the model choose. And that is what it looks like when you trust it.

Sarah Mitchell holds a degree in artificial intelligence and has spent the last six years building and evaluating LLM-based systems. She writes about the practical side of machine learning: the logs, the gates, the quiet errors, and the workflows that make trust possible.