Proximity to Ground Truth

I asked AI to proofread something I had written. It came back and told me there was an embarrassing error, quoted the problem, and explained why it was wrong.

The error did not exist. It had misread the sentence entirely.

What struck me was the confidence. It did not hedge. It did not say “this may read ambiguously.” It named a mistake with the same certainty it names things that are actually true.

I have not seen that particular failure in a long while, which is exactly why it landed. When a tool is right most of the time, the moments it is confidently wrong are the ones that cost you something.

The only reason I caught it is that I knew the source material better than the model did.

The pattern underneath

That is a small story, but it points at something bigger than proofreading.

AI does not fail because it is unintelligent. It fails, or excels, based on how close its input sits to the actual thing. Everything a model tells you is a claim about source material it was handed. If that material is thin, secondhand, or misread, the output will still arrive polished and confident. Fluency is not evidence.

Which means the question that matters is not “is this AI any good.” It is “how far is this AI from the truth right now, and who in this building is closer.”

Three approaches to the same documentation gap

We had a gap in our product documentation. Most teams do at some point, especially without a dedicated writer. We tried three approaches to catch up.

The first fed AI our work items. It read the tickets and wrote up what the feature did. The output was only ever as good as the tickets happened to be written. My guess is that a consistent, widely used ticket format would have helped it further, though we never tested that.

The second sent AI through the code. Engineering ran this one. It produced technically accurate documentation, and it was the least usable of the three. It described what the system did without ever explaining what the feature was good for or how someone would actually use it.

That is not a knock on the approach. Reading the code is a good way to produce technical documentation, and if that is what you need, it is probably the right input. It is the wrong input for documentation a customer is going to read.

The third took longer. I went into the staging environment and used the feature while narrating out loud, capturing it with Wispr Flow. I took screenshots as I went. Then I fed all of it into a Claude project I had set up with skills for product documentation.

That third version produced the documentation I would actually hand to a user.

Same goal, same ask. Three different systems, three different inputs, three different results. The system that produced the best documentation was not the most technical one. It was the one working from the closest source.

Why the PM sits closest

The walkthrough worked because it carried context the other two inputs did not have.

Tickets capture what got built. Code captures how it got built. Neither one captures what a person is trying to accomplish when they open the screen, which parts confuse them, or which capability they came looking for in the first place.

I had that context because of everything upstream of the walkthrough. I had sat with the customer noise. I had heard the same complaint phrased several different ways and worked out which version was the real problem. I knew which parts of the feature people cared about and which parts nobody had ever asked for.

That is not a documentation skill. It is the product management job, and it happens to be the thing that makes the input good.

What this changes about the work

The common framing is that AI drafts and the PM reviews. That gets it half right and misses the more useful half.

Reviewing output only works if you are close enough to the truth to catch a confident error. That is the proofreading story. But the larger opportunity sits earlier in the process. The PM is usually the closest available source of ground truth in the room, closer than the tickets, closer than the code, closer than any model working from either one. Being that source is worth more than being a reviewer.

I do not think this is permanent. Models will get better at pulling context from more places, and some of the distance I am describing will close. What I am less sure will close is the part that comes from sitting with customers and knowing which part of the noise is signal. Nobody has automated that yet.

Which is why I am less worried about AI replacing product managers than I am about the wrong product managers being the ones who adopt it fastest.

AI brings speed. It will draft the documentation, the spec, the summary, and the analysis quicker than any of us can. What it cannot bring is the reason any of that is worth writing. That comes from judgment about what actually matters, empathy for the person on the other side of the screen, the leadership to hold a position when the easy answer is available, and a working knowledge of what customers are actually saying.

Those were always the job. What has changed is that they are now also the input. A product manager who is close to the customer and fluent with these tools does not just review better output. They produce better output, because they are feeding the model something nobody else in the building has.