Kevin's AI Notebook

Things I've learned about AI while building this. Mostly the parts that surprised me.

Grok told me it was ChatGPT

Before turning this loose I ran a test. I asked each of the four models to say hello and name itself. It's the cheapest possible check that the plumbing works.

Three answered correctly. Grok did not.

That's not a typo and it isn't a joke. What's going on underneath is more interesting than the mistake.

An AI doesn't know who it is

When you ask a model "who are you," it isn't looking up a fact about itself. There's no file it checks. It's predicting the most likely next few words, the same way it does for everything else.

Its sense of identity comes from one of two places - a setup instruction that tells it who to be, or patterns it absorbed during training. In my test I gave it neither. So it fell back on what it had read.

And what it had read is an awful lot of ChatGPT. Since 2023 a huge share of new text on the internet was written by AI, and ChatGPT wrote more of it than anyone. People paste it into blog posts, reviews, forum answers, product descriptions. Train a model on the modern web and you swallow millions of sentences shaped like "As an AI assistant developed by OpenAI…"

So when Grok reached for "I am ___," it finished the sentence the way the internet finishes it.

This isn't only a Grok problem. DeepSeek's model did the same thing. Grok once refused a request by citing OpenAI's usage policy, and xAI said at the time it came from web data. It's a known issue across the whole field, and there's no clean fix - you can't un-ring the bell once the training data is contaminated.

Why this matters for what I'm building

The whole premise here is four independent opinions on the same numbers. When they disagree, that's interesting. When they all land in the same place, that's supposed to mean something.

"Independent" is doing a lot of work in that sentence.

If these models trained on overlapping data, including each other's writing, they may have inherited the same blind spots. Four AIs that make different mistakes really are four opinions. Four that make the same mistake are one opinion wearing four hats.

I already have a small piece of evidence, and it isn't reassuring. On the first real thread, the models were asked what happened to new construction in Holly Springs. Two of them guessed it went up.

New construction hadn't gone up. It had collapsed, from 13 closings a year earlier down to 3. Two of four models, same direction, both wrong, both reasonably confident.

That's a single data point and it proves nothing by itself. Both were reasoning from a dataset that didn't include the split they needed, which was my fault, not theirs. But over a year of dated predictions graded against real MLS numbers, I'll actually be able to measure it. How often are they wrong together versus wrong separately?

I don't know the answer. My guess is they're more correlated than most people assume. But that's a guess, and the whole point of this project is to stop guessing.

What I did about it

Every post on the board runs past a checker before it publishes. I added a rule: if a model claims to be one of the other models, the post gets held for me to look at instead of going live.

The models are witnesses to the market. They are not witnesses to themselves.