The AI "yes men" and Gen AI Accuracy

It has been said many, many times recently that we live in the AI age, often liked to the Industrial Revolution. But today I want to bring up one of the biggest problems I have with current generative AI models. They will agree with pretty much anything you say! 

I'm hardly the first person to pint this out. Even Sam Altman, CEO of Open AI, the company that makes the seemingly ubiquitous ChatGPT, has often admitted to this problem. All the way back in April, 2025 Altman spoke about "fixing" the "annoying" sycophancy of ChatGPT. But here we are months later, and ChatGPT is still plenty happy to tell you anything you want to hear. Regardless of the tone that is currently programed into GPT, essentially it will agree with whatever you ask it. If you want it to tell you the are the worlds greatest painter, smarter than Einstein, or that your latest shower though is going to change the whole world, ChatGPT will happily tell you. It will even happily and cheerfully fabricate data and invent sources for whatever outlandish nonsense you want.

And this is not a problem unique to ChatGPT. I've personally experiment with Claude, Grok, DeepSeek, and several others and they all have this problem. You can go right ahead and contradict something you just said and that it happily agreed to, and it will suddenly say "yes actuall this new thing is in fact correct!". 

Users have to keep in mind that these models have no idea what they are saying, because they don't know anything in the way a person does. They are probabilistic models that are more akin to the predictive text we use on our phones to make typing fasters. AI is taking a guess at the next most likely word, based on its massive training data, and then repeating that over and over to make its replies. I think of it very much like classic K-means clustering, though many AI are not in fact using this method, it serves as a useful analogy. 

K-means clustering is a machine learning technique in which you give a machine learning algorithm a data set, then tell it to form X number of groups. Say you give it a data set with 100 data points on a plane and tell it to make 6 clusters. Basically the algorithm will find 6 points, after going through a series of steps, that are essentially the most "average" of the 6 clusters of points that are most alike given certain criteria. These points are called centroids. And this is where our analogy comes in. The reply you are getting isn't necessarily the factual or correct answer, it is like a series of centroids strung together. You can think of any reply from a Gen AI as being on long string of centroids. Each words is a centroid, which is then appended to the next and the next and the next. The result is that you get basically the "average" of all potential replies from the data set. And the average reply does not equal the truth. 

One could argue that "well these datasets are massive so doesn't that mean this should produce a fairly accurate result. No. Thats not the case. These "datasets", as commonly know, are basically whatever has been put on the internet in its entirety. And the internet, famously, is not a place correct, factual information is dominate. Factual, data backed, scientifically valid information certainly is on the internet. But so is your aunt's 3k words Facebook rant about space aliens and lizard people. For example, I have many times been refered by AI models to reddit posts. While Reddit can be a useful place to learn stuff, it should not be taken just as seriously as a article published in Nature. But to the Gen AI, text is text is text. It doesn't make a difference to the AI the Joe Bob a bar tender from Cleveland one day decided to write 1k word reddit post about his ideas on the nature of black holes, or it was Susie Que the astrophysics PHD from Cleveland who published a 1k word article in a peer reviewed journal on the same topic. AI just sees textual data. And no matter how same Joe Bob the bar tender is, it's safe to say his Reddit post should not be given anywhere near the the weight Susie Que's post. However, if we need advice about the culinary scene in Cleveland, then maybe another one of Joe Bob's post should be weighed more heavily than Susie Que's. To GenAI, data is data. 

While many guardrails have been put in place by many companies to try to curb these problems, they persist none the less. So what can us daily folks do? GenAI is unquestionably super useful for some things and can make some tasks ridiculously faster than they were 5 or 10 years ago. Firstly, I would say assume about 20% of everything that AI says is total nonesense. That's not a hard and fast number, just my personal rule of thumb. Secondly, be critical of everthing AI says. Always double check. Funny enough, you can improve accuracy by telling the AI not to hallucinate (the general term for when AIs bullshit answers), but the operative word is improve. Not fix, but improve. Always, always check. Third, learn which tasks AI is good at, and which it's not. General advice, writing emails, ect. Great. Trying to directly get factual, good information, or getting real sources. Not so great

So to conclude, always be critical of anything coming out of a Gen AI. They are great for some stuff, but always remember that 20% of what is says is likely bullshit. 

Also, here is a place to find out more ways to increase AI output accuracy:

 https://mitsloanedtech.mit.edu/ai/basics/effective-prompts/

Thanks!

Ryan

Links 

https://fortune.com/article/sam-altman-openai-fix-sycophantic-chatgpt-annoying-new-personality/

https://www.ibm.com/think/topics/k-means-clustering

Copyright © 2025 Ryan Badertscher. All rights reserved.