In this case it is more a feature being called a bug

If we ignore the other poster, do you think the logic in my previous comment is circular?

That was what I was trying to say, I can see that the wording is ambiguous.

I agree, it’s a massive issue. It’s a very complex topic that most people have no way of understanding. It is superb at generating text, and that makes it look smarter than it actually is, which is really dangerous. I think the creators of these models have a responsibility to communicate what these models can and can’t do, but unfortunately that is not profitable.

If a solution is correct then a solution is correct. If a correct solution was generated randomly that doesn’t make it less correct. It just means that you may not always get correct solutions from the generating process, which is why they are checked after.

It’s not circular. LLMs cannot be fluent because fluency comes from an understanding of the language. An LLM is incapable of understanding so it is incapable of being fluent. It may be able to mimic it but that is a different thing. (In my opinion)

It’s not a bug, it’s a natural consequence of the methodology. A language model won’t always be correct when it doesn’t know what it is saying.