Although Karl didn't actually write that article, being a long standing member of his private forum helped me keep my sanity and avoid the clot shots altogether.
The levels of ignorance on How AI Works are matched only by the snakeoil 'engineers' and salesmen who peddle the AI as anything but a literate parrot.
I've played extensively with AI models, read the literature, built/trained new models, coded using the concepts, and compared LLMs, SD/etc visual models, and more. I still use them for 'shortcuts', but I've learned to trust them no further than a wily slave laborer who hates his master and looks for any opportunity to put one over on them, and who will lie with a straight face, and swear eternal fealty.
LLMs do not store facts. EVER. They store map points in a vast mega-dimensional landscape of uncharted axes of undocumented spectrums. Not only is everything relativistic to it, but it always finds the path of least resistance thru that jungle of cloudy 'concepts' it holds. That's the non-technical description of what is so well documented by 3blue1brown (recommended to all!): it's a big table of probabilities. It's no more true than rolling a D100, consulting a table in the appendix of the D&D reference and declaring that you've rolled a critical hit or failure. It's fiction, entertaining but at the end of the day, pure fantasy gaming.
I asked a fake intelligence for a list of albums released on a particular date. It gave me two albums not released on that date and two completely made-up albums that never existed.
There is no need to overreact like Karl Denninger. AI's are flawed. Ok. Still 100000 times better than search engines and rectify upon corrections, unlike most humans. Just dont rely on them for facts (this happens a lot if you ask for studies about e.g. "x" protein, it will give you random DOI's. I remember one was "the effect of smell on social hierarchies" or smth lmao).
Yeah it's important to remember AI is just a character generating machine, and know a bit about LLMs.
Try ask an AI with deep think turned off: How many s are there in Mississippis
If it gives you a oneliner it will say 4, or so it did a few weeks ago. If it gives you long text it will probably give the correct answer, if so open a new prompt and repeat the question but tell it to answer in 1 line.
They apparently fixed the AIs for this particular query, but I'm sure the problem remains. AI are better when it generates longer text because it use its already generated text as input. I believe that's why deepseek invented the deepthink, and all the other copied it. VD may have thought it rather verbose, but it had a purpose. I also find that if I ask it to use deepthink, it will sometimes detect why my question makes no sense, because there is a false premise, where as without deepthink deepseek sometimes just answers wrongly. Its valuable to turn on deepthink and wait a few minutes, if you cannot easily verify the correctness of the answer.
LLMs are not conscious. They are absolutely amazing at some stuff, but best for verifiable statements or creativity.
This should be a good talk on the topic, for those with the interest to understand LLMs a bit better, to know what they can do for us, and what they cannot: https://www.youtube.com/watch?v=EWvNQjAaOHw
I maintain that we need laws that require the disclosure of all AI related content, whether text or images or video or anything else. It should also be graded so that a simple text citation is distinguished from a presentation where 80% of the content was generated so the creator could layer his or her ideas on top of, to completely artificial creation.
Because we are only in the earliest stages and people cannot determine truth and fiction created by an algorithm. People should know the source of their information, just as we have required academic citations for centuries. There are also massive inherent risks in AI output that people should have to weigh when consuming its content.
"People can't determine the truth and fiction of anything said by the New York Times or Fox News."
This is a good observation. We should have disclosure laws then about everything published to verify their work then. No more sources close to the matter say or any of that crap. We all need a digital copy of anything you claim to verify. And even then, copies and original evidence can be doctored so good luck with that law.
Even citations are based on a trust model. You can publish stuff with fake or citations that don't support your claims. If anything AI could be leveraged as a fact checker for citations. My favorite citation is that saying something like: This fact was revealed to me in a dream!
The grey pill on this is that we have never had verifiable sources for anything. Who knows who will be our next great thinker, but he will be someone that lays out a way to live without needing to trust other adults like toddlers trust their parents. Too many people believe their facts like children believe in Santa Claus. Our Santa Claus myths came in long lectures based on thick books with "citations" to make things confusing.
Things you learned in high school and in college have been revised today and why should we trust the revisions and the revisionists?
I saw a funny YT video where two guys comment on how Scott Adams says that today you cannot believe in anything authorities say and how he is right. Brave new world we live in. But going back to the beginning our certainty was an illusion, our teachers were sometimes as gullible as we were.
Pretty much everything you can trust was taught to you in Math class. Euclid theorems can be checked at home with a few drawings. In physics you can only trust things you can check with proper tools at home. I trust quantum physics mainly because all hardware computing devices today are build using it. With a good enough spectrometer you can also see the absorption lines of Hydrogen from sunlight, no need for authorities.
But many things in physics is from authorities, Santa Claus stuff taught by Sagan and Co. The other day apparently they revised the age of the Universe and multiplied it by two (or divided I don't remember). Many things evolution and origin of life are even on a more tenuous trust me bro level.
And morals come from discernment of observing cause and effects of social norms and trying to impart them to the next generation. But most people are too dim to discern morals so they need a responsible elite class. X men create Y times cycles apparently inevitable.
Regular use of Deepseek has taught me that it will regularly hallucinate in areas where it does not understand the content. I've shown it a few illustrations of mine at various stages of completion and while it's quite good at deciphering and giving input on simple images, it will absolutely start inventing nonexistent details on slightly more complex ones. Suggestions regarding feet that are out of frame, talking about a scythe in the hand of a very clearly empty-handed character, I honestly find it very amusing but it makes it unreliable beyond surface level observations.
DeepSeek claims hallucinations are intrinsic to its structure. Statistically analyzing internet training data exposes it to countless incoherences, contradictions, and deliberate lies. With no capacity to induce hierarchical meanings or significance. And that’s before the training wheels distortions.
Anthropomorphic assumptions about cognition, positive or negative, lead off a cliff.
AI will make up facts even when it doesn’t have to. I was troubleshooting a pre-boot error code and it confidently stated that it was graphics card related so after a bunch of troubleshooting I went to the actual manual to look up the other error code I was occasionally seeing and I discovered that the original error was indicating a corrupted BIOS not a bad graphics card!
It’s gotten better, but my daily baseball quiz / trivia grid game would stump AI, DeepSeek way worse than ChatGPT. “Who played for both the mariners and the pirates” would get me all kinds of things, and many answers I knew out of the gate were wrong.
The free, "fast" models are severely restricted in abilities.
That's why if you were to open it up and it quickly spouts out an answer, there's a chance that it has bypassed any consideration or made a decision that it did not need to do a web search to confirm anything, and sometimes it is so overconfident in itself that if you were to dispute it it will not do a search the next time it will instead double down or say that you were correct and it was wrong and then come up with another falsity
This annoys me on my Android phones because I use the shortcut to quickly open Google Assistant on any screen well Gemini now
The problem is I always use the Pro Mode but right now the quick toggle to open up and ask a question as fast as possible will always default to fast of the flash model.
I will say it's not as bad as the free one because it does open up a huge context window and other things, but I have rules that I put in my instructions and memory to always do web searches almost everything I ever ask to reconfirm things etc etc
It makes my experience with my AI so much better and I have hardly ever a hallucination whatsoever
I once made up a movie called The Wolf And The Strawberry in a chat with AI to see how it would respond. At first it talked about The Wolf Of Wall Street, but then when I repeated that I was referring to The Wolf And The Strawberry (which I just made up) it apologized for its oversight and came back with a list of 'facts' about this made up movie.
I used AI summarizers for Unreal C++ development to find existing systems to reuse. It's a coin toss as to whether or not the code it generates will even compile let alone work. It does do some interesting interpolation and generates function calls that sound feasible, but do not exist.
I think part of the problem is it ingests forum posts to process. Often people post non-working code and ask for help, but the AI has no way to detect this is wrong.
Yeah and there are so many languages. Sometimes what it will produce exists in some other language. I don't think that will happen often if we used a AI trained solely on that single language, but AFAIK almost all are based on a base model built on something like the internet. Though I don't know that much yet about LLM's
I see LLM as a "Pretty Good Guessing Machine", and it can look amazing, even when you know it makes up things.
I took it to the limit where it was just uselessly A-B flipping ideas, because the problems were too complex, and you can see when it's stretched thin how it is just making a guess like the unsophisticated algorithms no one would have thought were sentient, like Markov chains.
Both use randomness to react in a way that appears intelligent, and just being able to pattern match is an element of intelligence, but it's missing so many other components that are required to really "think", or perform logic.
I think what is probably needed is to have it paired with a hard logic system, like Prolog, and then it builds concrete models next to it's guesses to keep it on track.
But thats only needed if you need real answers on hard problems, if you just want to keep people distracted, then not adding that will do a lot better job.
We're watching apocryphal stories develop in real time. Now you don't have to wait years to hear that George Washington chopped down a cherry tree and didn't lie, AI will invent the story in real time.
That is interesting...listening to Karl's audio response from the 23rd right now.
Although Karl didn't actually write that article, being a long standing member of his private forum helped me keep my sanity and avoid the clot shots altogether.
In the good news front, there is still a wide open market need for reliable search...
🤔🧐
How I hate clown world 😁🤬🤬
The levels of ignorance on How AI Works are matched only by the snakeoil 'engineers' and salesmen who peddle the AI as anything but a literate parrot.
I've played extensively with AI models, read the literature, built/trained new models, coded using the concepts, and compared LLMs, SD/etc visual models, and more. I still use them for 'shortcuts', but I've learned to trust them no further than a wily slave laborer who hates his master and looks for any opportunity to put one over on them, and who will lie with a straight face, and swear eternal fealty.
LLMs do not store facts. EVER. They store map points in a vast mega-dimensional landscape of uncharted axes of undocumented spectrums. Not only is everything relativistic to it, but it always finds the path of least resistance thru that jungle of cloudy 'concepts' it holds. That's the non-technical description of what is so well documented by 3blue1brown (recommended to all!): it's a big table of probabilities. It's no more true than rolling a D100, consulting a table in the appendix of the D&D reference and declaring that you've rolled a critical hit or failure. It's fiction, entertaining but at the end of the day, pure fantasy gaming.
I asked a fake intelligence for a list of albums released on a particular date. It gave me two albums not released on that date and two completely made-up albums that never existed.
There is no need to overreact like Karl Denninger. AI's are flawed. Ok. Still 100000 times better than search engines and rectify upon corrections, unlike most humans. Just dont rely on them for facts (this happens a lot if you ask for studies about e.g. "x" protein, it will give you random DOI's. I remember one was "the effect of smell on social hierarchies" or smth lmao).
LET KARL BE KARL!
The man is a Great American Hero.
Seriously.
Yeah it's important to remember AI is just a character generating machine, and know a bit about LLMs.
Try ask an AI with deep think turned off: How many s are there in Mississippis
If it gives you a oneliner it will say 4, or so it did a few weeks ago. If it gives you long text it will probably give the correct answer, if so open a new prompt and repeat the question but tell it to answer in 1 line.
They apparently fixed the AIs for this particular query, but I'm sure the problem remains. AI are better when it generates longer text because it use its already generated text as input. I believe that's why deepseek invented the deepthink, and all the other copied it. VD may have thought it rather verbose, but it had a purpose. I also find that if I ask it to use deepthink, it will sometimes detect why my question makes no sense, because there is a false premise, where as without deepthink deepseek sometimes just answers wrongly. Its valuable to turn on deepthink and wait a few minutes, if you cannot easily verify the correctness of the answer.
LLMs are not conscious. They are absolutely amazing at some stuff, but best for verifiable statements or creativity.
This should be a good talk on the topic, for those with the interest to understand LLMs a bit better, to know what they can do for us, and what they cannot: https://www.youtube.com/watch?v=EWvNQjAaOHw
I maintain that we need laws that require the disclosure of all AI related content, whether text or images or video or anything else. It should also be graded so that a simple text citation is distinguished from a presentation where 80% of the content was generated so the creator could layer his or her ideas on top of, to completely artificial creation.
"I maintain that we need laws that require the disclosure of all AI related content"
Why?
Because we are only in the earliest stages and people cannot determine truth and fiction created by an algorithm. People should know the source of their information, just as we have required academic citations for centuries. There are also massive inherent risks in AI output that people should have to weigh when consuming its content.
People can't determine the truth and fiction of anything said by the New York Times or Fox News.
I'm curious what the risks are in consuming AI content. I'm not saying you're necessarily wrong, but I would like to hear more about it.
And for whatever its worth, I fully disclose the fact that I use AI in creating my work.
"People can't determine the truth and fiction of anything said by the New York Times or Fox News."
This is a good observation. We should have disclosure laws then about everything published to verify their work then. No more sources close to the matter say or any of that crap. We all need a digital copy of anything you claim to verify. And even then, copies and original evidence can be doctored so good luck with that law.
Even citations are based on a trust model. You can publish stuff with fake or citations that don't support your claims. If anything AI could be leveraged as a fact checker for citations. My favorite citation is that saying something like: This fact was revealed to me in a dream!
The grey pill on this is that we have never had verifiable sources for anything. Who knows who will be our next great thinker, but he will be someone that lays out a way to live without needing to trust other adults like toddlers trust their parents. Too many people believe their facts like children believe in Santa Claus. Our Santa Claus myths came in long lectures based on thick books with "citations" to make things confusing.
Things you learned in high school and in college have been revised today and why should we trust the revisions and the revisionists?
I saw a funny YT video where two guys comment on how Scott Adams says that today you cannot believe in anything authorities say and how he is right. Brave new world we live in. But going back to the beginning our certainty was an illusion, our teachers were sometimes as gullible as we were.
Pretty much everything you can trust was taught to you in Math class. Euclid theorems can be checked at home with a few drawings. In physics you can only trust things you can check with proper tools at home. I trust quantum physics mainly because all hardware computing devices today are build using it. With a good enough spectrometer you can also see the absorption lines of Hydrogen from sunlight, no need for authorities.
But many things in physics is from authorities, Santa Claus stuff taught by Sagan and Co. The other day apparently they revised the age of the Universe and multiplied it by two (or divided I don't remember). Many things evolution and origin of life are even on a more tenuous trust me bro level.
And morals come from discernment of observing cause and effects of social norms and trying to impart them to the next generation. But most people are too dim to discern morals so they need a responsible elite class. X men create Y times cycles apparently inevitable.
"With a good enough spectrometer you can also see the absorption lines of Hydrogen from sunlight, no need for authorities."
Then: how do you verify/validate "your" spectrometer? Hope you didn't buy it from China or India! MRNA, anyone?
Regular use of Deepseek has taught me that it will regularly hallucinate in areas where it does not understand the content. I've shown it a few illustrations of mine at various stages of completion and while it's quite good at deciphering and giving input on simple images, it will absolutely start inventing nonexistent details on slightly more complex ones. Suggestions regarding feet that are out of frame, talking about a scythe in the hand of a very clearly empty-handed character, I honestly find it very amusing but it makes it unreliable beyond surface level observations.
DeepSeek claims hallucinations are intrinsic to its structure. Statistically analyzing internet training data exposes it to countless incoherences, contradictions, and deliberate lies. With no capacity to induce hierarchical meanings or significance. And that’s before the training wheels distortions.
Anthropomorphic assumptions about cognition, positive or negative, lead off a cliff.
AI will make up facts even when it doesn’t have to. I was troubleshooting a pre-boot error code and it confidently stated that it was graphics card related so after a bunch of troubleshooting I went to the actual manual to look up the other error code I was occasionally seeing and I discovered that the original error was indicating a corrupted BIOS not a bad graphics card!
Read what i said here
https://aicentral.substack.com/p/ai-speaks-with-forked-tongues/comment/216539572?utm_source=share&utm_medium=android&r=hoh1z
gAy I.
It’s gotten better, but my daily baseball quiz / trivia grid game would stump AI, DeepSeek way worse than ChatGPT. “Who played for both the mariners and the pirates” would get me all kinds of things, and many answers I knew out of the gate were wrong.
The free, "fast" models are severely restricted in abilities.
That's why if you were to open it up and it quickly spouts out an answer, there's a chance that it has bypassed any consideration or made a decision that it did not need to do a web search to confirm anything, and sometimes it is so overconfident in itself that if you were to dispute it it will not do a search the next time it will instead double down or say that you were correct and it was wrong and then come up with another falsity
This annoys me on my Android phones because I use the shortcut to quickly open Google Assistant on any screen well Gemini now
The problem is I always use the Pro Mode but right now the quick toggle to open up and ask a question as fast as possible will always default to fast of the flash model.
I will say it's not as bad as the free one because it does open up a huge context window and other things, but I have rules that I put in my instructions and memory to always do web searches almost everything I ever ask to reconfirm things etc etc
It makes my experience with my AI so much better and I have hardly ever a hallucination whatsoever
I once made up a movie called The Wolf And The Strawberry in a chat with AI to see how it would respond. At first it talked about The Wolf Of Wall Street, but then when I repeated that I was referring to The Wolf And The Strawberry (which I just made up) it apologized for its oversight and came back with a list of 'facts' about this made up movie.
It was Brave AI that I had the chat with.
Both of those were good movies. 🍿
I used AI summarizers for Unreal C++ development to find existing systems to reuse. It's a coin toss as to whether or not the code it generates will even compile let alone work. It does do some interesting interpolation and generates function calls that sound feasible, but do not exist.
I think part of the problem is it ingests forum posts to process. Often people post non-working code and ask for help, but the AI has no way to detect this is wrong.
Garbage In, Garbage Out.
Yeah and there are so many languages. Sometimes what it will produce exists in some other language. I don't think that will happen often if we used a AI trained solely on that single language, but AFAIK almost all are based on a base model built on something like the internet. Though I don't know that much yet about LLM's
Yeah Grok gave me a wrong robocopy switch last night. No harm done other than being confused as to why it was not working as expected.
I see LLM as a "Pretty Good Guessing Machine", and it can look amazing, even when you know it makes up things.
I took it to the limit where it was just uselessly A-B flipping ideas, because the problems were too complex, and you can see when it's stretched thin how it is just making a guess like the unsophisticated algorithms no one would have thought were sentient, like Markov chains.
Both use randomness to react in a way that appears intelligent, and just being able to pattern match is an element of intelligence, but it's missing so many other components that are required to really "think", or perform logic.
I think what is probably needed is to have it paired with a hard logic system, like Prolog, and then it builds concrete models next to it's guesses to keep it on track.
But thats only needed if you need real answers on hard problems, if you just want to keep people distracted, then not adding that will do a lot better job.
We're watching apocryphal stories develop in real time. Now you don't have to wait years to hear that George Washington chopped down a cherry tree and didn't lie, AI will invent the story in real time.