AI Hallucinations at the FDA

Elsa is inventing scientific studies and research

On the one hand, being an experienced user of three different forms of generative AI, I really don’t think that AIsourcing new drug approvals is a good idea, given the tendency of AI to invent fictitious studies and nonexistent research:

Robert F. Kennedy Jr., the Secretary of Health and Human Services, has made a big push to get agencies like the Food and Drug Administration to use generative artificial intelligence tools. In fact, Kennedy recently told Tucker Carlson that AI will soon be used to approve new drugs “very, very quickly.” But a new report from CNN confirms all our worst fears. Elsa, the FDA’s AI tool, is spitting out fake studies.

CNN spoke with six current and former employees at the FDA, three of whom have used Elsa for work that they described as helpful, like creating meeting notes and summaries. But three of those FDA employees told CNN that Elsa just makes up nonexistent studies, something commonly referred to in AI as “hallucinating.” The AI will also misrepresent research, according to these employees.

“Anything that you don’t have time to double-check is unreliable. It hallucinates confidently,” one unnamed FDA employee told CNN.

And that’s the big problem with all AI chatbots. They need to be double-checked for accuracy, often creating even more work for the human behind the computer if they care about the quality of their output at all. People who insist that AI actually saves them time are often fooling themselves, with one recent study of programmers showing that tasks took 20% longer with AI, even among people who were convinced they were more efficient.

On the other hand, we need to keep in mind that we’re talking about an institution that has approved countless vaccines without ever once requiring a single proper double-blind study of their safety.

So from a practical perspective, is there really any difference between the FDA relying on AI-invented studies and the studies that are paid for by the manufacturers who stand to making millions, if not billions, from receiving FDA permission to put their chemicals on the market?

And speaking of irrelevant studies, what does it have to do with literally any other occupation if programmers take 20 percent longer on their tasks with AI? If you look at the number of songs I’ve recorded with AI versus the number of songs I’ve recorded organically, it’s evident that AI speeds things up by a factor of 1000x.

One author with whom I am acquainted has increased his writing output by a factor of 5x.

So, the idea that programmers might waste a lot of time unnecessarily playing around with AI to the point that it slows them down says absolutely nothing about the utility or the time-effectiveness of utilizing AI for everyone else.