Primer.ai reposted this
Two weeks ago, National Institute of Standards and Technology (NIST) published a report on risk management of generative AI. The report lays out a dozen risks that cover everything from the risk of insulting people with offensive language to helping terrorists design bombs. The one that stands out is the risk of hallucinations. As the report defines, it is "the production of confidently stated but erroneous or false content." NIST is spot on. Fixing hallucinations is the top priority for the AI community, both industry and academia. Thousands of papers have been published on the subject this year alone. My own team at Primer has been laser focused on detecting and mitigating them. It's a solid report and very readable even for non-experts. I especially like the subtle points made about how hallucinations can sometimes be desirable, for example if you're asking the generative model to be creative. One detail did catch my eye. NIST uses the term "confabulation" rather than "hallucination", noting that some commenters say "the terms 'hallucination' and 'fabrication' anthropomorphize [generative AI]." It's a fair point, though its debatable whether "confabulation" is any less anthropomorphic. We should all settle on whether to change the term soon. Even Merriam-Webster Inc. has weighed in, defining hallucination as a type of AI error. There may be no going back at this point. NIST report: https://lnkd.in/g4db_SNR Hallucination papers: https://lnkd.in/gxgNKB3G Primer's latest progress on hallucination mitigation: https://lnkd.in/gAR23c3G Merriam-Webster definition of hallucination: https://lnkd.in/gv3jsy_w