38 | AI Risk Primer — Hallucinations (Part 1)
“AI Risk Primer” Article Series
As I’ve given talks about AI to a variety of nonprofit audiences recently, I’ve tried to pay keen attention to the questions I’m asked by attendees. Not surprisingly, they’re widely varied, but there are a few key themes. In particular, I’ve received many questions about the ways that using new AI capabilities and tools could backfire—the risks of generative AI.
While I’m not an expert in AI or its risks, I have been paying attention to the topic from many viewpoints. Doing so, I’ve been working to understand potential risks from the lens of what nonprofit staff might want to do with AI so that I can help you safely navigate paths forward.
This short series of “AI Risk Primer” articles will provide you with some grounding in what some of the most concerning risks are, what you should avoid doing, and the potential implications if you misstep.1 Today’s article examines AI hallucinations, and I’ll look at data privacy and copyright considerations in future briefs.
As a note: I’m exploring how to increase my use of ChatGPT as a writing partner, and how to gain value from the experience. Much of the content of this article was generated by ChatGPT at my prompting during an extensive ‘dialogue’ we had. You can read the exchange here. But as my closing soapbox highlights, I own anything that’s misguided, ambiguous, poorly worded, or just plain wrong!
The Risk: Generative AI Hallucinations
First, let’s take a look at what Generative AI is and how it works. Generative AI operates on a fundamentally probabilistic basis. Unlike traditional software that follows strict, deterministic rules, generative AI models predict the most likely output based on patterns learned from vast datasets. For example, when asked to write a fundraising email, the AI doesn’t just follow a template; it generates the content by calculating what a successful fundraising email should look like based on its training. This probabilistic nature means that while it can produce highly creative and varied outputs, it can also lead to less predictable and sometimes inaccurate results. For non-technical business professionals, think of it as a very advanced and creative assistant, whose suggestions are based on a wide range of possibilities rather than a single, defined path.
In the realm of artificial intelligence, the term 'hallucinations' takes on a unique meaning, diverging from its traditional association with human sensory experiences. In AI, hallucinations refer to instances where a machine learning model generates incorrect, unrealistic, or nonsensical outputs. This phenomenon often occurs when an AI, such as a language model or an image generator, encounters scenarios or data combinations it hasn't been adequately trained to handle. The AI, striving to produce a coherent output based on its training, ends up 'filling in the gaps' with fabricated information or images. These hallucinations are not just simple errors; they reflect the AI's attempt to make sense of unfamiliar or complex inputs using its existing knowledge base. Understanding and mitigating such hallucinations is a crucial challenge in AI development, as it directly impacts the reliability and trustworthiness of AI systems, especially in critical applications.
The nasty truth about AI hallucinations is that they are hard to detect—they sound every bit as convincing as a factual account. There have been many accounts this year of embarrassing occasions when someone took AI-generated information forward, only to learn that it was false (i.e., a hallucination).
The phenomenon is so broadly experienced that the Cambridge Dictionary – the world’s most popular online dictionary for learners of English – has updated its definition of hallucinate to account for the new meaning and crowned it Word of the Year for 2023.2
While specific instances of nonprofits suffering from AI hallucinations are not widely documented, the potential for such missteps is real. Imagine a scenario where an AI tool generates a funding proposal with inaccurately high success rates of a program, based on misinterpreted data. If used without verification, this could mislead donors and damage the organization's credibility. Another plausible situation could involve an AI-created social media campaign with incorrect historical references. If these AI-generated items were posted without review, they could cause great embarrassment to the organization. These hypothetical examples underline that it’s crucially important for you, the speaker/author/creator, review and fact-check AI-generated content.
The Approach: How Do I Avoid Triggering Hallucinations?
Using Traditional Internet Sources or Going to AI Chatbots
First, it’s important to decide between using a Generative AI tool or turning to a traditional internet search or reference site.
Using Standard Internet Search Tools (e.g., Google, Bing)
When to Use
For factual, well-established information.
For quick access to public domain data and statistics.
When seeking sources that need to be cited in official documents.
Advantages
Provides access to a vast range of information from various sources.
Information is often more easily verifiable.
Sample Tasks
Researching Funding Opportunities: A nonprofit staff member looking for grant opportunities or potential donors would benefit from using internet search tools to find the most current opportunities, application guidelines, and eligibility criteria.
Verifying Current Events or News: For creating timely and relevant content or statements related to current events, a staff member needs to use internet search tools to ensure they have the most up-to-date and accurate information.
Using Reference Sites (e.g., Wikipedia, Britannica)
When to Use
For general overviews on a wide range of topics.
For preliminary research or to gain a basic understanding of a subject.
Advantages
Summarizes information from multiple sources.
Easy to navigate for quick learning.
Sample Tasks
Preparing for Educational Workshops: When developing materials for educational workshops on topics like environmental conservation or public health, reference sites can provide comprehensive overviews and verified information that are essential for accurate content creation.
Background Research for Program Development: In the initial stages of developing a new program or service, reference sites are valuable for gaining a broad understanding of the subject matter, such as understanding the historical context of a community issue or the basics of a new policy area.
Using Generative AI Tools (e.g., ChatGPT)
When to Use
For tasks requiring creativity and synthesis of ideas.
For drafting content, brainstorming, or exploring different perspectives.
When personalized or conversational output is needed.
Advantages
Can generate unique content based on specific prompts.
Useful for ideation and generating diverse viewpoints.
Sample Tasks
Drafting Grant Proposals or Reports: Generative AI can assist in drafting grant proposals or reports by providing creative suggestions for structuring content, generating descriptive narratives, or even helping with initial drafts based on the program's goals and objectives.
Creating Customized Communication for Donors: For creating personalized thank-you letters, updates, or engagement materials for donors, generative AI can help tailor messages that resonate on a personal level, ensuring each communication feels unique and thoughtful.
To be continued…
I have more to say about Hallucinations—how to improve your prompting to minimize them and what you can do next. Coming in the next edition of StrefaTECH!
I’m a fan of the awful but frequently insightful question, “What’s the worst that could happen?” I’ll get into that important perspective in the section on possible outcomes of triggering an AI risk.
Some notable examples of publicly embarrassing hallucinations:
In a major launch presentation, the Google Bard chatbot incorrectly stated that the James Webb Space Telescope had captured the world’s first images of a planet outside our solar system. (CNET)
A lawyer used ChatGPT to prepare a filing, but the cases in the AI-generated materials were completely made up. (Forbes)