1 of 41

Empower Yourself!

Critical Evaluation of �AI Generated Content

Framework for Critiquing Results Generated by Artificial Intelligence

FLUF Test Framework

Dr. Jennifer Parker

2 of 41

Introduction to FLUF

Traditionally, online search results have been critiqued by frameworks like SIFT (Caulfield, 2019); CARRDSS (Valenza, 2004); CRAAP (Blakeslee, 2004); and 5 Key Questions (Thoman & Jolls, 2003). Prior to the FLUF Test (Parker, 2023), little guidance existed to critically evaluate or assess AI generative results.
The FLUF Test was developed as a primary tool for critically evaluating content generated through artificial intelligence. Using the FLUF indicators, AI users create better prompts and critique AI generative results (Parker, 2023).
The FLUF Test indicators include: Format, Language, Usability, and Fanfare. “FLUF” indicators are “look fors” in the AI generated results.
The goal of the FLUF test is to have “zero” FLUF, or zero evidence of infractions. The FLUF test uses a simple rubric of plus (+) or minus (-), which translates to a zero and one. Each issue or infraction found with an AI generative result is assessed a “plus” or receives a scores of one.
The total infractions are tallied to get the FLUF score. The goal is to have a zero, or no infractions and “Zero FLUF”. When a score results, the user is encouraged to reprompt, regenerate, and repeat until the AI generative result scores a zero.

3 of 41

What is FLUF?

9

In order to critique your results, you will want to familiarize yourself with the indicators of the FLUF test.

You want to have zero infractions – or zero "wrong" in your critique.

In FORMAT, review the layout and length.

Layout - If it doesn't follow the formal writing patterns or format, it gets a +

Length - If there are lots of extra words to extend the word count, it gets a +

In LANGUAGE, review the tone, phrasing, and repetition of the generated content

Tone - If it lacks personal style, tone, or human elements, it gets a +

Phrasing - If the syntax or semantics are off, or it presents information in an awkward way, it gets a +

Repetition – if the passage lacks succinct presentation of ideas, or has run-on sentences or repetition of ideas, thoughts, and/or phrases, it gets a +

In the area of USABILITY, examine consistency and credibility

Consistency – if there are inconsistencies in content it gets a +

Credibility – if you cannot determine whether there are credible references, information cannot be authenticated or validated, or it lacks citations/sources for documentation, then it gets a +

The final area is FANFARE, where we explore anecdotes and jargon.

Anecdotes – depending on the intention of the writing, the absence of human examples, analogies, metaphors, or comparisons gets a +

Jargon – again, depending on the intention of the writing, when the writing repeats vocabulary, technical language, or common findings without sharing new information or content, it gets a plus. If the jargon is presented in an informal way without specificity, it gets a +.

4 of 41

Got FLUF?

Re-Prompt

Regenerate

Repeat

Until you have

“Zero FLUF”

5 of 41

Underpinnings For This Work

Respect the 80/20 Rule of the Process
The 80/20 Rule	Pareto Principle (1906) The balance between human insight and technological capability 80% research and regeneration of online sources 20% human critique, creativity, and culmination to create a final output
Always Critically Evaluate All Online Content
Frameworks for Critical Evaluation of Online Resources	Multiple frameworks CRAAP (Blakeslee, 2004) – currency, relevance, authority, accuracy, purpose CARRDSS (Valenza, 2004) – credibility, accuracy, reliability, relevance, date, sources, scope SIFT (Caulfield, 2019) – stop, investigate, find, trace 5 Key Questions (Thorman & Jolls, 2003) – creator, techniques, perceptions, bias, purpose
Always Critically Evaluate All AI Generated Content
Frameworks for Critical Evaluation of AI Generated Results	FLUF Test (Parker, 2023) FLUF Indicators – Format, Language, Usability, Fanfare FLUF Your Prompt - who, what, where, when, why, how + format, language, usability, fanfare Got FLUF? reprompt, regenerate, repeat until you get ZERO FLUF! Combine AI Results & Human Creativity and Critique to Generate a Final Product
Utilize Authentic Activities that Integrate Research, Critical Evaluation, and Human Creativity and Innovation
Authentic Activities that Embrace Integration	iSearch Framework (Macrorie, 1988) Elements of the iSearch template: Topic/Issue or Challenge/Problem of Practice What I Know / Want to Know The Story of My Search Search Results Significance of the Research Experience My Growth as a Researcher Works Cited

Photo by Scott Graham on Unsplash

iSearch: Based on the iSearch Model. Macrorie, K. (1988). The I-Search Paper--Revised Edition of" Searching Writing.". Heinemann Educational Books Inc., 70 Court St., Portsmouth, NH 03801.

Source Information:

iSearch: Based on the iSearch Model. Macrorie, K. (1988). The I-Search Paper--Revised Edition of" Searching Writing.". Heinemann Educational Books Inc., 70 Court St., Portsmouth, NH 03801.

CRAAP: The CRAAP test was originally created by Sarah Blakeslee and her team of librarians from California State University - Chico’s Meriam Library in 2004.

CARRDSS: Former K-12 school librarian, now Rutgers Professor Dr. Joyce Valenza (2004) is credited with promoting the CARRDSS framework.

SIFT: The SIFT method was created by Mike Caulfield. All SIFT information on this page is adapted from his materials with a CC BY 4.0 license. Blog post on June 19, 2019

SIFT (The Four Moves)

5 Key Questions: Elizabeth Thoman and Tessa Jolls (2003) established The Center for Media Literacy and developed the 5 Key questions as part of a media kit. A Framework for Learning and Teaching in a Media Age Developed and written by Elizabeth Thoman Founder and Tessa Jolls President / CEO Center for Media Literacy www.medialit.org © 2003, 2005 Center for Media Literacy For terms of usage, go to www.medialit.org/medialitkit. © 2003 Center for Media Literacy / www.medialit.org Literacy for the 21st Century / Orientation & Overview PG 18; Literacy for the 21st Century An Overview & Orientation Guide To Media Literacy Education Part I: Theory CML MediaLit Kit™�

FLUF Test: Created in 2023 by Dr. Jennifer Parker, a Faculty Development Coordinator at the University of Florida as a framework for critical evaluation of content generated by Artificial Intelligence. For more information visit https://www.drjenniferparker.com/fluf-test-for-artificial-intelligence.html.

6 of 41

It’s not all about the technology…

The balance between human insight and technological capability in artificial intelligence

80% research and regeneration
20% human critique, creativity, and culmination

Human Component: The 80/20 Rule�Pareto Principle

AI isn’t all about the technology

Pareto Principle 1906

Using AI can help brainstorm that task we've put off, lighten the load of a time consuming task, increase our productivity, or act as a tutor.

However, there is a delicate balance between human insight and technological capability that is crucial to our use of artificial intelligence.

This concept is illustrated by the 80/20 rule, which highlights the interdependence of human expertise and AI capabilities.

In practice, this rule is widely acknowledged in the industry, emphasizing the need for a blend of human and AI contributions.

Experts agree that while AI plays a significant role, human expertise should remain the predominant influence, adhering to the 80/20 ratio in combining these elements.

As you explore AI tools like ChatGPT, CoPilot, Gemini, and Claude to name just a few, always give the human element – YOU - the "final say" by polishing up the results before submission.

Although AI and online research can lay the foundation (about 80% of the content), the human component should remain at the forefront where higher-order thinking expands, organizes, and critiques the final submission for the target.

This is where the FLUF test comes into play.

7 of 41

Human Component:

Be as descriptive as you can…

Use frameworks like The FLUF Test to improve your prompts or critique your results.

Visit the FLUF Website.

8 of 41

YOUR RESULTS are dependent upon YOUR ENTRY of good search terms or prompts.

Human Component:

Searching

9 of 41

Human Component:

FLUF Your Prompt

Prompt Element	Prompt information
Who	In the role of a college professor
What	Create a syllabus
Where	University of Florida
When	In the Fall 2024 semester
Why	For an EME 5054: Foundations of Educational Technology course
How	Hybrid format with both Face-to-Face and Online requirements
Format	At least six pages
Language	In English�
Usability	Using peer reviewed literature�
Fanfare	Including jargon and technical language from the International Society for Technology in Education
From the information above, generate the prompt Assume the role of a college professor and create a syllabus for a Fall 2024 course on EME 5054: Foundations of Educational Technology. The course is hybrid and offers both face-to-face and online requirements. The syllabus must be at least six pages, in English, and cite peer reviewed literature as a foundation for the course materials. Include educational technology vocabulary found in the International Society for Technology in Education.

Make a copy of the template

10 of 41

FLUF Look Fors: Critiquing AI Results

This Photo by Unknown author is licensed under CC BY.

11 of 41

Example: "Introduction to Soils"��Prompt: "Create a college level course on "Introduction to Soils in the Environment"

FLUF Format Infractions:

LAYOUT is an outline and I need an essay.
LENGTH needs to be 750 words in APA format.

Format

LLMs

12 of 41

Format�IMAGES

Is the image portraying what is described?

Are there any issues with the contents of the photo?

Is the image showing the right tools? equipment? dress? procedure?

Does that oar go in the boat or in the water? Does that girl have three legs? Is the guy standing in the boat or on the water? What are they sitting on?

13 of 41

Example: "Introduction to Soils"��Prompt: "Create a college level course on "Introduction to Soils in the Environment"

FLUF Language Infractions:

Tone - If it lacks personal style, tone, or human elements, it gets a +.
Phrasing - If the syntax or semantics are off, or it presents information in an awkward way, it gets a +.
Repetition – if the passage lacks succinct presentation of ideas, or has run-on sentences or repetition of ideas, thoughts, and/or phrases, it gets a +.

Language�LLMs

14 of 41

Language�IMAGES

Are the terms and vocabulary labeled correctly? Is it written in English?

15 of 41

Example: "Introduction to Soils"��Prompt: "Create a college level course on "Introduction to Soils in the Environment"

FLUF Usability Infractions:

Consistency – if there are inconsistencies in content it gets a +
Credibility – if you cannot determine whether there are credible references, information cannot be authenticated or validated, or it lacks citations/sources for documentation, then it gets a +

Usability�LLMs

16 of 41

Usability�IMAGES

This Photo by Unknown author is licensed under CC BY.

Citing AI Images (From Grammarly)

An example of how to cite an AI image in APA would look like this:

Figure 5

Portrait of Jean Baudrillard in postmodern style

[IMAGE]

Note. Image generated with the prompt “Jean Baudrillard in postmodern style” by OpenAI, ChatGPT, 2023 (https://chat.openai.com/chat).

Look for Creative Commons

Limits to Copilot Images (15 Boosts/Images)

Cite AI Generated Images

17 of 41

Fanfare�LLMs

FLUF Fanfare Infractions:

Depending on the intention of the writing:

Anecdotes – the absence of human examples, analogies, metaphors, or comparisons gets a +
Jargon - another form of repetition, cliché, or assumption in perhaps a condescending tone. Another example might be the absence of jargon associated with the profession (e.g. legal jargon, construction codes, etc.)

18 of 41

Fanfare�IMAGES

Is the image using appropriate terms? Is the image a true representation of the scenario? Are the logos correct?

19 of 41

FLUF: Here's How It Works

20 of 41

Generate AI Results and Apply FLUF

The FLUF Test is ongoing.

At a minimum, FLUF Test the following:

Initial Prompt
Regenerations with Prompt Adjustments
AI + Human – Final Iteration

21 of 41

The Goal: "Zero FLUF"

We are looking for Zero FLUF, or zero evidence of infractions. A ZERO means a good AI generated result and a "go" for using it in your work.

22 of 41

Example 1: You’ve Got FLUF!

Using ChatGPT, you prompt the AI tool to generate content.

Upon review, you have the following thoughts on the results:

+ Format - passage is too long (Length)

- Language - ok

+ Usability - sources can’t be validated (Credibility)

- Fanfare - ok

Works Cited

Adams, Michael. The Layers of Soil and Their Importance. Soil Science Journal, 2020.
Brown, Sarah. The Role of Soil in the Water Cycle. Environmental Studies Quarterly, 2022.
Davis, Laura. Biodiversity in Soil Ecosystems. Nature and Ecology Reports, 2023.
Evans, Robert. Sustainable Practices for Soil Conservation. Green Earth Publishing, 2016.
Garcia, Carlos. Impact of Soil Pollution on Human Health. Journal of Environmental Health, 2017.
Harris, Patricia. Organic Farming and Soil Health. Agricultural Innovations, 2021.
Johnson, Emily. Understanding the Soil Texture Triangle. Farming Science Review, 2021.
Lee, Jennifer. Global Soil Erosion Crisis. World Agriculture Today, 2018.
Smith, Richard. Bulk Density and Soil Compaction. Soil Mechanics Journal, 2019.
Williams, David. Soil and Environmental Sustainability. Earth Sciences Review, 2018.

X

Looks like you’ve got to re-prompt and re-generate your results.

23 of 41

FLUF SCORE

2

In this example, we see two infractions or issues with the AI generated results. The FLUF Score = 2.

We want ZERO FLUF…

We will need to re-prompt our AI tool.

24 of 41

Ways to address FLUF in FORMAT

If you have an issue with LENGTH, redirect your AI tool with some updated or additional prompts.

Prompt / Re-prompt might say:

“Reduce the essay length to 3-5 sentences”
“Summarize the results in 3 paragraphs”
“Expand the essay length to 750 words”
“Increase the word count to 1200 words”

25 of 41

Ways to address FLUF in USABILITY

If you have an issue with CREDIBILITY, redirect your AI tool with some updated or additional prompts

Prompt / Re-prompt might say:

“Remove all information that did not come from a peer reviewed article”
“Only include information from a peer reviewed article”
“Share the sources of the information”

26 of 41

Example 2: You’ve Got FLUF!

You prompt the AI tool to generate content. Upon review, you have the following thoughts on the results:

- Format - ok

+ Language – there are spelling and grammatical errors, or it is written at an elementary level

- Usability - ok

+ Fanfare – the technical jargon or standards of the field, association, or organization is not mentioned

Looks like you’ve got to re-prompt and re-generate your results.

27 of 41

FLUF SCORE

2

In this example, we see two infractions or issues with the AI generated results. The FLUF Score = 2.

We want ZERO FLUF…

We will need to re-prompt our AI tool.

28 of 41

Ways to address FLUF in LANGUAGE

If you have an issue with TONE, redirect your AI tool with some updated or additional prompts.

Prompt / Re-prompt might say:

“Improve the tone of the syllabus with a more friendly manner”
“Change the tone to a more professional one”

29 of 41

Ways to address FLUF in FANFARE

If you have an issue with JARGON, redirect your AI tool with some updated or additional prompts.

Prompt / Re-prompt might say:

“Use appropriate acronyms for the professional organization”
“Include national standards”
“Add quality standards and technical terms”
“Replace acronyms with full spelling of organizations”

30 of 41

Got FLUF?

Re-Prompt

Regenerate

Repeat

Until you get

31 of 41

FLUF Test: The Steps

32 of 41

How to use the FLUF Test

Review FLUF indicators
Create a prompt
Generate results and FLUF test
Update prompt; regenerate; FLUF test
Repeat until happy with results and zero FLUF
Combine AI Results & Human Creativity and Critique for Final Product

TEMPLATE: Make A Copy

33 of 41

Step 1: Review FLUF Indicators

FORMAT

LANGUAGE

USABILITY

FANFARE

34 of 41

Step 2: Create a Prompt

Key Elements:

Who
What
Where
When
Why
How
Format
Layout
Usability
Fanfare

35 of 41

Step 3: Generate Results and FLUF Test

Copy/Paste prompt

Copy/Paste AI Results

Score FLUF and Add comments

Recommend Revisions for Regeneration

36 of 41

Step 4: Update Prompt, Regenerate, FLUF Test

Update Prompt

Regenerate

FLUF Test

Recommend

37 of 41

Step 5: Repeat Until Zero FLUF

Update Prompt

Regenerate

FLUF Test

Recommend

38 of 41

Step 6: Combine AI Results with Human Creativity and Critique for Final Product

39 of 41

Declare Your AI Use

Scroll to the bottom of this article to see an example of AI declaration Embracing Lawn Ornaments: A Starter Guide - UF/IFAS Extension Sarasota County (ufl.edu)

“During the preparation of this work, the author used ChatGPT to help build the blog post. After using this tool/service, the author reviewed and edited the content, and takes full responsibility for the content of the publication. “

40 of 41

Resources

FLUF Test Experience Template (All Steps) Make A Copy	Sample Completed FLUF Test - Pharmacy Example	FLUF Prompt Template (Make A Copy)

41 of 41

For More Information

Contact:

JENNIFER PARKER, ED.D.

drjenniferparker@gmail.com

@drjennparker