JavaScript isn't enabled in your browser, so this file can't be opened. Enable and reload.

1 of 58

Getting Started with AI:�Research, Writing, & Critical Evaluation

UNIVERSITY OF FLORIDA

CENTER FOR TEACHING EXCELLENCE

Getting Started with Artificial Intelligence in Teaching and Learning: Research, Writing, & Critical Evaluation © 2023-2024 by University of Florida, Center for Teaching Excellence is licensed under CC BY-NC-ND 4.0

http://tinyurl.com/GSWAI-2-NOTES

2 of 58

INTRODUCTIONS TO TODAY'S PRESENTERS

JOHN JORDI, PHD

Instructional Assessment Coordinator

Center for Teaching Excellence

JENNIFER PARKER, ED.D.

Faculty Development Coordinator

Center for Teaching Excellence

LEOTA O'MALLEY, M. Ed.

Online Course Quality Specialist

Center for Teaching Excellence

MICHAEL BARBER, PHD

Assistant Director

Center for Teaching Excellence

KIMBERLY HEAL, M.S.

Graduate and Post-Doc Development Coordinator

Center for Teaching Excellence

http://tinyurl.com/GSWAI-2-NOTES

2

3 of 58

SESSION 2:

RESEARCH, WRITING, AND CRITICAL EVALUATION

VIRTUAL

90 MINUTES

SESSION 2 OF 4

http://tinyurl.com/GSWAI-2-NOTES

4 of 58

Session 2�GSWAI: Research, Writing, and Critical Evaluation

AGENDA

Agenda:

Using AI Tools with Writing Tasks
Ethical Considerations
Prompt Creation
AI Policies and the Syllabus - Hands On
Using AI Tools for Research
Critical Evaluation of AI Generated Content - FLUF Test
Citing AI Use
Research and AI - Hands On

5 of 58

Thoughts on Student Use of AI

Not Happy

Excited

Not Sure

5

6 of 58

Your Comfort Level with Using AI

Not Singing Yet

Rock Star

Karaoke Singer

6

7 of 58

Is AI the new Google? Will it become our new "go to" for information?

7

8 of 58

Artificial Intelligence in Text

Writing

Sample Tasks

Syllabus
Email
Report or Summary
Policies
Essays
Letters
Proposals
Case Study

8

9 of 58

Ethical Considerations

An AI generator cannot pass a course
AI contributions must be attributed and accurate
AI use should be open and documented
AI detectors are questionable at best

9

10 of 58

Food For Thought...

The balance between human insight and technological capability in artificial intelligence

80% research and regeneration
20% human critique, creativity, and culmination

Human Component: The 80/20 Rule�Pareto Principle

Using AI can help brainstorm that task we've put off, lighten the load of a time consuming task, increase our productivity, or act as a tutor.

However, there is a delicate balance between human insight and technological capability that is crucial to our use of artificial intelligence.

This concept is illustrated by the 80/20 rule, which highlights the interdependence of human expertise and AI capabilities.

In practice, this rule is widely acknowledged in the industry, emphasizing the need for a blend of human and AI contributions.

Experts agree that while AI plays a significant role, human expertise should remain the predominant influence, adhering to the 80/20 ratio in combining these elements.

As you explore AI tools like ChatGPT, CoPilot, Gemini, and Claude to name just a few, always give the human element – YOU - the "final say" by polishing up the results before submission.

Although AI and online research can lay the foundation (about 80% of the content), the human component should remain at the forefront where higher-order thinking expands, organizes, and critiques the final submission for the target.

This is where the FLUF test comes into play.

Pareto, Vilfredo (1906). Pareto Principle. Attributed by Joseph M. Juran (1940).

11 of 58

The Tools

11

12 of 58

How are AI prompts like Google search terms?

12

13 of 58

YOUR RESULTS are dependent upon YOUR ENTRY of good search terms or prompts.

Human Component:

Search Terms and Prompts

14 of 58

Human Component:

Critical Evaluation of AI Generated Results

Use frameworks like The FLUF Test to critically evaluate your AI generative results and improve your prompts.

Visit the FLUF Website.

15 of 58

Start by choosing a topic and generating a prompt.
Be as specific and descriptive as possible.
Include who, what, where, when, why, and how for the AI tool to generate your results.

The Art of the Prompt

15

16 of 58

TIP: If you want "peer reviewed research" then include that detail in your prompt.

The Importance of the Prompt: Research vs. Freely Available on the Web

16

AI generated content can be like Wikipedia – how do you know it's valid and reliable? It's only as good as the prompt it's given – if you want peer reviewed research or proven methods, tell it that

Example

Vet Med – Snake Venom Article
In the example, we show on the left the results from ChatGPT when prompted to "Describe the optimal treatment for severely envenomated dog caused by an eastern diamondback rattlesnake."
The results share the use of a rattlesnake vaccine – which is a long-standing joke and trick question in Veterinary Medicine - because there is no such thing / or if there is it is a marketing ploy that doesn't work.
By expanding the prompt to include "research based, proven methods" improves the results significantly.
In the new result, the Rattlesnake Vaccine disappears from the results.

17 of 58

Build a Prompt

Act like a [specify a role]
I need a [what do you need?]
You will [enter a task]
In the process, you should [enter details]
Avoid [enter exclusion]
Input the final result in a [select a format]
Here is an example: [enter an example]

17

18 of 58

Who?

What?

Where?

When?

Why?

How?

Documenting the Research Process: From Curation to Critical Evaluation of Resources Using Library Databases, Generative AI, and Beyond © 2023 by Dr. Jennifer Parker- University of Florida Center for Teaching Excellence  licensed under CC BY-NC-SA4.0 

Prompt Element	Prompt information
Who
What
Where
When
Why
How
Format
Language
Usability
Fanfare
From the information above, generate the prompt

18

19 of 58

FLUF Proof �Your Prompt

Prompt Element	Prompt information
Who	A fifty year old white woman with red hair and tortoise shell glasses
What	Working
Where	In her office at the University of Florida
When	During the day
Why	Grading exams for end of term
How	Using the computer
Format	Animation
Language	In English�
Usability	To be used as a selfie�
Fanfare	For social media for personal or profile use�
From the information above, generate the prompt Create a black and white image of a 50 year old red headed woman wearing tortoiseshell glasses. She is working in her office at the University of Florida during the day grading end-of-term exams on the computer. The image will be used as a selfie or profile picture.

Make a copy of the template

20 of 58

How is the reaction to generative AI

similar to the reaction when other tools were created (calculators, the Internet)?

20

21 of 58

AI Policies

Policy Considerations:

Conditions for use
Acknowledge the use of AI in work
Warnings and guidelines for critical evaluation
Accountability for AI output
Ethical and responsible use
Feedback on AI policies and use

21

An AI cannot pass a course

AI contributions must be attributed and true

AI use should be open and documented

Here are six key questions that your AI course policy must address:

Under what conditions is the use of AI allowed or prohibited?
How should students acknowledge the contributions of AI in their work?
What warnings and clear rules should be established regarding the propensity of AI to generate deceptive data (i.e. hallucinate)?
How should student accountability for AI output be managed?
How will you emphasize and motivate the ethical and responsible use of AI among your students?
How will you engage your students to provide feedback or collaborate with you on your policy at the beginning of the course? (and what feedback will you ask students for at the end of the course?)

22 of 58

Artificial Intelligence in Text

Your Turn

Write an AI Course Policy

Decide the six key questions about an AI policy.
Write a prompt to add an AI Policy given those answers for your syllabus.
Use the template as a guide

22

23 of 58

Start from scratch vs.

Uploading content?

That is the question...

23

24 of 58

Artificial Intelligence in Text

Your Turn

Warming Up your Syllabus

Change the tone or style of writing using AI
Prompt the tool to "sound less contractual but maintain a professional tone"
Prompt the tool by explaining who the audience is.

24

25 of 58

Using the AI Prompt Cookbook

Explore the AI Cookbook

25

26 of 58

Artificial Intelligence in Text

Your Turn

Writing Case Studies

AI Prompt Cookbook
Write an authentic case study about [individual and scenario] that allows readers to analyze how [theories/concepts]
Write three scenarios about [topic] that a college [subject] teacher could use to assess whether students can [learning objective]. Ask a question at the end of each scenario that assesses [learning objective].

26

27 of 58

But I want peer reviewed research – can AI do that?

27

28 of 58

The Tools

28

29 of 58

Reminder: �Guiding Principles

The balance between human insight and technological capability in artificial intelligence

80% research and regeneration
20% human critique, creativity, and culmination

Human Component: The 80/20 Rule�Pareto Principle

Using AI can help brainstorm that task we've put off, lighten the load of a time consuming task, increase our productivity, or act as a tutor.

However, there is a delicate balance between human insight and technological capability that is crucial to our use of artificial intelligence.

This concept is illustrated by the 80/20 rule, which highlights the interdependence of human expertise and AI capabilities.

In practice, this rule is widely acknowledged in the industry, emphasizing the need for a blend of human and AI contributions.

Experts agree that while AI plays a significant role, human expertise should remain the predominant influence, adhering to the 80/20 ratio in combining these elements.

As you explore AI tools like ChatGPT, CoPilot, Gemini, and Claude to name just a few, always give the human element – YOU - the "final say" by polishing up the results before submission.

Although AI and online research can lay the foundation (about 80% of the content), the human component should remain at the forefront where higher-order thinking expands, organizes, and critiques the final submission for the target.

This is where the FLUF test comes into play.

Pareto, Vilfredo (1906). Pareto Principle. Attributed by Joseph M. Juran (1940).

30 of 58

INSERT COPYRIGHT

31 of 58

Prompt:

"I am writing a report (or blog or whatever) about [insert topic]. Give me a list of key facts and statistics I should include in the report."

Perplexity

Cite Sources

Research, paraphrase, and cite sources.

31

32 of 58

33 of 58

Consensus

Research, citations, filters, analysis, and excerpts.

Prompts:

“Does creatine improve cognition?“
“Write the introduction to a literature review on the effects of particulate air pollution on the economy“
“Find me five papers about visceral fat and the connection to cancer published after 2000 and before 2020“
“I need papers on the mmr vaccine and its connection to autism. I only want papers from prestigious journals that are meta analysis’ or RCTs“
“What was the sample size of the control group in this experiment?“
“I want a list of effective ways to prevent homelessness“
“Write a blog for a health website on the benefits of Zone 2 cardio exercise for longevity“
"Create 5 different cases that would test vet med students' ability to create a treatment plan for a dog with tachycardia."

Research Tool

33

34 of 58

35 of 58

Jenni

Write, Cite, and Edit

Outline builder, in-text citations, and Research Library

Prompts:

“Does creatine improve cognition?“
“Write the introduction to a literature review on the effects of particulate air pollution on the economy“
“Find me five papers about visceral fat and the connection to cancer published after 2000 and before 2020“
“I need papers on the mmr vaccine and its connection to autism. I only want papers from prestigious journals that are meta analysis’ or RCTs“
“What was the sample size of the control group in this experiment?“
“I want a list of effective ways to prevent homelessness“
“Write a blog for a health website on the benefits of Zone 2 cardio exercise for longevity“
"Create 5 different cases that would test vet med students' ability to create a treatment plan for a dog with tachycardia."

35

36 of 58

37 of 58

Quillbot

Change Languages

Translate, cite, check grammar, or paraphrase.

Prompt: (paraphrase & translate)

"The study of meteorology dates back to ancient civilizations, but it was not until the 17th century that it emerged as a science. Pioneers like Galileo Galilei and Evangelista Torricelli contributed to its development by inventing instruments like the thermometer and barometer. "

37

38 of 58

39 of 58

40 of 58

Artificial Intelligence in Text

Authority and Credibility

Confessions of an AI User

Explore the blog and the Disclaimer
Use AI to generate a disclaimer

40

41 of 58

Are AI generated results like Wikipedia – where I need to critically evaluate the information?

41

42 of 58

Critical Evaluation of AI

Documenting the Research

iSearch, CRAAP and FLUF

Prompt Generation
iSearch Process
Critical Evaluation of Online Sources
Critical Evaluation of AI Generated Content
iSearch Paper

42

43 of 58

Documenting the Research Process

44 of 58

ISearch Elements

Know and understand the elements of the iSearch template:

Topic/Issue or Challenge/Problem of Practice
What I Know / Want to Know
The Story of My Search
Search Results
Significance of the Research Experience
My Growth as a Researcher
Works Cited

Based on the iSearch Model �Macrorie, K. (1988). The I-Search Paper--Revised Edition of" Searching Writing.". Heinemann Educational Books Inc., 70 Court St., Portsmouth, NH 03801.

45 of 58

Get Started: Make A Copy

RUBRIC AND PAPER TEMPLATE

Make a copy of the following:

Review/Make a Copy of the Assessment Rubric and the Template

https://docs.google.com/document/d/1w9MDKz7AJsWGZYU7azbQWNNL_rkXuFdh/copy

46 of 58

Critical Evaluation Frameworks �for Online Sources

CRAAP (Blakeslee, 2004)	CARRDSS (Valenza, 2004)	SIFT (Caulfield, 2019)	5 Key Questions (Thorman & Jolls, 2003)
• Currency • Relevance • Authority • Accuracy • Purpose	• Credibility • Accuracy • Reliability • Relevance • Date • Sources • Scope	• Stop • Investigate • Find • Trace	1. Creator 2. Techniques 3. Perceptions 4. Bias 5. Purpose

Spotting Bogus Sites, Fake News, Propaganda

With fake news, propaganda, and openly editable content like Wikipedia, where do you begin?
Let's start with the major frameworks – CRAAP, CARRDSS, SIFT, and the 5 Key questions. These are three of the most prominent ways to assess what you find on the internet for its usefulness.

Depending on your goals, one of these should be a fit for your own toolbox. As we move through the frameworks, jot down some ideas you may have about how the elements may apply to your own online research. Consider which one will be most helpful to you in your own research journey.

-----------------------------------------------------------------------------------------------------------------------------------------------------------

The CRAAP test was originally created by Sarah Blakeslee and her team of librarians from California State University - Chico’s Meriam Library in 2004.

Former K-12 school librarian, now Rutgers Professor Dr. Joyce Valenza (2004) is credited with promoting the CARRDSS framework.

The SIFT method was created by Mike Caulfield. All SIFT information on this page is adapted from his materials with a CC BY 4.0 license. Blog post on June 19, 2019

SIFT (The Four Moves)

Elizabeth Thoman and Tessa Jolls (2003) established The Center for Media Literacy and developed the 5 Key questions as part of a media kit.�

© 2003 Center for Media Literacy / www.medialit.org Literacy for the 21st Century / Orientation & Overview PG 18

Literacy for the 21st Century An Overview & Orientation Guide To Media Literacy Education Part I: Theory CML MediaLit Kit™ A Framework for Learning and Teaching in a Media Age Developed and written by Elizabeth Thoman Founder and Tessa Jolls President / CEO Center for Media Literacy www.medialit.org © 2003, 2005 Center for Media Literacy For terms of usage, go to www.medialit.org/medialitkit

CRAAP

• Currency: Is the source up to date?

• Relevance: Does the source connect to what you are trying to research?

• Authority: Are the authors respected in their field for being knowledgeable about the topic?

• Accuracy: Is the research supported by evidence and free from errors?

• Purpose: Is the source free from bias or motive by the author, or issues with validity and reliability?

CARRDSS

• Credibility: Are the authors respected in their field for being knowledgeable about the topic?

• Accuracy: Is the research supported by evidence or data and free from errors?

• Reliability: Does the source provide trusted information that is validated in other places?

• Relevance: Does the source connect to what you are trying to research?

• Date: Is the source up to date?

• Sources: Does the source document where information came from by detailing citations?

SIFT

• Stop: Before reading, does the article makes you feel comfortable and confident about the source and

the information.

• Investigate: Does the author have a solid reputation as an expert

• Find: Are there other sources that support the findings of this article?

• Trace: Are you able to find this information across other sources and can you verify the author has

presented information in an unbiased and factual way?

Five Key Questions of Media Literacy

1. Who created this message?

2. What techniques are used to attract my

attention?

3. How might different people understand

this message differently from me?.

4. What lifestyles, values and points of view

are represented in, or omitted from, this

message?

5. Why was this message sent?

47 of 58

Critical Evaluation Frameworks for AI Generated Content

FLUF TEST (Parker, 2023)
Format – Layout and Length Language – Tone, Phrasing, Repetition Usability – Credibility, Consistency Fanfare – Anecdotes, Jargon

48 of 58

49 of 58

The Goal: "Zero FLUF"

We are looking for Zero FLUF, or zero evidence of infractions. A ZERO means a good AI generated result and a "go" for using it in your work.

50 of 58

Got FLUF?

Re-Prompt

Regenerate

Repeat

Until you get

51 of 58

Groq

Speedy reply and a drop-down menu for length, format, and audience.

FLUF test meets AI

Prompt:

"Create a 750 word essay about homelessness..."

Modify:

Change the audience to "professional"
Change the length to 3 sentences

51

52 of 58

53 of 58

54 of 58

Additional AI Resources

Syllabus Resources - Google Docs	Syllabi Policies for AI Generative Tools - Google Docs	Sample AI Policy
AI Prompt Cookbook - Google Docs	Prompt �Generation	Disclaimers of AI Generated Content
UF Faculty Handbook for Adding AI to your Course	Documenting Research: iSearch, FLUF, Prompts	FLUF Test
ISearch Paper	Meteorology sample paper

54

55 of 58

Getting Started with Artificial Intelligence in Teaching & Learning

An Online Learning Series

The “Getting Started with AI in Teaching and Learning” is a four-part series that takes a deep dive into the use of generative Artificial Intelligence (AI) in teaching and learning.

Next In the Series

55

56 of 58

SESSION 3:

PRESENTATIONS AND VISUAL IMAGERY

VIRTUAL

90 MINUTES

SESSION 3 OF 4

57 of 58

SESSION 4:

ASSESSMENTS

VIRTUAL

90 MINUTES

SESSION 4 OF 4

58 of 58

Q AND A

CONTACT US:

CTE@aa.ufl.edu