1 of 21

Making Friends w/ AI

Carol-Lynn Perez

Senior Lecturer

Communication Studies

2 of 21

Today’s Agenda

  • Spring 2023
    • Outcome of not having an AI policy
  • Brainstorming
    • How to incorporate AI into current curriculum
  • New Assignment
    • COMPARE AND CONTRAST ChatGPT/AI Assignment
    • 4 parts to this assignment
  • Students Perspectives

​

2

8/18/2023

3 of 21

Description of Assignment

​

  • 1 sentence description:
    • Comparative Analysis and Critical Evaluation of AI-Generated Responses Based on Varied Prompts Yielding Differential Outcomes
    • The outcome of the assignment was unexpected
    • In short, I have incorporated AI into an assignment to teach students:

First, I wanted students to learn about the course material

Second, I wanted students to recognize the strengths and weaknesses of AI

3

8/18/2023

4 of 21

Invention is born out of necessity

  • Comm 179 – 39 students enrolled in Sp23
  • 9 out of 39 were detected
  • 23% of the total class
  • 7/9 admitted to using AI
  • 1 adamantly denied using AI despite a 100% match
  • 1 no response at all

​

4

8/18/2023

5 of 21

Brainstorming for AI Practical Application

Connect with the University, which has no official stance on AI as of Spring 2023

  • Read current articles from
    • Chronicle of High Education
    • NASPA
    • Inside Higher Ed
  • Brainstormed what I wanted students to learn
    • Two-fold idea previously mentioned
      • First, I wanted students to learn about the course material
      • Second, I wanted students to recognize the strengths and weaknesses of AI

​

5

8/18/2023

6 of 21

The catalyst for change

  • Compare and Contrast Research Paper
    • 15% of the total grade
    • Assignment utilized course textbooks and outside research
    • I had a list of detailed list of questions students should confront in the paper
    • I asked them to go beyond reporting facts, and to use interpretation and depth of analysis
    • Properly organized paper with college-level writing
    • Formatting was very specific

​

6

8/18/2023

7 of 21

Overview

This 4-part assignment will gave students an opportunity to play around with ChatGPT and to compare and contrast two different movements that were covered in the course text. They were required to run a number of different prompts related to two movements of their choice using ChatGPT or another AI generator. They were required to read the entirety of the assignment before they began running prompts through ChatGPT so they could understand learning goals and requirements.

​

7

8/18/2023

8 of 21

Purpose

Incorporate ChatGPT into the classroom at a relatable level, evaluate what was generated by ChatGPT, look for strengths and weaknesses in both the prompts that students create and what was generated by artificial intelligence, and draw global conclusions about both the content generated and their experience. They also demonstrated their knowledge of the materials and concepts covered in the class and illustrated this understanding by referencing course materials and outside research in Section 3, including using Castells book and other online sources.

8

8/18/2023

9 of 21

Example Prompt

Write a 1,000 word cohesive essay based on Iceland Revolution and Occupy Wall Street including the following information: Compare and contrast the movements to include social media strategy, political and cultural contexts, what set each movement in motion, what makes the movements similar and different, and did one movement affect the other.

​

9

8/18/2023

10 of 21

Formatting & Organization

List the prompts they used (no less than 10), and then they should highlight/bold the prompt you ultimately use for this assignment as shown below.

​

They listed the responses in order, as I want to see the progression of their prompts and how those changed as they moved through the assignment.

​

10

8/18/2023

11 of 21

Trends and Observations

​

  • Students were frustrated

​

  • Running different prompts did not wildly change the AI output

​

  • Limitations were observed by students

​

​

​

11

8/18/2023

12 of 21

Evaluation of AI Response

They were required to evaluate the strengths and weaknesses of the response.

​

I asked that they copy and paste an unedited response and then do it a second time and make notes in the actual response itself. ��They were required to delineate their writing from AI’s writing using underline, bold or italicize in the body of the paper in so I can see their input, evaluation and analysis.

​

12

8/18/2023

13 of 21

Evaluation Con’t

I asked them to pick apart the paper in detail, using information they have learned from the class, and from research online using credible websites. ��They also added information if AI has generated a vague response, given inaccurate information, or has omitted pertinent information.

13

8/18/2023

14 of 21

Evaluation Con’t

In other words, I wanted them to go section by section, line by line, and evaluate the response generated by AI and give me both a critical evaluation AND factual information you have learned in the class so far.

​

They were required to cite their sources in the text of the paper, and explain why they have corrected the AI generated information.

​

14

8/18/2023

15 of 21

Dissecting AI Responses

  1. AI presented *some* factual information
  2. Writing format, syntax and sentence structure were correct but bland
  3. Students were frustrated when they noticed limitations
  4. Students recognized AI has many strengths, but has a lot of weaknesses

​

15

8/18/2023

16 of 21

Student Insights

Eye-opening, challenging, frustrating

Sterile responses

Unforgettable experience

17 of 21

“This process of attempting to use AI to create an academically appropriate and successful essay has been incredibly eye-opening, challenging, and honestly frustrating. Through this experience, I have reflected and drawn conclusions highlighting how using AI for such things only allows for surface-level analysis and explanation, resulting in a broad and unimpactful narrative that can be challenging to comprehend and even harder to connect with.”

“

Student 1

​

”

17

8/18/2023

18 of 21

“Despite none of the claims made being 100% false, there was an overall lack of human intellect with the responses generated. With ChatGPT, I felt as if the generated responses seemed extremely sterile in their information and layout.”�

“

Student 2

​

”

18

8/18/2023

19 of 21

“This assignment has allowed me to explore ChatGPT and analyze the perks and flaws of it. If I had to use ChatGPT, I would use it as a guide in helping me figure out what points to talk about in my paper. Even though its responses can be vague, I could conduct the additional research on my own to prove that the claims made are true. This is definitely an experience that I will never forget.”�

“

Student 3

​

”

19

8/18/2023

20 of 21

Questions?

21 of 21

Thank you

Carol-Lynn Perez​

carol-lynn.perez@sjsu.edu

www.carollynnperez.com