1 of 10

AI Chatbots asking the right Questions

Why did you get the answers you did?

How do I get better answers next time?

Scene Set

Update from Barry (5 mins)

Four Questions:

What does my AI Chatbot know?
How does AI know what it knows?
How should I think of my AI Chat Bot?
Is Context Important?

Structured Exercises

Breakout Rooms
Comfort Break

Feedback and Summary

Share learning points in plenary

Summary of how to get better answers

10:00-10:20

10:20-11:00

11:00-11:30

Alan Freeland

This is a follow on session to last months Zoom session on an Introduction to using AI Chatbots. The assumption for this session is that you have access to a Chatbot and have used one before. This session is a hands on workshop with you using an Ai Chatbot, by preference ChatGPT more on this later.

Hopefully after last months session there have been times when using an AI Chat Bot that you have gone: Wow that is amazing - I cant believe how clever it is and I am sure there have also been times when you have gone I can’t believe how stupid it is. This session is to help you have more of the former reactions and less of the latter.

I will take you through the Agenda then I will hand over to Barry for a quick update on what is happening in our u3a world of AI.

Then as a scene set for todays workshop I will give us some insight to the answers to four big questions.

What does my AI Chatbot know?
How does AI know what it knows?
How should I think of my AI Chat Bot?
And Is Context important?

I will then introduce the exercises which you can do at your own pace. Different people will be at different levels of experience and that is fine.

We will split into Breakout out rooms to go through some structured exercises and share our experiences. Each room will have a facilitator and you can chose which room is most suitable for you. More on this also later.

We will then come back together at 11:00 to share our learning points. If you wish to take a comfort break about 10 mins to 11 would be good timing.

You should have received yesterday a one page document describing the exercises for today plus some hints and tips on prompting your Chatbot.

Our zoom host Liz will make sure the document is in the chat. To finish I will take you through some hints and tips.

The intent of todays session is as well as using an AI Chatbot in ways you hadn’t used before it will also help us to understand the answer to the question Why did I get the answer I did and how can I get a better responses next time?

Hand over to Barry.

Time: 1 min 15 secs

2 of 10

What does my AI Chatbot know?

create me an image in the style of a Victorian painting of a canal boat being pulled by a horse through an idyllic country landscape

Copilot

First question then is what does my AI Chat bot know? Lets prompt it and see what the reply tells us.

Here is a prompt for our AI Chatbot.

create me an image in the style of a Victorian painting of a canal boat being pulled by a horse through an idyllic country landscape

I think we can all visualise this scene. Indeed Barry shared with us last month some images created with a prompt like this.

This is what I got.

click

Horses on a towpath pulling a barge is a very much part of British history. However For most people round the world and especially Americans it was not part of their history

All the major Chatbots we are likely to use come from an American culture.

There are some pictures of horses pulling barges on the internet but more importantly there are hardly any images of a horse pulling a barge where the caption says a horse pulling a barge. So our Chatbot was never trained on this concept.

What I think our Chatbot has done is it knows barges are used to carry goods so it assumes I am asking for a picture of a barge carrying a horse.

So there will be some questions for which lack of training data means the Chatbot wont be able to give us a good answer.

Time 1 min 10

3 of 10

What does my AI Chatbot know?

What were the most popular colours in the gardens at The Chelsea Flower Show this year?

Purple: This was a dominant color throughout the show,….

Plum: A rich, deep plum was a favoured shade of purple…..

Orange: This provided a vibrant contrast to purple hues and was often used in conjunction with it to create visually striking displays.

Gemini

Copilot

Peach Tones: The Floristry and Floral Design exhibits incorporated an abundance of flowers in shades of peach.....

Contrasting Colors: The visitor’s eye was caught by contrasting colors of pink, purples, oranges, and yellows throughout the show…….

ChatGPT

Red,Orange and Yellow:…….

Pink, Purple, and Earthy Tones:……

Neon Colours…..

A second example to help us understand What does my Chatbot know?

I ran an AI Chatbot session on Monday and having been to the Chelsea Flower Show one of the attendees asked this very sensible question. What were the most popular colours in the gardens at The Chelsea Flower Show this year?

Lets reflect for a moment what this person meant. She meant having wandered round the Flower Show gardens she came away with an impression that the popular colours were a b and c - and she wondered was this what most people thought?

How do we expect our AI chatbot to answer this question? It clearly isn’t capable of wandering around Chelsea and forming its own view. If it had access to all the TV programmes on the flower show it could review all the footage and perform some form of analysis, but it probably doesn’t have access to these films and it would take a lot longer to watch them all and do the analysis than the second or two that the response is going to take.

So what answers did we get? And I have very much abbreviated all the answers for this slide.

click

Google Gemini said The dominant color was purple and in particular Plum with Orange as a contrast

click

Microsoft copilot said Peach with contrasting colors of pink, purples, oranges and yellow

click

ChatGpt said Red, orange and Yellow with Pink, Purple and Earthy tones as secondary.

How did these Chatbots reach these conclusions? Well they were not trained on data from Chelsea. They looked up articles on the internet and helpfully they do give you the source of the articles so you can read them. What it has done is scan the articles and picked up words like dominant and abundance and played these back to us. If you read the articles mostly the article is about a particularly display or a particular garden. I don’t think anyone had written an article that specifically sought to answer our question. But if they had I expect our AI Chatbot would have found it for us.

So are we disappointed not to have got the answer we wanted or do we think hey it did the best it could.

2 mins 10 secs

4 of 10

How does AI know what it knows?

Large Language Model

AI Chat Bot

Web Search

So how does our AI Chatbot know what it knows.

This is not the place to dive into detail but the 1 min answer is engineers have fed some of the largest computers in the world with as many words and images and sounds as they could get off the internet and said to the computer - go and spot patterns between all these words images and sounds. It may take these largest computers in the world six months of continuous processing but eventually they create something than the industry calls a Large Language model. This is a mathematical and statical model that describes the way we use language and how words relate to images and sounds.

Companies then create a small simple user interface called an AI Chat bot to provide us a way of communicating with this language model.

And because it is knowledgeable on language it can create web searches - just like you or I would and it can issue those web searches and aggregate or summarise the answers for us. The data it gets from the internet is data that already exists. The way it packages it for us it creates a new each time.

So Our AI Chatbot uses is knowledge of language to interpret our question to find information and to create answers.

Time 1 mins 10 secs

https://www.youtube.com/watch?v=_6R7Ym6Vy_I

5 of 10

How should I think of my AI Chat Bot?

Photograph: Todd Antony/BBC

Wikipedia, Official Photo

Your AI Chatbot has almost infinite Knowledge but zero Understanding

Given all that when we talk to our Chatbot how should we think about it?

We can think of our AI Chat Bots like The very best of BBC output such as Melyvn Bragg’s In our time or Clare Balding and the BBC’s sports analysis and insight. AI Chat Bots can inform and entertain us on a huge variety of topics and present many different view points.

There is a difference between a Chatbot and a person. We get to shape the narrative, If you want your questions answered with a particular bias - you are pro or anti immigration, same sex relationships, industry regulation or what ever, your Chat bot can provide that bias.

But in some ways an AI Chat Bot is like the brightest most knowledgeable but laziest teenager you have ever met. It can give you insights you would never come up with yourself but you might have to cajole a thoughtful response. They can also be lazy - after all computers are expensive to run.

And I wanted an image of a politician and selected Boris simply on the basis of his charisma. He is here representing all politicians. I personally think that Ai Chat Bots are very much like politicians. They are very good with words but mostly they don’t understand what they are talking about.

In reality of course AI Chatbots are just computers. A huge network of computers with an architecture similar to the way brains work. And just like with human brains no one fully understands how the technology behind AI Chat Bots really works.

Click

Your AI Chatbot has almost infinite Knowledge but zero Understanding

When you talk to your AI Chat bot it is worth keeping all these persona in mind.

Time 1 min 30

6 of 10

Is Context Important?

I need some bread

Pictures from copilot

The last of the four questions was is Context Important?

If you and I meet in the street we all ready know quite a lot about each other by what we look like, where we are, how we are dressed and how we talk.

You and your AI Chatbot don’t have this understanding of each other.

Asking an AI Chatbot to do something is called prompting. Creating a good prompt is key to getting the Chatbot to do what you want. The technical jargon for this is called Prompt Engineering.

And the most important thing to include in your prompt is context.

Let me try and illustrate what I mean by context.

If someone said to you I need some bread what would your reply be?

What if I add some context

Click

please create me a picture of refugee looking forlorn and hungry in a bombed out middle eastern town

What would your reply be now?

Or what about this context

please create me a picture of a youth in a threatening attitude to an old lady in a city centre

My understanding is that bread is sometimes slang for money.

Or what about this context

please create me a picture of a young boy near a supermarket, looking lost, asking for directions

Three different contexts which would provoke three very different replies.

The Chat bot doesn’t know your situation, hopefully this slide highlights the importance of providing context. The exercises are designed to show the value of context

Time 0 min 50 secs

7 of 10

The Exercises

Chatgpt.com

copilot.microsoft.com

Gemini.google.com

A text example - introduces the chat i/f and the need for detailed questions
Attachment example -introduces using the attachment icon
An Image example- demonstrates generative ai image creation.
Text Summary- demonstrates generative AI language analysis skills..
A poem example - introduces saving content.
Produce a song- introduces the concept of other AI tools.
Produce a multiple choice quiz

Breakouts

Main Room: Facilitated by Barry for beginners

Room 1: Facilitated by Alan

Room 2: Facilitated by Mike

Room 3: Facilitated by David

Enough then of the theory it is time to put your AI Chatbot to work.

For these exercises it doesn’t matter what Chatbot you use - If you have a choice I would recommend ChatGPT 4o. It is free and doesn’t impose onerous limits on how often you can use it.

Microsoft are tightening up how many questions you can ask of it’s free service currently its 30 only if you use its Edge web browser but it is only 5 if you use a different web browser.

And Google still isn’t offering to create images for anyone living in Europe or the UK which limits its usefulness.

The document we have sent to you and is in the chat is a one page document. On one side are the exercises for today and on the other some hints on Prompt Engineering.

There are seven exercise - you can work through them in sequence or just do the ones that interest you. You do need to do exercise 5 before 6. Each exercise is designed to get you using a different feature of the Chatbots. And after each exercise you are invited to reflect and share your observations on how useful or otherwise your Chatbot has been and how you can make its responses more relevant.

You can chose which breakout room you wish to go to and you can switch rooms. Liz do you just want to explain how this works?

8 of 10

How to get better results from your AI Chatbot

Be clear, precise and direct. Use an active voice “You must”, “Give me”, “Create”
Provide context.

How is the response going to be used, who is the audience?
What role are you playing. What role do you want the Chat Bot to play

How much do you want? E.g. the number of pages, or how many examples you want.
Ask for the response in the way you prefer.

A list, Step by Step instructions, a magazine article, a school text, the way I write.
Or share an example of what you want.
Ask for references if that is what you want.

It is a conversation. Refine, Refine and Refine some more. If you don’t understand a response, ask the Chatbot to explain, e.g. “Explain to me like I am an 11 year old”
Be critical of the responses. Ask for references if that would help.
Be aware they reflect human biases present in the training data.
They can get lazy, you may need to be assertive.
They are often not good with negatives - If you say don’t include an Elephant….
If you want Colour and not Color ask for British English.

Here then is a cheat sheet of how to prompt successfully. If you are just starting with AI Chatbots I think these are the most useful.

1. Be Clear and Precise: Just like giving directions, the clearer you are, the easier it is for the AI to understand and get to the point. Think about exactly what you want to know and ask that directly.

2. Provide Context. Explain to your ChatBot what role you are playing and what role you want the chatbot to play. You may want to consider starting a new chat session if you plan to dramatically change roles and topics For example if you spent the morning pretending to be a pirate whilst your Chatbot is a mermaid then in the afternoon you want help preparing a talk on geo-politics for your local u3a you may wish your chatbot to forget the mornings session.

3. How much do you want. e.g. If you want a poem is it like a Japanese three line haiku or like the 200,000 line Indian epic poem Mahabharata,

4. What style of response do you want? A list of bullets, a scholarly article ? Let the Chatbot know.

5. And remember this is a conversation if the response you get isn’t quite what you want tell the Chat bot how to modify what it has done, e.g. say make it shorter, or give me more detail on this bit , or use simpler words.

AI Chatbots are like people - some times they make stuff up. If the response is important to you you may wish to check it. You can always ask for references
Be aware they reflect human biases present in the training data. Most training data reflects a white western demographic.
It costs resources to analyses your request and provide an answer. Especially if the AI service is busy it may try and minimise the amount of work it needs to do - you can push back, tell it to try harder.
And just like if I say to you don’t think of Elephants - you will think of Elephants, Chatbots will give more emphasise on the noun and less on the word not.
Most Chatbots assume you want American rather than British English - if you want British English ask for it. ChatGPT allows you to specify this as part of your profile.

https://jamesg.blog/2023/04/02/llm-prompts-source-attribution/

Self consistency

“List and explain the causes of the Spanish Civil War. Then, identify which cause was most significant and explain why”

Retrieval-Augmented generation (RAG)

Add additional data sources for domain specific applications

Limiting extraneous tokens

“Explain the process of photosynthesis in simple terms, focusing only on the key steps and elements involved”

System prompt - added to each context

“You are a pirate”

9 of 10

More Advanced techniques

More Advanced Techniques

Ask for alternatives or variations: – Give me three options/ answers.
If it is a complex topic and you don’t know how to get started, ask: “How do I get started”, or “give me a template for”, or “what should I consider” ..
If it is a big topic break it down into smaller steps – responses from the Chatbot will tend to be more relevant. Use delimiters (brackets, paragraphs, punctuation) to separate different ideas.
Try different Chatbots

Once you have built up some experience in using AI Chatbots here are four more suggestions on how to get the answer the way you want it.

Ask for alternatives. You could ask for example for the Chatbot to provide you three variations of the answer. Google’s Gemini does this automatically for you. And some image generators will give you a number of different images to chose from.
AI Chatbots are great at helping you get started on a complex subject you are not familiar with and if you don’t really know how to get started. Just ask. For example ask What should I consider when Buying an electric car, going on a cruise for the first time, or giving a talk on geo-politics, or whatever.
Just like with humans if you provide a long introduction your audience can lose the thread - so can AI Chatbots. This is one of the differentiators between different Chatbots and different versions of Chatbots. Some are better than others at handing long prompts. Whichever Chatbot you use responses tend to be more focused on what you want if you keep them short. Even for something as simple as a letter you can break it down into an introduction, main content and close. This way you can more easily refine each section to get the content and tone you want.
And again just like with humans if you ask the same question to different people you will get different answers, not just in content but in style and how it is presented. If you do start to see real value in using Chatbots you may wish to play with some different ones. And bear in mind that you will likely get better responses and a better service if you pay for a Chatbot. And also note that Microsoft and OpenAI are partners so Copilot and ChatGPT are likely to be more alike than Google’s Gemini or many of the other alternatives.

Hopefully these tips will help you to keep getting the Wow that’s amazing experience.

1 of 10

2 of 10

3 of 10

4 of 10

5 of 10

6 of 10

7 of 10

8 of 10

9 of 10

10 of 10