1 of 21

When Robots Learn to Write, What Happens to Learning?

Bill Hart-Davidson, Ph.D.

2 of 21

About Me

Bill Hart-Davidson, Ph.D. (Purdue, 1999)

Professor in Dept. of Writing, Rhetoric & American Cultures

I study writing, broadly construed, as human activity. I tend to be interested in behavior(s) more than texts. This is because I am fascinated by writing and its centrality to our organizational lives.

If writing gets things done in the world, maybe better writing can make things better.

I would like to thank Burt Bargerstock and Julie Crowgey, for inviting me to speak today. I am really honored to spend some time this morning with you and to share some thoughts that I hope launch more engaging conversations today, tomorrow, and beyond. Much of the prior work I’m going to reference today, however briefly, has been done in close collaboration with colleagues, and especially Dr. Ryan Omizo of Temple University. So whenever I say “we” throughout this presentation I mean to acknowledge him and various other collaborative partners who have contributed along the way. I work best as part of a team and I’ve been very fortunate to have great colleagues to work with in my career.

My field is rhetoric and writing, which is both ancient and relatively new depending on which of those two terms you choose to emphasize. I study writing as it happens in the world, not so much after it has coalesced into a text. I am especially interested in the writing people do that helps those people do things like learn, care for others with chronic illnesses, etc. Another strand of my research looks at writing in the context of primary care and how better writing can improve health outcomes. That is health services research and it is clinical - literally, we have a clinical trial in the field funded by an NIH RO1 currently.

But today, I’m honored to explore the interesting dynamics of Large-Language Models and writing among a group of fellow travelers - writers, learners, researchers, teachers and mentors! I’ve been thinking about these things for many years, have made some writing software and have done studies that use LLMs myself. I hope I can share some of what I have learned today in a helpful way.

3 of 21

Writing is a Human Activity

Grant

Letter

of Intent

Most of the writing that we do is part of a stream of communication events.

Our focus in grant seeking, for instance, is on proposing, not writing a great proposal. If we could skip it, we would!

2003

Let me say just a bit more about what I mean by studying writing as a kind of human activity and give you an example. When I was a PhD student at Purdue, I was lucky enough to attend a talk by a visiting researcher - Britt-Louise Gunnarson - who was a sociolinguist. She had been working on creating visualizations of writing activity as a way to try and understand it as a social phenomenon rather than just an individual one. One thing she emphasized that really stuck with me and sparked my own research interest was the way writing as an activity is diachronic. Every piece of writing is part of a stream of activity that precedes and often profoundly shapes it. And if we pick any one particular text to focus on, we can identify other literate events upstream and downstream that constitute the goal-oriented activity that these events are part of.

The image here is from a study I did early in my career. I was trying to understand how successful grant writing teams did their work and what we might discover that could account for their success, which in the case of grant writing is securing funding. One realization I made and that still registers some surprise today is the idea that successful teams don’t focus on writing good proposals. They focus on good propos-ing. Getting to the goal of an effective outcome in the most effective and efficient way means that some of the things that make a proposal effective occur well “upstream” of the proposal document.

There is much that is and will remain deeply human about the activity of writing. Even when we have robots to help.

4 of 21

Topics for Today

A little about Large Language Models (LLMs) and how they work…or

What the heck just happened to make GPT-3 so much better at drafting longer texts that resemble human-drafted ones? Spoiler: Transformers.

A little about writing (as humans do it and as robots do it)

Writing is intentional, goal-directed activity that sometimes results in a text. LLMs can simulate some parts of this activity better than others.

Four Proposals for Writing and Learning with AI in the Loop

How should we be thinking differently about the writing tasks we ask students to do and how we evaluate them?

5 of 21

This is ChatGPT - GPT in a chatbot

A chatbot interface connected to the GPT-3 LLM produced by OpenAI

Trained with human feedback to be responsive to dialogic queries

Capable of producing texts that are similar in content, tone, and formatting to those that a human would make.

No Meat

So what happened recently? Why are we here? One reason is this guy… ChatGPT, a chatbot released toward the end of last year. It is built on something called a Large Language Model, an LLM, that has been in development for years prior, at least since 2017 or so for GPT. Hence the 3, behind GPT.

ChatGPT, as you’ve no doubt heard and or seen for yourself, can do some interesting things. (list above)

Here I’ve asked the ChatBot to give me a recipe for Nachos. And it has done a pretty remarkable job, to my eye. For one thing, it’s got all the semantic chunks right. Not just the words, but look at the way it has given me the Ingredients and then a set of Instructions as two distinct pieces. It’s marked some ingredients as optional. The instructions are sequential while the ingredients are unordered. The ingredient list includes quantities.

And…something else…the formatting. It *looks* like a recipe. What does that tell me? This thing knows that blank space and line terminations are important to this genre. This helps a recipe not only work as a kind of intersubjective (from one entity to another) message, but also as a usable object to guide my actions when I’m in the kitchen. Remarkable, given that the prompt I gave it was very simple.

But there are some curious things about this recipe too. One is… no meat. Huh. That works for me. It would make some of my vegan friends happy. But it would get me run right out of the house at some of my relatives’ places in Texas! So…what’s going on here?

6 of 21

How Does ChatGPT Know What to Write?

The LLM, GPT-3, has been trained on a very large collection of texts. Hundreds of billions of words. After processing, these are called “tokens” in LLM parlance.
The “T” in GPT stands for “transformer.” The model converts the words to tokens and the tokens to a very large graph that allows for more efficient computation than just working on a big string of words does.
GPT-3 is known as an autoregressive model. Based on one string of tokens, it predicts what comes next using probabilities derived from its training corpus.
GPT-3 is a “zero-shot” classifier, which means it does not need any examples of the thing it is trying to recreate. This contrasts with one-shot or few-shot models.
ChatGPT has other layers of training too - one from feedback that evaluates responses and is used to refine future ones. This makes it a “deep learning” application.

7 of 21

This is Lex - GPT3 in a Word Processor

No Formatting

Clickbaity Titles

Still No Meat

This is a different application called Lex. It is a word processor - similar to Google docs but a little more simple - with GPT built in. This, rather than a standalone Chatbot is probably how most of us will interact with LLMs soon enough. It will just be part of our writing tools…and everywhere. So instead of just typing words, you can also ask it for help generating some based on a prompt and/or what you’ve already written.

I asked it the same thing I asked ChatGPT, for a Nachos recipe. You’ll see it gave me the same words, pretty much, but without the formatting. This tells me that the deep learning training layer for ChatGPT has improved the results for genres like recipes compared to what comes from the more “raw” result of the LLM.

In Lex, you can ask it to do something I’ve always had a bit of trouble with: come up with a title. You click the little icon and it suggests some.

All of these read as kind of clickbaity to me! Funny, but it reveals something else that is tricky about LLMs…they have no idea what you are up to! So they are guessing. Here it might think I’m posting to a food blog and want to get clicks. Of course it doesn’t think that at all. What it has in its training corpus is an abundance of semantically validated titles from the web! But if I’m instead making a shopping list for the store, I probably don’t need a title urging me to “Unleash Your Inner Chef”... lol.

8 of 21

Transformer - More than Meets the Eye!

“An attention function can be described as mapping a query and a set of key-value pairs to an output, where the query, keys, values, and output are all vectors. The output is computed as a weighted sum of the values, where the weight assigned to each value is computed by a compatibility function of the query with the corresponding key.”

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.

But…so…what happened this Fall? Or how did robots get so good at writing so suddenly (apparently?)

The breakthrough that happened is not really one in writing per se, not is it terribly recent. As often happens, the innovation that has found an application in these text-generating applications is one in sequential modeling and computation.

The Google Brain team and folks who have gone on to become part of the OpenAI group who released GPT-3 and ChatGPT created a new kind of sequential modeler that was vastly more efficient than the previous kinds. The previous kinds were recursive. The looked at a string of words - or tokens as they are called after processing - performed some classification on those tokens and then computed the probability of what the next token in the string would be.

You don’t have to be in computer science to see how computationally expensive this approach gets very quickly as the tokens pile up. Recursive models can only look at so much text and crunch so many possibilities before it becomes infeasible in human lifetimes to get high-quality results.

The transformer works faster because it’s not looking at words. It transforms the tokens into vectors and evaluates a graph, a much more computationally efficient object. Let’s geek out a little bit with a simple example of what I’m talking about. This comes from some work I did a few years ago.

9 of 21

Texts: not text strings but networks that grow over time

Let’s say we want to produce something like a clickbaity headline. First we transform a text from a string of words into a network graph. In the example here, we are contrasting a word-based (dictionary) approach for analysis of titles to a vector approach. After some text processing, words become “tokens.” Each token is a node in the graph linked to the next one in order. In this model here, if a word repeats, we don’t add another token, we add a link. This logic and how you create edges in the graph matter a great deal. They produce “embeddings” or commonly repeated patterns of tokens that recur.

This process increases the information we know about each node. But more importantly, we start to see a more global structure that corresponds to the whole title. That title’s structure can be further represented in the adjacency matrix you see on the right, which is just a binary representation of the graph.

Now, regardless of the semantic value of that structure, we can go looking for similarity between it and other structures and classify those as instances of the thing we’re looking for. In this example, we might see that many titles about different topics share this structure (and with a deep learning approach, we might see that titles like this may outperform those that don’t look like this). If so, we can start to make an inference between this repeatable pattern and the stability of the genre it represents - a clickbait headline. Where we have a strong link between those, we can be confident that we can train a model to detect that signal in a large corpus. We’re not looking for words at that point, but rather for the pattern.

We can also swap out some of the tokens in this structure with similar ones as long as they fit the pattern. In this way, we can start to build whole text structures not just one word at a time, but one chunk or as we call them “rhetorical move” at time.

10 of 21

Hedges and the Hedge-o-Matic

Greenhouse gas emissions from human activities may continue to affect Earth’s climate for decades and even centuries. Humans are likely adding carbon dioxide to the atmosphere at a rate far greater than it is removed by natural processes, creating a long-lived reservoir of the gas in the atmosphere and oceans that is driving the climate to a warmer and warmer state.

Greenhouse gas emissions from human activities continue to affect Earth’s climate for decades and even centuries. Humans are adding carbon dioxide to the atmosphere at a rate far greater than it is removed by natural processes, creating a long-lived reservoir of the gas in the atmosphere and oceans that is driving the climate to a warmer and warmer state.

Source: https://www.globalchange.gov/climate-change/whats-happening-why

Actual

Text

Hedge signals added

My colleague Ryan and I showed this was possible with an important rhetorical move in scientific writing: the propositional hedge.

A propositional “hedge” is an important - and maybe the most important - signal that scientists use to communicate the findings of their work. The function of this move is to match the strength of the claim with the strength of the available evidence and to do so with care and precision. Most of the time, science communication calls for a conservative or cautious approach. This is because science, as a form of reasoning, seeks to disprove things, show them to be not true, rather than definitely prove them.

But in cases where we want to communicate clearly, a hedge tends to make things less certain as well. This makes the use of hedges a very tricky, but very important thing to master. Perhaps you agree?

For us, hedges are a key way to detect scientific content. Finding hedges in the right concentration in a selection of text is a reliable signal that we are dealing with scientists talking to other scientists. So we set out to show that hedges were reliably detectable and something we could study in greater detail. We did that by developing a little gadget we call the Hedge-o-Matic

11 of 21

The Hedge-o-Matic reliably identifies propositional hedges

Source: https://www.globalchange.gov/climate-change/whats-happening-why

HoM: http://hedgeomatic.cal.msu.edu/hedgeomatic/

Actual Text

Hedge signals added

Let me mention a few interesting, if very geeky, things that were something of a surprise to folks in our disciplines and related ones that the HoM helps us to see. One is that hedge signals are not simple keyword IDs, though they can be strongly associated with particular kinds of words like modal verbs.

Hedge moves are NOT carried in the content words, alone, that are typically the focus for other kinds of content analysis techniques. In fact, hedge moves often depend - as this example shows - on what are commonly called “stopwords” - words that it in other kinds of analysis are thrown out as part of text-processing steps meant to boost the signal-to-noise ratio of content words.

Our work on the HoM beginning in 2015 showed we can reliably detect hedges in unseen sentences. We can also feed this information back to writers to help them better calibrate their claims when writing. We did all this with a comparably tiny but very high-quality (hand curated) training set of hedge sentences, taking advantage of similar textual dynamics and transformational techniques as we see in GPT-3.

12 of 21

Topics for Today

A little about Large Language Models (LLMs) and how they work…or

What the heck just happened to make GPT-3 so much better at drafting longer texts that resemble human-drafted ones? Spoiler: Transformers.

A little about writing (as humans do it and as robots do it)

Writing is intentional, goal-directed activity that sometimes results in a text. LLMs can simulate some parts of this activity better than others.

Four Proposals for Writing and Learning with AI in the Loop

How should we be thinking differently about the writing tasks we ask students to do and how we evaluate them?

13 of 21

Are (writing) Robots a Threat?

Maybe.

Writing activity is important as social behavior. When robots make texts, it simulates some of this behavior. And that might cause problems…e.g. Grant proposals.

But the bigger issue for me is…

Writing is good practice. Robots can do part of that practice now really easily. It may mean that humans will miss practice and miss some learning as a result.

Photo Credit: Toy Robot on a table by Flickr user Antti Nissinen.

Are these writing robots a threat? Well… maybe.

Because they were not really (or not only?) trying to solve any particular problem related to writing but rather just saw the connection between writing and parallel processing tasks for computing, the OpenAI folks have unleashed more questions than answers upon the world, including those in my own field.

But…we’ve seen this moment coming for a while, it is fair to say. And many of the fears we see echoed are not new in our scholarship. Some are quite old.

To put things succinctly, robots can pose problems. For one, writing activity is important in its own right. It has meaning as a kind of behavior. And so “outsourcing” it in some cases is just…not going to mean the same thing. Take a research grant proposal. If you knew I used an AI program to write the proposal, would you still believe I could do the project? …maybe, maybe not, right? Because the proposal is not really the end goal for either of us. It is a way for us to come to an agreement about which idea, which team, etc. has the right stuff to get good results. A better question might be: in what ways would AI writing tools be an acceptable aid for proposing?

Another problem AI writing tools pose is that they can be a way for students to skip practice. Writing is a useful and helpful way to practice things that lead to learning. And if a robot does the practice, it might learn. But the person likely won’t learn much.

14 of 21

You are Here… or “New Writing Process Just Dropped!”

So where do we stand today?

Well, for a growing number of folks and soon maybe a majority of us for the majority of the time we write something, the writing process may look like this. This is a change. And I do think it is an inflection point of sorts.

Now, historically, drafting was always out-sourceable for people in positions of power. Deans can ask Associate Deans to draft things for their signature, for instance (ask me how I know!). :)

But as technology tends to do, a broad group of people can now perhaps have a robot do a first draft or produce a sample and they can begin from the review step of the process. On net, this is not a bad thing. And it could be a very good thing, ultimately. But we have some caveats.

It’s important to remember that a robot isn’t inventing from thin air. They are reassembling texts from a database. It’s other people’s writing, and truly the statistically most repetitive parts of others’ writing, that shows up in those drafts most of the time.

And… we should always remember that good writing cycles are iterative, almost always. So one pass through this is almost never enough. This is going to be true whether a robot is involved or not. And the more nuance we need, the more cycles of review and revision we tend to go through. Review and revision are and will continue to be, I think, very important human skills in the writing process. Because if they are not, these activities won’t build trust in the same way. We see this already with form letters or spam emails that are delivered in bulk. You know, I know, we know…regardless of how correct the grammar is in the prose, those actions don’t mean much to anyone. And so they go unread, or auto-filtered out of our inboxes all together!

15 of 21

Writing is Intentional Human Action

Writing is behavior. That behavior sometimes results in a text. Sometimes those texts are shared and sometimes they are not. But…
People use writing to do things OTHER than make texts. Writing is usually a means, not the end.
The situations where making a text is the GOAL are the exception, not the rule. Like poetry.
In the words of writing studies researcher Anthony Pare - “We don’t write writing.”

16 of 21

Topics for Today

A little about Large Language Models (LLMs) and how they work…or

What the heck just happened to make GPT-3 so much better at drafting longer texts that resemble human-drafted ones? Spoiler: Transformers.

A little about writing (as humans do it and as robots do it)

Writing is intentional, goal-directed activity that sometimes results in a text. LLMs can simulate some parts of this activity better than others.

Four Proposals for Writing and Learning w/AI in the Loop

How should we be thinking differently about the writing tasks we ask students to do and how we evaluate them?

17 of 21

Writing and Learning - Four Proposals

Whenever we ask students to write, we should consider that as asking them to practice something. Writing is not a transparent window on thinking. It’s a social activity that has benefits for learning. But it takes practice to realize those benefits.
Students need more deliberate practice in other parts of the writing process - not just drafting. Criterion-referenced review and revision are especially important.
Show your work. That should be our new (old?) mantra. We need to see the practice, and students need feedback on it.
Where LLMS, or AI are part of our workflow, we should have a consent and disclose approach.

Consent should precede the use of LLMs in many writing situations, including teaching and learning, as well as academic publishing
Disclosure should be(come) a regular part of our practice if we use LLMs, for example in the method sections of a research article.

18 of 21

Consent and Disclosure

Where LLMs are part of our work, we should…

Make sure that we are transparent about when and how people are asked to contribute their work to systems that will reuse it for training purposes
Make sure that we understand when and how they were used to enhance the writing process and take not to be deceptive about those
Develop conventions for disclosing the use of LLMs as we do with other computer-enhanced tools, e.g. in methods sections, etc.
Develop ground rules for letting folks know what is and is not out of bounds for using LLMs in classroom (i.e. practice) situations - e.g. “I want you to do this drafting by hand…” etc.

Across the functions of a place like Syracuse or Michgian State, we need to get intentional about developing practices for consent and disclosure when we are working with robots.

We ought to take care to not use other peoples work without them knowing it may be part of an AI training set or subject to review in some fashion that ultimately amounts to reuse of their work without consent.

We should disclose when we are using AI tools and how. Perhaps in our methods sections of research articles? And in other ways, as appropriate for the situation and genres we are working in.

And as teachers, we should share some ground rules for when LLMs are out of bounds and when they are allowable. Like using a calculator in math class, for one example. When we mark them off limits for drafting, we should talk with students about what we are asking them to practice by and with writing and why.

19 of 21

When Robots Learn to Write, What Happens to Learning?

Bill Hart-Davidson, Ph.D.

Thank you!

20 of 21

References & Further Reading

Aristotle. (2022). Art of Rhetoric. J.H. Freese, trans. Loeb Classical Library. Harvard UP.

Erdős, P., & Rényi, A. (1961). On the strength of connectedness of a random graph. Acta Mathematica Hungarica, 12(1), 261-267.

Gunnarsson, B. L. (1997). The writing process from a sociolinguistic viewpoint. Written communication, 14(2), 139-188.

Hart-Davidson, W. (2003, October). Seeing the project: Mapping patterns of intra-team communication events. In Proceedings of the 21st annual international conference on Documentation (pp. 28-34).

Hart-Davidson, W., Spinuzzi, C., & Zachry, M. (2007, October). Capturing & visualizing knowledge work: Results & implications of a pilot study of proposal writing activity. In Proceedings of the 25th annual ACM international conference on Design of communication (pp. 113-119).

Hart-Davidson, W., & Omizo, R. (2017). Genre signals in textual topologies. In Topologies as techniques for a post-critical rhetoric (pp. 99-123). Palgrave Macmillan, Cham.

Kaufer, D., Geisler, C., Vlachos, P., & Ishizaki, S. (2006). Mining textual knowledge for writing education and research: The DocuScope project. Writing and digital media, 115-129.

Omizo, R., & Hart-Davidson, W. (2016). Hedge-O-Matic. enculturation, 7.Omizo, R., & Hart-Davidson, W. (2016). Finding genre signals in academic writing. Journal of Writing Research, 7(3), 485-509.

21 of 21

References, cont.

Omizo, R. M., & Hart-Davidson, W. (2017, August). Digging text viz: an archaeological review of ACM digital library text visualizations publications (1991--2003). In Proceedings of the 35th ACM International Conference on the Design of Communication (pp. 1-13).

Omizo, R., Hart-Davidson, W., Nguyen, M. T., Clark, I., McDuffie, K., & Ridolfo, J. (2016). You Can Read the Comments Section Again: The Faciloscope App and Automated Rhetorical Analysis. DH Commons Journal.

Omizo, R. M. (2019). Participation and the Problem of Measurement. In The rhetoric of participation: interrogating commonplaces in and beyond the classroom. Computers and Composition Digital Press/Utah State University Press Logan, UT.

Omizo, R. M. (2020). Machining topoi: Tracking premising in online discussion forums with automated rhetorical move analysis. Computers and Composition, 57, 102578.

Omizo, R. (2022). Reprogramming the Faciloscope. Reprogrammable Rhetoric: Critical Making Theories and Methods in Rhetoric and Composition, 108.

Pare, A. Pare, A. (2009). Writing Matters: Back to the Future with Rhetoric. Education Canada, 49(4), 4-8.

Plato. (1952). Plato's Phaedrus. Cambridge :University Press.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.