1 of 3

Text data – ubiquitous, dense, labour intensive

ChatGPT uses

    • Creating - new texts, based on preceding texts
      • New episodes in storylines about an alternative futures (parevo.org)
    • Summarising - existing texts
      • Headlines, condensed texts
    • Comparing – multiple texts
      • Most significant difference, most optimistic, most likely, most realistic
    • Analysing -individual texts
      • Actor and relationship extraction
      • Causal relationship extraction – Steve Powell’s work
      • Sentiment analysis – adjective and adverb extraction

Meta-use: beta.pickaxeproject.com - for prompt development

Rick Davies, MERL Tech meeting on ChatGPT et al, March 2023

2 of 3

Evaluating ChatGPT output

  • Reliability: Consistency of output, with each new use of same prompt
  • Internal validity: Consistency of content generated within one prompt use
  • External validity: Consistency with other known facts (in prompts or elsewhere)
  • Use choices:
    • Exploration – regenerate response to same prompt
    • Exploitation - revise existing prompt, continue dialogue
  • Parameter settings
    • Which LLM
    • Temperature
    • Presence and frequency penalties

​

Rick Davies, MERL Tech meeting on ChatGPT et al, March 2023

3 of 3

Using ChatGPT as a tool for the analysis of text data

https://mande.co.uk/

​

Rick Davies, MERL Tech meeting on ChatGPT et al, March 2023