1 of 9

Does Generative AI know statistics

PyCon 2025 – Pittsburgh

Louis Luangkesorn

2 of 9

Question

  • Does Generative AI know how to conduct an analysis of data
  • Try this!
  • Questions:
    • Does Generative AI explore
    • Does it evaluate what it finds out
    • Does it get the code right?
    • Can it explain what it learns

3 of 9

RAND Health Insurance Expriment

  • Data is included in statsmodels package
  • Read in the description and the metadata to Generative AI
  • Ask how to analyze this

4 of 9

How to analyze

Here is a data set of an health insurance experiment with variable names

How should I analyze the relationship between outpatient visits and the other variables

​

5 of 9

Linear regression interpretation prompt

Interpret the results from the linear regression summary based on the health insurance experiment

6 of 9

Poisson regression implementation prompt

how should I set up the analysis using a Poisson regression in python with statsmodels

7 of 9

Poisson regression interpretation prompt

Interpret the results from the Poisson regression summary based on the health insurance experiment

​

8 of 9

Negative binomial interpretation prompt

Interpret the results from the negative binomial regression summary based on the health insurance experiment

​

9 of 9

Study communications prompts

  • Present a summary of the results to people who work in community public health who are not statisticians
  • Present a summary of the results to people who are community organizers but are not familiar with public health, insurance, or statistics
  • Present a summary of the results of the negative binomial regression analysis of the health insurance experiment to an audience of public health analysts
  • Present a summary of the results of the negative binomial regression analysis of the health insurance experiment to an audience of community leaders

​