Pre-read: Can advanced AI help predict the future?

Limits to prediction (Spring 2024)

Arvind Narayanan

The predictive models we’ve encountered in this course so far are simple ones such as logistic regression and random forests; the only exception where deep learning was helpful was for weather prediction — weather being a physical rather than social system.

Do advanced AI techniques such as deep learning, transformers, and foundation models have any role in predicting social outcomes? If so, in which settings are they useful? If not, why not? That is what we will explore for this class.

The first two readings are about the technical basics. The foundation models paper was written in 2021 — before ChatGPT but after most of the core innovations that made ChatGPT possible. The paper didn’t invent the concept but rather surveys the ongoing paradigm shift happening in many areas of AI. It is a mammoth paper; you need to read only the introduction section. The next reading is a blog post that explains large language models in particular in an intuitive way.

The next item on the menu is the main one. In Using sequences of life-events to predict human lives, the authors build a deep learning (transformer) model for life trajectories, possibly the first time this has been done successfully for these types of prediction tasks. It is no coincidence that the work was enabled by a registry dataset that is massively larger than typical survey datasets. One useful heuristic is that if there is a lot of structure in the data, and enough samples to learn that structure, then deep learning can be helpful. (Note that you can find a link to the arXiv preprint at the top of the page. Princeton doesn’t seem to provide access to the paywalled version.)

Some things to keep in mind as you read the paper: what are the similarities between strings of text and life trajectories that motivated the application of transformers? What are the differences? Did they create a foundation model? Why do they pick the performance metric that they do? Do you have an intuitive feel for how good the reported performance is? The authors summarize their results by saying “accurate individual predictions are indeed possible”, and explicitly contrast this with the message of the Fragile Families paper (reference 19 in their paper). Do you believe this claim, considering that they don’t actually attempt to compare their findings with the Fragile Families paper?

The final reading is about a completely different route by which advanced AI might improve prediction. Frankly, I don’t know what to make of this paper, which should make for an interesting class discussion.