🦜Stochastic Parrots Day Reading List🦜

On March 17, 2023, Stochastic Parrots Day organized by T Gebru, M Mitchell, and E Bender and hosted by The Distributed AI Research Institute (DAIR) was held online commemorating the 2nd anniversary of the paper’s publication. Below are the readings which popped up in the Twitch chat in order of appearance.

1. On the Dangers of Stochastic Parrots *the paper of the day - happy birthday paper*: https://dl.acm.org/doi/10.1145/3442188.3445922

& https://magazine.scienceforthepeople.org/vol24-2-dont-be-evil/stochastic-parrots/

2. Algorithms of Oppression: http://algorithmsofoppression.com/
3. Climbing Towards NLU: https://aclanthology.org/2020.acl-main.463.pdf 
4. Bounty Everything: Hackers and the Making of the Global Bug Marketplace: https://datasociety.net/library/bounty-everything-hackers-and-the-making-of-the-global-bug-marketplace/
5. Enchanted Determinism: Power without Responsibility in Artificial Intelligence: https://estsjournal.org/index.php/ests/article/view/277
6. https://betterwithout.ai/
7. Are language models Scary? https://betterwithout.ai/scary-language-models
8. Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning: https://arxiv.org/abs/1912.10389
9. Algorithmic injustice: a relational ethics approach:https://www.sciencedirect.com/science/article/pii/S2666389921000155
10. Situating Search: https://dl.acm.org/doi/10.1145/3498366.3505816
11. Watermarking LLMs: https://arxiv.org/pdf/2301.10226.pdf
12. Article from OpenAI founder on not sharing data: https://www.theverge.com/202https://vetspecialists.co.uk/fact-sheets-post/myositis-fact-sheet/3/3/15/23640180/openai-gpt-4-launch-closed-research-ilya-sutskever-interview
13. Vampyroteuthis Infernalis A Treatise, with a Report by the Institut Scientifique de Recherche Paranaturaliste: https://www.upress.umn.edu/book-division/books/vampyroteuthis-infernalis
14. Deb Raji's article on ground truth/lies: https://www.technologyreview.com/2020/12/10/1013617/racism-data-science-artificial-intelligence-ai-opinion/
15. A Watermark for Large Language Models: https://huggingface.co/spaces/tomg-group-umd/lm-watermarking 🙂
16. Artificial Consciousness is Impossible: https://.com/artificial-consciousness-is-impossible-c1b2ab0bdc46
17. Data Governance in the Age of Large-Scale Data-Drive Language Technology - https://arxiv.org/abs/2206.03216
18. Dr. Bender has also written on some of these points, specifically addressing AI hype and claims that LLMs "have intelligence", in a really accessible way here: https://medium.com/@emilymenonbender/on-nyt-magazine-on-ai-resist-the-urge-to-be-impressed-3d92fd9a0edd
19. Weapons of Math Destruction: https://blogs.scientificamerican.com/roots-of-unity/review-weapons-of-math-destruction/
20. Time article about how workers for identifying toxic language for OpenAI were underpaid: https://time.com/6247678/openai-chatgpt-kenya-workers/
21. Microsoft lays off AI ethics team (what could go wrong?) https://techcrunch.com/2023/03/13/microsoft-lays-off-an-ethical-ai-team-as-it-doubles-down-on-openai/
22. Birhane and Raji critique ChatGPT, Galactica, and the Progress Trap https://www.wired.com/story/large-language-models-critique/
23. The Black Box Society The Secret Algorithms That Control Money and Information: https://www.hup.harvard.edu/catalog.php?isbn=9780674970847
24. Reading on invisibilized labor for automation & content moderation: Sarah Roberts - Behind the Screen; Gray and Suri - Ghost Work: https://yalebooks.yale.edu/book/9780300261479/behind-the-screen/
25. Atlas of AI: https://yalebooks.yale.edu/book/9780300264630/atlas-of-ai/; visual aid: https://anatomyof.ai/img/ai-anatomy-map.pdf
26. Social Turkers (art): https://lauren-mccarthy.com/social-turkers
27. Design Justice: https://mitpress.mit.edu/9780262043458/design-justice/
28. Film about content moderators: https://en.wikipedia.org/wiki/The_Cleaners_(2018_film)
29. Speculative short film about a world in which people “plug in” to work: https://en.wikipedia.org/wiki/Sleep_Dealer
30. Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision: https://dl.acm.org/doi/abs/10.1145/3415186 
31. Milagros Miceli on the power over data annotators in AI work: https://dl.acm.org/doi/abs/10.1145/3415186
32. Content moderation and data-labeling work: The Gig is Up by Canadian filmmaker Shannon Walsh: https://thegigisup.ca/
33. Ghost work - the invisible work force that powers the web: https://ghostwork.info/
34. A Future for Intersectional Black Feminist Technology Studies: https://sfonline.barnard.edu/safiya-umoja-noble-a-future-for-intersectional-black-feminist-technology-studies/
35. Bully Boss: "If you don't complete this (unethical) request, I'll fire you and hire someone else who will." The importance of why tech workers should unionize, especially in the age of A.I. bias. https://adactio.com/articles/18676
36. On the intersection of race and google search: https://www.invisibleculturejournal.com/pub/google-search-hypervisibility/release/1
37. https://research.utexas.edu/wp-content/uploads/sites/3/2015/10/mechanical_turk.pdf
38. Data workers: Please let us know if there's anything the LaborTech Research Network can do to support you (labortechresearchnetwork.org/).
39. Thea Ricofrancis on the cost of lithium batteries -- https://logicmag.io/nature/what-green-costs/
40. Noble: Towards a Critical Black Digital Humanities: https://www.jstor.org/stable/10.5749/j.ctvg251hk.5
41. TRK calculator by Caroline Sinders: https://carolinesinders.com/trk/
42. On the hidden costs of low wage workers: https://en.wikipedia.org/wiki/Nickel_and_Dimed
43. On food poverty in Britain: https://policy-practice.oxfam.org/resources/below-the-breadline-the-relentless-rise-of-food-poverty-in-britain-317730/
44. How Europe Underdeveloped Africa is a classic -- https://www.versobooks.com/books/2785-how-europe-underdeveloped-africa
45. Climate Leviathan has a lot of great insight about how academics became activists inside the climate fight, as well as a critique of the economic/authoritarian aspects of the current responses to climate: https://www.versobooks.com/books/3138-climate-leviathan
46. Artisanal Intelligence: https://polclarissou.com/boudoir/posts/2023-02-03-Artisanal-Intelligence.html
47. Addressing the tropes of AI: Betterimagesofai.org
48. Map of AI Myths and responses to these: https://www.aimyths.org/
49. Possibilities and risks of AI re: automated structuring of museum collection data: https://trainingthearchive.ludwigforum.de/en/interviews-en/ 
50. ChatGPT Is Dumber Than You Think: https://www.theatlantic.com/technology/archive/2022/12/chatgpt-openai-artificial-intelligence-writing-ethics/672386/
51. The Hierarchy of Knowledge in Machine Learning and Related Fields and Its Consequences: https://www.youtube.com/watch?v=DccuM7kGWss
52. Can we improve large language models if we train them according to values?: https://cdn.openai.com/palms.pdf
53. CFP for TextGenEd Teaching with Text Generation Technologies: https://wac.colostate.edu/repository/collections/cfp-textgened/ 
54. Coding Literacy How Computer Programming Is Changing Writing: https://mitpress.mit.edu/9780262036245/coding-literacy/
55. https://www.deepl.com/translator vs google translate
56. Real estate agents say they can’t imagine working without ChatGPT now: https://www.cnn.com/2023/01/28/tech/chatgpt-real-estate/index.html
57. Classifier by OpenAI to distinguish between AI-written text vs. human-written text: https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text
58. What can luddites teach us about the relationship between work and technology today? https://www.versobooks.com/blogs/5065-breaking-things-at-work-a-verso-roundtable
59. The first book written with GPT-4: https://www.linkedin.com/feed/update/urn:li:ugcPost:7041773192712998912/?commentUrn=urn%3Ali%3Acomment%3A%28ugcPost%3A7041773192712998912%2C7041868475782352896%29&dashCommentUrn=urn%3Ali%3Afsd_comment%3A%28704
60. Design From the Margins: https://www.belfercenter.org/publication/design-margins
61. You’re Not Going to Like How Colleges Respond to ChatGPT: https://slate.com/technology/2023/02/chat-gpt-cheating-college-ai-detection.html
62. Resisting AI An Anti-fascist Approach to Artificial Intelligence: https://bristoluniversitypress.co.uk/resisting-ai
63. Towards the Sociogenic Principle: Fanon, The Puzzle of Conscious Experience, of “Identity” and What it’s Like to be “Black”: http://www.coribe.org/pdf/wynter_socio.pdf
64. Policy Elements of Decision-making Algorithms: https://internetinitiative.ieee.org/newsletter/december-2018/policy-elements-of-decision-making-algorithms
65. https://soletlab.asu.edu/coh-metrix/ - "Coh-Metrix is a program that uses natural language processing to analyze discourse."
66. Mimi Onuoha - Library of Missing Datasets (art) https://mimionuoha.com/the-library-of-missing-datasets
67. More Than “If Time Allows”: The Role of Ethics in AI Education by C Fiesler: https://dl.acm.org/doi/10.1145/3375627.3375868
68. Disruptive Fixation: School Reform and the Pitfalls of Techno-Idealism, by Christo Sims: https://press.princeton.edu/books/hardcover/9780691163987/disruptive-fixation
69. Harnessing GPT-4 so that all students benefit. A nonprofit approach for equal access: https://blog.khanacademy.org/harnessing-ai-so-that-all-students-benefit-a-nonprofit-approach-for-equal-access/
70. The AI Crackpot Index: https://www.madeofrobots.com/files/TheAICrackpotIndex.html
71. A Human in a Machine World https://medium.com/@kaolvera/a-human-in-a-machine-world-e8f2166507f
72. ChatGPT Is a Bullshit Generator Waging Class War: https://www.vice.com/en/article/akex34/chatgpt-is-a-bullshit-generator-waging-class-war
73. Journalists recommended by MMitchell: Melissa Heikkilä / MIT Tech Review - James Vincent / Verge - Khari Johnson / WIRED - Karen Hao
74. A good overall critique of ChatGPT, blog post: https://www.danmcquillan.org/chatgpt.html
75. Ethan Zuckerberg writing for Prospect about ChatGPT and the problem of bullshit: https://www.prospectmagazine.co.uk/science-and-technology/tech-has-an-innate-problem-with-bullshitters-but-we-dont-need-to-let-them-win
76. Research finds 60% of UK media coverage about Artificial Intelligence is industry-led: https://www.youtube.com/watch?v=t1TEXcrhe1Q
77. All-knowing machines are a fantasy: https://iai.tv/articles/all-knowing-machines-are-a-fantasy-auid-2334
78. https://rethinkmedia.org/
79. The Centre for Ethics at the University of Toronto (multiple playlists on AI and ethics): https://www.youtube.com/@CentreforEthics/ 
80. Machine Habitus: Toward a Sociology of Algorithms: (Google Books link)
81. https://whichlight.notion.site/AI-Projects-8a3316193f564fe3840c970639bec005
82. The Artificial Intelligence Act: https://artificialintelligenceact.eu/
83. https://www.vice.com/en/article/ak3w5a/openais-gpt-4-is-closed-source-and-shrouded-in-secrecy
84. UNESCO Ethics of Artificial Intelligence: https://www.unesco.org/en/artificial-intelligence/recommendation-ethics
85. How truthful can LLMs be: a theoretical perspective with a request for help from experts on Theoretical CS: https://www.lesswrong.com/posts/E6jHtLoLirckT7Ct4
86. Using speculative fiction to examine refugee rights in Denmark (fake Job Center app): https://dl.acm.org/doi/abs/10.1145/3461778.3462003
87. AI/ML Media Advocacy Summit Keynote: Steven Zapata: https://www.youtube.com/watch?v=puPJUbNiEKg
88. The viral AI avatar app Lensa undressed me—without my consent:  https://www.technologyreview.com/2022/12/12/1064751/the-viral-ai-avatar-app-lensa-undressed-me-without-my-consent/
89. Gender Shades: https://www.media.mit.edu/projects/gender-shades/overview/
90. GPT-4 System Card: https://cdn.openai.com/papers/gpt-4-system-card.pdf
91. Copyright Registration Guidance: Works Containing Material Generated by Artificial Intelligence: https://www.federalregister.gov/documents/2023/03/16/2023-05321/copyright-registration-guidance-works-containing-material-generated-by-artificial-intelligence
92. Race After Technology: https://www.ruhabenjamin.com/race-after-technology
93. Ursula K. Le Guin -- these technologies are not inevitable: http://www.ursulakleguinarchive.com/Note-Technology.html
94. Reconstructing Training Data from Trained Neural Networks:  https://arxiv.org/abs/2206.07758
95. Extracting Training Data from Large Language Models: https://arxiv.org/abs/2012.07805
96. Controversial technology Pushback against AI policing in Europe heats up over https://www.globaltimes.cn/page/202110/1237232.shtml
97. This Algorithm Could Ruin Your Life: https://www.wired.com/story/welfare-algorithms-discrimination/
98. AI & Equality < Human Rights Toolbox >: https://aiequalitytoolbox.com/
99. Why AI Models are not inspired like humans: https://www.kortizblog.com/blog/why-ai-models-are-not-inspired-like-humans
100. A Relational Theory of Data Governance: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3727562
101. AI Incident Database: https://incidentdatabase.ai/
102. AI Incidents Database: https://partnershiponai.org/workstream/ai-incidents-database/
104. Awful AI: https://github.com/daviddao/awful-ai
105. Glaze for identifying AI artwork: https://glaze.cs.uchicago.edu/
106. Against Predictive Optimization: https://predictive-optimization.cs.princeton.edu/?utm_source=pocket_saves
107. Computer vision recognition that suggests what something is not: https://ridiculous.software/probably_not/
108. AI Data Laundering: How Academic and Nonprofit Researchers Shield Tech Companies from Accountability: https://waxy.org/2022/09/ai-data-laundering-how-academic-and-nonprofit-researchers-shield-tech-companies-from-accountability/
109. Replika users fell in love with their AI chatbot companions. Then they lost them https://www.abc.net.au/news/science/2023-03-01/replika-users-fell-in-love-with-their-ai-chatbot-companion/102028196
110. Against Predictive Optimization: On the Legitimacy of Decision-Making Algorithms that Optimize Predictive Accuracy: https://predictive-optimization.cs.princeton.edu/
111. Model Cards: https://arxiv.org/abs/1810.03993
112. Data Sheets for Data Sets: https://arxiv.org/abs/1803.09010
113. https://incidentdatabase.ai/apps/incidents/ 
114. Ethical Guidelines of the German Informatics Society: https://gi.de/ethicalguidelines
115. https://en.wikipedia.org/wiki/Computers_are_social_actors
116. Snap Judgment: https://chriscombs.net/2022/07/09/snap-judgment/
117. Global Indigenous Data Alliance: https://www.gida-global.org/
118. Equality labs: https://www.equalitylabs.org/
119. AI researcher on the drone debate "For me, it would be terrible if my work contributed to the death of people": https://www.stern.de/amp/digital/technik/-for-me--it-would-be-terrible-if-my-work-contributed-to-the-death-of-people--9548216.html 
120. The Tech Worker Handbook: https://techworkerhandbook.org/
121. Examining Responsibility and Deliberation in AI Impact Statements and Ethics Reviews: https://dl.acm.org/doi/abs/10.1145/3514094.3534155
122. Tech Workers Coalition: https://techworkerscoalition.org/
123. What Choice Do I Have? https://weallcount.com/2022/06/20/what-choice-do-i-have/
124. Indigenous Protocol and Artificial Intelligence Position Paper: https://spectrum.library.concordia.ca/id/eprint/986506/
125. Indigenous Data Sovereignty and Policy: https://www.taylorfrancis.com/books/oa-edit/10.4324/9780429273957/indigenous-data-sovereignty-policy-maggie-walter-tahu-kukutai-stephanie-russo-carroll-desi-rodriguez-lonebear
126. A Survey on Bias and Fairness in Machine Learning (gathering 23 sources of “bias”): https://arxiv.org/pdf/1908.09635.pdf 
127. Understanding data: Praxis and Politics https://datapraxis.net/
128. Artificial Unintelligence: https://mitpress.mit.edu/9780262537018/artificial-unintelligence
129. Artificial Intelligence: A Guide for Thinking Humans https://melaniemitchell.me/aibook/
130. Stuff of Bits https://mitpress.mit.edu/9780262546522/the-stuff-of-bits/
131. Automating Equality https://us.macmillan.com/books/9781250074317/automatinginequality
132. Data Feminism: https://mitpress.mit.edu/9780262547185/data-feminism/
133. Cloud Ethics: https://www.dukeupress.edu/cloud-ethics
134. Constructing Certainty in Machine Learning: On the performativity of testing and its hold on the future - https://osf.io/zekqv/
135. Nodes of Certainty for AI Engineers: https://www.tandfonline.com/doi/abs/10.1080/1369118X.2021.2014547?journalCode=rics20
136. Steven Wolfram's approachable explanation https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/
137. For those who asked, my website is mirabellejones.com - I’m an artist and PhD Fellow at UCPH in CS - Relevant projects: Artificial Intimacy (chatbots made using GPT-3 fine-tuned on social media data - we have a workshop coming up! Please email me if interested.), Embodying the Algortihm (performance artists use GPT-3 instructions for performances - CHI ‘23 paper on the way with C. Neumayer and I. Shklovski), It’s Time We Talked (can we use deep fake to explore alternate timelines?) Zoom Reads You (what would it be like for a LLM to narrate your Zoom presence?)