1 of 28

Thinking through and writing about research ethics beyond “broader impact”

FAccT 2021: Translation tutorial

Kate Sim, Amelia Hassoun, Andrew Brown

University of Oxford

This work is licensed under a Creative Commons

Attribution-NonCommercial 4.0 International License.

Hi everyone, welcome to our tutorial: Thinking through and writing about research ethics beyond broader impact. We want to begin by acknowledging the difficulties of organizing a virtual conference and want to thank the program committee for doing such a wonderful job. We also want to acknowledge how exhausting it can be to stare at a screen all day, so thank you all for making the time to join us. And we encourage you to pace yourself as you see fit.

If you need subtitles turned on, turn it on on the bottom of the screen. They are not available during the breakout room.

With that, let’s get started. [each introduce]. So, what do a sociologist, an anthropologist, and a computer vision researcher have to say about research ethics?

This tutorial came out of shared observations we made in our respective fields about research ethics--mainly how we feel constrained by the way “AI ethics” “research ethics” and “broader impact” are talked about.

This tutorial joins the ongoing conversations in the FAccT and wider sociotechnical communities about what it means to “do” research ethics for AI/ML projects.

We invite AI/ML researchers—especially, masters/doctoral students, junior, and early career researchers--to think more expansively, holistically, and critically about research ethics as it applies to their projects. [next]

2 of 28

Workshop agenda

Common misconceptions

5m

Lifecourse activity

Breakout groups, followed by *5m break*

25m

Group discussion

Key takeaways, surprises, and challenges

20m

Next steps

10m

Imaginative exercise

10m

What are your research best/worst case scenarios?

A lifecourse approach to research ethics

15m

Setting a framework; followed by *5m break*

3 of 28

Common misconceptions:

4 of 28

ethics = broader impact

5 of 28

ethics = formula

Another common misconception is that research ethics is formulaic. There is a single formula which will guarantee an ethics-proof research design You are guaranteed to be ethical, you never think about it again, and you can move on.

AI/ML researchers are not alone in voicing this sentiment; researchers in the social sciences and humanities share this feeling, especially against institutionalized ethics committees (i.e., IRB).

There is no one way to “do” research ethics. It’s not a formula that will grant you an ethics-proof research design. It also is not the goal of ethics review to stamp a project as “ethical.”

Instead, research ethics is a process. We invite you to think about research ethics as a process with signposts are intended to help you make appropriate and relevant judgement calls throughout the course of your project so that you and your project can be accountable to the choices you make.

6 of 28

ethics = censorship

There also exists the concern that ethics procedures will censor academic freedom.

This is simply not true. In fact, we’d argue that having an ethics process in peer review/conference process enriches the rigor and integrity of academic research. For example, at NeurIPS last year, the committee made it clear that submissions would not rejected solely on the basis of broader impact statements (see Abuhamad & Rheault 2020). Thinking back to all the controversial projects in the past, ethical concerns exist in tandem with serious methodological and analytical limitations and flaws (examples)

So what the equating of ethics with censorship tells us is that ethics is considered to be separate from methods. What we hope to discuss with you today is that ethics is inseparable from methods.

7 of 28

ethics = too subjective

8 of 28

no humans = no ethics

9 of 28

Exercise: Think about your current project or a project you’d like to do in the future.

10 of 28

What would it look like if that project went perfectly?

What steps would go right? What would the results be? Who would benefit?

What’s the best case scenario?

11 of 28

Now think about what it would look like if your worst nightmare came to pass.

What would go wrong? What would happen as a result? Who would be negatively affected (including you)?

What’s the worst case scenario?

12 of 28

Imagine each project as having a life course.

It is up to the researcher to think about the conditions, assumptions, projections, and aspirations that shape their research design.

In Sociology, the life course approach encourages us to think about how individual’s choices and actions interact dynamically with their immediate surroundings and larger social structures. Applied to research ethics, taking a lifecourse approach to our research allows us to take four factors into consideration:

(1) the choices a researcher makes;

(2) how those choices are shaped by their immediate environment and social structures. Think about your department, funding bodies, disciplinary culture and norms;

(3) how those environments and systems are part of a larger historical and social context. The pandemic, obviously, comes to mind. But also, there has been a drive in recent years from multiple communities to really think about what it means to reflect on and reimagine the role of technology in our lives.

And finally (4) time: Our research builds on the works of others and will have a life beyond completion. And in ways we don’t expect or have control over.

Taking a life course approach to our research thus allows us to consider the individual, the group, the social, and the temporal dynamics that shape our projects. If research has a life course, then it is up to the researcher to do their due diligence in ensuring that the lifecourse of their project is a meaningful one.

13 of 28

14 of 28

Life Course: Before the Project

Informed consent
Opting out
Identifiability
Representativeness
Positionality
Data quality

So we offer a lifecourse framework for thinking about research ethics. It takes into consideration the life course of the research project, but also takes seriously the lifecourse of the people involved, including the researcher, the people they work with, and the people who participate.

We’ve broken down research lifecourse into three phases: the before, during, and after. In each phase, we’ll highlight a few principles. Again, everything will be written out in the worksheet you will see later.

In the BEFORE phase:

This is where the researcher faces some of the more important ideas about research ethics, and has to make a set of decisions that will shape how the research unfolds.

Informed consent is a concept traditioanlly used in human-subject research but it still offers a helpful framework for data-driven research. So the question is less about can individuals consent or opt out, but rather about the conditions of consent and opting out. For example, if you are using a dataset of profile images from, let’s say, a dating app, think about how that use conflicts with the app users’ expectations of privacy.
Another big concern is around Identifiability of each data point and the representativeness of the dataset. There is a tendency to just de-identify the entire dataset, but there is a trade-off between deidentification and representativeness.

If you’re using a dataset about Tik Tok users, for example, you might remove all personally identifiable information, including age. But this means that your dataset now appears homogenized and can reproduce assumptions about Tik Tok users as primarily young people.

Representativeness of the dataset refers to the dataset as a whole, but also to the social groups that are represented in your dataset.

For example, in a dataset about professions, you may have gender parity. But, your female data points might have traditionally feminized and devalued professions while male data points have masculinized professions. In short, “teachers” for women; “professors” for men.

So, the choices we make as researchers at this phase of the research have a real repercussion on how we are representing the real world, and how those representations might have a real impact on the people involved. [next]

15 of 28

Life Course: During the Project

Ongoing review
Exposure to risk
Exploitation
Access to study information
Data storage & transfer

16 of 28

Life Course: After the Project

Accessibility of results
Usability
Immediate impacts
Future applications
Possible misuses
Limitations

17 of 28

The tutorial will continue after a 5 minute break

18 of 28

Research Life Course Activity

�- We will split you into breakout rooms and give each group a machine learning case study to practice applying the life-course approach.��- Once you enter the breakout room, we will give you a link to a life-course worksheet with details on the case study and questions to answer with your group. Fill out the worksheet together and feel free to type right on the sheet itself! It’s all yours. ��- After your breakout group discussion, we will all return to the big group session and share some of your answers to the questions, raise any difficulties/challenges you had, and discuss any interesting findings. ��

19 of 28

ML Case Studies

Face Recognition

Action Recognition

Simulated Environments

Salsa Dancing

20 of 28

Face Recognition

Goal - Recognise if any two face images correspond to the same person�
Method - train a machine learning algorithm on a dataset of human faces�
Potential Uses - Testing new machine learning algorithms, Recognising people in photo-albums/TV-shows , Biometric Identification, Surveillance�
More info - The dataset contains many face images of over 500,000 well-known celebrities, downloaded from search-engines. The names are anonymised. Statistics provided with an example dataset are shown below [1]

[1] MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition. Guo Y., Zhang L., Hu X., He X., Gao J. ECCV ‘16

21 of 28

Action Recognition

Goal - Recognise specific human actions in videos�
Method - train a machine learning algorithm on a dataset of videos of specific human actions�
Potential Uses - Identifying and classifying human behaviour for recommender systems; Automatically detecting suspicious activities for law enforcement�
More info - All videos are downloaded from YouTube e.g. researcher queried “basketball dribble” to YouTube and downloaded the first few videos. The dataset does not contain the names of the people in the videos. The dataset contains over 200,000 videos of 4,000 actions. Example videos for some action classes are shown on the right.

Salsa Dancing

Cleaning an oven

Basketball Dribble

22 of 28

Simulated Environment

Goal - Train a machine learning algorithm to drive a car�
Method - Train the algorithm in a simulated environment such as the game “Grand Theft Auto”. Give the agent the task of driving without crashing and to obey the highway code in the game environment.�
Potential Uses - Self-driving vehicles. Have the agent use its knowledge learnt in the simulated environment in the real world! Engineers could expect the agent to be able to drive in environments ‘similar’ to those in the simulated environment�
More info - The data is generated from what the agent does in the human-made simulated environment. This is an example of a powerful machine learning technique called ‘Reinforcement Learning’ (think AlphaGo [3]). This research would involve the development and advancement of these techniques.

[1] https://deepdrive.voyage.auto/ [2]https://www.youtube.com/channel/UCfzlCWGWYyIQ0aLC5w48gBQ�[3] Silver D., Schrittwieser J., Simonyan K., et al. Mastering the game of Go without human knowledge. Nature 2017

Images [1] [2]

23 of 28

Life course activity: instructions

Split into breakout rooms. Each breakout room gets a worksheet containing (1) instructions; (2) case study description; and (3) lifecourse checklist.

Read through the instructions, the case study description, and skim over the checklist. Use the checklist as a guide to discuss ethical concerns in relation to the case study.��In each group, assign roles: A facilitator to keep the discussion going. A timekeeper to keep track of time. A rapporteur to relay your group’s discussion to everyone. In particular: � a) The question that you found most difficult to answer?� b) The question that you found most novel -- that you hadn’t encountered or thought of before� c) The key takeaway or talking point from the experience; the most interesting answer you had

Reconvene as a big group to discuss key takeaways and observations. ��During the breakout group, if you need to ask any questions, “ask-for-help” by pressing the question mark on the bottom of your Zoom screen and we will come join the breakout room to answer!��The breakout room will last for 25 minutes, this is followed by a 5 minute break, then the group discussion. We will message everyone with timings in Zoom.

24 of 28

Interactive worksheet:

https://bit.ly/38ah944

25 of 28

Discussion

26 of 28

What conventions stop us from easily being able to write about and share the ethical implications of our research?

What could we do to change this?

27 of 28

Some ideas we have:

For everyone:

circulating and sharing standardised, comprehensive, accessible checklists/guides that describe ethics review practice in these broader terms.
streamlining our language so that everyone can share a common vocabulary around ethics review practice, therefore able to talk to each other generatively.
being more honest about our project limitations and benefits.
building these conversations and trainings into departmental practice.

For reviewers/teachers/Sr. researchers:

expanding the “broader impacts” section in CfPs, grants, research statements to reflect this broader ethics review.
adding this kind of section to all kinds of project proposals (e.g. PhD applications, grants and funding applications).
training reviewers in this broader ethics review practice.
teaching students and researchers how to do ethics review.

28 of 28

Kate Sim

kate.sim@oii.ox.ac.uk

Amelia Hassoun

amelia.hassoun@oii.ox.ac.uk

Andrew Brown

abrown@robots.ox.ac.uk