1 of 10

INVESTIGATING AI:

ACCOUNTABILITY, ETHICS AND HUMAN IMPACT

Garance Burke in conversation with Djordje Padejski

2 of 10

Once limited to policing the nation’s boundaries, the Border Patrol has built a surveillance system stretching into the country’s interior that can monitor ordinary Americans’ daily actions and connections for anomalies instead of simply targeting wanted suspects. Started about a decade ago to fight illegal border-related activities and the trafficking of both drugs and people, it has expanded over the past five years.

my first question here is how much did we know about? this type of work before? you're reporting?

Many people have been interested in the ways in which border patrol has been doing its work. Particularly under this administration we've seen more partnerships with local law enforcement, and what we were really focused on is this program that uses a very specific technology that can read your license plate and then uses that information and other data to try to predict who on the roads is suspicious. And so we were very focused particularly on this kind of technology. We found is that the Border Patrol's use of this system has grown over the last 5 years.

it started about a decade ago and has really expanded in recent years so, this is a network of cameras that can scan license plates, They figure out the numbers and letters and then that is input into a system that in turn has an algorithm that flags who is deemed to be suspicious according to the Border Patrol.

Now the border patrol would not share with us how this algorithm works. um, but we do know that it relies on information about where drivers are taking specific. routes what routes they travel. These license plate readers are along both the Southern and Northern border the we also found that this technology stretches farther into the interior than we had anticipated. Even scanning folks’ license plates who are traveling near Chicago, Gary, Indiana. Michigan in addition to you know, Arizona, Texas

3 of 10

4 of 10

5 of 10

Another series that my colleague Sally Ho and I wrote about looked at AI-powered tools that some city and county governments are using to help predict whether kids are sent to foster care (similar algorithms also are being applied to families in the UK, and South Korea).Today, child welfare agencies in at least 26 states and Washington, D.C., have considered using algorithmic tools, and jurisdictions in at least 11 have deployed them, according to the American Civil Liberties Union.

But little was known about how technology was used to make crucial decisions about children and families.

We wanted to understand the algorithm’s outcomes in Allegheny County, Pennsylvania, where we had heard some concerns it could harden racial biases into the child welfare system.

Our first story looked at a Carnegie Mellon University team’s statistical analysis of the county’s own data which found that the algorithm in its first years of operation showed a pattern of flagging a disproportionate number of Black children for a “mandatory” neglect investigation, when compared with white children.

We not only found that Allegheny’s algorithm in its first years of operation showed a pattern of flagging a disproportionate number of Black children for a “mandatory” neglect investigation, when compared with white children. But we followed up with another story that also uncovered that parents with disabilities were concerned that the tool singled them out as well.

Here is how the algorithm works:

If someone reports concern that a child is being neglected at home, they’re fed through a hotline’s screening process where this tool calculates the child’s potential risk and assigns a score.

Social workers then decide whether to investigate what is going on in the child’s home.

The Allegheny Family Screening Tool is specifically designed to predict the risk that a child will be placed in foster care in the two years after they are investigated.

It uses personal data collected from birth, Medicaid, substance abuse, mental health, jail and probation records, among other government data sets.

Then, it calculates a risk score of 1 to 20: The higher the number, the greater the risk.

County officials said that social workers can always override the tool, and called the CMU’s team’s research “hypothetical” since it looked at what the algorithm recommended on its own.

We spelled out that because family court hearings are closed to the public and the records are sealed, we couldn’t identify first-hand any families who the algorithm recommended be mandatorily investigated for child neglect.

By centering the story on the experience of a former therapist who quit her county job over related concerns, as well as that of a family lawyer trying to defend her clients by battling against an algorithm she can’t see, the story hit harder. We saw some quick impacts, the state of Oregon stopped using a related algorithm. The White House lifted up some of our findings in the first AI Bill of Rights.

For the second story, we collaborated with ACLU researchers who got access to the county data and essentially reengineered the algorithm and found that when it flagged people who accessed county services for mental health and other behavioral health programs, that could add up to three points to a child’s risk score, a significant increase on a scale of 20.

We put in extensive records requests to get the fields and weights underpinning several algorithms deployed by child welfare agencies, including some marked “CONFIDENTIAL,” offering rare insight into the mechanics driving these emerging technologies. Among the factors they have used to calculate a family’s risk, whether outright or by proxy: race, poverty rates, disability status and family size. They include whether a mother smoked before she was pregnant and whether a family had previous child abuse or neglect complaints.

And most importantly we found a family who was willing to share their story with us.

And we learned that the U.S. Department of Justice had opened up a federal civil rights investigation as a result of our work.

6 of 10

In 2024, I led a series called the AI Campaign that's exploring the influence of artificial intelligence in this crucial election year around the globe, where verifying content whether it is AI-generated or not, is crucial.

Many elections officials are really concerned about AI's potential to create content that could mislead voters and as generative AI tools really grow in sophistication, officials are worried that they could steer voters to avoid the polls or just generally erode the public's trust in what they see and hear.

So we're most interested in who are the human beings who are deploying these AI tools for campaigns and what exactly are they using?

So the first couple stories focused on Brad Parscale, who's a digital campaign operative who helped engineer Donald Trump's 2016 presidential victory, who now has a new AI-powered platform that he says will really transform transform campaigning.

He says that his AI-powered tools are going to usher in a wave of conservative victories worldwide. And that's everything from AI that can generate customized emails, gauge voter sentiment amplify the social media posts of anti-woke influencers, etc.

The second story dealt with how the then Biden campaign is considering using AI and still smarting from having been outmaneuvered by Donald Trump and his allies in 2016. So they're actually just still grappling with how to use AI to help them motivate voters and also identify deceptive content. But they did tell us that they were actually using AI for fundraising and to help them find and motivate voters and to better identify and defeat disinformation, for example.

7 of 10

8 of 10

9 of 10

GENERATIVE AI

A term for AI systems capable of creating text, images, video, audio, code and other media in response to queries. Humans can interact with generative AI models in a seemingly natural way but the models aren't reliably capable of distinguishing between facts and falsehoods or fantasy. Generative AI systems often are powered by large language models. They sometimes generate inaccurate or fabricated responses to queries, an effect AI scientists call hallucination. If using the term hallucination, describe it as an issue associated with the technology that produces falsehoods or inaccurate or illogical information. Some in the field prefer the term confabulation or simpler terms to describe the inaccuracies that don't draw comparisons with human mental illness.

AI is a broad umbrella term that can be used to describe anything from a computer program to the software that powers a robot.

Who here is using AI? (Take a few answers as to how people are already using it).

AI: such as visual perception, speech recognition, decision-making, and translation between languages.

Stylebook Online can be and is updated as needed year-round. We may update online in six months – OR, in two months.

We have been doing standards and style training within the AP and throughout the industry on this.

Longer definitions of those entries can be found in what we call the Top 10 AI terms, which are aimed at giving journalists an independent basic assessment of the data science to date.

For example, look at this one on how we define generative AI, which are the systems powering chatbots and image-generation tools.

You’ll note the boilerplate language includes that the models do not produce facts and are known for producing falsehoods.

Artificial Intelligence: computer systems, software or processes that can emulate aspects of human work and cognition.

Machine Learning: an AI process in which computer systems identify patterns in datasets to make or refine the decisions and predictions that they generate without being explicitly programmed to do so. E.g. face recognition, language translation and self-driving cars.

Algorithm: detailed computational instructions that describe how to solve a problem or perform a specific task, think a checklist or a recipe.

Training Data: a dataset used to teach an algorithm or a machine learning model how to make predictions.

10 of 10

[place holder slide]

Coverage of artificial intelligence tools should aim to show:

How these systems work.
Where they are deployed.
How well they perform.
Whether they are regulated.
Who benefits and makes money as a result.
Which communities may be negatively impacted by such tools.

FOLLOW JOURNALISTIC

BEST PRACTICES