Ethics of Autonomous and

Intelligent Systems

​

Kevin W. Lu

2024-09-26

Outline

1. Ethical and legal considerations in design
2. Modern engineering codes of ethics: self-regulation
3. Common violations
4. Resolving conflicts among the guidelines
5. Principles of product liability law
6. Designing against foreseeable uses and misuses
7. Anticipating the effects of change in a product

2

G. Voland, Engineering by Design, Chapter 8: Ethics and Product Liability

1. Ethical and Legal

Considerations in Design

• Engineers are legally and morally liable for the consequences of their work
• Unethical actions often are unintentional
• Ethical codes provide guidelines for dealing with dilemmas in which the ethical action may not be obvious
• Legal constraints and practical guidelines include
• Professional registration
• Federal, state, or local laws, regulations, and ordinances
• Contract law
• Tort (non-contract) law

3

The Code of Hammurabi

• The sixth Babylonian king, Hammurabi circa 1810 BC — 1750 BC, enacted the code, and partial copies exist on a 2.25-m (7.38-ft) stone stele and various clay tablets, dating back to about 1754 BC
• The code consists of 282 laws including those for builders such as

229. If a builder builds a house for someone, and does not construct it properly, and the house falls in and kills its owner, then the builder shall be put to death

230. If it kills the son of the owner, the son of the builder shall be put to death

231. If it kills a slave of the owner, then the builder shall pay, slave for slave, to the owner of the house

232. If it ruins goods, the builder shall make compensation for all that has been ruined, and inasmuch as the builder did not construct properly the house and it fell, the builder shall re-erect the house from his own means

233. If a builder builds a house for someone, even though he has not yet completed it; if then the walls seem toppling, the builder must make the walls solid from his own means

4

Marine Safety Code

• Continued disasters and high loss of life prompted congressional action through the passage of the Act of February 28, 1871 applied to all steam vessels and sought to protect their crews as well as their passengers
• It retained the useful functions of the Act of 1838 and the Steamboat Act of May 30, 1852, and added new requirements that provided a comprehensive Marine Safety Code, which forms the basis of the present marine safety code
• The Act of 1871 created the Steamboat Inspection Service that was merged with the Bureau of Navigation in 1932 to form the Bureau of Navigation and Steamboat Inspection, reorganized into the Bureau of Marine Inspection and Navigation in 1936, temporarily transferred to the US Coast Guard in 1942, and permanently transferred to the Coast Guard in 1946
• It established a Supervisory Inspector General directly responsible to the US Secretary of the Treasury, extended licensing requirements to all masters and chief mates, provided for the revocation of licenses, authorized periodic inspection, and gave the Board of Supervisory Inspectors the authority to prescribe nautical rules of the road

5

Industry 4.0

6

https://en.wikipedia.org/wiki/Industry_4.0

Benefits of Industrial IoT (IIoT)

• Improving operational efficiency
• Improving productivity
• Creating new business opportunities
• Reducing downtime
• Maximizing asset utilization
• Ability to sell products as a service
• Reducing asset lifecycle costs
• Enhancing worker safety
• Enhancing product innovation process
• Better understanding of customer demand

7

Morgan Stanley Research on the IIoT

Black Swan and Gray Rhino

• "A black swan is an event, positive or negative, that is deemed improbable yet causes massive consequences"
• "A gray rhino is a highly probable, high impact yet neglected threat: kin to both the elephant in the room and the improbable and unforeseeable black swan"
• "Gray rhinos are not random surprises, but occur after a series of warnings and visible evidence"

8

Flaws in Automated Systems

The Boeing 737 MAX Maneuvering Characteristics Augmentation System (MCAS) used to rely on one of two angle of attack (AOA) sensors and stabilizers on the tail to push the nose back down to avoid a stall

9

Source: The New York Times

2. Modern Engineering Codes of Ethics: Self-Regulation

Professional engineering societies such as the National Society of Professional Engineers (NSPE) and the Institute of Electrical and Electronics Engineers (IEEE) set codes of ethics to

• Encourage engineers to behave according to the accepted standards of the profession
• Provide guidelines about these standards and their application
• Assure lawmakers that engineering societies can be trusted to regulate the actions of their own members
• Encourage professional societies to support those members who do act in an ethical manner but then suffer negative consequences, e.g., loss of employment because of this action

10

NSPE Code of Ethics (CoE)

Fundamental canons, including rules of practice:

Engineers, in the fulfillment of their professional duties, shall

1. Hold paramount the safety, health, and welfare of the public
2. Perform services only in areas of their competence
3. Issue public statements only in an objective and truthful manner
4. Act for each employer or client as faithful agents or trustees
5. Avoid deceptive acts
6. Conduct themselves honorably, responsibly, ethically, and lawfully so as to enhance the honor, reputation, and usefulness of the profession

11

https://www.nspe.org/resources/ethics/code-ethics

NSPE Professional Obligations

• Engineers shall be guided in all their relations by the highest standards of honesty and integrity
• Engineers shall at all times strive to serve the public interest
• Engineers shall avoid all conduct or practice that deceives the public
• Engineers shall not disclose, without consent, confidential information concerning the business affairs or technical processes of any present or former client or employer, or public body on which they serve
• Engineers shall not be influenced in their professional duties by conflicting interests
• Engineers shall not attempt to obtain employment or advancement or professional engagements by untruthfully criticizing other engineers, or by other improper or questionable methods
• Engineers shall not attempt to injure, maliciously or falsely, directly or indirectly, the professional reputation, prospects, practice, or employment of other engineers. Engineers who believe others are guilty of unethical or illegal practice shall present such information to the proper authority for action
• Engineers shall accept personal responsibility for their professional activities, provided, however, that engineers may seek indemnification for services arising out of their practice for other than gross negligence, where the engineer's interests cannot otherwise be protected
• Engineers shall give credit for engineering work to those to whom credit is due, and will recognize the proprietary interests of others

12

https://www.nspe.org/resources/ethics/code-ethics

IEEE Code of Ethics

We, the members of the IEEE, in recognition of the importance of our technologies in affecting the quality of life throughout the world, and in accepting a personal obligation to our profession, its members and the communities we serve, do hereby commit ourselves to the highest ethical and professional conduct and agree:

1. To uphold the highest standards of integrity, responsible behavior, and ethical conduct in professional activities.
2. To treat all persons fairly and with respect, to not engage in harassment or discrimination, and to avoid injuring others.
3. To strive to ensure this code is upheld by colleagues and co-workers.

13

http://www.ieee.org/about/corporate/governance/p7-8.html

NCEES

The National Council of Examiners for Engineering and Surveying (NCEES) provides leadership in professional licensure of engineers and surveyors through excellence in

• Uniform laws, i.e., Model Laws
• Licensing standards, e.g., Fundamentals of Engineering Exam

and Principles and Practice of Engineering Exam

• Professional ethics, i.e., Model Rules

to safeguard the health, safety, and welfare of the public and to shape the future of professional licensure

14

3. Common Ethical Violations

• Failing to protect the public
• Unethical disclosure of facts and information
• Failure to include all pertinent information in professional reports
• Performing work for which one is not qualified
• Expressing a professional opinion that is not founded on both adequate knowledge of facts and technical competence in the field
• Issuing a statement or other communication without identifying all interested parties
• Failing to act as a faithful agent or trustee of one's employer or client

15

Institutional Review Board

• An institutional review board (IRB), also known as an independent ethics committee (IEC), ethical review board (ERB), or research ethics board (REB), is a type of committee that applies research ethics by reviewing the methods proposed for research to ensure that they are ethical
• Such boards are formally designated to approve (or reject), monitor, and review biomedical and behavioral research involving humans
• They often conduct some form of risk-benefit analysis in an attempt to determine whether or not research should be conducted
• The purpose of the IRB is to assure that appropriate steps are taken to protect the rights and welfare of humans participating as subjects in a research study
• Along with developed countries, many developing countries have established national, regional, or local Institutional Review Boards in order to safeguard ethical conduct of research concerning both national and international norms, regulations, or codes

16

Artificial Intelligence (AI)

Artificial Intelligence (AI) refers to the theory and development of computer systems able to perform tasks that normally require human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages

• Peter W. Singer, "Isaac Asimov’s Laws of Robotics Are Wrong," The Brookings Institution, 2009-05-18
• Ulrike Barthelmess and Ulrich Furbach, "Do We Need Asimov's Laws," 2014-04-29
• One Hundred Year Study on AI—2016 report
• Eight tenets released by Partnership on AI
• IEEE global initiative on ethics of autonomous and intelligent systems

Extended Intelligence (EI) or Intelligence Augmentation (IA) refers to the effective use of information technology in extending or augmenting human intelligence

• MIT Media Lab Human Dynamics Project on Social AI and Extended Intelligence
• Augmenting Human Intelligence by MIT Technology Review Custom in partnership with Brainspace, 2016-06-13
• IBM Research Response to Request for Information (RFI) on Preparing for the Future of Artificial Intelligence by the Office of Science and Technology Policy (OSTP) 2016-06-27

​

17

AI Tenets

The tenets of the Partnership on AI founded by Amazon, Apple, DeepMind, Google, Facebook, IBM, and Microsoft

1. We will seek to ensure that AI technologies benefit and empower as many people as possible.
2. We will educate and listen to the public and actively engage stakeholders to seek their feedback on our focus, inform them of our work, and address their questions.
3. We are committed to open research and dialogue on the ethical, social, economic, and legal implications of AI.
4. We believe that AI research and development efforts need to be actively engaged with and accountable to a broad range of stakeholders.
5. We will engage with and have representation from stakeholders in the business community to help ensure that domain-specific concerns and opportunities are understood and addressed.
6. We will work to maximize the benefits and address the potential challenges of AI technologies, by:
1. Working to protect the privacy and security of individuals.
2. Striving to understand and respect the interests of all parties that may be impacted by AI advances.
3. Working to ensure that AI research and engineering communities remain socially responsible, sensitive, and engaged directly with the potential influences of AI technologies on wider society.
4. Ensuring that AI research and technology is robust, reliable, trustworthy, and operates within secure constraints.
5. Opposing development and use of AI technologies that would violate international conventions or human rights, and promoting safeguards and technologies that do no harm.
7. We believe that it is important for the operation of AI systems to be understandable and interpretable by people, for purposes of explaining the technology.
8. We strive to create a culture of cooperation, trust, and openness among AI scientists and engineers to help us all better achieve these goals.

18

AI Thematic Pillars

• Safety-critical AI
• Fair, transparent, and accountable AI
• Collaborations between people and AI systems
• AI, labor, and the economy
• Social and societal influences of AI
• AI and social good

19

https://www.partnershiponai.org/thematic-pillars

4. Resolving Conflicts among

the Guidelines

• Engineers' foremost responsibility is to act in the best interest of the public
• All other considerations are secondary to this goal

20

Life 3.0

In his book, Prof. Max Tegmark of MIT and cofounder of the Future Life Institute (FLI) argues that we shouldn’t passively ask "what will happen?" as if the future is predetermined (or undirected), but instead ask what we want to happen and then try to create that future (i.e., beneficial intelligence)

21

Red Herring

• Max Tegmark, Life 3.0: "The fear of machines turning evil is another red herring. The real worry isn’t malevolence, but competence. A superintelligent AI is by definition very good at attaining its goals, whatever they may be, so we need to ensure that its goals are aligned with ours."
• A red herring refers to a particularly strong kipper, a fish (typically a herring) that has been strongly cured in brine (a high-concentration solution of salt in water) and/or heavily smoked with its flesh turning reddish and smelling
• In a figurative sense, a red herring is a logical fallacy or a literary device that misleads or distracts from a relevant or important issue
• The earliest reference to using a red herring for distracting hounds in pursuit of a hare is an article published on 1807-02-14 by William Cobbett 1763—1835 in Political Register
• The Gentleman's Recreation (1686) by Nicholas Cox referred to using herrings that was not to distract the hounds from a trail, rather to guide horses and hounds along a trail according to Michael Quinion

22

AlphaGo Zero to AlphaZero

Removing the Go knowledge from the Go engine made a better Go engine and, at the same time, an engine that could play shogi and chess [Wikipedia, DeepMind Blog]

23

Questions about Consciousness

24

Emotional Intelligence

Emotional intelligence (EI), also known as emotional quotient (EQ), is the capability of individuals to

• Recognize their own emotions and those of others
• Discern between different feelings and label them appropriately
• Use emotional information to guide thinking and behavior
• Manage and/or adjust emotions to adapt to environments or achieve one's goal(s)

25

The Torch Bearers

• Anna Hyatt Huntington 1876—1973 designed Los Portadores de la Antorcha (The Torch Bearers) from 1949 to 1954 with the theme of passing the lit torch of ethics, learning, and values from one generation to the next
• Donated the original aluminum cast sculpture to la Universidad Complutense de Madrid (est. 1293) in Spain on May 15, 1955, and one of the replicas to Stevens in April 1964

26

5. Principles of Product Liability Law

• The majority of product liability laws are determined at the state level and vary widely from state to state
• The claims most commonly associated with product liability are negligence, strict liability, breach of warranty, and various consumer protection claims
• Three major types of product defects
• Manufacturing defects
• Design defects
• Failure-to-warn defects
• Under strict liability, the manufacturer is liable if the product is defective, even if the manufacturer was not negligent in making that product defective
• Strict liability thus causes manufacturers to evaluate the full costs of their products including the litigation costs

27

Privity Protection No Longer

• Referring to a direct contractual relationship between two parties, privity used to shield manufacturers from lawsuits by requiring that the product had to have been purchased by buyer directly from the defendant
• If the manufacturer sold its products through a distributor, only the distributor (who had sold the product directly to the buyer) could be sued in the event of a mishap
• The distributor could claim that the product was sold in good faith and therefore was not responsible for the injury or loss
• The following cases removed privity protection from cases involving negligence, implied warranty (contract law), or strict liability (tort law)
• MacPherson v. Buick Motor Company (1916)
• Henningsen v. Bloomfield Motors, Inc. (1960)
• Greenman v. Yuba Power Products, Inc. (1963)
• By reason of its existence and sale, every product is now expected to be safe to use, and most product liability lawsuits are now filed under the tort category of strict liability

28

6. Design Against Misuses

• Anticipate foreseeable uses and misuses of the product by the user, e.g., texting while driving
• Although one can not expect to eliminate all possible misuses of a product, the attempt must be made
• Design against any misuse of a product and make it impossible for such misuses to occur

29

Levels of Driving Automation

30

https://www.sae.org/standards/content/j3016_201806

Self-Driving Car Accidents

• Cars driven under traditional human control are currently involved in approximately 1.18 fatalities for every 100,000,000 mi (160,934,400 km) driven
• Distance driven by Tesla Model S cars at the time of the 2016-05-07 driver fatality in Williston, Florida, was 130,000,000 mi (209,214,720 km)

31

Source: The New York Times

IEEE Standards for Wireless Access in Vehicular Environments (WAVE)

• IEEE 1609.0-2019 — IEEE Guide for WAVE Architecture
• IEEE P1609.2b — IEEE Standard for WAVE Security Services for Applications and Management Messages Amendment 2: Protocol Data Unit (PDU) Functional Types and Encryption Key Management
• IEEE P1609.2.1 WAVE Certificate Management Interfaces for End-entities
• IEEE 1609.3-2016 — IEEE Standard for WAVE Networking Services
• IEEE 1609.4-2016/Cor 1 IEEE Standard for WAVE Multi-channel Operation
• IEEE 1609.11-2010 — IEEE Standard for WAVE Over-the-Air Electronic Payment Data Exchange Protocol for Intelligent Transportation Systems (ITS)
• IEEE 1609.12-2016 — IEEE Standard for WAVE Identifier Allocations

​

32

False Information

• Web contents can be false, fabricated, manipulated, or misleading
• The International Federation of Library Associations and Institutions (IFLA) published an Infographic "How to spot fake news" on 2017-01-27
• Facebook prioritizes news reports in its news feed from publications that users have rated in Facebook surveys as trustworthy
• Google published a paper titled "Knowledge-based Trust: Estimating the Trustworthiness of Web Sources" on 2015-02-12 about a new method of scoring webpages based on the accuracy of the facts presented through understanding a page’s context without the use of third-party signals

33

Web 3.0

34

InterPlanetary File System (IPFS)

• Juan Benet started Protocol Labs in May 2014, and published original 2014 papers on the InterPlanetary File System (IPFS) and Filecoin
• IPFS is a protocol and network designed to create a content-addressable, peer-to-peer method of storing and sharing hypermedia in a distributed file system
• Juan Benet's 2016 TEDx Talk on the next internet revolution
• Filecoin is an open-source, public, cryptocurrency and digital payment system intended to be a blockchain-based cooperative digital storage and data retrieval method
• OrbitDB is a serverless, distributed, peer-to-peer database that uses
• IPFS as its data storage
• IPFS Pubsub to automatically sync databases with peers
• CRDTs (conflict-free replicated data type) for conflict-free database merges

35

7. Anticipating the Effects of

Change in a Product

• All products age with resultant changes in their characteristics that may result in injury or loss
• Engineers must
• Anticipate the change that may occur in a product during its useful lifetime
• Consider the disposal of a product after its useful life has ended
• Protect both people and the environment from the disposal of a hazardous product

​

36

General Data Protection Regulation

• The General Data Protection Regulation (GDPR) was designed to
• Harmonize data privacy laws across Europe
• Protect and empower all European Union (EU) citizens data privacy
• Reshape the way organizations across the region approach data privacy
• It also addresses the export of personal data outside the EU
• Among their provisions, the rules include the right to be forgotten that people can ask companies to remove certain online data about them
• The rules also require anyone under 16 to obtain parental consent before using popular digital services
• Effective 2018-05-25, if companies do not comply, they could face fines up to 4% of their annual global revenue or €20 Million, whichever is greater

37

Privacy Management

Privacy management for fair and authorized processing of personally identifiable information (PII)

• Collection, storage, use, organization, recording, alignment, combination, disclosure by transmission, consultation, erasure, destruction, alteration, etc.
• Any data that identifies an individual or from which identity or contact information of an individual can be derived

38

OASIS Technical Committees

• OASIS is Organization for the Advancement of Structured Information Standards
• Privacy Management Reference Model (PMRM)
• Privacy by Design Documentation for Software Engineers (PbD-SE)
• Cross-Enterprise Security and Privacy Authorization (XSPA)

39

https://www.oasis-open.org

Privacy Management Sequence

40

https://en.wikipedia.org/wiki/Unified_Modeling_Language

Privacy by Design

41

Ethically Aligned Design (EAD)

• IEEE Ethics in Action in Autonomous and Intelligent Systems
• IEEE Financial Playbook Version 1.0: Trusted Data and Artificial Intelligence Systems (AIS) for Financial Services
• IEEE Ethically Aligned Design: A Vision for Prioritizing Human Well-Being With Autonomous and Intelligent Systems, First Edition (EAD1e)
• IEEE 7000 series of standards
• IEEE Ethics Certification Program for Autonomous and Intelligent Systems (ECPAIS)
• The Open Community for Ethics in Autonomous and Intelligent Systems (OCEANIS)

42

IEEE Standards and Projects

7000-2021 Standard Model Process for Addressing Ethical Concerns During System Design

7001-2021 Standard for Transparency of Autonomous Systems

7002-2022 Standard for Data Privacy Process

P7003 Standard for Algorithmic Bias Considerations

P7004 Standard for Child and Student Data Governance

P7004.1 Recommended Practices for Virtual Classroom Security, Privacy and Data Governance

7005-2021 Standard for Transparent Employer Data Governance

P7006 Standard for Personal Data Artificial Intelligence Agent

7007-2021 Ontological Standard for Ethically Driven Robotics and Automation Systems

P7008 Standard for Ethically Driven Nudging for Robotic, Intelligent and Autonomous Systems

7009-2024 Standard for Fail-Safe Design of Autonomous and Semi-Autonomous Systems

7010-2020 Recommended Practice for Assessing the Impact of Autonomous and Intelligence Systems on Human Well-Being

P7010.1 Recommended Practice for Environmental Social Governance (ESG) and Social Development Goal (SDG) Action Implementation and Advancing Corporate Responsibility

P7011 Standard for the Process of Identifying and Rating the Trustworthiness of News Sources

P7012 Standard for Machine Readable Personal Privacy Terms

P7013 Inclusion and Application Standards for Automated Facial Analysis Technology

P7014 Standard for Ethical considerations in Emulated Empathy in Autonomous and Intelligent Systems

P7015 Standard for Data and Artificial Intelligence (AI) Literacy, Skills, and Readiness

P7016 Standard for Ethically Aligned Design and Operation of Metaverse Systems

P7016.1 Standard for Ethically Aligned Educational Metadata in Extended Reality (XR) and Metaverse

P7017 Recommended Practice for Design-Centered Human-Robot Interaction (HRI) and Governance

P7018 Standard for Security and Trustworthiness Requirements in Generative Pretrained Artificial Intelligence (AI) Models

P7019 Standard for the Implementation and Governance of Autonomous and Intelligent Systems Within the

Practice of Earth Law

​

43

EAD Pillars → General Principles

44

EAD General Principles → Chapters

45

EAD Conceptual Framework

The autonomous and intelligent systems shall be trustworthy, provable, and accountable; and shall align to explicitly formulated human values from principles to practice

46

Summary

• Codes of ethics provide guidelines for dealing with various types of professional dilemmas
• The ancient code of Hammurabi for builders and the United States marine safety code are early guidelines for acceptable engineering practice
• Professional engineers must follow specific legal obligations and requirements
• The NSPE Code of Ethics can be helpful if one is confronted by an ethical choice or dilemma
• Design engineers must be familiar with product liability law to understand the legal boundaries for socially acceptable products
• Manufacturing defects in a product is localized to a single unit, whereas warning and design defects affect the entire product population
• Negligence, strict liability and implied warranty, and express warranty and misrepresentation are the three major categories of product liability lawsuits
• Privity no longer serves to protect manufacturers from lawsuits
• One must design against foreseeable uses and misuses of a product
• Possible changes in a product during its useful lifetime must be anticipated in the design process

47