Generative AI Risk Assessment.xlsx

	A	B	C	E
1			Make a copy and edit as you please!
2	Generative AI Risk Assessment
3
4	Vendor Name:
5	Website:
6	Product/Service:
7	Date Completed:
8
9	Generative AI Internal Questions
10
11		Questions		Answers
12	1	What is the business justification for generative AI?
13	2	What is the intended use case for generative AI?
14	3	Will the generative AI service be accessible to customers?
15	4	Have we communicated with customers that we will be incorporating generative AI services into our offerings?
16	5	Does the use case comply with internal policies or standards related to generative AI?
17	6	Do we have any contractual limitations/restrictions that prohibit the use of generative AI with customers, business partners, or otherwise integrating into our services?
18	7	Do we need to amend contracts or Terms and Conditions to limit our liability resulting from use of generative AI services?
19	9	Please describe how you expect the third-party to use AI to facilitate their services, including any specific AI technologies or vendors. If you are unsure, or if the possible use has not been finalized, please describe the possible scope or determining factors.
20	11	Can or does the vendor comply with all appropriate laws, rules, and regulations pertaining to the Company?
21
22	Generative AI Vendor Questions
23
24		Questions		Answers
25	1.0	General
26	1.1	In what ways will our purchased product depend on Generative AI?
27	1.2	Will the use of generative AI support any customer interactions or services?
28	1.3	Do you have current customers in our industry utilizing Generative AI? If not, how will your platform adapt?
29	2.0	Data Privacy, Retention and Deletion
30	2.1	Will our company's use of your product remain anonymous?
31	2.2	Does your product offer transparent notice to an end-user that they will be interacting with a generative AI system?
32	2.3	Is there an option to opt-out of the generative AI features?
33	2.4	If an end-user opts to not use the Generative Artificial Intelligence portion(s) of your product, to what degree will you product still function?
34	2.5	Will data we input into the model be used to train the model?
35	2.6	Can you provide evidence that opt outs have been removed from training the model?
36	2.7	Are prompts attributable to specific users or organizations (when no identifying information is included in the prompt)?
37	2.8	Will our inputs to the model remain segmented from that of other companies?
38	2.9	Does the vendor moderate prompts (inputs) for the purpose of evaluating inappropriate content or bias?
39	2.10	Are there different data retention policies for the user interface versus the API?
40	2.11	How long will our inputs/prompts be retained if submitted via the user interface? Via the API?
41	2.12	Is the Data Retention period configurable?
42	2.13	Can our data be deleted on request, even if temporarily retained?
43	2.14	Where will our data reside?
44	2.15	Do we have the option to choose country residency for our data?
45	2.16	Will our data traverse international boundaries?
46	2.17	Do we have the option to restrict usage of the generative AI platform by region or country?
47	2.18	Will personally identifiable information sent to your product be anonymized?
48	3.0	Model Training, Validation & Maintenance
49	3.1	What foundational model is used?
50	3.2	Will you provide the common name of the foundational model? (e.g. GPT-3.5, LLAMA, etc.)
51	3.3	Does the generative AI feature rely on an Large Language Model (LLM)?
52	3.4	Do you have controls in place to ensure the foundational model was not trained with prohibited or biased content?
53	3.5	Do you check the provenance of the data in the training set to ensure its use is permitted?
54	3.6	Is the model data de-identified, aggregated and anonymized?
55	3.7	Does your model training process include human reinforcement?
56	3.8	Do you publish a change log for the model?
57	3.9	Have you performed any independent audits or validation of AI model outputs? Are you willing to share those results?
58	4.0	Information Security
59	4.1	Are penetration tests or other types of security assessments conducted on the model?
60	4.2	Do you regularly share / post the results of these independent tests?
61	4.3	Does the AI system or provider have change/release management processes to identify, reduce/mitigate, and manage toxic outputs, hallucinations, and biases during system development and operation?
62	4.4	Do you ensure that third-party libraries and components integrated into the AI system are current and secure?
63	4.5	Are you able to provide evidence of the implementation, effectiveness, and maintenance of security controls within the AI system?
64	4.6	Do you maintain evidence of baseline configurations and any subsequent changes made to the AI system?
65	4.7	Do you log security-related events (unplanned or externally initiated event) in the AI system? (e.g breaches, exfiltration of data, attempted exploits)
66	4.8	Is there a formalized training to ensure stakeholders remain current on best practices, emerging threats, and risks associated with AI?
67	4.9	What measures are in place to prevent unintentional leakage of private or sensitive information?
68	4.10	Are safeguards implemented to protect the AI system against malicious input or prompt injections?
69	4.11	Do you maintain a configuration management plan, and can you provide evidence of its implementation?
70	5.0	Technology Integration
71	5.1	Can the generative AI system be integrated via an API?
72	5.2	Do you offer private (in-house) and public options for use of the generative AI system?
73	5.3	Do you allow third-parties to develop plugins or add-ons for sale to the public?
74	6.0	3rd Party Risk/Usage
75	6.1	Do you utilize subcontractors in the development of or support of your generative AI system?
76	6.2	Will subcontractors have access to any of our data that we submit to the generative AI system, or that has been captured in logs?
77	6.3	Is the generative AI service delivered via a cloud service provider (CSP)?
78	6.4	Will you disclose the critical subservice providers that support the platform?
79	7.0	Legal, Regulatory and Compliance
80	7.1	Will you disclose future discoveries in the model that may materially impact the output?
81	7.2	Is the AI model continuously scanned (security scans, bias scans, etc)?
82	7.3	Do you maintain policies and procedures related to user privacy, biases, and explainability.
83	7.4	Are your services compliant with relevant data protection regulations (e.g., GDPR, HIPAA) and industry-specific regulations (e.g., finance, healthcare)?
84	7.5	Can you provide documentation of compliance or audits?
85	7.6	Is your service compliant with regulations such as GLBA, CCPA, GDPR or other data protection laws/regs regarding the handling of personal identifying information?
86	7.7	Do you have a Data Processing Agreement (DPA) that outlines data protection responsibilities?
87	7.8	Who owns the data generated by the generative AI system?
88	7.9	Is ownership clearly defined in the Terms and Conditions?
89	7.10	Are there any restrictions on how we can use the AI-generated content?
90	7.11	Is there a potential that copyrighted or other protected information could be produced as outputs from the generative AI systems?
91	7.12	Do you provide mechanisms to trace or audit the AI-generated results?
92	7.13	Do you assume liability in the event that the output of the generative AI system contradicts laws or regulations?
93	7.14	Is there an indemnification clause in the contract to protect clients in such situations?
94	7.15	Do you provide guaranteed SLAs for uptime of the generative AI service?
95	7.18	Should we need to terminate the relationship, will you be able to remove our data from your generative AI system in a reasonable period of time?
96
97	Additional Considerations
98	1	Additional Due Diligence	The FS-ISAC Artificial Intelligence Working Group has developed numerous resources to assist with learning the fundamentals, taxonomy, threats and countermeasures relevant to AI and Generative AI systems, with a focus on the Financial Services sector. In addition, there are numerous commercial and government entities that have created resources to facilitate deeper analysis of AI/generative AI system such as Shared Assessments (SIG), NIST, MITRE and others.)
99	2	Model Risk Analysis	When the output of the generative AI systems contributes to financial modeling or decisioning processes, consider seeking guidance from model risk experts in your organization.
100	3	Federal Guidance, Standards and Frameworks	While not specific to generative AI, NIST is the lead agency charged with developing AI risk assessment standards. Their AI Risk Management Framework may be a helpful additional resource. Particularly the section on Third-party entities (see the RMF Playbook, AI Actors, Third-party entities, https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook). NIST has published an Artificial Intelligence Risk Management Framework (AI RMF 1.0). The framework is designed to equip organizations with approaches on AI risk management. NIST specifies that the AI landscape and technologies continue to develop and frameworks will need to adjust accordingly.