[TEMPLATE] Checklist for SaaS Security Maturity Evaluation (COPY ME)

	A	C	D	E	F	G
1	Checklist for SaaS Security Maturity Evaluation			(make a copy to edit)
2	Created by IronCore Labs. Instructions can be found in the original blog post. We welcome suggestions to improve the model.
3	This work is licensed under a Creative Commons Attribution 4.0 International License.
4
5	Company:
6	Date:
7
8		Item	Instructions	Your Notes	Points
9	Basic -- these are the things that everyone should be doing and no one should be touting any more than they'd tout locking their front door. ONE POINT PER LINE.
10		SSL/TLS	Look for the words "SSL", "TLS", or "encryption in transit." If they don't use or claim to use SSL, stop now and fail them immediately.		0
11		Transparent Disk/DB Encryption	Look for any claims of "encrypted at rest" or "database encryption" or "disk encryption." As with SSL, if they don't do this, fail them immediately.		0
12		Secure Software Development Lifecycle	Watch for "Secure SDLC" or similar. This suggests they have built-in security reviews, automated security code scans, and other measures to stop security flaws from shipping. If they mention things like code scans, make sure this box is checked.		0
13		3rd-party Penetration Tests	Look for "PenTest" or "Pen Test" as keywords. We consider this basic because it's the first thing that most companies do when asked to prove their security is reasonable. If they claim SOC2 Type 2, you can also check this box.		0
14		Content Security Policy (CSP)	This is an interesting way to get a feel for how well a company is paying attention to their web security. It's very easy to add a permissive policy, though difficult to make a restrictive one. I suggest checking their application page (post login) using something like CSP-Evaluator. Currently filed under Basic to keep things simple. They have a policy or they don't.		0
15		Web Application Firewall (WAF)	Most companies use a WAF to detect and stop common attacks, though we don't always see companies mention it on their security page. It's basic security.		0
16		(Distributed) Denial of Service Protection	Look for any discussion of DoS or DDoS protection. Companies don't always mention this protection, but it should be basic for everyone.		0
17		Secured Datacenter	There really shouldn't even be a checkbox for this, but you see it on nearly every security page often sensationalizing it with talk of "armed guards." But these days most software runs on one of the major infrastructure providers (AWS/GCP/Azure) and they all have physically secure facilities. It's a gimme that pretty much everyone has today. No brownie points for mentioning it. In fact, if they make a big deal about it, they're bullshitting you and you should get very skeptical.		0
18		Vulnerability Management	Also called "Patch Management" this is also a basic measure that everyone should be doing. It's a little alarming if it isn't even mentioned given that 80% of an application's code is comprised of third-party software libraries (per the GitHub State of the Octoverse report). But it takes real work to stay on top of vulnerabilities in dependencies.		0
19				Subtotal	0
20	Intermediate -- these are the things that indicate someone is doing at least a notch above the baseline. THREE POINTS PER LINE.
21	Logins
22		Two-factor or Multi-factor Authentication	Also look for words like "2FA" and "MFA." This is borderline basic security these days. Note that you may consider adding a bonus point manually if they support FIDO U2F hardware tokens. In a future edition of this spreadsheet, we might require two-factor options that go beyond text messages.		0
23		Single Sign-On	Also called "SSO" this is a feature that's always required when selling to Enterprises. If you aren't an Enterprise and they don't offer it for your plan level, then no points here.		0
24		IP White and Black Lists	This is mostly seen in companies whose software is used in call centers or other settings where they only want people to log in when they're at work. But having the feature is an indicator of greater attention to security beyond the basics.		0
25		New Device Login Warning Emails	This feature is, to me, too little and too late, but notifying someone that their account is potentially compromised is still better than letting it happen without notice. We have a line item for a better approach below in the Advanced section.		0
26		Account Lockout After N Attempts	This should be a basic feature, but today it still marks a company that is putting extra attention into security beyond the basics.		0
27	Certifications
28		SOC2 Type 2 or ISO27001 Certification	SOC2 comes in two flavors: Type 1 and Type 2. Type 1 just means they have good policies. Type 2 means they've been audited to be sure they follow those policies. In general, certifications like these are an important marker of a security maturity level that's at least intermediate. It should also be surprising if a vendor sells to Enterprises but doesn't have SOC2 or something equivalent.		0
29	Responsible Disclosure
30		Bug Bounty Program	Companies with a good, but not necessarily advanced, security posture provide clear instructions for security researchers to alert them of issues in their software or service. And they incentivize those programs with monetary rewards for the reports.		0
31				Subtotal	0
32	Advanced -- these are rarer security options that mark a company as having a very strong security maturity. FIVE POINTS PER LINE.
33	Advanced Authentication Options
34		New Device Login Approvals	With this item, a login from a new device cannot be successful unless it is first approved by an existing device. This is a pro-active alert approach.		0
35	Data Transparency
36		Data Change Audit Trails	Companies with advanced security programs track changes to data, who changed the data, when and from where. And then they make that information available to their customers regardless of their plan type.		0
37		Data Access Audit Trails	A notch above just logging when something changes, this feature shows a company that is willing to let a customer know anytime their data is viewed -- even if by an internal employee. It's normal to see this level of logging only available to more expensive plans.		0
38		Real-time Security Insights	Instead of just having a UI or an endpoint where someone can fetch logs, this feature allows a company to get real-time insights into audit-trails about their data and other security events such as logins, permission changes, etc.		0
39	Application-layer Encryption (aka ALE)
40		End-to-end Encryption (E2EE)	We've set this up to award double points since, when it's done right, it offers the best protection of sensitive data possible for a cloud service today. But it's very difficult to do right, which is why we see it so rarely.		0
41		Virtual Customer Isolation	Also known as per-tenant encryption for multi-tenant SaaS applications, this means that each customers' data is encrypted with a per-customer key and that is done in such a way that you can't accidentally query over data from multiple customers in a single request.		0
42		Customer Managed Keys	Also look for Bring Your Own Key, Customer Held Keys, CMK, BYOK, and sometimes Enterprise Key Management. This builds on per-tenant encryption but puts the customer in charge of the keys that protect their data so they can revoke that access if needed.		0
43				Subtotal	0
44				Total	0	/70
45
46			Level for score of 0 out of possible 70	Comments
47		Security Maturity Level:	Weak
48			0-15: weak, 15-30: reasonable, 30+: exceptional
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100