Cloud security�from zero to one

Harsh Varagiya

agenda

​

• security challenges in cloud
• our cloud security journey at day 1
• implementing first line of defense
• detection engineering in cloud
• preventive controls in cloud
• data security in cloud
• key takeaways

​

3

• members only app for trustworthy users (750+ credit score)
• single app to track & manage all Credit Cards
• members earn CRED coins for paying bills
• we engage & reward our members for good behaviour
• cross-sell lifestyle & financial services

day 1: security is not an afterthought

• foundational security best practices were factored in right from day 1
• we deployed the initial architecture with security best practices in mind taking the help of AWS enterprise support team
• as the company grows, your architecture-level security mistakes will come back and haunt you!

how do you define security challenges in cloud?

​

Risk Based Approach vs Maturity Based Approach

risk based approach

define threats by conducting threat modelling in cloud

​

• identify your attack landscape within cloud (e.g. internal vs external)

​

• identify crown jewels (e.g. data, IP)

​

• identify must have functional business requirements (e.g. CI/CD, identity)

​

​

https://twitter.com/elrowan/status/1525989942261956608

​

defenders are entangled with integrating complex security tools, maintaining compliance posture and running behind latest industry buzzwords while overlooking the fundamentals

​

​

AWS security arsenal

​

​

An AWS organization consolidates your AWS accounts so that you can administer them as a single unit

It has one management account along with zero or more member accounts

Most of your workloads should reside in member accounts, except for some centrally managed processes that must reside in either the management account or in accounts assigned as delegated administrators for specific AWS services

security best practices which were baked in from day 1

• access management
• Using a SSO and Implementing 2FA
• Not creating individual user accounts
• Implementing RBAC

​

• detection
• Turning on cloudtrail, Guard duty and other logs
• Enabling trusted advisor to work on foundational security
• Creating cloudwatch alarms for detecting root account usage

​

• network
• Account and VPC level segregation of workloads
• Inbound restriction via security groups
• Using cloudfront CDN

​

• attack surface minimization
• Ensuring no public ec2 and s3 buckets exists

​

• service Deployment
• Complete automated deployment of microservices using IaC and Codepipeline
• Using managed container services

our approach towards building security from day 1

• small security team on day 1 which means limited bandwidth
• our approach
• DRY (Don’t Repeat Yourself)
• KISS(Keep It Simple & Stupid)
• utilize cloud native technologies wherever possible
• Less operational overhead
• Easy to implement security from day 1 at different layers

​

setting up first line of defense on cloud (zero to 1 story)

• defining Web Application Firewall approach

​

• must have requirements
• WAF should not be inline - single point of failure
• security team should not sit down writing WAF rules

​

• realizations we had after evaluating third party WAF’s
• licensing model of WAF were complex
• operational overhead - WAF rules were not managed
• a lot of them were selling AI/ML which was a blackbox to us
• most of the 3rd party products required us to move away from Cloudfront - design re-architecture

AWS enterprise support team to rescue

• AWS WAF Workshop (AWS moved to WAF v2 from WAF Classic)
• https://waf.sa.engineering/
• WAF v2 - marketplace rules, WAF logging was efficient

​

​

​

​

​

Log4j exploit case study

• Log4j caused great amount of panic as security teams did not have enough information on how to block log4j attacks
• hotpatch for log4j exploit was released within hours by aws of log4j exploit going public

​

​

​

Log4j exploit - SNS alert

• Log4j attack attempt detected by AWS WAF

​

​

AWS Advanced Shield

• did not enable on day 1 as shield basic is enabled for everyone who is behind cloudfront
• as we scaled up, business downtime was unaffordable and it was important to protect against high volume advanced DDoS attacks
• AWS Integration of WAF and Shield works flawlessly to protect against layer7 DDoS attacks.

​

​

security logging the right way from zero to one

• logging the right way
• setup AWS organisation from day 1
• centralised logging using organisation cloudtrail instead of individual trails
• protecting the log bucket
• setting up security data lake
• we used managed AWS Elastic search to setup a centralised security data lake on day 1
• over time, built an inhouse security data lake using Elastic on Graviton
• currently, we are ingesting >500GB/day with 90 days retention
• this helped us greatly to mature our detection controls

detection engg from zero to one - building DIAL

• we had cloudwatch alarms on malicious activities (e.g. Root account usage)
• limitations
• alerts were not actionable
• alerts can only be configured on selected API calls
• we started writing standalone scripts on AWS lambda to detect malicious activities using cloudtrail events as a trigger
• slowly over time, we matured home grown detection scripts to a full fledged cloud security misconfiguration detection tool which detects threats in real time called DIAL (https://github.com/CRED-CLUB/DIAL)

​

​

DIAL architecture & benefits

​

​

​

​

https://blog.cred.club/team-cred/tech/averting-cloud-misconfiguration-at-scale-with-dial/

​

​

​

​

benefits of DIAL

​

• detection time of DIAL for any malicious activity is < 5 seconds. Traditional SIEM detection time is > 5-10 minutes.

​

• completely stateless, infinitely scalable and cost effective

​

• modular which makes it easy to increase our security coverage to any of the AWS resources

​

• easy to deploy in multiple AWS accounts which we spawn

​

DIAL alerts

​

​

​

detection engg from zero to one - building Xenon

access keys leakage is one of the prominent cause for data breaches across the globe

ways access keys can get leaked?

• keys committed to a public repo
• instance profile keys - ec2 ssrf vulnerability
• social engineering/phishing
• stolen from system through endpoint compromise

​

​

​

Xenon - key features

• Xenon aims to detect instances whose credentials have been compromised, and then push out results to slack and IR tool for further investigation.
• real time detection of any compromised credentials for an instance
• alerts are sent to slack and IR for security team to investigate

​

detection engg from zero to one - network anomaly detection

• problems we had faced during COVID
• VPN becomes a key element with everyone working remotely
• VPN instance has to be kept in a demilitarised zone (DMZ)
• publicly hosted VPN host has a become a key target for attackers
• lack of threat visibility over VPN and internal traffic
• increasing number of attacks on VPN and zero days seen in the wild
• leveraging AWS VPC traffic mirroring to build network anomaly detection

​

VPC traffic mirroring

traffic Mirroring is an Amazon VPC feature that you can use to copy network traffic from an elastic network interface. You can then send the traffic to out-of-band security and monitoring appliances for:

• packet inspection
• Threat monitoring
• Network troubleshooting

​

​

anomaly detection using VPC Traffic Mirroring

​

​

​

​

https://aws.amazon.com/blogs/startups/cred-achieves-a-high-level-of-network-inspection-with-vpc-traffic-mirroring/

​

traffic mirroring use cases

• use VPC traffic mirroring for your key hosts and load balancers and mirror the traffic to suricata and zeek.
• analyse zeek network logs instead of flow logs for easy understanding of network activity
• generates stateful event logs for threat hunting
• detection use cases
• detects attack on VPN instance
• detects malicious traffic flowing through VPN
• detect Zero day attacks on VPN instance
• detects mass data extraction

​

​

alerts generated using suricata

​

preventive controls in AWS cloud

• while detection controls are always good to have, gradually we need to put preventive controls as well
• we started to put preventive control by placing account level restrictions via IAM
• gradually, we moved to using SCP in AWS organisations which is a better way to implement preventive controls
• few of the SCP use cases we have implemented
• block unused regions
• deny ability to change 's3 Block Public Access' config
• deny Backdoor ec2 Access API - EC2 Serial, SSM Session, SendSSHPublicKey
• block access to DeleteAccount API
• deny any account leaving AWS organisation
• deny ability to stop cloudtrail

​

​

data security

• securing our customers data is extremely important for us since we are operating in the fintech space
• we took a risk based approach towards securing our data infrastructure and data in cloud
• we performed threat modelling on our data stores - RDS, Redshift, s3 around these 5 pillars:
• Access control
• Users/Secrets management
• Encryption policies
• Logging and monitoring
• Backups
• key objective was to gain visibility on the above controls and find out existing gaps in the current process and come up with security recommendations to close the gaps
• we identified the gaps and gradually took the data security controls to maturity

​

​

key takeaways

• start with implementing cloud security best practices from day 1
• focus on cloud native technologies - keep operational effort minimal
• deploy the architecture with security best practices in mind - majority of the problems will get solved here
• as the company grows, your architecture level security mistakes will come back and haunt you!
• threat modelling - Do not stop thinking like an attacker
• assume that you are already compromised or will be compromised
• give importance to protecting your access keys
• tools will not magically solve your security problems

​