1 of 21

Serverless in Production

Michael “Sveder” Sverdlin

m@sveder.com

@msveder

2 of 21

Lambdas in Lightricks

  • Part of our API, data pipeline and devops pipeline.
  • Around 30 invocations per second across all our Lambdas
  • Difference between peak and slack of around 300-3000%

3 of 21

4 of 21

Designing and Building

5 of 21

Conjoined triangles of Streaming to Lambdas

Batch size

Lambda Memory

Alarms

Shards

Shards

Alarms

6 of 21

“Concurrent Executions”

  • 100 Concurrent executions?
  • Concurrent execution = shard count
  • Concurrent execution =

Invocation count * duration

  • Late night calls with AWS to get it higher
  • Don’t worry, default is 1000 nowadays

​

7 of 21

Startup and Memory

  • Memory and CPU scale together.
  • Affects Lambda cold start times.
  • Reduce your deployment package (on S3)
  • Java and C# cold start times slow, Python and NodeJS fast.
  • Don’t forget to keep warm (somehow).

8 of 21

Frameworks!

  • Serverless 0.56
  • Serverless 1.0
  • Zappa
  • 23 Svederless frameworks. We’re JS now.

http://blog.spotinst.com/2017/06/07/state-serverless-ecosystem/

9 of 21

Zappa

  • WSGI over Lambda (Python only)
  • Automatic dependency “Compilation”
  • Automatic warmup trigger.
  • Better env vars (KMS)

But

  • Kinda not really serverless (AG for example)

10 of 21

Testing

  • Run on Amazon Linux AMI
  • Python: python-lambda-local
  • Node: lambda-local
  • Atlassian’s LocalStack (for API)
  • Runscope + CircleCI

​

11 of 21

Maintaining and Monitoring

12 of 21

Alarms

  • Alarms on invocation timeouts.
  • Alarms on Lambda $ costs.
  • Alarms on minimum invocation?
  • Sentry is great, but be careful with security groups :)

​

​

13 of 21

Logging and Tracing

  • Cloudwatch logs suck
  • Easy to stream to ElasticSearch
  • Dashbird.io, iopipe.com
  • AWS X-ray - Meh
  • Log your input
  • Most importantly - Lambda’s invocation length means nothing, use external tools like Runscope

​

14 of 21

API Gateway

  • Not fun at all
  • Lambda = function, AG = web server
  • HTTP return codes
  • Creates an internal Cloudfront dist
  • Lambda env var suck. API Gateway mapping can be used as configuration.

​

15 of 21

API Gateway - stack traces

16 of 21

API Gateway - stack traces

17 of 21

API Gateway - stack traces

18 of 21

API Gateway - stack traces

19 of 21

API Gateway - stack traces

20 of 21

Security

  • API Gateway supports certificate pinning
  • API Gateway also has API Keys
  • You can also use WAF for blocking.

21 of 21

​

​

Q & A

​

Let’s talk:

m@sveder.com

@msveder