HOW TO MIGRATE
From Heroku to GCP / K8S
Node.js Paris, October 18th, 2018
1. What & why?
2. The K8S way
3. Deployment
4. Database
5. Env. vars
6. SSL
What
Algolia
Site Search crawler
Heroku & Docker compose
Algolia
Search Index
Front-end
Algolia
Database
Search Index
Front-end
Algolia & Site Search crawler
Database
Search Index
Front-end
Site Search Crawler
Crawler Architecture (1/3)
Search Index
Website
Workers
Crawler Architecture (2/3)
Search Index
Website
Database
Job Queue
Workers
Manager
Admin
API
Crawler Architecture (3/3)
Search Index
Website
Database
Job Queue
Workers
PDF Proxy
SPA Proxy
Manager
Admin
API
Deploy to Heroku
$ git push heroku
Local run 1/2
services:
db:
image: postgres:9.6.10
ports:
- "26432:5432"
redis:
image: redis:latest
ports:
- "36379:6379"
broker:
image: rabbitmq:3-management
ports:
- "26672:5672"
rendertron:
image: algolia/rendertron
ports:
- "23000:3000"
docker-compose.yml + shared .env file
Local run 2/2
$ docker-compose up -d
$ yarn dev
Why GCP/K8S
Compliance
Ownership
Cost
=
K8S
deploy and orchestrate�containerized applications
The K8S way
Differences
Deployment
Connecting to a database
Environment variables
HTTPS/SSL certificate
Deployment
Before K8S
01-redis.yml
02-rabbitmq.yml
03-cloudsql-proxy.yml
10-env-vars-common.yml
30-crawler-manager.yml
50-crawler-web.yml
70-crawler-worker.yml
99-crawler-ingress.yml
db-migrate.yml
Dockerfile
K8S: Dockerfile, YAML files and scripts
k8s-apply.sh*
k8s-auth.sh*
k8s-build.sh*
k8s-deploy.sh*
k8s-restart.sh*
k8s-run-job.sh*
Deploy to K8S
1/3
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: rabbitmq
spec:
replicas: 1
template:
metadata:
labels:
app: rabbitmq
spec:
restartPolicy: Always
terminationGracePeriodSeconds: 60
containers:
- name: rabbitmq
image: rabbitmq:3-management
imagePullPolicy: Always
ports:
- containerPort: 5672
- containerPort: 15672
E.g. 02-rabbitmq.yml
---
apiVersion: v1
kind: Service
metadata:
name: rabbitmq
# => exports env var RABBITMQ_SERVICE_HOST=10.7.250.108 (value may change)
spec:
type: NodePort
ports:
- port: 5672
name: "main"
# => exports env var RABBITMQ_SERVICE_PORT_MAIN=5672
- port: 15672
name: "ui"
# => exports env var RABBITMQ_SERVICE_PORT_UI=15672
selector:
app: rabbitmq
Deploy to K8S
2/3
---
kind: Deployment
replicas: 1
image: rabbitmq:3-management
---
kind: Service
# => exports RABBITMQ_SERVICE_HOST=10.7.250.108
- port: 5672
name: "main"
# => exports RABBITMQ_SERVICE_PORT_MAIN=5672
- port: 15672
name: "ui"
# => exports RABBITMQ_SERVICE_PORT_UI=15672
E.g. 02-rabbitmq.yml
Deploy to K8S
3/3
gcloud beta container clusters get-credentials "crawler-prod" \
--region "us-central1" --project "algolia-crawler"
DOCKER_IMAGE="gcr.io/algolia-crawler/crawler:latest"
docker build . -t ${DOCKER_IMAGE}
gcloud auth configure-docker
docker push ${DOCKER_IMAGE}
kubectl apply -f ./
kubectl set image --record deployment/mgr mgr=${DOCKER_IMAGE}
kubectl set image --record deployment/web web=${DOCKER_IMAGE}
kubectl scale deployment mgr web --replicas=0
kubectl create -f db-migrate.yml
until kubectl logs jobs/db-migrate; kubectl logs jobs/$1 2>&1 | grep -e "Done" >/dev/null; do sleep 1 ; done
kubectl scale deployment mgr web --replicas=1
E.g. deploy script
Connect to a Database
Heroku: one click + one env var
GCP/K8S
Environment variables
Heroku: web UI or CLI
Env. vars in K8S
=> sed and templates = your friends
SSL certificate
1/3
Heroku: one click
SSL certificate
2/3
GCP: “not our business”
SSL certificate
3/3
cert-manager:
Current status
As this project is pre-1.0, we do not currently offer strong guarantees around our API stability.
Conclusion
Migrating our distributed system to K8S/GCP was hard
K8S requires more custom tooling
K8S documentation is not ideal
Opportunity to improve the reliability of our system
Next Steps
Better monitoring & troubleshooting tools
Auto-scaling of workers
Auto-renewal of SSL certificate
Zero-downtime deployment
Thank you.