LIGHTNING STRIKES�CLOUD FORUM 2024

Lightning Round Rules

InstaCloud, the Turn-key Solution�Richard Guo, Stanford University

Cloud Credit Craze! Stanford’s Guide to Making it Rain (in the cloud)

�Lucrecia Kim-Boswell and Bruno Velazquez�Stanford University

​

Up Next: John bailey

Cloud Credit Craze

Revolutionizing Access to Google Cloud Credits at Stanford

​

Summer 2020: A Juggling Act

Summer 2020: A Juggling Act

• Multiple groups and individuals seeking assistance
• Varying needs and expectations
• ~50 sub-billing IDs for each project/researcher
• Manual set-up
• No shut off mechanism when credits run out

​

A Juggling Act

It was a lose-lose-lose situation

Solution: Burwood Credit Management System

Burwood introduced budget items and user management features that revolutionized how we track credits and usage

Turn off resources/disable billing or change to using ‘real dollars’

Automation

Easy to manage the credits - set up to suspend or continue billing

Ease of Management

Multiple departments ande centers running their own research credit grant.

​

Able to partner with multiple teams

​

Scaled from 50 - 400 projects

Solutions at Scale

Burwood Features and Impact

The result?

Lessons in Scaling Cloud Object Storage�

John Bailey�Washington University in St. Louis

​

​

Up Next: David Lacey

2024 Cloud Forum�Lightning Round Presentation

Lessons in Scaling Cloud Object Storage for Backups

John Bailey

Asst. Director, Cloud Systems

jwbailey@wustl.edu

WashU Hybrid Cloud Backup Architecture

Wasabi VDC

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

Primary Datacenter

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

Secondary Datacenter

​

​

​

​

Disk Backup Storage

VMware Cluster A

Veeam B&R

B

Disk Backup Storage

VMware Cluster B

AWS VDC

​

​

​

​

3

3

Physical Servers

Veeam B&R

A

Wasabi

Backup Storage

Veeam B&R

C

AWS EC2

Instances*

Data

Policy

​

2

1

1

Physical Servers

Offsite Tapes

1. Veeam copy jobs replicate a portion of backup data to Wasabi storage.
2. Wasabi data policy protects backups from ransomware attacks with data immutability.
3. AWS Veeam B&R instance ready to recover systems to cloud hosted AWS or Azure.

​

546.3 TB

​

Recent Challenges

• Data source = Block storage.
• Storage target = Object storage.
• Translation between the storage types means breaking files into 1MB objects (Veeam design.)

Recent Challenges

• Source data size: ~350TB.
• Backup data size: ~700TB.
• Versions, retention, immutability change rates, metadata objects
• Total number of object stored: 1.1 billion.
• Wasabi recommends no more than 100M objects in a storage bucket.

Corrective Action (Short term)

• More buckets!
• Instead of using 1 Wasabi bucket, use 20.
• Configure Veeam to use all 20 as a unified cloud copy target (repo.)

Corrective Action (Long term)

• Transition backup platforms from Veeam to Cohesity.
• Cohesity is designed from the ground up to use cloud object storage as a target.
• Cohesity writes to Wasabi in 32MB objects, rather than 1MB objects.

Key Lessons

• Understand complexities of:
• Source data store.
• Target data store.
• Tools that will move data form one to another (migration / backup tools.)
• Insist on details from vendors about scale.
• At what size will their product fall over?

Kubernetes in a Snap��David Lacey�The J Paul Getty Trust

​

Up Next: kari Robertson

September 17, 2019

David Lacey

Kubernetes In A Snap

Don’t deploy native Kubernetes��Do use vendor distributions��Do use Infrastructure as Code (IaC)

• Want a record of change
• Easy to create and maintain multiple instances���

Take the Easy Route

22

Kubernetes in a snap

Create a plan as your default configuration��Use variables and workspaces in the source code��Run your plan to create and manage clusters

• Change a variable to upgrade
• Can roll updates independently ���

Terraform Better Than ClickOps

23

Kubernetes in a snap

module "kubernetes" {� source = "terraform-aws-modules/eks/aws"� version = "17.24.0"�� cluster_name = var.kubernetes_cluster_name� cluster_version = var.kubernetes_version�� cluster_create_endpoint_private_access_sg_rule = true� cluster_endpoint_public_access = var.kubernetes_cluster_endpoint_public_access� cluster_endpoint_private_access = var.kubernetes_cluster_endpoint_private_access� cluster_endpoint_private_access_cidrs = var.kubernetes_cluster_endpoint_private_access_cidrs� enable_irsa = true� manage_aws_auth = var.kubernetes_manage_auth� subnets = local.private_subnet_ids� vpc_id = var.vpc_id� worker_additional_security_group_ids = [aws_security_group.eks_worker_sg.id]��worker_groups_launch_template = [� {� name = "default"� instance_type = var.kubernetes_worker_default_instance_type� asg_desired_capacity = var.kubernetes_worker_default_instance_count_minimum� asg_max_size = var.kubernetes_worker_default_instance_count_maximum� asg_min_size = var.kubernetes_worker_default_instance_count_minimum� kubelet_extra_args = "--node-labels=node.kubernetes.io/role=default --node-labels=node.kubernetes.io/lifecycle=normal"� tags = local.cluster_autoscaler_tags� }� ]�}

Terraform Code Snippet

24

Kubernetes in a snap

Allows you to maintain a record of truth��You need to bootstrap into a Kubernetes cluster��Manages the lifecycle of a service in Kubernetes clusters��Can manage ArgoCD with ArgoCD���

Use ArgoCD for GitOps

25

Kubernetes in a snap

my-app:� enabled: true� labels:� team: cloud-forum� values:� containers:� my-app:� image:� repo: registry.k8s.io/my-app� tag: "ce132cf42ea"� ports:� - container: 443� resources:� requests:� cpu: 50m� memory: 64Mi� name: my-app� ingress:� enabled: true� ingressClassName: internal-nginx� hosts:� - host: &myAppHost my-app.getty.edu� paths:� - "/"� tls:� - hosts:� - *myAppHost� secretName: my-app.getty.edu���

Helm Values Snippet

26

Kubernetes in a snap

Can use Spot.IO, Karpenter or Kubecost��Has the ability right size the worker nodes��Has the ability to scale up or down the worker nodes��Watchout for daemon sets and side cars being starved��

FinOps in Kubernetes

27

Kubernetes in a snap

High Level Kubernetes Workflow

28

Kubernetes in a snap

Thank you.

David Lacey

dlacey@getty.edu

Up Next: Richard guo

Attracting and Retaining Diverse Talent��Kari Robertson�University of California Office of the President

​

Attracting and Retaining Diverse Talent

Kari Robertson�Chief Technology Officer�University of California (UCOP)

I am not an expert on 

diversity, equity, or inclusion

...but this is an important topic

Attracting Applicants

Job Descriptions

• Straightforward
• Minimal Requirements
• Family Friendly
• Use Neutral Language
• Gender Bias Analyzer: https://gender-decoder.katmatfield.com/
• Supervisory Role Requirement
• Experience cultivating a respectful and diverse work environment

​

​

Champion

Headstrong

Driven

Leader

Self-Confident

Persistent

Active

Innovative

Strong

Dominant

Ambitious

Independent

Competitive

Decisive

Ambitious

-

Commitment

Enthusiastic

Empathic

Polite

Cooperative

Kind

Detail-Oriented

Gentle

Assistant

Nurturing

Supportive

Sensitive

Flexible

Compassionate

People-Oriented

Attracting Applicants

• Diversify recruitment channels, target affinity groups
• Highlight institution diversity initiatives
• Encourage employee referrals
• Utilize your professional network
• Actively invite individuals to apply

​

Selecting Candidates

Selecting Candidates

• Diverse interview/selection committee 
• All committee members must complete implicit bias training
• Multiple reviewers rate applicants
• Data driven selection process 
• Include an interview question on the value of diversity
• Provide examples of actions you have taken as a leader to promote diversity, equity, and inclusion
• Salary offers based on preset range

​

Staff Retention

Staff Retention

• Make fostering diversity an explicit goal
• Acknowledge holidays, enable flexible schedules
• Encourage attendance in affinity groups, webinars, etc
• Share resources highlighting diversity
• Interim/stipend opportunities require internal recruitment
• Rotate responsibilities for note taking, lunch organization, party planning

Challenge Your Assumptions

Challenge Your Assumptions

• Employment Gaps
• Conference Attendance
• Social Hours
• Visual Assumptions
• 'Being Technical'

Remember: I am not an expert on 

diversity, equity, or inclusion

...but I wanted to share these University of California guidelines

THANK YOU

Abstract

Research has proven that diverse teams are more productive due to the variety of skills, backgrounds, and perspectives. At University of California, we strive to create diverse teams and have developed pointers that can help any institution do the same. Let’s rethink what skills make for the ideal cloud engineer, leader, or student intern.

Up Next: Gabriel geise

InstaCloud, the Turn-key Solution��Richard Guo�Stanford University

​

Cardinal Cloud:

InstaCloud

University IT

Client Technology Solutions & Consulting

I need computing power from UIT …

47

Google’s First Server

The options

48

Laptop/Workstation

Farmshare

Cardinal Cloud

CTSC

Research Computing’s Clusters

Overhead:

Tons

None

Capability

Full Featured

Niche

The options

49

Laptop/Workstation

Farmshare

Cardinal Cloud

CTSC

Research Computing’s Clusters

Overhead:

Tons

None

Capability

Full Featured

Niche

UIT VM

The options

50

Laptop/Workstation

Farmshare

Cardinal Cloud

CTSC

Research Computing’s Clusters

Overhead:

Tons

None

Capability

Full Featured

Niche

UIT VM

The options

51

Laptop/Workstation

Farmshare

Cardinal Cloud

CTSC

Research Computing’s Clusters

Overhead:

Tons

None

Capability

Full Featured

Niche

UIT VM

The options

52

Laptop/Workstation

Farmshare

Cardinal Cloud

CTSC

Research Computing’s Clusters

Overhead:

Tons

None

Capability

Full Featured

Niche

UIT VM

The options

53

Laptop/Workstation

Farmshare

Cardinal Cloud

CTSC

Research Computing’s Clusters

Overhead:

Tons

None

Capability

Full Featured

Niche

UIT VM

The options

54

Laptop/Workstation

Farmshare

Cardinal Cloud

CTSC

Research Computing’s� Clusters

Overhead:

Tons

Capability

Full Featured

Niche

UIT VM

What’s Missing?

55

Laptop/Workstation

Farmshare

Cardinal Cloud

CTSC

Research Computing’s� Clusters

Overhead:

Tons

None

Capability

Full Featured

Niche

UIT VM

A low overhead, full featured computing platform

that’s also affordable

InstaCloud

• VMs
• Ubuntu, Windows, AL2/2023
• Containers
• GPUs (coming 2024)

• Automatic patching
• Automatic reboots
• Minsec compliant (low and moderate)
• No cloud management
• Cardinal Cloud benefits^*
• Bundled, consistent pricing

​

57

InstaCloud

Perfect for:

​

• Dev/Test environment
• Low stakes web service (no SLA)
• Short duration needs (< 1 year)
• Introductory AI/ML

58

InstaCloud

https://instacloud.stanford.edu

59

Thank You

60

Zero to GCC High in 8 Weeks

�Gabriel Geise�The Pennsylvania State University

​

Up Next: Shruthi Sreenivasa Murthy

Zero to Azure GCC High in 8 Weeks

and 6 easy steps.

​

Gabriel Geise

gbg3@psu.edu

​

Step 1.

This slide redacted

Step 2. Choose an Implementation Partner

• Gets you through the initial setup phase
• Helps to guide decision points
• Facilitates the timeline
• Follow-on helps for auditing of systems

Step 3. Design

• Start with a reference implementation
• Meeting with technical and security teams.
• Discuss needs, and outcomes.

​

Step 4. Implementation & Verification

• Partner generally responsible for implementation
• Initial accounts created for access

​

​

Step 5. Documentation

• What was built
• Settings in the directory
• Roles and people
• Process

Step 6. ???

Zero to Azure GCC High in 8 Weeks

and 6 easy steps.

​

Gabriel Geise

gbg3@psu.edu

​

Simplifying Cloud Operations for Research

�Shruthi Sreenivasa Murthy �St. Louis University

​

Simplifying Cloud Operations for Research� - Shruthi Sreenivasa Murthy

SCAER turns to RCG

The Sinquefield Center for Applied Economic Research (SCAER) is one of the leading research centers at SLU. The center is a part of the SLU Research Institute and brings together all the researchers at SLU who have a common interest in applied economic research.

​

Research Computing Group (RCG) is a collaboration between the Office the VP of Research and the Information Technology services at SLU. RCG builds on SLU’s ambition to develop and improve computing services, technology, services and support for research.

73

Project Discovery

74

Researchers at SCAER use large volumes of human mobility data to create useful research files for a wide range of studies in economics, public health, and finance related to patterns of social mobility. The underlying micro datafiles are very large, averaging approximately over 100 terabytes per year. Dataset includes human mobility data for the years 2019 to 2023.

​

​

Immediate Imperative - Preprocessing�

75

Immediate Imperative - Post Processing�

76

The solution we built

77

Putting the data into action

SCAER’s research directly aligns with SLU's broader mission of serving humanity. Some use cases that we extensively use this data for:

​

• By investigating human mobility patterns during natural disasters, SCAER researchers aim to better understand how communities respond and adapt to such events.
• They are leveraging mobility data to provide a novel perspective on accessibility to dialysis centers for patients with end-stage renal disease (ESRD)
• They are utilizing this human mobility data to study food insecurity, by tracking movement patterns, we can identify regions where populations frequently travel longer distances to access food, indicating potential "food deserts" or areas lacking affordable, nutritious food options.
• Researchers at the TGI have used the mobility spend data to examine the effect of MLB games on spending patterns and visits to various businesses in the St. Louis metro area. 

​

​

78

​

THANK YOU!

79

THANK YOU

TRACK DOWN THE LIGHTNING SPEAKERS THIS WEEK AND ASK THEM ABOUT THEIR WORK