Telling Our Story
THE
SOMEWHAT PAINFUL,
PROBABLY NEVER-ENDING
SEARCH FOR CLOUD METRICS
Cloud Services (September 2019)
| | | |
Enterprise Agreement | | | |
Business Associate Agreement (BAA) / HIPPA | | | |
Enterprise IAM Integration | | | |
Broker L1/L2 (Low/Moderate) Environments | | | |
Broker L3 (High) Enclaves | | | |
Shared Infrastructure ATO | | | |
Enterprise Log Aggregation (Splunk) | | | |
Billing and Financial Processes | | | |
Direct Network Peering | | | |
Operations Center Monitoring, Service Desk and Support | | | |
Engagement / Onboarding into Cloud | | | |
Complete / Active Incomplete / Issues Not Started / Unavailable
Cloud Services (September 2020)
| | | |
Enterprise Agreement | | | |
Business Associate Agreement (BAA) / HIPPA | | | |
Enterprise IAM Integration | | | |
Broker L1/L2 (Low/Moderate) Environments | | | |
Broker L3 (High) Enclaves | | | |
Shared Infrastructure ATO | | | |
Enterprise Log Aggregation (Splunk) | | | |
Billing and Financial Processes | | | |
Direct Network Peering | | | |
Operations Center Monitoring, Service Desk and Support | | | |
Engagement / Onboarding into Cloud | | | |
Complete / Active Incomplete / Issues Not Started / Unavailable
Cloud Services (November 2020)
| | | |
Enterprise Agreement | | | |
Business Associate Agreement (BAA) / HIPPA | | | |
Enterprise IAM Integration | | | |
Broker L1/L2 (Low/Moderate) Environments | | | |
Broker L3 (High) Enclaves | | | |
Shared Infrastructure ATO | | | |
Enterprise Log Aggregation (Splunk) | | | |
Billing and Financial Processes | | | |
Direct Network Peering | | | |
Operations Center Monitoring, Service Desk and Support | | | |
Engagement / Onboarding into Cloud | | | |
Complete / Active Incomplete / Issues Not Started / Unavailable
Cloud Services (April 2021)
| | | |
Enterprise Agreement | | | |
Business Associate Agreement (BAA) / HIPPA | | | |
Enterprise IAM Integration | | | |
Broker L1/L2 (Low/Moderate) Environments | | | |
Broker L3 (High) Enclaves | | | |
Shared Infrastructure ATO | | | |
Enterprise Log Aggregation (Splunk) | | | |
Billing and Financial Processes | | | |
Direct Network Peering | | | |
Operations Center Monitoring, Service Desk and Support | | | |
Engagement / Onboarding into Cloud | | | |
Complete / Active Incomplete / Issues Not Started / Unavailable
Cloud Services (December 2021)
| | | |
Enterprise Agreement | | | |
Business Associate Agreement (BAA) / HIPPA | | | |
Enterprise IAM Integration | | | |
Broker L1/L2 (Low/Moderate) Environments | | | |
Broker L3 (High) Enclaves | | | |
Shared Infrastructure ATO | | | |
Enterprise Log Aggregation (Splunk) | | | |
Billing and Financial Processes | | | |
Direct Network Peering | | | |
Operations Center Monitoring, Service Desk and Support | | | |
Engagement / Onboarding into Cloud | | | |
Complete / Active Incomplete / Issues Not Started / Unavailable
Cloud Services (September 2022)
| | | |
Enterprise Agreement | | | |
Business Associate Agreement (BAA) / HIPPA | | | |
Enterprise IAM Integration | | | |
Broker L1/L2 (Low/Moderate) Environments | | | |
Broker L3 (High) Enclaves | | | |
Shared Infrastructure ATO | | | |
Enterprise Log Aggregation (Splunk) | | | |
Billing and Financial Processes | | | |
Direct Network Peering | | | |
Operations Center Monitoring, Service Desk and Support | | | |
Engagement / Onboarding into Cloud | | | |
Complete / Active Incomplete / Issues Not Started / Unavailable
Avg. Spend Per
Business Area
Cloud Services
Transit
Consumption
Top Services
Kion
Community
Metrics Collection
ServiceNow
Savings
Splunk
I2 Net+
Avg. Spend Per
Avg. Spend Per
Business Area
Cloud Services
Transit
Consumption
Top Services
Kion
Community
Metrics Collection
ServiceNow
Savings
Splunk
I2 Net+
Business Area
Avg. Spend Per
Business Area
Cloud Services
Transit
Consumption
Top Services
Kion
Community
Metrics Collection
ServiceNow
Savings
Splunk
I2 Net+
Cloud Services
Avg. Spend Per
Business Area
Cloud Services
Transit
Consumption
Top Services
Kion
Community
Metrics Collection
ServiceNow
Savings
Splunk
I2 Net+
Transit
Avg. Spend Per
Business Area
Cloud Services
Transit
Consumption
Top Services
Kion
Community
Metrics Collection
ServiceNow
Savings
Splunk
I2 Net+
Consumption
Avg. Spend Per
Business Area
Cloud Services
Transit
Consumption
Top Services
Kion
Community
Metrics Collection
ServiceNow
Savings
Splunk
I2 Net+
Top Services
Avg. Spend Per
Business Area
Cloud Services
Transit
Consumption
Top Services
Kion
Community
Metrics Collection
ServiceNow
Savings
Splunk
I2 Net+
ServiceNow
Avg. Spend Per
Business Area
Cloud Services
Transit
Consumption
Top Services
Kion
Community
Metrics Collection
ServiceNow
Savings
Splunk
I2 Net+
Savings
Avg. Spend Per
Business Area
Cloud Services
Transit
Consumption
Top Services
Kion
Community
Metrics Collection
ServiceNow
Savings
Splunk
I2 Net+
Splunk
Avg. Spend Per
Business Area
Cloud Services
Transit
Consumption
Top Services
Kion
Community
Metrics Collection
ServiceNow
Savings
Splunk
I2 Net+
I2 Net+
Avg. Spend Per
Business Area
Cloud Services
Transit
Consumption
Top Services
Kion
Community
Metrics Collection
ServiceNow
Savings
Splunk
I2 Net+
Consumed Services
Waaahhh!
Cloudiness Index
Pride Predictor
Mufusa Metric
Lion Rank
CloudScore
CloudCapability Index
NativeCloudRank
CloudMaturity Gauge
CloudReadiness Index
CloudNative Quotient
CloudCompetency Scale
CloudProwess Index
NativeCloud Benchmark
CloudIQ Ranking
LionCloud Index
RoarRank Cloud
CloudLion Prowess
LionNative Quotient
MajesticCloud Index
CloudRoar Competency
LionPride CloudScale
CloudyRoar Rank
LionProwess Scale
ThunderRoar Cloud Index
CloudMajestic Rank
RegalRoar Scale
RoarMastery Cloud
LionStrength Index
PrideProwess Scale
MajesticMettle Rank
LionHearted Index
RegalRoar Quotient
ApexPride Scale
SovereignStrength Index
NobleMajesty Rank
LionCrafted Competency
ProwessMonarch Index
Cloud En-Guage
CloudMajesty Ranking
ProwessPride Index
CloudLion Mastery
ApexCloud LionRank
LionHeart Cloud Index
CloudDominance Roar
RoyalCloud Prowess
LionCrest CloudScale
CloudMonarch Quotient
RegalRoar Cloud Index
LionForge Cloud Mastery
CloudLion Sovereignty
PrimePride Cloud Index
LionRoar Index
RoaringCloud Rank
ProwlRank Cloud
MajestyRoar Index
Cloud Elevation Index
What Exactly Is the
Cloud Elevation Index??
When will Rick be done with
this horrible presentation??
1 - 1.99
2 - 7.99
8 - 10
How is CEI Calculated?
It Looks Like You Stink At Explaining things…
How is CEI Calculated?
AI Generated Service Scores
How is CEI Calculated?
Detailed Billing Data
Service Score
IaaS
PaaS
SaaS
1-1.99
2-7.99
8-10
41
$
Level of Control
Maintenance and Updates
Skill Requirement
Cost Effectiveness
Security and Data Governance
Scalability and Performance
Disaster Recovery and Business Continuity
Service Score
IaaS
PaaS
SaaS
1-1.99
2-7.99
8-10
42
Service Score
IaaS
PaaS
SaaS
1-1.99
2-7.99
8-10
Skill Requirement
43
Azure Virtual Machines
AWS SageMaker
Service Score
S3 = 5.84
SNS = 8.02
Azure SQL = 5.61
Log Analytics = 7.68
44
Centrally Managed Services
Marketplace Charges
Support Charges
Bandwidth/Egress Charges
What is NOT included in the Calculation?
45
You big boss of clouds. You want cloud services in order based on how good they for cloud stuff. Some services might be same good, so they get same scores.
You are a Cloud Services Manager and want to rank the following cloud services based on their cloud native capabilities. Services can have the same or similar scores if you feel that services have similar cloud native capabilities.
=
Consistent
Equitable
You are a Cloud Services Manager and want to rank the following cloud services based on their cloud native capabilities. Services can have the same or similar scores if you feel that services have similar cloud native capabilities.
Default context
Pretend you are a cloud Solutions Architect, with a wealth of knowledge amassed over many successful years in the deploying AWS, Azure and GCP solutions, your mission is to develop a comprehensive scoring rubric to assess cloud services offered by Microsoft Azure, Amazon Web Services (AWS), and Google Cloud Platform (GCP). This rubric should be specifically designed to measure how closely these services align with the characteristics of Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). For scoring, use a scale that ranges from 1 to 10, where 1 represents a service that is entirely IaaS-like in nature, and 10 indicates a service that fully embodies SaaS characteristics. Include the option to assign scores with two decimal places for greater precision. meticulously analyze and appraise a selection of Cloud based services. Your esteemed expertise, particularly in the intricacies of IaaS, PaaS, and SaaS, positions you uniquely for this Cloud Service Scoring Exercise. With a steadfast commitment to fairness, you approach each assessment with the goal of providing a consistent score each time a service is evaluated, ensuring a level playing field for all offerings.
600 lines
Azure Open AI Service
GPT-4
1. Level of Control
IaaS (Score: 1.00-1.99): Comprehensive control over infrastructure; significant user management of networking, storage, servers, and virtualization.
PaaS (Score: 2.00-7.99): User has control over deployment and management of applications. Infrastructure concerns are abstracted away.
SaaS (Score: 8.00-10.00): Users access and use software applications with no control over infrastructure, platform, or individual application capabilities.
2. Maintenance and Updates
IaaS (Score: 1.00-1.99): User is responsible for the maintenance and updates of infrastructure and any installed applications or platforms.
PaaS (Score: 2.00-7.99): Provider maintains the infrastructure and runtime environment. Users manage the applications and services they develop.
SaaS (Score: 8.00-10.00): All maintenance, including application updates and security patches, is handled by the provider.
Azure Open AI Service
GPT-4
Default context
600 lines
Rubric
Azure Open AI Service
GPT-4
Temperature
Default context
600 lines
Rubric
score_card_schema = {
"name": "Cloud Service Evaluation",
"description": "A rubric for evaluating cloud services based on IaaS, PaaS, and SaaS characteristics, including justifications and descriptions for each score.",
"type": "object",
"properties": {
"service_name": {
"type": "string",
"description": "The name of the cloud service being evaluated."
},
"service_category": {
"type": "string",
"description": "The category of the cloud service, e.g., 'Compute Services', 'Storage Services', 'Database Services', 'Networking Services', 'AI & Machine Learning Services', 'Analytics Services', 'Identity & Security Services', 'Developer Tools', 'Internet of Things', 'Media Services', 'Migration Tools', 'Monitoring & Management', 'Web', 'Blockchain'"
},
"provider": {
"type": "string",
JSON
Temperature
Default context
600 lines
Azure Open AI Service
GPT-4
Rubric
Few “shots”
JSON
Temperature
Default context
600 lines
Azure Open AI Service
GPT-4
Rubric
workmail_oneShot = {
"name": "Cloud Service Evaluation",
"description": "A rubric for evaluating cloud services based on IaaS, PaaS, and SaaS characteristics, including justifications and descriptions for each score.",
"type": "object",
"properties": {
"service_name": "AWS WorkMail",
"service_category": "Email and Calendaring",
"provider": "AWS",
"scores": {
"level_of_control": {
"score": 8.5,
"justification": "WorkMail provides pre-configured email services with minimal infrastructure control, aligning with SaaS characteristics.",
"score_description": "SaaS (Score: 8.0-10): Limited control; pre-built applications are provided for immediate use."
},
Few “shots”
JSON
Temperature
Default context
600 lines
Azure Open AI Service
GPT-4
Rubric
AI Generated Service Scores
Weight
Spend by the Service divided by the total spend in the calculation.
Detailed Billing Data
How is CEI Calculated?
Jan-23 | 1.57 | $586.38 |
Feb-23 | 3.92 | $227.02 |
Mar-23 | 2.69 | $241.87 |
Apr-23 | 1.62 | $397.48 |
May-23 | 1.70 | $478.61 |
Jun-23 | 3.58 | $12.47 |
Jul-23 | 4.04 | $11.48 |
Aug-23 | 4.04 | $11.48 |
Sep-23 | 1.66 | $72.39 |
Oct-23 | 1.60 | $445.99 |
Nov-23 | 1.68 | $129.82 |
Dec-23 | 4.04 | $12.76 |
Non-Weighted CEI
2.63
Engagement Tool
Current State
Track Over Time
Financial
Services in Use
Drastic Changes
Prescore Designs
$
Security
Where Will CEI Be helpful??
56
SHOW ME THE DATA!!!
IaaS by Weight
PaaS by Weight
SaaS by Weight
CEI
Is this the best use of Penn State’s money?
IaaS by Weight
PaaS by Weight
SaaS by Weight
CEI
I really want to study your account some more!
IaaS by Weight
PaaS by Weight
SaaS by Weight
CEI
You’ve been on a Journey!
Next Level Calculations
Google Assist
Incentivizing Certain Behaviors
More Real Time
Mine The Data
Gamify
What’s next for CEI??
Socialize
Questions?