InCommon Certificate Service Webinar

Certificate Discovery, Management, and Renewal

September 17, 2024

Welcome

[ 2 ]

1. Welcome

• Introductions

2. Presentation

3. Q&A

4. Closing

​

AGENDA

[ 3 ]

• We are recording this webinar, and you will receive the link to the recording via email.

​

• We’re taking questions and comments live using the Zoom Q&A function, so please send those messages during the presentation because we want this to be as interactive as possible.

• Also feel free to post messages in the chat. Just be sure when you are posting, your message is being sent to everyone, from the drop-down menu options.

​

​

​

​

Reminders

[ 4 ]

Meet Your Fellow Subscribers

​

American University

Bucks County Community College

Cal Poly Humboldt

Carleton College

Central Connecticut State University

Clemson University

Cleveland State University

Colorado State University

Cooper University Health Care

Cornell University

Duke University

George Washington University

Harvard University

Institute for Advanced Study

Iowa State University of Science and Technology

Kansas State University

LSU Health New Orleans

Mount Holyoke College

Nevada System of Higher Education

Northwestern University

Oak Ridge National Library

Oberlin College

Penn State

​

Here are some of the organizations attending today’s webinar:

Rice University

Rochester Institute of Technology

Southern Illinois University

St. Lawrence University

SUNY Geneseo

The University of Alabama at Birmingham

The University of Oklahoma Health Sciences Center

Trinity University

University of Alaska

University of California Berkeley

University of Florida

University of Minnesota

University of Mississippi

University of Nebraska System

University of North Carolina Charlotte

University of North Texas System

University of Pittsburgh

University of Rochester

University of Wyoming

Van Andel Institute

Virginia Commonwealth University

Washington & Lee University

Wellesley College

Woods Hole Oceanographic Institution

[ 5 ]

Speakers

Brian Epstein, Institute for Advanced Study

​

​

​

Clay Cooper, Rochester Institute of Technology

​

​

​

Paul Caskey, Internet2

Moderator

[ 6 ]

• Browser changes - 90-day certificate renewals

​

• Post Quantum Cryptography

​

• Certificate Discovery

​

• Automating Certificates

​

​

​

Today’s Topics

[ 7 ]

Institute for Advanced Study (IAS)

[ 8 ]

Size comparison and profiles of IAS vs RIT

Institute for Advanced Study (IAS)

• Founded in 1930
• 4 Schools
• Historical Studies
• Mathematics
• Natural Sciences
• Social Science
• 28 Faculty
• 20 Faculty Emeriti
• 200 Members
• 130 Staff
• 25 IT Staff
• 600 acre campus
• In Princeton, NJ, but not affiliated with Princeton University

Rochester Institute of Technology (RIT)

• Founded in 1829
• 11 Colleges
• Private doctoral university: High research activity (R2)
• Students and Employees
• 20,570 Students
• 1,547 Faculty
• 2,642 Staff
• Campuses
• Rochester, NY
• Dubai, UAE
• Dubrovnik, Croatia
• Zagreb, Croatia
• Pristina, Kosovo
• Weihai, China

[ 9 ]

Main drivers to certificate lifecycle automation

• 90 day certificates
• Post Quantum Cryptography

[ 10 ]

90 day certs

• Google Chromium project added to their Root Program Policy 2023-03
• In a future policy update or CA/Browser Forum Ballot Proposal, we intend to introduce:
• … our proposed maximum subordinate CA certificate validity is three (3) years.
• a reduction of TLS server authentication subscriber certificate maximum validity from 398 days to 90 days.
• We’ve done this before - in September 2020 we moved to 398 day certificates maximum
• Estimates put the change in second half of 2024
• Manual certificate process no longer viable

[ 11 ]

Quick history of modern cryptography

[ 12 ]

RSA

Elliptic Curve Cryptography

Quantum Computers

• Use qubits instead of classical bits
• Qubits can be in more than one state at the same time
• The state of a qubit is unknown until you observe it
• Qubits are fragile and interference can put them into an error state
• Error rates slow down the development of quantum computers

Shor’s Algorithm

• Makes factoring large semiprimes obtainable
• Another algorithm for solving the discrete logarithm problem
• And yet another for the period finding problem
• With proper quantum computer, RSA, DH, ECC, ECDH all become obsolete

Fourier Transform

Fourier Transform

3

5

Factoring 21 = 3 x 7

Post Quantum Cryptography

[ 21 ]

Quantum Factored

4 bit semiprime (2001): 15

5 bit semiprime (2012): 21

6 bit semiprime (2019 - failed): 35

​

Need to factor

1024 bit semiprime:

​

148580318425290417426752236620200656154903589692206628993323925127909644107437906684050027551844776278596512160601382625659982180758999544221868254722043619979845267456568698673226637753806670656171820422430405644979121166118132380586800025752259640730121738156822296246476504443847811940638639921190244907229

​

Need to factor

2048 bit semiprime:

​

23042155144807033264822777505847352979234760665383921887127664352782280852194120959365586434538328089186185273125706082069878978067782668670768363438482616137159112282139687893167423457466458802531248068491920362991080654721527620276216893955001892785769536572672674398163063893122113035627939070111414300285284659709274691084071399479406171139467536646372377297327570376498053909770429654959125201782358647860679882638386416987498248726270464804391684357189386521326968154447830980281514623449772865907495447946085856235827634344068188496211319432792505796581502095794388382929003408629452106790235491702341492875849

​

Need to factor

4096 bit semiprime:

​

895042969348578296981915910336163678732453124759533145586557604952895760317776316569158676150827486124282107892547967130332914027774243839644745256753400595926775963454953677271868790785113833576640691167921680046506876548102235826951106601104321424115479228484934247680630146283879498848910449797060157700850496337813043477561293777835401086614295601519110057733873752161156353028368807727894927096506522360815072003186182464302107329240801651864447332432227480806121176546660163477961207908730137818046855075860397970872099230128151889729620508160890782546593607782476115722199666190983162590520741299726377775088716806070106496493062874472415479023496763767874917569948669544950361493264214164541577312583731983304075229287865529835971053244670543287945198597007230421194138598762200939985241659753438228960525439886371016023854489945527427016801283327238740264623041454543856910545747681154748200516832481353745896101528766394364192199682267695484131145194250945616529523978521590091466213527524706661778437750247039074736641799486281849406242236762484545429225759181886778743331440696334172440423765256156865851502929465513715146197453647468878018363628603489424424030802769700952150094244596873055650438391281230331589881854277

Post-Quantum Cryptography (PQC) - where are we?

​

​

• Cloudflare has rolled out PQC to the majority of its domains
• experts-exchange.com
• gitlab.com
• Chromium supporting X25519Kyber768 hybrid ECC+PQC
• Chromium based browsers (Edge, Opera, Brave, etc) support
• Firefox support
• BoringSSL
• nginx
• Go
• Rust
• Microsoft's quantum-resistant cryptography is here - FIPS 203 - (2024-09-09)

[ 28 ]

What do we need to do?

1. Don’t panic.
2. Educate yourself and your company on the risks.
3. Understand your environment.
1. Where do you use encryption?
2. What type of encryption do you use?
3. Can you update? Are your vendors working on implementing PQC?
4. Realize that you should be auditing your encryption usage anyway.

Certificate and Cipher Scanning

[1]epfw:~$ sudo nmap -sS -Pn --script=ssl-cert pps4.ias.edu -pT:25

Starting Nmap 7.92 ( https://nmap.org ) at 2024-09-05 08:22 EDT

Nmap scan report for pps4.ias.edu (172.16.90.164)

Host is up (0.017s latency).

​

PORT STATE SERVICE

25/tcp open smtp

| ssl-cert: Subject: commonName=pps.ias.edu/organizationName=Institute for Advanced Study/stateOrProvinceName=New Jersey/countryName=US

| Subject Alternative Name: DNS:pps.ias.edu, DNS:pps3.ias.edu, DNS:pps4.ias.edu, DNS:pps5.ias.edu, DNS:smtp.ias.edu

| Issuer: commonName=InCommon RSA Server CA 2/organizationName=Internet2/countryName=US

| Public Key type: rsa

| Public Key bits: 2048

| Signature Algorithm: sha384WithRSAEncryption

| Not valid before: 2024-03-20T00:00:00

| Not valid after: 2025-04-20T23:59:59

| MD5: 3eb6 cb29 11be a1ef cc50 fb0f 988b ac67

|_SHA-1: 1b57 e825 9f5d 7866 334e 63ca 2fc3 ae3a b2d3 29c4

​

Nmap done: 1 IP address (1 host up) scanned in 0.48 seconds

[1]epfw:~$

​

[ 32 ]

Certificate and Cipher Scanning

[1]epfw:~$ sudo nmap -sS -Pn --script=ssl-enum-ciphers pps4.ias.edu -pT:25

Starting Nmap 7.92 ( https://nmap.org ) at 2024-09-05 08:18 EDT

Nmap scan report for pps4.ias.edu (172.16.90.164)

Host is up (0.014s latency).

​

PORT STATE SERVICE

25/tcp open smtp

| ssl-enum-ciphers:

| TLSv1.2:

| ciphers:

| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A

| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A

| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A

| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A

| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (secp256r1) - A

| TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A

| TLS_RSA_WITH_3DES_EDE_CBC_SHA (rsa 2048) - C

| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A

| TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 2048) - A

| TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A

| TLS_RSA_WITH_AES_256_CBC_SHA256 (rsa 2048) - A

| TLS_RSA_WITH_RC4_128_SHA (rsa 2048) - C

| compressors:

| NULL

| cipher preference: client

| warnings:

| 64-bit block cipher 3DES vulnerable to SWEET32 attack

| Broken cipher RC4 is deprecated by RFC 7465

|_ least strength: C

​

Nmap done: 1 IP address (1 host up) scanned in 2.46 seconds

[1]epfw:~$

​

​

[ 33 ]

What do we need to do?

1. Plan
1. What can you upgrade?
2. What can’t you upgrade?
3. Risk Analysis
2. Execute your Plan
3. Automation

Module-Lattice-based Key-Encapsulation Mechanism FIPS-203

13. Qualifications. In applications, the security guarantees of a KEM only hold under certain conditions (see NIST SP 800-227 [1]). One such condition is the secrecy of several values, including the randomness used by the two parties, the decapsulation key, and the shared secret key itself. Users shall, therefore, guard against the disclosure of these values. While it is the intent of this standard to specify general requirements for implementing ML-KEM algorithms, conformance to this standard does not ensure that a particular implementation is secure. It is the responsibility of the implementer to ensure that any module that implements a key establishment capability is designed and built in a secure manner. Similarly, the use of a product containing an implementation that conforms to this standard does not guarantee the security of the overall system in which the product is used. The responsible authority in each agency or department shall ensure that an overall implementation provides an acceptable level of security. NIST will continue to follow developments in the analysis of the ML-KEM algorithm. As with its other cryptographic algorithm standards, NIST will formally reevaluate this standard every five years. Both this standard and possible threats that reduce the security provided through the use of this standard will undergo review by NIST as appropriate, taking into account newly available analysis and technology. In addition, the awareness of any breakthrough in technology or any mathematical weakness of the algorithm will cause NIST to reevaluate this standard and provide necessary revisions.

Current New Certificate Process

• IT group makes a request via ticketing system
• Central IT creates certificate
• Generate Key and CSR
• Submit to cert-manager (Sectigo via InCommon)
• Approve certificate for signing (1 year max)
• Central IT puts certificate, private key, and intermediate certificates in password safe
• Name of entry includes expiration date
• Trusted IT employees have access to shared password safe
• Central IT updates ticket with location of stored certificate and key
• IT group manually installs certificate and key

[ 37 ]

Current Certificate Renewal Process

• Central IT receives expiration notice from Sectigo/InCommon
• Central IT replies to original ticket asking if the certificate is still required
• On confirmation of need, Central IT follows procedure for New Certificate Generation
• Generate new keys, no key re-use
• Update security or type of key based on current best practices

[ 38 ]

Issues with current procedure

• Manual process
• Time consuming
• Human error

[ 39 ]

Certificate Automation

• Certificate creation
• Automatic Certificate Management Environment (ACME)
• certbot
• Sectigo integration
• Certificate deployment
• Sectigo’s solution (Network Agents)
• Write your own
• Some vendors may help
• Manual deployment (yuck!)

[ 40 ]

Rochester Institute of Technology (RIT)

​

​

[ 41 ]

Size comparison and profiles of IAS vs RIT

Institute for Advanced Study (IAS)

• Founded in 1930
• 4 Schools
• Historical Studies
• Mathematics
• Natural Sciences
• Social Science
• 28 Faculty
• 20 Faculty Emeriti
• 200 Members
• 130 Staff
• 25 IT Staff
• 600 acre campus
• In Princeton, NJ, but not affiliated with Princeton University

Rochester Institute of Technology (RIT)

• Founded in 1829
• 11 Colleges
• Private doctoral university: High research activity (R2)
• Students and Employees
• 20,570 Students
• 1,547 Faculty
• 2,642 Staff
• Campuses
• Rochester, NY
• Dubai, UAE
• Dubrovnik, Croatia
• Zagreb, Croatia
• Pristina, Kosovo
• Weihai, China

[ 42 ]

Values

• Reduce manual work
• Enable certificate service for all
• Ensure certificates are issued to the appropriate admins
• Use off-the-shelf, community supported tools where possible
• If custom tools are needed, favor centralized, expert-built tools over individual tools

[ 43 ]

Current Process

Login to Cert Manager

Paste

Copy

Collect cert from email notice

Cert Admin Approval

Manager Approval

Submit request

Paste

Copy

Paste

Copy

Paste

Copy

Update request with order number

Close request

[ 44 ]

Where do our certs come from?

[ 45 ]

ACME Options

[ 46 ]

ACME - RFC 8555

[ 47 ]

ACME - RFC 8555

​

[ 48 ]

Distributed responsibilities

​

College of

Computing Sciences IT

​

computing.rit.edu

compsci.rit.edu

pluto.cs.rit.edu

mars.cs.rit.edu

green.it.rit.edu

College of

Business IT

​

mba.rit.edu

econ.rit.edu

business-api.rit.edu

server-alpha.business.rit.edu

server-beta.business.rit.edu

​

Central IT

​

​

*.rit.edu

*.main.ad.rit.edu

*.ad.rit.edu

*.ntid.edu

​

[ 49 ]

Distributed responsibilities

​

College of

Computing Sciences IT

​

computing.rit.edu

compsci.rit.edu

pluto.cs.rit.edu

mars.cs.rit.edu

green.it.rit.edu

College of

Business IT

​

mba.rit.edu

econ.rit.edu

business-api.rit.edu

server-alpha.business.rit.edu

server-beta.business.rit.edu

​

Central IT

​

​

*.rit.edu

*.main.ad.rit.edu

*.ad.rit.edu

*.ntid.edu

​

[ 50 ]

Sectigo ACME

​

[ 51 ]

Sectigo ACME

​

[ 52 ]

ACME Options

[ 53 ]

How to ensure security?

If HTTP-01 is good enough for Let’s Encrypt, then it’s good enough for me!

[ 54 ]

Introducing ACME Proxy

Or: How hard can RFC 8555 be?

[ 55 ]

ACME Proxy

[ 56 ]

ACME Proxy

​

[ 57 ]

ACME Proxy

​

certbot:

sudo certbot certonly \

--server http://localhost:8000/directory/ \

--domain test1.acme-local.rit.edu \

--email frodough@rit.edu \

--eab-kid <HMAC_KEY_ID> \

--eab-hmac-key <HMAC_KEY>

​

​

acme.sh:

acme.sh --register-account \

--server http://localhost:8000/directory \

--eab-kid <EAB_KID> \

--eab-hmac-key <EAB_HMAC_KEY>

[ 58 ]

ACME Proxy

​

[ 59 ]

ACME Proxy

​

[ 60 ]

ACME Proxy - Behind the Scenes

[ 61 ]

Supporting machinery - RIT ps-acme

​

[ 62 ]

Supporting machinery - acme-toolkit

​

$ ./acme-toolkit.py

acme-toolkit.py is a swiss-army toolkit for ACME.

usage: eabbinder.py command [arguments]

​

Commands:

help shows this help!

bind-account allows you to perform EAB

without using a client

lookup-account shows account ID from

public/private key pair

pem-to-jwk converts a pem formatted key

to a jwk

jwk-to-pem converts a jwk formatted key

to a pem

[ 63 ]

Challenges

• Clients are written to support Let’s Encrypt’s server, not the RFC
• Let’s Encrypt’s server is not fully RFC compliant
• Heterogenous applications
• Apache HTTP Server
• Apache Traffic Server
• IIS
• HAProxy
• Kubernetes Nginx Ingress via cert-manager (not Sectigo Cert Manager)
• SMTP servers
• MySQL
• Hardware lights-out management
• Java/Tomcat/Jetty
• Vendor-provided appliances
• Load-balancing via DNS
• Load-balancing via network unicast

[ 64 ]

Advice

• Standardize on a webserver product and configuration!
• Use a load balancer regardless of the number of app servers!
• No load balancer? Use your standardized webserver as a proxy in front of difficult apps:
• Jetty/Tomcat
• Applications that self-manage certs via a form inside the application

​

​

​

The goal is not to find tools that support ACME or REST!

The goal is to find a configuration that allows you to set it-and-forget it!

[ 65 ]

ACME Proxy

Thank you!

​

Dave Lafferty (RIT) - ACME Proxy developer

Mike Bruckner (RIT) - ps-acme module developer

GitHub - grindsa/acme2certifier

[ 66 ]

Q&A

[ 67 ]

Q & A

We’re taking questions and comments live using the Zoom Q&A function.

[ 68 ]

Closing

[ 69 ]

Thank You

• To Our Speakers

​

• And to All of You for Being Here

​

• Have a great rest of the day!

[ 70 ]