I’m at the Open Security Conference right now. The osco is an Open Space conference about information security. I’m also part of the organisation team and did a season about a private ACME compatible CA with HSM. Since the conference language is english, I will also write my summary here in english.
I thank NitroKey for providing me two HSM 2 for this session, so that I could show you the setup with hardware security modules. I choose the HSM 2 because it is Open Source, cheap and fast. But you can use other compatible modules.
Conference Session
A private ACME compatible CA with HSM The PDF with the slides explains the concept and the reason to set up an own private PKI. But there was also a demonstration, there we build a PKI in a small lab environment together. At the end, we had a running webserver with a valid certificate issued via ACME from out own intermediate certificate authority.
You want it to try yourself? You find the needed steps in this blog post. You need a lab with a VM for an air gap system, an intermediate certificate authority, a webserver and your client.
Implementation
You need two NitroKey HSM 2 for this lab. This setup is inspired by the NitroKey documentation and the step ca documentation. These documentations are also a good start to look for advanced topics like backups, high availability or other provisioners.
Prepare the air gap system
Set up a VM with debian bookworm. You can use another distribution, if you want. But you will have to adapt some commands or the package names accordingly. Connect the HSM module, that will contain your root key, to your VM.
After the installation of the basic system, login as root and install the needed packages.
1
apt install libengine-pkcs11-openssl gnutls-bin opensc pcscd opensc-pkcs11
With these packages the openssl commands can access the HSM.
Initialize the HSM 2 for the root ca
Please be aware, that this step will delete all previous data of the HSM! This is also the moment, you have to decide about a backup strategy with a Device Key Encryption Key or n-of-m schemes. This will not be possible after the initialization anymore.
In this example, I won’t use a backup.
You will be asked for a SO-PIN (Security Officer PIN) and a PIN for usage. Choose an appropriate label for your environment, so you can recognize your root ca.
1
pkcs11-tool --module /usr/lib/x86_64-linux-gnu/opensc-pkcs11.so --init-token --init-pin --label="RootCA" --login
Create the private key for the root ca in the HSM 2
After the HSM is set up, we can create a private key on it for our root ca. I use an elliptic curve algorithm. They are faster on the HSM 2 from NitroKey and the keys are - at the same security level - shorter than classic ones. The label is a name for your key. I’m not very creative here and just call it root. You will be asked for the user PIN.
1
pkcs11-tool -l --keypairgen --key-type EC:secp384r1 --label root
The output contains the ID of your key. We need the ID later. So store it in your notes.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Key pair generated:
Private Key Object; EC
label: root
ID: 7f5957cbbb66dc7d0921b4ff4be8c68cff57960a
Usage: sign, derive
Access: none
Public Key Object; EC EC_POINT 384 bits
EC_POINT: 046104490d98dfc96daca53264a66942d0f40d7dee74a0a751ad5bf1fedfed496cacfd03d498e1952c9a9f694eb3393d17f2f952581a2252026a83098776d28780198d5852931f6506d4423f3c98c888eefdebcc9b04bc4eb63be9e2013e8cc042c166
EC_PARAMS: 06052b81040022
label: root
ID: 7f5957cbbb66dc7d0921b4ff4be8c68cff57960a
Usage: verify, derive
Access: none
Create the certificate for the root ca
Next we create a folder structure for our root certificate authority.
1
2
3
4
5
pki_dir=/opt/certificate-authority
mkdir $pki_dir && cd $pki_dir
mkdir certs config crl newcerts intermediate intermediate/certs intermediate/crl intermediate/csr intermediate/newcerts
touch index.txt intermediate/index.txt
cd config
We are now in the config folder of our root ca. Open the configuration for the creation of the root certificate with a text editor
1
nano create_root_cert.ini
Paste the following configuration into the file. You should replace CA_homelab with another name, that suites your environment. Also adapt the section req_distinguished_name to your needs.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
[ ca ]
default_ca = CA_homelab
[ CA_homelab ]
# Directories and config files
dir = /opt/certificate-authority
certs = $dir/certs
crl_dir = $dir/crl
new_certs_dir = $dir/newcerts
database = $dir/index.txt
serial = $dir/serial
# Use SHA-2 instead of deprecated SHA-1
default_md = sha512
# Other configs
name_opt = ca_default
cert_opt = ca_default
default_days = 375
preserve = no
policy = policy_strict
[ policy_strict ]
# Only sign matching intermediate certificates
countryName = match
stateOrProvinceName = match
organizationName = match
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
[ req ]
# Request config
default_bits = 4096
distinguished_name = req_distinguished_name
string_mask = utf8only
prompt = no
# Again SHA-2
default_md = sha512
[ req_distinguished_name ]
C = DE
ST = Bayern
O = Homelab
OU = Homelab Certificate Authority
CN = Homelab Root CA
[ v3_ca ]
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints = critical, CA:true
keyUsage = critical, digitalSignature, cRLSign, keyCertSign
As you can see, we configured the keyUsage and basicConstraints so the resulting certificate is a certificate authority certificate.
Save the file and create the certificate. Please replace the YOUR_KEY_ID_HERE with your key ID. You wrote that down, when you generated the key inside the HSM.
1
openssl req -config create_root_cert.ini -engine pkcs11 -keyform engine -key YOUR_KEY_ID_HERE -new -x509 -days 3650 -sha512 -extensions v3_ca -out ../certs/root.crt
After that you have a secret key, stored in your HSM, and a public certificate on disk. You can have a look at the certificate with the following command.
1
openssl x509 -noout -text -in ../certs/root.crt
Prepare the intermediate ca
Set up a VM with debian bookworm. You can use another distribution, if you want. But you will have to adapt some commands or the package names accordingly. Connect the HSM module, that will contain your intermediate key, to your VM.
After the installation of the basic system, login as root and install the needed packages. They are the same as on the air gap system
1
apt install libengine-pkcs11-openssl gnutls-bin opensc pcscd opensc-pkcs11
With these packages the openssl commands can access the HSM on the intermediate VM.
Initialize the HSM 2 for the intermediate ca
Please be aware, that this step will delete all previous data of the HSM! So skip it, if you only test the setup and have only one HSM device for root ca and intermediate ca, which is not a good idea for production. This is again the moment, you have to decide about a backup strategy with a Device Key Encryption Key or n-of-m schemes. This will not be possible after the initialization anymore.
In this example, I won’t use a backup.
You will be asked for a SO-PIN (Security Officer PIN) and a PIN for usage again. Choose an appropriate label for your environment, so you can recognize your intermediate ca.
1
pkcs11-tool --module /usr/lib/x86_64-linux-gnu/opensc-pkcs11.so --init-token --init-pin --label="IntermediateCA" --login
Create the private key for the intermediate ca in the HSM 2
After the HSM is set up, we can create a private key on it for our intermediate ca. Again we will use an elliptic curve algorithm. The label is a name for your key. I just call it intermediate. You will be asked for the user PIN.
1
pkcs11-tool -l --keypairgen --key-type EC:secp384r1 --label intermediate
The output contains the ID of your key. We need the ID later. So store it in your notes.
1
2
3
4
5
6
7
8
9
10
11
12
13
Key pair generated:
Private Key Object; EC
label: intermediate
ID: 7d02839fee4987c62639b86970438b09f8e5801a
Usage: sign, derive
Access: none
Public Key Object; EC EC_POINT 384 bits
EC_POINT: 0461044c7aea5f45c271847242103856df876661bbe4b75c1fc76684dc98b6003f8ddb2487d1e540514b61f3f52e1df28998bee5ad0fc4b1a2ec33dd8850252b30a603ca1bd026cb4833a77088ab0b9c3caccd7c9d0b43027965224d7257ddd9fdd609
EC_PARAMS: 06052b81040022
label: intermediate
ID: 7d02839fee4987c62639b86970438b09f8e5801a
Usage: verify, derive
Access: none
Create the certificate signing request for the intermediate ca
Next we create a folder structure to create a certificate signing request for our intermediate certificate authority.
1
2
3
4
5
pki_dir=/opt/certificate-authority
mkdir $pki_dir && cd $pki_dir
mkdir certs config crl newcerts intermediate intermediate/certs intermediate/crl intermediate/csr intermediate/newcerts
touch index.txt intermediate/index.txt
cd config
We are now in the config folder of our intermediate ca certificate. Open the configuration for the creation of the intermediate certificate signing request with a text editor
1
nano create_intermediate_csr.ini
Paste the following configuration into the file. You should replace CA_homelab with another name, that suites your environment. Also adapt the section req_distinguished_name to your needs.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
[ req ]
default_bits = 4096
distinguished_name = req_distinguished_name
string_mask = utf8only
prompt = no
# Again SHA-2
default_md = sha512
[ v3_ca ]
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints = critical, CA:true
keyUsage = critical, digitalSignature, cRLSign, keyCertSign
# Again SHA-2
default_md = sha512
[ req_distinguished_name ]
C = DE
ST = Bayern
O = Homelab
OU = Homelab Certificate Authority
CN = Homelab Intermediate CA
As you can see, we configured the keyUsage and basicConstraints so the resulting certificate is a certificate authority certificate in the same way as in the root ca.
Save the file and create the certificate signing request. Please replace the YOUR_KEY_ID_HERE with your key ID. You wrote that down, when you generated the key inside the HSM for the intermediate ca.
1
openssl req -config create_intermediate_csr.ini -engine pkcs11 -keyform engine -key YOUR_KEY_ID_HERE -new -sha512 -out ../intermediate/csr/intermediate.csr
You can have a look at your request with the following command.
1
openssl req -text -noout -verify -in ../intermediate/csr/intermediate.csr
Copy the CSR to your air gap system in a secure way.
Sign the certificate signing request
The CSR from the intermediate ca has to be copied to the air gap system. We log in to the air gap system
Next we have to find the exact location of the key inside the HSM of the root ca. The model, manufacturer and serial depend on your HSM. This one work for the HSM 2 from NitroKey. The token is the label you gave your HSM at initialization followed by a string that indicates, that the UserPIN should be used to authenticate.
1
p11tool --list-all pkcs11:model=PKCS%2315%20emulated;manufacturer=www.CardContact.de;serial=DENK0105969;token=RootCA%20%28UserPIN%29
The output looks like the following snippet.
1
2
3
4
5
6
7
Object 0:
URL: pkcs11:model=PKCS%2315%20emulated;manufacturer=www.CardContact.de;serial=DENK0105969;token=RootCA%20%28UserPIN%29;id=%7F%59%57%CB%BB%66%DC%7D%09%21%B4%FF%4B%E8%C6%8C%FF%57%96%0A;object=root;type=public
Type: Public key (EC/ECDSA-SECP384R1)
Label: root
Flags: CKA_EXTRACTABLE;
ID: 7f:59:57:cb:bb:66:dc:7d:09:21:b4:ff:4b:e8:c6:8c:ff:57:96:0a
You have to note URL of your root key. I labeled mine root so in this example I use the URL of the Object 0 with the label root.
Switch to the config folder of your root ca and edit the config for the signing of your CSR.
1
2
cd /opt/certificate-authority/config/
nano sign_intermediate_csr.ini
Paste the following configuration into the file. You should replace CA_homelab with another name, that suites your environment. Also replace the private_key with the URL of your root key.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
[ ca ]
default_ca = CA_homelab
[ CA_homelab ]
# Directories and configs
dir = /opt/certificate-authority
certs = $dir/certs
crl_dir = $dir/crl
new_certs_dir = $dir/newcerts
database = $dir/index.txt
serial = $dir/serial
# Root key on NitroKey and root certificate
private_key = pkcs11:model=PKCS%2315%20emulated;manufacturer=www.CardContact.de;serial=DENK0105969;token=RootCA%20%28UserPIN%29;id=%7F%59%57%CB%BB%66%DC%7D%09%21%B4%FF%4B%E8%C6%8C%FF%57%96%0A;object=root;type=public
certificate = $certs/root.crt
# Again SHA-2
default_md = sha512
name_opt = ca_default
cert_opt = ca_default
default_days = 375
preserve = no
policy = policy_loose
[ policy_loose ]
countryName = optional
stateOrProvinceName = optional
localityName = optional
organizationName = optional
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
[ v3_intermediate_ca ]
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints = critical, CA:true, pathlen:0
keyUsage = critical, digitalSignature, cRLSign, keyCertSign
As you can see, we allowed the signed certificate to act as certificate authority. Save the file and sign the request.
1
openssl ca -config sign_intermediate_csr.ini -engine pkcs11 -keyform engine -extensions v3_intermediate_ca -days 1825 -notext -md sha512 -create_serial -in ../intermediate/csr/intermediate.csr -out ../intermediate/certs/intermediate.crt
After the signing is done and the certificate is created, you can have a look at it with the following command.
1
openssl x509 -noout -text -in ../intermediate/certs/intermediate.crt
Now we will create a certificate chain, since that is needed on the intermediate ca.
1
cat ../intermediate/certs/intermediate.crt ../certs/root.crt > ../intermediate/certs/chain.crt
Copy the root certificate, the intermediate certificate and the certificate chain to your laptop or other system you work on. They will be needed later. Copy all certificates to your intermediate ca system.
At this moment the root ca can be shut down and the system and the root HSM can be stored in a safe or other secure location.
Compile the step-ca
The release of the step-ca binary does not have the support for PKCS #11. So we can either use the container image that supports it (step-ca:hsm) or we compile the binary ourselves. Since we don’t want to run unchecked container images in production, we take the compilation route.
I assume you have Debian on your laptop, or you have another VM to compile the code. In a very secure intermediate ca system, I would not compile it on the ca system itself, because you want to harden that system and don’t want a lot of development tools on it. But for this demo it would be ok to use the intermediate ca.
First clone the smallstep project called certificates
1
2
git clone git@github.com:smallstep/certificates.git
cd certificates
When we prepare the build tools. Some can be installed with apt, but go has to be fetched and installed manually.
1
2
3
4
5
6
7
8
sudo apt install make gcc ack libpcsclite-dev pkg-config
GO_VERSION=1.23.1
curl -LO https://golang.org/dl/go$GO_VERSION.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go$GO_VERSION.linux-amd64.tar.gz
cat<<EOF | sudo tee /etc/profile
export PATH=\$PATH:/usr/local/go/bin
EOF
source /etc/profile
After the source command the go command is available in your current open bash. You can build the step-ca binary with make. The CGO_ENABLED flag is very important, since this enables the support for HSM 2 via PKCS #11.
1
make bootstrap && make build GO_ENVS="CGO_ENABLED=1"
Depending on your system it can take some time to prepare and compile everything. When the make command finishes, you find your binary in the bin folder of the certificates project. Copy that binary to the local bin folder of your intermediate ca system and make it executable.
It should look like this.
1
2
$ ls -l /usr/local/bin/step-ca
-rwxr-xr-x 1 root root 49845136 Oct 1 07:55 /usr/local/bin/step-ca
Install step-ca
Go to the intermediate ca system and make sure, that step-ca is executable.
1
step-ca --version
Services should run with the lowest privileges possible, to minimize the impact of attacks. So first create a system user for the step-ca and create a folder structure for the service.
1
2
3
4
5
6
useradd -d /etc/step-ca/ -s /bin/false -c "smallstep ca user" --system step
mkdir -m 0700 /etc/step-ca/
mkdir -m 0700 /etc/step-ca/db
mkdir -m 0700 /etc/step-ca/config
mkdir -m 0700 /etc/step-ca/certs
mkdir -m 0700 /etc/step-ca/secrets
The configuration needs the URL of your intermediate certificate. We used a similar command for the root ca HSM. This time use the label of your intermediate HSM.
1
p11tool --list-all pkcs11:model=PKCS%2315%20emulated;manufacturer=www.CardContact.de;serial=DENK0105969;token=IntermediateCA%20%28UserPIN%29
Note the URL of the key with the correct label. Mine was intermediate.
1
2
3
4
5
6
Object 0:
URL: pkcs11:model=PKCS%2315%20emulated;manufacturer=www.CardContact.de;serial=DENK0105969;token=RootCA%20%28UserPIN%29;id=%7D%02%83%9F%EE%49%87%C6%26%39%B8%69%70%43%8B%09%F8%E5%80%1A;object=intermediate;type=public
Type: Public key (EC/ECDSA-SECP384R1)
Label: intermediate
Flags: CKA_EXTRACTABLE;
ID: 7d:02:83:9f:ee:49:87:c6:26:39:b8:69:70:43:8b:09:f8:e5:80:1a
Now create a configuration for the step-ca. Normally you could initialize the config with step-ca init but with a config file, you can use automation tools like Ansible or Puppet to securely deploy your intermediate ca.
1
nano /etc/step-ca/config/ca.json
Put the following input into the file and save it.
The key value has to be replaced with your URL except, that you have to replace the type=public with type=private. That’s important, because you want to access the private key and not the public key.
The uri also has to be changed. The token has to be the label you gave the HSM of your intermediate ca.
And you have to provide your pin-value. This is needed, since the step-ca needs the usage rights to send CSRs into the HSM and get certificates out of it.
You may also want to change the name of your ACME provisioner. The name will also be used in the URL for requesting certificates.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
{
"root": "/etc/step-ca/certs/root.crt",
"federatedRoots": null,
"crt": "/etc/step-ca/certs/intermediate.crt",
"key": "pkcs11:id=%7D%02%83%9F%EE%49%87%C6%26%39%B8%69%70%43%8B%09%F8%E5%80%1A;object=intermediate;type=private",
"address": ":443",
"insecureAddress": "",
"kms": {
"type": "pkcs11",
"uri": "pkcs11:module-path=/usr/lib/x86_64-linux-gnu/opensc-pkcs11.so;token=IntermediateCA%20%28UserPIN%29?pin-value=my-secret-user-pin"
},
"dnsNames": [
"localhost",
"ca"
],
"logger": {
"format": "text"
},
"db": {
"type": "badgerv2",
"dataSource": "/etc/step-ca/db",
"badgerFileLoadingMode": ""
},
"authority": {
"claims": {
"minTLSCertDuration": "5m",
"maxTLSCertDuration": "168h",
"defaultTLSCertDuration": "168h",
"disableRenewal": false,
"minHostSSHCertDuration": "5m",
"maxHostSSHCertDuration": "1680h",
"defaultHostSSHCertDuration": "720h",
"minUserSSHCertDuration": "5m",
"maxUserSSHCertDuration": "48h",
"defaultUserSSHCertDuration": "16h"
},
"provisioners": [
{
"type": "ACME",
"name": "homelab"
}
]
},
"tls": {
"cipherSuites": [
"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",terminal
"TLS_AES_128_GCM_SHA256",
"TLS_AES_256_GCM_SHA384",
"TLS_CHACHA20_POLY1305_SHA256"
],
"minVersion": 1.2,
"maxVersion": 1.3,
"renegotiation": false
}
}
This config file contains the user PIN of your HSM. Therefore, you should only allow the service user access to it.
1
chmod 0600 /etc/step-ca/config/ca.json
Now place the certificates you got from your root ca into the correct folder.
1
2
3
4
5
6
7
ls -lah /etc/step-ca/certs/
total 20K
drwx------ 2 step step 4.0K Oct 1 07:54 .
drwx------ 6 step step 4.0K Oct 1 07:50 ..
-rw------- 1 step step 1.9K Oct 5 11:37 chain.crt
-rw------- 1 step step 952 Oct 5 11:37 intermediate.crt
-rw------- 1 step step 936 Oct 5 11:20 root.crt
The certificates should also only be changeable by the service itself. You don’t want to deliver the wrong root certificate to your clients.
1
chmod 0600 /etc/step-ca/certs/*
Also include the root certificate in the truststore, so that the ca can connect to TLS encrypted site with a certificate from out PKI. On Debian we use the ca-certificates package for that.
1
2
3
4
apt install ca-certificates
mkdir /usr/local/share/ca-certificates/homelab
cp /etc/step-ca/certs/root.crt /usr/local/share/ca-certificates/homelab/
update-ca-certificates
Finally, we change the owner of the whole step-ca installation to the service user.
1
chown -R step:step /etc/step-ca
Next we set up the binary. You copied it to /usr/local/bin/step-ca. Now we allow it to use a low port number without being root.
1
setcap 'CAP_NET_BIND_SERVICE=ep' /usr/local/bin/step-ca
We could start the service now, but for a stable service we should create a systemd service. Open the config file.
1
nano /etc/systemd/system/step-ca.service
Now paste the following content into it and save it.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
[Unit]
Description=step-ca service
Documentation=https://smallstep.com/docs/step-ca
Documentation=https://smallstep.com/docs/step-ca/certificate-authority-server-production
After=network-online.target
Wants=network-online.target
StartLimitIntervalSec=30
StartLimitBurst=3
ConditionFileNotEmpty=/etc/step-ca/config/ca.json
[Service]
Type=simple
User=step
Group=step
Environment=STEPPATH=/etc/step-ca
WorkingDirectory=/etc/step-ca
ExecStart=/usr/local/bin/step-ca config/ca.json
ExecReload=/bin/kill --signal HUP $MAINPID
Restart=on-failure
RestartSec=5
TimeoutStopSec=30
StartLimitInterval=30
StartLimitBurst=3
; Process capabilities & privileges
AmbientCapabilities=CAP_NET_BIND_SERVICE
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
SecureBits=keep-caps
NoNewPrivileges=yes
; Sandboxing
ProtectSystem=full
ProtectHome=true
RestrictNamespaces=true
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
PrivateTmp=true
PrivateDevices=true
ProtectClock=true
ProtectControlGroups=true
ProtectKernelTunables=true
ProtectKernelLogs=true
ProtectKernelModules=true
LockPersonality=true
RestrictSUIDSGID=true
RemoveIPC=true
RestrictRealtime=true
SystemCallFilter=@system-service
SystemCallArchitectures=native
MemoryDenyWriteExecute=true
ReadWriteDirectories=/etc/step-ca/db
[Install]
WantedBy=multi-user.target
Basically we tell step-ca to use the configuration folders and files we created earlier.
Again we don’t want anyone else to modify the service. Therefore, we change the owner and usage rights.
1
2
chown -R step:step /etc/systemd/system/step-ca.service
chmod 0600 /etc/systemd/system/step-ca.service
The last setup step on the intermediate ca is to start the pcsd and the step-ca service. Look at the output of the status command, if everything is running as expected.
1
2
3
4
5
6
7
systemctl start pcscd
systemctl enable pcscd
systemctl status pcscd
systemctl start step-ca
systemctl enable step-ca
systemctl status step-ca
To make our demo work, we need the ca recognize webserver as a domainname, we have to add it to the /etc/hosts file.
1
nano /etc/hosts
The demo webserver
Go to the VM with the webserver.
The webserver needs to connect to the ca. So add the IP of the ca to the /etc/hosts file.
1
nano /etc/hosts
Install the webserver nginx and the ca-certificates package.
1
apt install nginx ca-certificates
Copy the root certificate to the ca-certificates folder and update the certificates. I copied mine wie ssh to /tmp/root.crt and when to the folder. But this depends on how you can copy files to your VM. This step is needed for the ACME client to connect to the TLS protected ACME endpoint of the intermediate CA.
1
2
3
mkdir /usr/local/share/ca-certificates/homelab
cp /tmp/root.crt /usr/local/share/ca-certificates/homelab/
update-ca-certificates
Make two folders for the key and cert for nginx.
1
2
mkdir -p /etc/ssl/private/
mkdir -p /etc/ssl/certs
Then open the config for nginx.
1
nano /etc/nginx/sites-enabled/default
Paste the following content into it and save it.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
server {
listen 80 default_server;
listen [::]:80 default_server;
# listen 443 ssl default_server;
# listen [::]:443 ssl default_server;
# ssl_certificate /etc/ssl/certs/webserver.crt;
# ssl_certificate_key /etc/ssl/private/webserver.key;
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
server_name webserver;
location / {
try_files $uri $uri/ =404;
}
}
Now nginx would be ready, if there were any certificates. We can start it already, so the ACME client can use it for the challenge.
1
2
3
systemctl start nginx
systemctl enable nginx
systemctl status nginx
After that, install acme.sh. You can also use other ACME clients, if you want to. acme.sh is written in bash and has the ability to use nginx for challenges. It is a bad idea to pipe a bash script directly into a root shell. So download it first, look into it and execute it after you don’t find any strange things.
1
2
3
4
wget https://get.acme.sh/ -O install.sh
nano install.sh
sh install.sh
source "/root/.acme.sh/acme.sh.env"
After acme.sh is installed, we can call it to issue a certificate for the webserver. The following command will use the nginx mode, to solve challenges. After the renewal of a certificate, nginx will be reloaded with the reloadcmd. The server points to the intermediate ca with the acme provisioner called homelab. If you changed the name of the provisioner, you have to change it in that URL, too.
1
acme.sh --issue -k ec-384 --nginx --reloadcmd "systemctl reload nginx.service" --server https://ca/acme/homelab/directory --days 1 -d webserver --key-file /etc/ssl/private/webserver.key --fullchain-file /etc/ssl/certs/webserver.crt
After the command is executed successfully we enable TLS in our nginx config.
1
nano /etc/nginx/sites-enabled/default
Then change the content.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
server {
listen 80 default_server;
listen [::]:80 default_server;
listen 443 ssl default_server;
listen [::]:443 ssl default_server;
ssl_certificate /etc/ssl/certs/webserver.crt;
ssl_certificate_key /etc/ssl/private/webserver.key;
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
server_name webserver;
location / {
try_files $uri $uri/ =404;
}
}
After that, reload nginx.
1
2
systemctl reload nginx
systemctl status nginx
Browser setup
Open your Firefox and goto settings. Search for certificates and click on show certificates. Import the root certificate to the certificate authorities. Restart Firefox.
Now edit your /etc/hosts file so that webserver is the hostname of your webserver VM.
Open https://webserver in Firefox.
Yeah. The chain of trust is valid now.
Optional: Install step-cli
There is also a step-cli that can help you with different tasks like fetching the root certificate, requesting certificates or verifying stuff. If you want to install it, you should fetch cosign first, to verify the packages.
1
2
3
4
5
6
cosign_version=2.4.0
wget https://github.com/sigstore/cosign/releases/download/v${cosign_version}/cosign_${cosign_version}_amd64.deb
wget https://github.com/sigstore/cosign/releases/download/v${cosign_version}/cosign_${cosign_version}_amd64.deb-keyless.pem
wget https://github.com/sigstore/cosign/releases/download/v${cosign_version}/cosign_${cosign_version}_amd64.deb-keyless.sig
dpkg -i cosign_${cosign_version}_amd64.deb
cosign verify-blob --certificate cosign_${cosign_version}_amd64.deb-keyless.pem --signature cosign_${cosign_version}_amd64.deb-keyless.sig --certificate-identity "keyless@projectsigstore.iam.gserviceaccount.com" --certificate-oidc-issuer "https://accounts.google.com" cosign_${cosign_version}_amd64.deb
After you installed and verified the cosign package, you can download, verify and install the step-cli.
1
2
3
4
5
6
7
step_version=0.27.4
wget https://dl.smallstep.com/gh-release/cli/gh-release-header/v${step_version}/step-cli_${step_version}-1_amd64.deb
wget https://github.com/smallstep/cli/releases/download/v${step_version}/step-cli_${step_version}-1_amd64.deb.pem
wget https://github.com/smallstep/cli/releases/download/v${step_version}/step-cli_${step_version}-1_amd64.deb.sig
cosign verify-blob --certificate step-cli_${step_version}-1_amd64.deb.pem --signature step-cli_${step_version}-1_amd64.deb.sig --certificate-identity-regexp "https://github\.com/smallstep/workflows/.*" --certificate-oidc-issuer https://token.actions.githubusercontent.com step-cli_${step_version}-1_amd64.deb
dpkg -i step-cli_${step_version}-1_amd64.deb
step version
Conclusion
In this longer article we learned about Public Key Infrastructure and how to set up a private certificate authority with support for ACME and hardware security modules. With this information you should be able to set up your own lab to test it.
Please contact me, if you have any questions or there is something wrong. I tried to convert my notes for the session to an article as goog as possible, but there is surely room for improvement. So feedback is welcome.
See you soon,
seism0saurus