TLS Server Signature Algorithm
In this post we’ll look at what the TLS Server Signature Algorithm is for.
This guide can get technical at times so it’s recommended that you understand the follow terms as they are assumed knowledge:
- RSA (Rivest–Shamir–Adleman)
- ECC (Elliptic-Curve-Cryptography)
- Symmetric Cryptography (i.e. AES)
- Asymmetric Cryptography (Public/Private Key pair)
- DH & ECDH (Diffie-Hellman & Elliptic-Curve Diffie-Hellman)
- Hash Function (i.e. SHA1, SHA256, SHA512)
- CA (Certificate Authority)
- Host Certificate (Certificate used for a web-server)
Contents
- SSLScan Results
- What are Server Signature Algorithms?
- Server Signature Algorithm Breakdown
- Edge Case 1
- Edge Case 2
- Conclusion
SSLScan Results
Below we have the Server Signature Algorithm(s) and SSL Certificate sections from a SSLScan of a web server:
SSL Certificate Section Breakdown
The Signature Algorithm: sha384WithRSAEncryption line indicates that the Private Key used to sign this Host Certificate is an RSA key, and the hash function used for the integrity check is SHA384. This line DOES NOT tell us anything about the Host Certificate, only what was used to sign it (i.e. the CA).
The line RSA Key Strength: 4096 tells us what type of algorithm was used to create the Host Certificate Keys. In this case it was RSA with a 4096-bit key. This is important because we now know that the Private Key for this Host Certificate is an RSA key, rather than an ECC key.
What are Server Signature Algorithms used for?
When using TLS, messages sent by the client and server are encrypted using symmetric keys. To do this the client and server both need to be able to calculate the same symmetric keys. This is achieved by exchanging Public keys (using DH or ECDH) during the TLS connection setup. These Public keys are NOT the same as the Host Certificate Public key, these keys are ephemeral (temporary) and are randomly generated at the start of each new connection to provide Perfect Forward Secrecy.
During the connection setup phase, the server must generate a Public/Private key pair and send the Public key to the client. However, since this Public key has just been generated, it’s not safe to sent to the client as is. This is because the client has no proof that the key was generated by the server. To solve this the server hashes and signs this new ephemeral Public key using the Private key of the Host Certificate.
Below is a diagram of how the signing and verification works. The ephemeral Public key as well as some other information sent during the TLS connection setup is the input data for the Hash Function. The resulting hash is then encrypted using the Private key of the Host Certificate to produce a signature (left diagram). The ephemeral Public key and the signature are then sent to the client. The client can verify that that ephemeral Public key is from the server by checking the signature. The signature verification can be done using the Host Certificate Public key (right diagram).
We can now ready to understand the Server Signature Algorithm(s) section of the SSLScan. The algorithms listed are the algorithms that the server supports for signing the ephemeral Public Key in the process that was explained above.
Server Signature Algorithm Breakdown
Below we have the same scan results as above just with one of the algorithms highlighted. Let’s break down what the highlight algorithm means.
rsa_pkcs1_sha256 means that the ephemeral Public key will be encrypted using RSA. PKCS1 indicates the standard that is being used. SHA256 is the hashing algorithm that is used for the integrity check of the data.
Notice that all of the Server Signature Algorithms have something in common. They all use RSA. So why are they all RSA and not ECDSA or DSA?
Remember earlier we said the Host Certificate uses RSA by observing the SSL Certificate section. This means the Host Private key is an RSA key. Therefore, when we encrypt something using the Host Private key, it must use RSA encryption.
To think about it another way, let’s imagine that one of the algorithms listed in the Server Signature Algorithm(s) section said it uses ECDSA instead of RSA. This would imply that the ephemeral Public key sent during the TLS setup was signed using an ECC key. This is not possible because the Host Private key is an RSA key not an ECC key.
Edge Case 1
SSLScan False Positives
Unfortunately SSLScan is not perfect, and sometimes we get incorrect results. Let’s look at how to identify false positives and found out the correct results.
Have a look at the SSLScan results of adventofcode.com below. See if you can identify what is wrong:
From the SSL Certificate section we can see that this Host Certificate is RSA. Because of this, and what we learnt about what the Signature Algorithms are for, it doesn’t make sense for there to be ECDSA or DSA algorithms.
If this happens then there has most likely been an error with SSLScan. We can verify the results using manually using openssl. Let’s see how to do that now.
Verifying False Positives
Using openssl we can connect to a target and specify the Signature Algorithm we wish to use. If you’re unsure how to do this, I have a quick guide with all the commands here:
Testing dsa_sha256:
Testing ecdsa_secp256r1_sha256:
Testing rsa_pkcs1_sha256:
Skipping over some of the output:
At the end of the Certificate we can see the information about the Server Signature. The Peer signing digest: is set to SHA256, which is the hash function we specified. The Peer signature type: is set to RSA which is also what we specified.
From the above outputs we can see that the server does indeed only supported RSA Signature Algorithms (as it should). We now know there were false positives within the SSLScan.
Edge Case 2
Dual Certificates
Below is the SSLScan results of facebook.com below. Just like the first edge case we can see that there are signature algorithms that don’t match the host certificate. The Host Certificate is ECC and it doesn’t make sense for there to be RSA algorithms under the Server Signature Algorithm(s) section.
Using the same process as the first edge case, let’s see what’s happening:
Testing rsa_pkcs1_sha256:
1
openssl s_client -connect facebook.com:443 -sigalgs "RSS-PSS+SHA256"
Skipping to the end of the certificate output:
Testing ecdsa_secp256r1_sha256:
1
openssl s_client -connect facebook.com:443 -sigalgs "ECDSA+SHA256"
Skipping to the end of the certificate output:
This output is perplexing. Why did both signature algorithms work when the certificate is ECC?
How does this make sense?
As always, the devil is in the details. If we closely examine the output from both commands you can see that the certificates returned are different!
This makes more sense. The server is sending us a different certificate based on what signature algorithm is used. When we use an RSA Signature Algorithm, we get an RSA Certificate. When we use an ECC Signature Algorithm, we get an ECC Certificate.
Conclusion
- The Signature Algorithm is used for encrypting and signing the ephemeral public key during the TLS connection setup.
- SSLScan is not 100% reliable. OpenSSL should be used to verify results manually if the results don’t appear to make sense.