How TLS works, and when it can be decrypted

A common question on any team that runs its own servers goes like this: “We own the server and we have the certificate, so can’t we capture the packets and read the HTTPS traffic in Wireshark?”

The answer depends on what “certificate” means, and on which key exchange the connection used. Getting from that question to a full answer takes in most of how TLS works, so these notes follow that path.

What TLS provides

TLS gives a connection three properties:

Client authentication is optional, and is the basis of mutual TLS. TLS doesn’t hide everything, though. IP addresses, ports, timing and message sizes stay visible, and so does the server name unless Encrypted Client Hello is in use.

TLS normally runs on top of TCP. QUIC, which carries HTTP/3, uses the TLS 1.3 handshake inside its own transport.

The two files

A TLS server has two separate things.

server.crt is public. The server sends it to every client that connects. It contains the domain names it’s valid for, validity dates, the public key, and a signature from whoever issued it.

server.key is secret. It stays on the server or load balancer.

So a certificate alone can’t decrypt anything, because every observer on the network can already see it. When engineers say “decrypt it with our certificate”, they mean the certificate and private key together.

A test pair can be generated and inspected like this:

openssl req -x509 -newkey rsa:2048 -nodes -keyout server.key -out server.crt -days 365 -subj "/CN=localhost"
openssl x509 -in server.crt -noout -text

In our test, server.crt was 1,115 bytes and server.key 1,704 bytes. The output shows rsaEncryption as the key type and sha256WithRSAEncryption as the signature.

How a certificate earns trust

Anyone can create a certificate. What makes one trusted is a chain back to a root that the client already has.

Root CA          stored in the OS or browser trust store
   | signs
Intermediate CA  sent by the server
   | signs
Leaf certificate api.example.com, sent by the server

To validate a certificate, a client checks that:

1. Each certificate’s signature verifies against the next one up.
2. The chain ends at a trusted root.
3. Each certificate is within its validity dates.
4. The hostname matches a Subject Alternative Name on the leaf.
5. Key usage and other constraints allow this use.

A server has to send its intermediates. A missing intermediate is a common failure, and it can work in some browsers while failing in API clients, because some clients fetch or cache intermediates and others don’t.

Lifetimes are shrinking. Under CA/Browser Forum ballot SC-081v3, the maximum lifetime for public certificates fell to 200 days on 15 March 2026. It falls to 100 days in March 2027 and 47 days in March 2029 (DigiCert). In practice, that makes automated renewal a requirement.

Internal systems often run a private CA instead, which only their own machines trust.

The building blocks

TLS combines a few types of cryptography, each with one job.

Public-key operations are far slower than symmetric encryption, so TLS uses them only to set things up. The speed can be measured directly:

openssl speed -seconds 1 rsa2048
openssl speed -seconds 1 -evp aes-128-gcm
openssl speed -seconds 1 -evp chacha20-poly1305

AES is fast here because the CPU has dedicated AES instructions. ChaCha20 is designed to be fast in software, which matters on hardware without them. RSA’s split is also why servers carry more load than clients: the private-key side is the slow one.

RSA and Diffie-Hellman with tiny numbers

Real keys are too large to follow by hand. These examples use the real math with tiny numbers. None of this is secure.

RSA setup:

• Primes: $p = 7$, $q = 13$
• Modulus: $N = 7 \times 13 = 91$
• Public key: $e = 5$
• Private key: $d = 29$

The pair works because $(7-1)(13-1) = 72$, and $5 \times 29 = 145 = 2 \times 72 + 1$.

• Encrypt: $C = M^5 \bmod 91$
• Decrypt: $M = C^{29} \bmod 91$

Python’s pow(32, 29, 91) computes the second, and returns 2.

Static RSA: the old key exchange

In older TLS 1.2 suites such as TLS_RSA_WITH_AES_128_GCM_SHA256, the client encrypts a secret directly to the server’s public key.

CLIENT                                                        SERVER
  |--- ClientHello: ClientRandom = 10 ------------------------->|
  |<-- ServerHello: ServerRandom = 20, certificate (e=5, N=91) -|
  |                                                             |
  |  Client picks a secret: PMS = 2                             |
  |  Encrypts it: 2^5 mod 91 = 32                               |
  |--- ClientKeyExchange: 32 ---------------------------------->|
  |                                                             |
  |                       Server decrypts: 32^29 mod 91 = 2     |

Both sides now share the pre-master secret, 2. Real TLS 1.2 mixes it with both randoms in a pseudo-random function to make a master secret, and then the session keys. For the toy version, addition stands in for that function:

Session key = 10 + 20 + 2 = 32

If the cipher simply adds the key to each byte, then "HI" (72, 73) is sent as 104 and 105.

Why a private key decrypts this traffic

A capture contains 10, 20, 32 and then [104, 105]. With only the public key, finding the input to $M^5 \bmod 91 = 32$ is the RSA problem, which is infeasible at real key sizes. With the private key, it’s three steps:

1. Decrypt: $32^{29} \bmod 91 = 2$.
2. Derive: $10 + 20 + 2 = 32$.
3. Subtract: 104 − 32 = 72 (H), and 105 − 32 = 73 (I).

Wireshark does the same thing with real values. It has one serious consequence: anyone who obtains the key later can decrypt every old capture. Traffic recorded years earlier stays readable for as long as someone keeps the recording.

Ephemeral key exchange and forward secrecy

Ephemeral Diffie-Hellman fixes that. Both sides create throwaway values for each connection. With public values $g = 5$ and $p = 23$:

1. The client picks a secret $a = 4$ and sends $A = 5^4 \bmod 23 = 4$.
2. The server picks a secret $b = 3$ and sends $B = 5^3 \bmod 23 = 10$.
3. The client computes $10^4 \bmod 23 = 18$.
4. The server computes $4^3 \bmod 23 = 18$.

Both reach 18, which never crosses the network. The values 4 and 3 are thrown away afterwards.

The server’s private key now does a different job: it signs the handshake, proving the server’s identity. It doesn’t encrypt anything that could unlock the session. So a capture plus server.key still can’t reach 18. This property is forward secrecy: a later key leak doesn’t expose past sessions.

TLS 1.3 removed static RSA entirely. RFC 10015 (July 2026) also says clients must not offer and servers must not select RSA key exchange in TLS 1.2, citing the lack of forward secrecy and exposure to Bleichenbacher’s attack.

The TLS 1.3 handshake

TLS 1.3 completes in one round trip, against two for full TLS 1.2. The client guesses a key exchange group and sends its key share in the first message.

CLIENT                                                SERVER
  |--- ClientHello: random, key share, suites, SNI ------>|
  |                                                       |
  |<-- ServerHello: random, key share --------------------|
  |    ---- everything below is encrypted ----            |
  |<-- EncryptedExtensions -------------------------------|
  |<-- Certificate ---------------------------------------|
  |<-- CertificateVerify ---------------------------------|
  |<-- Finished ------------------------------------------|
  |                                                       |
  |--- Finished ----------------------------------------->|
  |=== application data ==================================|

Each message has a purpose:

• ClientHello offers versions, cipher suites, groups and key shares.
• ServerHello picks one and returns the server’s key share. Both sides can now derive handshake keys.
• EncryptedExtensions carries settings that don’t need to be in the clear.
• Certificate is encrypted in TLS 1.3, unlike TLS 1.2.
• CertificateVerify is a signature over the handshake so far, which proves possession of the private key.
• Finished is a MAC over the handshake, which detects tampering.

If the client guessed the wrong group, the server sends a HelloRetryRequest, which adds a round trip.

A real session

The handshake can be printed with -msg:

openssl s_server -accept 8443 -cert server.crt -key server.key -www -tls1_3
echo Q | openssl s_client -connect localhost:8443 -tls1_3 -msg

Our session, with sizes taken from the output:

OpenSSL reported 2,425 bytes read and 1,529 bytes written, with TLS_AES_256_GCM_SHA384 and the group X25519MLKEM768. The signature type was rsa_pss_rsae_sha256.

Two things stand out. The hellos are large because they carry post-quantum key shares. And the output shows ChangeCipherSpec records, even though TLS 1.3 doesn’t need them: they’re kept for compatibility with middleboxes that expect TLS 1.2-shaped traffic.

The key schedule

TLS 1.3 doesn’t use one key. It builds a tree of secrets with HKDF, using two operations: HKDF-Extract concentrates input into a secret, and HKDF-Expand-Label stretches a secret into labelled outputs (RFC 8446 §7.1).

Pre-shared key, or zeros
        |
        v
  Early Secret          -> early traffic keys, for 0-RTT
        |
   (EC)DHE shared secret mixed in
        v
  Handshake Secret      -> "c hs traffic", "s hs traffic"
        |
        v
  Master Secret         -> "c ap traffic", "s ap traffic"
                           exporter and resumption secrets

Each output also mixes in a hash of the handshake transcript so far. Any change to an earlier message therefore produces different keys, and the handshake fails.

The directions have separate keys. From each traffic secret, TLS derives a key and an IV. A long-lived connection can roll forward with a KeyUpdate message, which derives the next secret from the current one (RFC 8446 §7.2).

Records: how data is carried

After the handshake, data travels in records. Each holds up to $2^{14}$ bytes of plaintext.

| type | version | length | encrypted payload + inner type + padding | tag |

In TLS 1.3, the outer header always says “application data” and a legacy version, and the real content type is hidden inside the encryption.

Each record has a unique nonce, made by XORing a per-direction IV with a 64-bit sequence number. The sequence number isn’t sent. Both sides count, so a dropped, repeated or reordered record fails authentication and ends the connection.

Optional padding can hide the true length, but most traffic doesn’t use it. Sizes and timing can still reveal a lot.

Cipher suites

The naming shows how much the protocol simplified.

TLS 1.3 negotiates key exchange and signatures separately, and defines five suites: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_AES_128_CCM_SHA256 and TLS_AES_128_CCM_8_SHA256. All are AEAD, and all of TLS 1.3’s key exchanges provide forward secrecy unless a plain pre-shared key is used on its own.

Resumption and 0-RTT

A full handshake costs a round trip and public-key work, so TLS can resume. After a handshake, the server can send a session ticket. On reconnecting, the client presents it, and both sides derive keys from a shared resumption secret, usually with a fresh key exchange too.

TLS 1.3 also offers 0-RTT, where the client sends data in its first flight. It saves a round trip, but this early data can be replayed by an attacker who captures it. So it’s only safe for requests that are harmless to repeat, such as an idempotent read (RFC 8446 §8). Early data also has weaker forward secrecy than the rest of the connection.

Extensions that matter

• SNI (Server Name Indication) names the requested host, so one IP address can serve many sites. It’s sent in the clear in a normal ClientHello.
• ALPN negotiates the application protocol, such as h2 or http/1.1.
• Encrypted Client Hello (draft) encrypts the sensitive parts of the ClientHello, including SNI.
• Client certificates enable mutual TLS, where the server also verifies the client. It’s common for service-to-service traffic.

Post-quantum key exchange

A large enough quantum computer could break today’s elliptic-curve and RSA key exchange. Recorded traffic could then be decrypted later, which is called “harvest now, decrypt later”. So key exchange has moved first.

X25519MLKEM768 is a hybrid: it combines classical X25519 with the post-quantum ML-KEM-768. The connection stays safe if either one holds. The cost is size, which is visible in the 1,444-byte ClientHello above.

Client adoption has moved quickly. Cloudflare publishes current figures on Radar. Certificates and signatures are a separate, harder migration, because post-quantum signatures are much larger.

When traffic can be read

Inline termination

NGINX, Envoy or a load balancer holds the certificate and key and ends TLS. It runs the key exchange itself, so it sees plaintext. It can then forward traffic in the clear, or start a new TLS connection to the backend. This is the normal production answer.

Key logging

A TLS library can write its secrets to a file, usually named by SSLKEYLOGFILE. Wireshark reads the file and decrypts a capture. The format is RFC 9850. Each line holds a label, the 32-byte ClientRandom, and a secret:

CLIENT_RANDOM <client random> <master secret>              TLS 1.2
CLIENT_HANDSHAKE_TRAFFIC_SECRET <client random> <secret>   TLS 1.3
SERVER_HANDSHAKE_TRAFFIC_SECRET <client random> <secret>
CLIENT_TRAFFIC_SECRET_0 <client random> <secret>
SERVER_TRAFFIC_SECRET_0 <client random> <secret>
EXPORTER_SECRET <client random> <secret>

Support depends on the library. In our test, Apple’s curl, built with SecureTransport and LibreSSL, created an empty log file, while openssl s_client -keylogfile worked. A key log unlocks every session in it, so it needs the same care as a password file.

Reading at the application boundary

Plaintext exists on both sides of TLS, before encryption and after decryption. Debug logging and tools attached to library functions such as SSL_write read it there, with no capture or keys involved. This needs access to the host.

Forcing static RSA

Older guides suggest limiting a server to static RSA so a passive sensor can use the private key. That removes forward secrecy for all traffic, and RFC 10015 now forbids it. It’s only useful as a lab demonstration.

A walkthrough

All of this runs on localhost, with a throwaway certificate.

Log TLS 1.3 secrets

Start a server:

openssl s_server -accept 8443 -cert server.crt -key server.key -www -tls1_3

Connect from a second terminal and log the secrets:

echo Q | openssl s_client -connect localhost:8443 -tls1_3 -keylogfile k13.log | grep -E 'Negotiated|Cipher is'
cut -d' ' -f1 k13.log

Result in our test: X25519MLKEM768, TLS_AES_256_GCM_SHA384, and all five TLS 1.3 labels listed above.

See static RSA

openssl s_server -accept 8443 -cert server.crt -key server.key -www -tls1_2 -cipher AES128-GCM-SHA256
echo Q | openssl s_client -connect localhost:8443 -tls1_2 -cipher AES128-GCM-SHA256 -keylogfile k12.log | grep -E 'Cipher is|Extended master'

Result in our test: OpenSSL 3.6.5 still allowed this when asked explicitly, with AES128-GCM-SHA256, Extended master secret: yes, and a CLIENT_RANDOM line in the log.

Decrypt a capture

sudo tcpdump -i lo0 -w rsa.pcap port 8443      # lo on Linux
tshark -r rsa.pcap -o "tls.keys_list:127.0.0.1,8443,http,server.key" -Y http
tshark -r tls13.pcap -o "tls.keylog_file:k13.log" -Y http

Expected: the first tshark command shows HTTP from the static RSA capture, and the second shows it from TLS 1.3. Running the private-key command on the TLS 1.3 capture should show nothing, which is forward secrecy in action.

Common failures

openssl s_client -connect host:443 -servername host -showcerts shows the chain a server is actually sending.