Skip to content

Optimize (D)TLS 1.3 handshakes and disable session tickets - #1654

Open
aronrosenberg wants to merge 1 commit into
paullouisageneau:masterfrom
aronrosenberg:optimize_tls13_handshakes
Open

aronrosenberg wants to merge 1 commit into
paullouisageneau:masterfrom
aronrosenberg:optimize_tls13_handshakes

Conversation

@aronrosenberg

@aronrosenberg aronrosenberg commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Key exchange groups

The OpenSSL transports only offered P-256, unlike the GnuTLS and Mbed TLS backends, whose defaults already offer P-256, X25519, P-384 and more. This has two effects:

  • (D)TLS 1.3: Chrome and libwebrtc send an X25519 key share in their ClientHello, which a libdatachannel server answered with a HelloRetryRequest, causing an extra round trip to occur. This happens with TLS WebSockets on OpenSSL 3.x, and with DTLS on OpenSSL 4.1, which negotiates DTLS 1.3 by default.
  • (D)TLS 1.2: the client's groups also restrict the server's certificate curve (RFC 8422 Section 5.3), so servers with a P-384 certificate were unreachable.

Both transports now offer P-256:X25519:P-384

  • P-256 first, as it is a MUST for WebRTC (RFC 8827) and (D)TLS 1.3 (RFC 8446), so no compliant peer needs a retry.
  • X25519, a SHOULD in RFC 8446 and RFC 9325, which browsers offer first.
  • P-384, for P-384 certificates under (D)TLS 1.2.

If the OpenSSL provider lacks a group (X25519 in FIPS mode, for instance), the transports fall back to P-256 alone.

Session tickets

Session tickets are now disabled in all backends, and the session cache in OpenSSL, since WebRTC DTLS sessions are never resumed. With OpenSSL 3.x, this removes about 480 bytes from the server's DTLS 1.2 handshake. Disabling session tickets matches libwebrtc's behavior.

Logging

The handshake log line now includes the negotiated version, for example DTLS handshake finished (DTLSv1.3).

Tests

The new Key exchange groups test runs libwebrtc's default groups, the old P-256-only list and the new list against each other, as client and server, over (D)TLS 1.2 and 1.3. It checks the negotiated group, where HelloRetryRequests occur, and handshakes with a P-384 certificate. It runs in memory and skips on non-OpenSSL backends.

The OpenSSL DTLS and TLS transports offered the P-256 key exchange
group only. A (D)TLS 1.3 client sends a key share for its first group
only and browsers pick X25519, so a libdatachannel server answered
every browser with a HelloRetryRequest, costing a round trip. This
already happened for TLS 1.3 WebSockets with OpenSSL 3.x and now
happens for DTLS 1.3 with OpenSSL 4.1. In (D)TLS 1.2, the groups also
restrict the curve of the server's ECDSA certificate (RFC 8422 Section
5.3), so servers with a P-384 certificate could not be reached at all.

Both transports now offer P-256:X25519:P-384, most preferred first.
P-256 is a MUST for WebRTC (RFC 8827) and (D)TLS 1.3 (RFC 8446), so
leading with it means no compliant peer needs a retry. X25519 is a
SHOULD for (D)TLS 1.3 (RFC 8446) and in BCP 195 (RFC 9325), and is
what browsers offer first. P-384 covers servers with P-384
certificates under (D)TLS 1.2. If the OpenSSL provider lacks a group,
X25519 in FIPS mode for instance, P-256 alone is used as before. The
GnuTLS and Mbed TLS defaults already include all three groups.

Session tickets and the server session cache are now disabled, since
in WebRTC DTLS sessions are never resumed. With OpenSSL 3.x, this
removes about 480 bytes from the server's DTLS 1.2 handshake.

The handshake log line now includes the negotiated (D)TLS version.

A new "Key exchange groups" test runs a Chrome-like configuration
(libwebrtc's default groups), the previous P-256-only configuration
and the new one against each other as client and server, over
(D)TLS 1.2 and 1.3. It checks the negotiated group, where
HelloRetryRequests occur, and handshakes with a server whose
certificate uses P-384.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant