OpenVPN client instance : process startup fails if "fragment size" option is set

[r000m5]

Describe the bug

Openvpn client instance process on OPNsense 24.7.6-amd64 won't start if "fragment size" option is set when connecting to a valid server.
I didn't try with any previous opnsense version.

To Reproduce

1. configure a client instance with these parameters
   
   
   

2. Make sure the tunnel gets established with a valid server (the issue is not reproducible otherwise). Example of a working openvpn 2.6.12-bookworm0 on debian Bookworm :

proto udp6
lport 1195
dev tun1
dev-type tun
script-security 3
keepalive 10 60
persist-tun
persist-key
topology subnet 
server 192.168.168.0 255.255.255.0
server-ipv6 fdde:6c68:3589::/64
client-config-dir ccd
#disable-dco
tun-mtu 1420
#fragment 1220
mssfix
tls-server
dh /etc/openvpn/keys/dh2048.pem
ca /etc/openvpn/keys/vpn-s2s-ca.crt 
cert /etc/openvpn/keys/vpn-s2s.pem
key /etc/openvpn/keys/vpn-s2s.key 
log /var/log/openvpn-s2s.log
verb 5 
route-ipv6 2001:db8:3053:100::/57
route-ipv6 fde6:66ba:24ff::/48

/etc/openvpn/ccd/home.acme.com

iroute 192.168.0.0 255.255.224.0
iroute-ipv6 2001:db8:3053:100::/57
iroute-ipv6 fde6:66ba:24ff::/48

3. Check that the tunnel is established and that connectivity through the tunnel is OK

4. Set "fragment size" to 1220 on both client and server (I read on openvpn forums that they have to be identical on client and server side)

5. restart openvpn on the client and on the server side

6. opnsense client instance won't start, it fails :

2024-10-20T19:02:30	Error	openvpn_client6	Cannot open TUN/TAP dev /dev/ovpn6: No such file or directory (errno=2)	
2024-10-20T19:02:30	Warning	openvpn_client6	GDG: problem writing to routing socket: No such process (errno=3)	
2024-10-20T19:02:30	Warning	openvpn_client6	WARNING: if you use --mssfix and --fragment, you should use the "mtu" flag for both or none of of them.	
2024-10-20T19:02:30	Warning	openvpn_client6	NOTE: the current --script-security setting may allow this configuration to call user-defined scripts	
2024-10-20T19:02:30	Warning	openvpn_client6	WARNING: No server certificate verification method has been enabled. See http://openvpn.net/howto.html#mitm for more info.

Expected behavior

openvpn process is started on both the server and client side

Describe alternatives you considered

No alternative :

• if PMTUD is broken somewhere on the internet, TCP sessions that need fragmentation will fail if openvpn is unable to fragment packets.
• if I set tun-mtu to 1500 to avoid fragmentation, DCO is broken (openvpn recommends to set tun-mtu to 1420 with dco)

Relevant log files

/var/log/openvpn/latest.log on the client side

2024-10-20T19:02:30	Error	openvpn_client6	Cannot open TUN/TAP dev /dev/ovpn6: No such file or directory (errno=2)	
2024-10-20T19:02:30	Warning	openvpn_client6	GDG: problem writing to routing socket: No such process (errno=3)	
2024-10-20T19:02:30	Warning	openvpn_client6	WARNING: if you use --mssfix and --fragment, you should use the "mtu" flag for both or none of of them.	
2024-10-20T19:02:30	Warning	openvpn_client6	NOTE: the current --script-security setting may allow this configuration to call user-defined scripts	
2024-10-20T19:02:30	Warning	openvpn_client6	WARNING: No server certificate verification method has been enabled. See http://openvpn.net/howto.html#mitm for more info.

/var/log/openvpn-s2s.log on the server side shows no error

2024-10-20 18:49:42 us=841121 Connection Attempt MULTI: multi_create_instance called
2024-10-20 18:49:42 us=841288 2001:db8:284:d700::2 Re-using SSL/TLS context
2024-10-20 18:49:42 us=841452 2001:db8:284:d700::2 WARNING: if you use --mssfix and --fragment, you should use the "mtu" flag for both or none of of them.
2024-10-20 18:49:42 us=841902 2001:db8:284:d700::2 Control Channel MTU parms [ mss_fix:0 max_frag:0 tun_mtu:1250 tun_max_mtu:0 headroom:126 payload:1600 tailroom:126 ET:0 ]
2024-10-20 18:49:42 us=842261 2001:db8:284:d700::2 Data Channel MTU parms [ mss_fix:0 max_frag:0 tun_mtu:1420 tun_max_mtu:1600 headroom:136 payload:1768 tailroom:562 ET:0 ]
2024-10-20 18:49:42 us=842311 2001:db8:284:d700::2 Fragmentation MTU parms [ mss_fix:1103 max_frag:1191 tun_mtu:1420 tun_max_mtu:1600 headroom:136 payload:1768 tailroom:562 ET:0 ]
RWWWRRRWR2024-10-20 18:49:42 us=886847 2001:db8:284:d700::2 VERIFY OK: depth=1, CN=Easy-RSA CA
2024-10-20 18:49:42 us=887233 2001:db8:284:d700::2 VERIFY OK: depth=0, C=FR, CN=home.acme.com
WR2024-10-20 18:49:42 us=887984 2001:db8:284:d700::2 peer info: IV_VER=2.6.12
2024-10-20 18:49:42 us=888025 2001:db8:284:d700::2 peer info: IV_PLAT=freebsd
2024-10-20 18:49:42 us=888046 2001:db8:284:d700::2 peer info: IV_TCPNL=1
2024-10-20 18:49:42 us=888073 2001:db8:284:d700::2 peer info: IV_MTU=1600
2024-10-20 18:49:42 us=888105 2001:db8:284:d700::2 peer info: IV_NCP=2
2024-10-20 18:49:42 us=888139 2001:db8:284:d700::2 peer info: IV_CIPHERS=AES-256-GCM:AES-128-GCM:CHACHA20-POLY1305
2024-10-20 18:49:42 us=888171 2001:db8:284:d700::2 peer info: IV_PROTO=990
2024-10-20 18:49:42 us=888204 2001:db8:284:d700::2 peer info: IV_LZO_STUB=1
2024-10-20 18:49:42 us=888238 2001:db8:284:d700::2 peer info: IV_COMP_STUB=1
2024-10-20 18:49:42 us=888269 2001:db8:284:d700::2 peer info: IV_COMP_STUBv2=1
2024-10-20 18:49:42 us=888397 2001:db8:284:d700::2 TLS: move_session: dest=TM_ACTIVE src=TM_INITIAL reinit_src=1
2024-10-20 18:49:42 us=888524 2001:db8:284:d700::2 TLS: tls_multi_process: initial untrusted session promoted to trusted
WRR2024-10-20 18:49:42 us=909356 2001:db8:284:d700::2 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, peer certificate: 2048 bits RSA, signature: RSA-SHA256, peer temporary key: 253 bits X25519
2024-10-20 18:49:42 us=909455 2001:db8:284:d700::2 [home.acme.com] Peer Connection Initiated with [AF_INET6]2001:db8:284:d700::2:26690
2024-10-20 18:49:42 us=909528 home.acme.com/2001:db8:284:d700::2 MULTI_sva: pool returned IPv4=192.168.168.2, IPv6=fdde:6c68:3589::1000
2024-10-20 18:49:42 us=909659 home.acme.com/2001:db8:284:d700::2 OPTIONS IMPORT: reading client specific options from: ccd/home.acme.com
2024-10-20 18:49:42 us=909863 home.acme.com/2001:db8:284:d700::2 MULTI: Learn: 192.168.168.2 -> home.acme.com/2001:db8:284:d700::2
2024-10-20 18:49:42 us=909896 home.acme.com/2001:db8:284:d700::2 MULTI: primary virtual IP for home.acme.com/2001:db8:284:d700::2: 192.168.168.2
2024-10-20 18:49:42 us=909928 home.acme.com/2001:db8:284:d700::2 MULTI: Learn: fdde:6c68:3589::1000 -> home.acme.com/2001:db8:284:d700::2
2024-10-20 18:49:42 us=909970 home.acme.com/2001:db8:284:d700::2 MULTI: primary virtual IPv6 for home.acme.com/2001:db8:284:d700::2: fdde:6c68:3589::1000
2024-10-20 18:49:42 us=910011 home.acme.com/2001:db8:284:d700::2 MULTI: internal route 192.168.0.0/19 -> home.acme.com/2001:db8:284:d700::2
2024-10-20 18:49:42 us=910048 home.acme.com/2001:db8:284:d700::2 MULTI: Learn: 192.168.0.0/19 -> home.acme.com/2001:db8:284:d700::2
2024-10-20 18:49:42 us=910086 home.acme.com/2001:db8:284:d700::2 MULTI: internal route fde6:66ba:24ff::/48 -> home.acme.com/2001:db8:284:d700::2
2024-10-20 18:49:42 us=910165 home.acme.com/2001:db8:284:d700::2 MULTI: Learn: fde6:66ba:24ff::/48 -> home.acme.com/2001:db8:284:d700::2
2024-10-20 18:49:42 us=910204 home.acme.com/2001:db8:284:d700::2 MULTI: internal route 2001:db8:3053:100::/57 -> home.acme.com/2001:db8:284:d700::2
2024-10-20 18:49:42 us=910237 home.acme.com/2001:db8:284:d700::2 MULTI: Learn: 2001:db8:3053:100::/57 -> home.acme.com/2001:db8:284:d700::2
2024-10-20 18:49:42 us=910359 home.acme.com/2001:db8:284:d700::2 Data Channel MTU parms [ mss_fix:1104 max_frag:1192 tun_mtu:1420 tun_max_mtu:1600 headroom:136 payload:1768 tailroom:562 ET:0 ]
2024-10-20 18:49:42 us=910413 home.acme.com/2001:db8:284:d700::2 Fragmentation MTU parms [ mss_fix:1104 max_frag:1192 tun_mtu:1420 tun_max_mtu:1600 headroom:136 payload:1768 tailroom:562 ET:0 ]
2024-10-20 18:49:42 us=910560 home.acme.com/2001:db8:284:d700::2 Outgoing dynamic tls-crypt: Cipher 'AES-256-CTR' initialized with 256 bit key
2024-10-20 18:49:42 us=910625 home.acme.com/2001:db8:284:d700::2 Outgoing dynamic tls-crypt: Using 256 bit message hash 'SHA256' for HMAC authentication
2024-10-20 18:49:42 us=910669 home.acme.com/2001:db8:284:d700::2 Incoming dynamic tls-crypt: Cipher 'AES-256-CTR' initialized with 256 bit key
2024-10-20 18:49:42 us=910735 home.acme.com/2001:db8:284:d700::2 Incoming dynamic tls-crypt: Using 256 bit message hash 'SHA256' for HMAC authentication
2024-10-20 18:49:42 us=910863 home.acme.com/2001:db8:284:d700::2 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
2024-10-20 18:49:42 us=910904 home.acme.com/2001:db8:284:d700::2 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
2024-10-20 18:49:42 us=911016 home.acme.com/2001:db8:284:d700::2 SENT CONTROL [home.acme.com]: 'PUSH_REPLY,tun-ipv6,route-gateway 192.168.168.1,topology subnet,ping 10,ping-restart 60,ifconfig-ipv6 fdde:6c68:3589::1000/64 fdde:6c68:3589::1,ifconfig 192.168.168.2 255.255.255.0,peer-id 0,cipher AES-256-GCM,protocol-flags cc-exit tls-ekm dyn-tls-crypt,tun-mtu 1420' (status=1)
W2024-10-20 18:49:43 us=1398 MULTI: Learn: fde6:66ba:24ff:103::118 -> home.acme.com/2001:db8:284:d700::2
rW2024-10-20 18:49:43 us=1801 home.acme.com/2001:db8:284:d700::2 Data Channel: cipher 'AES-256-GCM', peer-id: 0
2024-10-20 18:49:43 us=1872 home.acme.com/2001:db8:284:d700::2 Timers: ping 10, ping-restart 120
2024-10-20 18:49:43 us=1908 home.acme.com/2001:db8:284:d700::2 Protocol options: protocol-flags cc-exit tls-ekm dyn-tls-crypt

Environment

Software version used and hardware type if relevant, e.g.:

OPNsense 24.7.6-amd64
FreeBSD 14.1-RELEASE-p5
OpenSSL 3.0.15

KVM virtual machine.

[OPNsense-bot]

Thank you for creating an issue.
Since the ticket doesn't seem to be using one of our templates, we're marking this issue as low priority until further notice.

For more information about the policies for this repository,
please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.

The easiest option to gain traction is to close this ticket and open a new one using one of our templates.

[fichtner]

Basically OpenVPN daemon decides to switch from DCO to TUN/TAP fallback due to the feature use which doesn’t work if we predefine the interface as DCO.

[r000m5]

Thanks for your reply.
First of all sorry for the "incomplete" tag, I tried to follow the template but I guess I made a mistake.

When you say "the feature use which doesn’t work if we predefine the interface as DCO", I guess you mean that the "fragment" option doesn't work with a predefined DCO interface.

If that's the case do you have any idea why ? Would that be a configuration mistake I made ? Or maybe not yet implemented since DCO is still experimental ? Or it should work but we're facing a bug here ?

[fichtner]

I honestly don’t know what the reason here is WRT DCO fragment use. If you switch from DCO to TUN it should work fine in that regard. OpenVPN surely has a reason and we‘re missing a validation to prevent the misconfiguration(even though the rules here are arcane).

Cheers,
Franco

[fichtner]

Also one of the reasons here is that DCO is implicit unless explicitly disabled or implicitly disabled due to incompatible features. Now the other side might also not use DCO then and both sides don‘t even have to agree on DCO as it is compatible with TUN and the other way around.

[r000m5]

WRT ?
I use DCO because of the poor performance of openvpn when DCO is not used (low bandwidth).
Indeed I didn't notice that DCO doesn't support fragment, now I know thank you (https://community.openvpn.net/openvpn/wiki/DataChannelOffload/Features)

[r000m5]

DCO doesn't support fragment : https://community.openvpn.net/openvpn/wiki/DataChannelOffload/Features

[fichtner]

Ok, will add a validation for it.