Connected UDP sockets not cleaned up from reverse_sk map on abort
After PR #39486, a BPF program was attached to cgroup/sock_release to clean up the corresponding entries in cilium_lb4_reverse_sk and cilium_lb6_reverse_sk when a socket is released.
However, when a connected UDP socket is destroyed by the Cilium agent, the socket’s address and port are cleared to zero by the kernel before the BPF hook runs. As a result, the cleanup cannot locate and remove the corresponding entry from the reverse_sk map. https://elixir.bootlin.com/linux/v6.15.3/source/net/ipv4/udp.c#L2167-L2168
I tested the following cases. Cleanup worked correctly for TCP sockets and regular close() on connected UDP sockets, but failed in the cases of connected UDP abort().
| Case | Protocol | IP Family | Type | Result |
|---|---|---|---|---|
| 1 | TCP | IPv4 | Regular close |
|
| 2 | TCP | IPv4 | Abort |
|
| 3 | TCP | IPv6 | Regular close |
|
| 4 | TCP | IPv6 | Abort |
|
| 5 | UDP (Connected) | IPv4 | Regular close |
|
| 6 | UDP (Connected) | IPv4 | Abort |
|
| 7 | UDP (Connected) | IPv6 | Regular close |
|
| 8 | UDP (Connected) | IPv6 | Abort |
|
Test Logs
# TCP IPv4 Regular close (✅ Cleaned up)
# Before
$ bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_lb4_reverse_sk
key: c9 9c 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: fd ea 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: ef 08 25 01 00 00 00 00 0a f4 01 19 0c ea 00 00 value: 0a 60 78 01 0c ea 00 0e
key: a2 7f 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: 82 7f 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
Found 5 elements
# After
$ bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_lb4_reverse_sk
key: c9 9c 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: fd ea 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: a2 7f 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: 82 7f 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
Found 4 elements
# Datapath debug log
<...>-4040702 [009] ...11 1975090.344066: bpf_trace_printk: sock4_delete_revnat deletes cilium_lb4_reverse_nat_sk cookie=19204335 address=10.244.1.25 port=3306
# TCP IPv4 abort (✅ Cleaned up)
# Before
$ bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_lb4_reverse_sk
key: 92 83 25 01 00 00 00 00 0a f4 01 19 0c ea 00 00 value: 0a 60 78 01 0c ea 00 0e
key: c9 9c 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: fd ea 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: a2 7f 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: 82 7f 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
Found 5 elements
# After
$ bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_lb4_reverse_sk
key: c9 9c 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: a7 1f 25 01 00 00 00 00 0a f4 03 14 0c ea 00 00 value: 0a 60 78 01 0c ea 00 0e
key: fd ea 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: a2 7f 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: 82 7f 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
Found 5 elements
# Datapath debug log
<...>-4066157 [002] ...11 1975497.742971: bpf_trace_printk: sock4_delete_revnat deletes cilium_lb4_reverse_nat_sk cookie=19235730 address=10.244.1.25 port=3306
# Cilium agent log
time=2025-11-07T02:18:25.352735969Z level=info msg="Forcefully terminated sockets" module=agent.controlplane.loadbalancer-reconciler.socket-termination filter="{DestIp:10.244.1.25 DestPort:3306 Family:2 Protocol:6 States:8126 DestroyCB:0x3588740}" success=1
# TCP IPv6 regular close (✅ Cleaned up)
# Before
$ bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_lb6_reverse_sk
key:
97 c3 25 01 00 00 00 00 fd 00 00 10 02 44 00 03
00 00 00 00 00 00 7c b5 0c ea 00 00 00 00 00 00
value:
fd 00 00 10 00 96 00 00 00 00 00 00 00 00 66 c5
0c ea 00 12
Found 1 element
# After
$ bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_lb6_reverse_sk
Found 0 elements
# Datapath debug log
<...>-4139332 [007] ...11 1976372.017012: bpf_trace_printk: sock6_delete_revnat deletes cilium_lb6_reverse_nat_sk cookie=19252119 address=fd00:0010:0244:0003:0000:0000:0000:7cb5 port=3306
# TCP IPv6 abort (✅ Cleaned up)
# Before
$ bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_lb6_reverse_sk
key:
02 4c 25 01 00 00 00 00 fd 00 00 10 02 44 00 01
00 00 00 00 00 00 81 2f 0c ea 00 00 00 00 00 00
value:
fd 00 00 10 00 96 00 00 00 00 00 00 00 00 66 c5
0c ea 00 12
Found 1 element
# After
$ bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_lb6_reverse_sk
key:
05 f3 25 01 00 00 00 00 fd 00 00 10 02 44 00 03
00 00 00 00 00 00 67 57 0c ea 00 00 00 00 00 00
value:
fd 00 00 10 00 96 00 00 00 00 00 00 00 00 66 c5
0c ea 00 12
Found 1 element
# Datapath debug log
mysql-4148665 [005] ...11 1976777.900637: bpf_trace_printk: sock6_delete_revnat deletes cilium_lb6_reverse_nat_sk cookie=19221506 address=fd00:0010:0244:0001:0000:0000:0000:812f port=3306
# Cilium agent log
time=2025-11-07T02:40:15.434612284Z level=info msg="Forcefully terminated sockets" module=agent.controlplane.loadbalancer-reconciler.socket-termination filter="{DestIp:fd00:10:244:1::812f DestPort:3306 Family:10 Protocol:6 States:8126 DestroyCB:0x3588740}" success=1
# UDP IPv4 regular close (✅ Cleaned up)
# Before
$ bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_lb4_reverse_sk
key: 1a df 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: 63 9c 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: 5b 24 25 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: 48 eb 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: 47 58 26 01 00 00 00 00 0a f4 01 c9 00 35 00 00 value: 0a 60 00 0a 00 35 00 04
Found 5 elements
# After
$ bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_lb4_reverse_sk
key: 1a df 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: 63 9c 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: 5b 24 25 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: 48 eb 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
Found 4 elements
# Datapath debug log
nginx-3979156 [010] .N.11 1980112.671733: bpf_trace_printk: sock4_delete_revnat deletes cilium_lb4_reverse_nat_sk cookie=19290183 address=10.244.1.201 port=53
# UDP IPv4 abort (❌ Not cleaned up)
# Before
$ bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_lb4_reverse_sk
key: 10 22 27 01 00 00 00 00 0a f4 00 08 00 35 00 00 value: 0a 60 00 0a 00 35 00 04
key: 63 9c 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: 15 f6 26 01 00 00 00 00 0a f4 03 91 00 35 00 00 value: 0a 60 00 0a 00 35 00 04
key: 48 eb 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: b4 1f 26 01 00 00 00 00 0a f4 00 08 00 35 00 00 value: 0a 60 00 0a 00 35 00 04
Found 5 elements
# After
bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_lb4_reverse_sk
key: 10 22 27 01 00 00 00 00 0a f4 00 08 00 35 00 00 value: 0a 60 00 0a 00 35 00 04
key: 63 9c 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: 15 f6 26 01 00 00 00 00 0a f4 03 91 00 35 00 00 value: 0a 60 00 0a 00 35 00 04
key: 48 eb 24 01 00 00 00 00 ac 13 00 05 19 2b 00 00 value: 0a 60 00 01 01 bb 00 01
key: b4 1f 26 01 00 00 00 00 0a f4 00 08 00 35 00 00 value: 0a 60 00 0a 00 35 00 04
Found 5 elements
# Datapath debug log
nginx-317512 [008] ...11 1981476.790945: bpf_trace_printk: sock4_delete_revnat deletes cilium_lb4_reverse_nat_sk cookie=19341840 address=0.0.0.0 port=0
nginx-317513 [006] ...11 1981478.823930: bpf_trace_printk: sock4_delete_revnat deletes cilium_lb4_reverse_nat_sk cookie=19330581 address=0.0.0.0 port=0
nginx-317515 [009] ...11 1981476.790945: bpf_trace_printk: sock4_delete_revnat deletes cilium_lb4_reverse_nat_sk cookie=19275700 address=0.0.0.0 port=0
# Cilium agent log
cilium-qstqw cilium-agent time=2025-11-07T03:58:54.794369751Z level=info msg="Forcefully terminated sockets" module=agent.controlplane.loadbalancer-reconciler.socket-termination filter="{DestIp:10.244.0.8 DestPort:53 Family:2 Protocol:17 States:65535 DestroyCB:0x3588740}" success=2
cilium-qstqw cilium-agent time=2025-11-07T03:58:56.827406813Z level=info msg="Forcefully terminated sockets" module=agent.controlplane.loadbalancer-reconciler.socket-termination filter="{DestIp:10.244.3.145 DestPort:53 Family:2 Protocol:17 States:65535 DestroyCB:0x3588740}" success=1
# UDP IPv6 regular close (✅ Cleaned up)
# Before
$ bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_lb6_reverse_sk
key:
e8 ec 25 01 00 00 00 00 fd 00 00 10 02 44 00 01
00 00 00 00 00 00 0e 95 00 35 00 00 00 00 00 00
value:
fd 00 00 10 00 96 00 00 00 00 00 00 00 00 88 d4
00 35 00 15
key:
33 7d 26 01 00 00 00 00 fd 00 00 10 02 44 00 01
00 00 00 00 00 00 49 f1 00 35 00 00 00 00 00 00
value:
fd 00 00 10 00 96 00 00 00 00 00 00 00 00 88 d4
00 35 00 15
key:
d8 b6 26 01 00 00 00 00 fd 00 00 10 02 44 00 01
00 00 00 00 00 00 0e 95 00 35 00 00 00 00 00 00
value:
fd 00 00 10 00 96 00 00 00 00 00 00 00 00 88 d4
00 35 00 15
Found 3 elements
# After
$ bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_lb6_reverse_sk
Found 0 elements
# Datapath debug log
nginx-234450 [013] ...11 1980515.085113: bpf_trace_printk: sock6_delete_revnat deletes cilium_lb6_reverse_nat_sk cookie=19314392 address=fd00:0010:0244:0001:0000:0000:0000:0e95 port=53
nginx-234449 [010] ...11 1980515.085185: bpf_trace_printk: sock6_delete_revnat deletes cilium_lb6_reverse_nat_sk cookie=19262696 address=fd00:0010:0244:0001:0000:0000:0000:0e95 port=53
nginx-234451 [001] ...11 1980515.089559: bpf_trace_printk: sock6_delete_revnat deletes cilium_lb6_reverse_nat_sk cookie=19299635 address=fd00:0010:0244:0001:0000:0000:0000:49f1 port=53
# UDP IPv6 abort (❌ Not cleaned up)
# Before
$ bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_lb6_reverse_sk
key:
6e 89 26 01 00 00 00 00 fd 00 00 10 02 44 00 01
00 00 00 00 00 00 0e 95 00 35 00 00 00 00 00 00
value:
fd 00 00 10 00 96 00 00 00 00 00 00 00 00 88 d4
00 35 00 15
key:
44 d5 26 01 00 00 00 00 fd 00 00 10 02 44 00 01
00 00 00 00 00 00 0e 95 00 35 00 00 00 00 00 00
value:
fd 00 00 10 00 96 00 00 00 00 00 00 00 00 88 d4
00 35 00 15
key:
78 e3 26 01 00 00 00 00 fd 00 00 10 02 44 00 01
00 00 00 00 00 00 0e 95 00 35 00 00 00 00 00 00
value:
fd 00 00 10 00 96 00 00 00 00 00 00 00 00 88 d4
00 35 00 15
key:
9a 7e 26 01 00 00 00 00 fd 00 00 10 02 44 00 01
00 00 00 00 00 00 0e 95 00 35 00 00 00 00 00 00
value:
fd 00 00 10 00 96 00 00 00 00 00 00 00 00 88 d4
00 35 00 15
key:
5b f2 26 01 00 00 00 00 fd 00 00 10 02 44 00 01
00 00 00 00 00 00 49 f1 00 35 00 00 00 00 00 00
value:
fd 00 00 10 00 96 00 00 00 00 00 00 00 00 88 d4
00 35 00 15
Found 5 elements
# After
$ bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_lb6_reverse_sk
key:
6e 89 26 01 00 00 00 00 fd 00 00 10 02 44 00 01
00 00 00 00 00 00 0e 95 00 35 00 00 00 00 00 00
value:
fd 00 00 10 00 96 00 00 00 00 00 00 00 00 88 d4
00 35 00 15
key:
44 d5 26 01 00 00 00 00 fd 00 00 10 02 44 00 01
00 00 00 00 00 00 0e 95 00 35 00 00 00 00 00 00
value:
fd 00 00 10 00 96 00 00 00 00 00 00 00 00 88 d4
00 35 00 15
key:
78 e3 26 01 00 00 00 00 fd 00 00 10 02 44 00 01
00 00 00 00 00 00 0e 95 00 35 00 00 00 00 00 00
value:
fd 00 00 10 00 96 00 00 00 00 00 00 00 00 88 d4
00 35 00 15
key:
9a 7e 26 01 00 00 00 00 fd 00 00 10 02 44 00 01
00 00 00 00 00 00 0e 95 00 35 00 00 00 00 00 00
value:
fd 00 00 10 00 96 00 00 00 00 00 00 00 00 88 d4
00 35 00 15
key:
5b f2 26 01 00 00 00 00 fd 00 00 10 02 44 00 01
00 00 00 00 00 00 49 f1 00 35 00 00 00 00 00 00
value:
fd 00 00 10 00 96 00 00 00 00 00 00 00 00 88 d4
00 35 00 15
Found 5 elements
# Datapath debug log
<...>-262236 [012] ...11 1980729.365195: bpf_trace_printk: sock6_delete_revnat deletes cilium_lb6_reverse_nat_sk cookie=19302766 address=fd00:0010:0244:0001:0000:0000:0000:0e95 port=0
nginx-262233 [013] ...11 1980729.365149: bpf_trace_printk: sock6_delete_revnat deletes cilium_lb6_reverse_nat_sk cookie=19322180 address=fd00:0010:0244:0001:0000:0000:0000:0e95 port=0
nginx-262234 [001] ...11 1980729.365148: bpf_trace_printk: sock6_delete_revnat deletes cilium_lb6_reverse_nat_sk cookie=19325816 address=fd00:0010:0244:0001:0000:0000:0000:0e95 port=0
nginx-262235 [014] ...11 1980729.365149: bpf_trace_printk: sock6_delete_revnat deletes cilium_lb6_reverse_nat_sk cookie=19299994 address=fd00:0010:0244:0001:0000:0000:0000:0e95 port=0
nginx-262238 [004] ...11 1980731.855290: bpf_trace_printk: sock6_delete_revnat deletes cilium_lb6_reverse_nat_sk cookie=19329627 address=fd00:0010:0244:0001:0000:0000:0000:49f1 port=0
# Cilium agent log
time=2025-11-07T03:46:27.360769519Z level=info msg="Forcefully terminated sockets" module=agent.controlplane.loadbalancer-reconciler.socket-termination filter="{DestIp:fd00:10:244:1::e95 DestPort:53 Family:10 Protocol:17 States:65535 DestroyCB:0x3588740}" success=4
time=2025-11-07T03:46:29.850947547Z level=info msg="Forcefully terminated sockets" module=agent.controlplane.loadbalancer-reconciler.socket-termination filter="{DestIp:fd00:10:244:1::49f1 DestPort:53 Family:10 Protocol:17 States:65535 DestroyCB:0x3588740}" success=1