Skip to content

Connection Reset Issue - Solved by Downgrading to v0.8.6

Environment:

kube-vip version affected: v1.0.0, v0.8.9
kube-vip version working: v0.8.6
Architecture: ARM64 (RK3588 SoCs)
Kubernetes: k3s v1.27.7+k3s2
OS: Debian 11/12 (bullseye/bookworm)
Cluster: 3 control-plane nodes, 3 worker nodes

Symptoms:

Frequent read tcp X.X.X.X:XXXXX->VIP:6443: read: connection reset by peer errors
kubectl commands failing intermittently, especially long-running operations (helm install, kubectl
apply large manifests)
Short commands (kubectl get namespaces) working fine
Connection resets occurring during data transfer, not at connection establishment

Root Cause Identified: Found critical panics in leader election in kube-vip logs: E0908 19:55:41.632956 1 clusterLeaderElection.go:202] "Observed a panic" panic="" stacktrace=< goroutine 51 [running]: github.com/kube-vip/kube-vip/pkg/cluster.(*Cluster).StartCluster.func4() /src/pkg/cluster/clusterLeaderElection.go:202 +0x384

These panics cause:

kube-vip process restarts during active connections
Temporary VIP loss during crashes
TCP connections being reset mid-transfer

Solution: Downgrade to kube-vip v0.8.6 completely resolved the issue.

The working configuration: image: ghcr.io/kube-vip/kube-vip:v0.8.6 args: ["manager"] env:

name: cp_enable
value: "true"
name: address
value: "192.168.0.100"
name: vip_arp
value: "true"

... other env vars

securityContext: capabilities: add: ["NET_ADMIN", "NET_RAW"]

Versions tested:

❌ v1.0.0: Frequent connection resets + leader election panics
❌ v0.9.2: Same connection reset issues
❌ v0.8.9: Same connection reset issues
✅ v0.8.6: Completely stable, no connection issues

Recommendation: For production deployments, especially on ARM64 architecture, avoid v1.0.0 and v0.8.9 until leader election panic is fixed. v0.8.6 is stable and recommended.

This might be related to the ARM64-specific build or leader election timing issues on certain hardware. The problem is reproducible and consistently resolved by downgrading.