Skip to content

use a NodePort service as ExternalName in another service type ExternalName for configuring default-backend

What happened:

I'm trying to use an ExternalName service as a default backend for an Ingress to handle situation when the deployment has no active endpoints. I get 504 Gateway Time-out error. In controller logs:

W1010 15:04:46.362237       7 controller.go:1216] Service "default/web" does not have any active Endpoint.
2024/10/10 15:05:07 [error] 1294#1294: *41828 [lua] balancer.lua:348: balance(): error while setting current upstream peer [echo.default.svc.cluster.local]:8080: invalid IPv6 address while connecting to upstream, client: 192.168.49.1, server: hello-world.example, request: "GET / HTTP/1.1", host: "hello-world.example"
2024/10/10 15:05:12 [error] 1294#1294: *41828 upstream timed out (110: Operation timed out) while connecting to upstream, client: 192.168.49.1, server: hello-world.example, request: "GET / HTTP/1.1", upstream: "http://0.0.0.1:80/", host: "hello-world.example"

What you expected to happen:

The request should be passed to the default backend service. It should not treat the external name as an IPv6 address. AFAIK, it works when an external name service is used in Ingress. Why is default backend handled differently?

NGINX Ingress controller version (exec into the pod and run nginx-ingress-controller --version.): v1.11.2

Kubernetes version (use kubectl version): v1.31.0

Environment:

Initially I experienced this in EKS, then I reproduced it in Minikube.

  • Cloud provider or hardware configuration: Minikube v1.34.0

  • OS (e.g. from /etc/os-release): Ubuntu 24.04

  • Kernel (e.g. uname -a): 6.8.0-45

  • How was the ingress-nginx-controller installed: minikube addons enable ingress

How to reproduce this issue:

Install minikube

minikube start

Install the ingress controller

minikube addons enable ingress

Install hello-app with 0 replicas

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: web
  name: web
  namespace: default
spec:
  replicas: 0
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - image: gcr.io/google-samples/hello-app:1.0
        imagePullPolicy: IfNotPresent
        name: hello-app

kubectl expose deployment web --type=NodePort --port=8080

Install echo server

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: echo
  name: echo
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: echo
  template:
    metadata:
      labels:
        app: echo
    spec:
      containers:
      - image: jmalloc/echo-server
        imagePullPolicy: IfNotPresent
        name: echo
kubectl expose deployment echo --type=NodePort --port=8080

Create ExternalName service that points to echo service

apiVersion: v1
kind: Service
metadata:
  name: echo-en
  namespace: default
spec:
  externalName: echo.default.svc.cluster.local
  type: ExternalName
  ports:
  - protocol: TCP
    port: 8080
    targetPort: 8080

Create an ingress for hello-app with echo service external name as default backend

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/default-backend: echo-en
  name: example-ingress
spec:
  ingressClassName: nginx
  rules:
    - host: hello-world.example
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web
                port:
                  number: 8080

Send a request

❯ curl --resolve "hello-world.example:80:$( minikube ip )" -i http://hello-world.example
HTTP/1.1 504 Gateway Time-out
Date: Thu, 10 Oct 2024 15:35:28 GMT
Content-Type: text/html
Content-Length: 160
Connection: keep-alive

<html>
<head><title>504 Gateway Time-out</title></head>
<body>
<center><h1>504 Gateway Time-out</h1></center>
<hr><center>nginx</center>
</body>
</html>