Forwarded Headers and Kubernetes Ingress Nginx

1. Setup environment at local VM
2. Redirect within Spring Boot (Servlet)
3. The proxy_redirect directive
4. References

GitHub: https://github.com/qqbuby/spring-ingress-nginx-xf/

1. Setup environment at local VM

1.1. Setup cert for HTTPS

Create a self-signed cert with openssl

$ openssl req -x509 \
  -nodes \
  -newkey rsa:4096 \
  -days 3650 \
  -keyout loca.io.ca.key \
  -out local.io.ca.crt \
  -subj "/C=CN/ST=Shanghai/L=Shanghai/O=Global Security/OU=IT Department/CN=*.local.io" \
  -addext "subjectAltName=DNS:local.io,DNS:*.local.io"

Import to CA root certificates

$ sudo mkdir /usr/local/share/ca-certificates/extra
$ sudo cp local.io.crt /usr/local/share/ca-certificates/extra/
$ sudo update-ca-certificates

Install cert as secret to kubernetes

$ kubectl create -n default secret tls local.io --cert=local.io.crt --key=local.io.key

1.2. Install echo server

$ kubectl apply -f echoserver/

1.3. Setup Nginx as Proxy Server

# install nginx server
$ sudo apt-get install nginx -y
# remove the default site (80)
$ sudo rm /etc/nginx/site-enabled/*
$ sudo cp etc/nginx/sites-enabled/*.conf /etc/nginx/sites-enabled/
$ sudo cp etc/nginx/conf.d/forwarded.conf /etc/nginx/conf.d/
$ sudo cp etc/nginx/proxy_params /etc/nginx/
$ sudo nginx -s reload

1.4. Setup `hosts` for hostname `local.io`

$ cat << EOF >> /etc/hosts
> 127.0.0.1 local.io
> 127.0.0.1 default.local.io
> 127.0.0.1 native.local.io
> 127.0.0.1 framework.local.io
> 127.0.0.1 relative.local.io
> EOF

1.5. Test echo server

$ curl -i https://local.io:8083
HTTP/1.1 200 OK
Server: nginx/1.14.2
Date: Fri, 29 Oct 2021 11:16:07 GMT
Content-Type: text/plain
Transfer-Encoding: chunked
Connection: keep-alive



Hostname: echoserver-9d94d584f-q855h

Pod Information:
    -no pod information available-

Server values:
    server_version=nginx: 1.13.3 - lua: 10008

Request Information:
    client_address=10.244.0.115
    method=GET
    real path=/
    query=
    request_version=1.1
    request_scheme=http
    request_uri=http://local.io:8080/

Request Headers:
    accept=*/*
    host=local.io:8083 (1)
    user-agent=curl/7.64.0
    x-forwarded-for=127.0.0.1, 10.244.0.1
    x-forwarded-host=local.io:8083 (2)
    x-forwarded-port=80 (3)
    x-forwarded-proto=https (4)
    x-forwarded-scheme=https
    x-original-forwarded-for=127.0.0.1
    x-real-ip=127.0.0.1
    x-request-id=8b0d54087c07c53de1adb5f10a7b9398
    x-scheme=https

Request Body:
    -no body in request-

1	The Host request header specifies the host and port number of the server to which the request is being sent. If no port is included, the default port for the service requested is implied (e.g., 443 for an HTTPS URL, and 80 for an HTTP URL).
2	The X-Forwarded-Host header is a de-facto standard header for identifying the original host requested by the client in the Host HTTP request header. Host names and ports of reverse proxies (load balancers, CDNs) may differ from the origin server handling the request, in that case the X-Forwarded-Host header is useful to determine which Host was originally used.
3	The *X-Forwarded-Port* header is not a standard header, but also usually used to identify the port that a client used to connect to your proxy or load balancer. In the above example, the orignal `port` was `8083`, but the `x-forwarded-port` was `80` writed by Ingress Nginx Controller in Kubernetes.
4	The X-Forwarded-Proto header is a de-facto standard header for identifying the protocol (HTTP or HTTPS) that a client used to connect to your proxy or load balancer.

2. Redirect within Spring Boot (Servlet)

2.1. Spring WebApp Demo Code

// src/main/java/com/example/springingressnginx/SpringIngressNginxApplication.java
@RestController
@SpringBootApplication
public class SpringIngressNginxApplication {

    public static void main(String[] args) {
        SpringApplication.run(SpringIngressNginxApplication.class, args);
    }

    @GetMapping(value = "/echo")
    public void echo(HttpServletRequest request, HttpServletResponse response) {
        Enumeration<String> headerNames = request.getHeaderNames();
        if (headerNames != null) {
            while (headerNames.hasMoreElements()) {
                String headerName = headerNames.nextElement();
                response.setHeader(String.format("x-echo-%s", headerName.toLowerCase()), request.getHeader(headerName));
            }
        }
    }

    @GetMapping(value = "/302")
    public void redirect(HttpServletResponse response) throws IOException {
        response.sendRedirect("/echo");
    }
}

#src/main/resources/application.properties
management.endpoint.health.probes.enabled=true
management.endpoint.health.group.health.include=readiness,liveness
management.endpoints.web.exposure.include=health,env
server.port=8080
server.servlet.context-path=/
#---
spring.config.activate.on-profile=native
server.forward-headers-strategy=native
#---
spring.config.activate.on-profile=framework
server.forward-headers-strategy=framework
#---
spring.config.activate.on-profile=relative
server.tomcat.use-relative-redirects=true

2.2. Deploy Spring WebApp into Kubernetes

$ mvn spring-boot:build-image
$ kubectl apply -f kubernetes/

2.3. Test WebApp that runs behind HTTP proxy

server.forward-headers-strategy=none

$ curl -i https://default.local.io:8083/echo
HTTP/1.1 200
Server: nginx/1.14.2
Date: Fri, 29 Oct 2021 11:17:48 GMT
Content-Length: 0
Connection: keep-alive
x-echo-host: default.local.io:8083
x-echo-x-request-id: ff430717d3bfc9c4dd13af52c38d0f6c
x-echo-x-real-ip: 127.0.0.1
x-echo-x-forwarded-host: default.local.io:8083
x-echo-x-forwarded-port: 80
x-echo-x-forwarded-proto: https
x-echo-x-forwarded-scheme: https
x-echo-x-scheme: https
x-echo-x-original-forwarded-for: 127.0.0.1
x-echo-user-agent: curl/7.64.0
x-echo-accept: */*

$ curl -iL https://default.local.io:8083/302
HTTP/1.1 302
Server: nginx/1.14.2
Date: Fri, 29 Oct 2021 11:17:50 GMT
Content-Length: 0
Connection: keep-alive
Location: https://default.local.io:80/echo

curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number

server.forward-headers-strategy=native

$ curl -i https://native.local.io:8083/echo
HTTP/1.1 200
Server: nginx/1.14.2
Date: Fri, 29 Oct 2021 11:18:19 GMT
Content-Length: 0
Connection: keep-alive
x-echo-host: native.local.io:8083
x-echo-x-request-id: 6cb7ad640e49d8fe1cdf61da56bd835a
x-echo-x-real-ip: 127.0.0.1
x-echo-x-forwarded-host: native.local.io:8083
x-echo-x-forwarded-port: 80
x-echo-x-forwarded-proto: https
x-echo-x-forwarded-scheme: https
x-echo-x-scheme: https
x-echo-x-original-forwarded-for: 127.0.0.1
x-echo-user-agent: curl/7.64.0
x-echo-accept: */*

$ curl -iL https://native.local.io:8083/302
HTTP/1.1 302
Server: nginx/1.14.2
Date: Fri, 29 Oct 2021 11:18:28 GMT
Content-Length: 0
Connection: keep-alive
Location: https://native.local.io:80/echo

curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number

server.forward-headers-strategy=framework

$ curl -i https://framework.local.io:8083/echo
HTTP/1.1 200
Server: nginx/1.14.2
Date: Fri, 29 Oct 2021 11:18:59 GMT
Content-Length: 0
Connection: keep-alive
x-echo-host: framework.local.io:8083
x-echo-x-request-id: 9671ac40270af9f3dc89cdfcb29ea77d
x-echo-x-real-ip: 127.0.0.1
x-echo-x-forwarded-scheme: https
x-echo-x-scheme: https
x-echo-x-original-forwarded-for: 127.0.0.1
x-echo-user-agent: curl/7.64.0
x-echo-accept: */*

$ curl -iL https://framework.local.io:8083/302
HTTP/1.1 302
Server: nginx/1.14.2
Date: Fri, 29 Oct 2021 11:19:01 GMT
Content-Length: 0
Connection: keep-alive
Location: https://framework.local.io:80/echo

curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number

server.tomcat.use-relative-redirects=true

$ curl -i https://relative.local.io:8083/echo
HTTP/1.1 200
Server: nginx/1.14.2
Date: Fri, 29 Oct 2021 11:19:59 GMT
Content-Length: 0
Connection: keep-alive
x-echo-host: relative.local.io:8083
x-echo-x-request-id: 42f3771e0a730d6d6b9dc745f17dd807
x-echo-x-real-ip: 127.0.0.1
x-echo-x-forwarded-host: relative.local.io:8083
x-echo-x-forwarded-port: 80
x-echo-x-forwarded-proto: https
x-echo-x-forwarded-scheme: https
x-echo-x-scheme: https
x-echo-x-original-forwarded-for: 127.0.0.1
x-echo-user-agent: curl/7.64.0
x-echo-accept: */*

$ curl -iL https://relative.local.io:8083/302
HTTP/1.1 302
Server: nginx/1.14.2
Date: Fri, 29 Oct 2021 11:20:02 GMT
Content-Length: 0
Connection: keep-alive
Location: /echo

HTTP/1.1 200
Server: nginx/1.14.2
Date: Fri, 29 Oct 2021 11:20:02 GMT
Content-Length: 0
Connection: keep-alive
x-echo-host: relative.local.io:8083
x-echo-x-request-id: 925a533fa0e52336f0eb1a216eb0a289
x-echo-x-real-ip: 127.0.0.1
x-echo-x-forwarded-host: relative.local.io:8083
x-echo-x-forwarded-port: 80
x-echo-x-forwarded-proto: https
x-echo-x-forwarded-scheme: https
x-echo-x-scheme: https
x-echo-x-original-forwarded-for: 127.0.0.1
x-echo-user-agent: curl/7.64.0
x-echo-accept: */*

3. The `proxy_redirect` directive

Proxy redirect

With the annotations nginx.ingress.kubernetes.io/proxy-redirect-from and nginx.ingress.kubernetes.io/proxy-redirect-to it is possible to set the text that should be changed in the Location and Refresh header fields of a proxied server response.

Setting "off" or "default" in the annotation nginx.ingress.kubernetes.io/proxy-redirect-from disables nginx.ingress.kubernetes.io/proxy-redirect-to, otherwise, both annotations must be used in unison. Note that each annotation must be a string without spaces.

By default the value of each annotation is "off".

Use proxy_redirect to correct the Location response header field.

# kubernetes/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: spring-ingress
  labels:
    app: spring-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/proxy-redirect-from: "default"
    nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_redirect https://local.io:80 https://local.io:8083;
      proxy_redirect http://local.io:443 https://local.io:8083;
spec:

Try again with server.forward-headers-strategy=default

$ curl -i https://default.local.io:8083/302
HTTP/1.1 302
Server: nginx/1.14.2
Date: Fri, 29 Oct 2021 11:34:05 GMT
Content-Length: 0
Connection: keep-alive
Location: https://default.local.io:8083/echo

Try again with server.forward-headers-strategy=native

$ curl -i https://native.local.io:8083/302
HTTP/1.1 302
Server: nginx/1.14.2
Date: Fri, 29 Oct 2021 11:34:05 GMT
Content-Length: 0
Connection: keep-alive
Location: https://native.local.io:8083/echo

Try again with server.forward-headers-strategy=framework

$ curl -i https://framework.local.io:8083/302
HTTP/1.1 302
Server: nginx/1.14.2
Date: Fri, 29 Oct 2021 11:34:05 GMT
Content-Length: 0
Connection: keep-alive
Location: https://framework.local.io:8083/echo