Custom default-backend
Background
Cloud-Platform created custom error pages to be used by the default backend for Nginx ingress-controller. The default backend is the default service that Nginx falls back to if it cannot route a request successfully. When the service in the Ingress rule does not have active endpoints, this ingress controller default-backend service will handle the response by serving the cloud-platform custom default error page.
However, some applications don’t want to use the cloud-platform custom default error pages from the Nginx ingress controller but need to be served their own custom error pages. This can be achieved by implementing own custom default backend in a namespace.
Create custom error page
First create a docker image containing custom HTTP error pages using the example from the ingress-nginx, or simplified version of that created by the cloud platform team.
Customised default backend
Using this custom-default-backend example from ingress-nginx, create a service and deployment of the error pages container in your namespace.
To create Deployment and Service manually use this below command:
$ kubectl -n ${namespace} create -f custom-default-backend.yaml
service "nginx-errors" created
deployment.apps "nginx-errors" created
This should have created a Deployment and a Service with the name nginx-errors.
$ kubectl -n ${namespace} get deploy,svc
NAME DESIRED CURRENT READY AGE
deployment.apps/nginx-errors 1 1 1 10s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/nginx-errors ClusterIP 10.0.0.12 <none> 80/TCP 10s
Defining annotations in Ingress file.
Final step is to use an Default Backend annotation in your Ingress. This annotation is of the form nginx.ingress.kubernetes.io/default-backend: <svc name> to specify a custom default backend. This <svc name> is a reference to a service inside of the same namespace in which you are applying this annotation. This annotation overrides the global default backend.
Default-backend service in the namespace will handle the error responses when both default-backend annotation and the custom-http-errors annotation is set. Custom-http-errors annotation configure which HTTP status codes Nginx should be forwarding to the default-backend.
If custom-http-errors is also specified globally, the custom-http-error values specified in this annotation along with custom default backend will override the global value for the given ingress hostname and path.
In the example below, custom error pages from custom-default-backend: nginx-errors service will be served for "404,415" errors which is set as custom-http-errors annotation.
Example Ingress with default-backend and custom-http-errors annotations:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: helloworld-rubyapp-ingress
annotations:
nginx.ingress.kubernetes.io/default-backend: nginx-errors
nginx.ingress.kubernetes.io/custom-http-errors: "404,415"
external-dns.alpha.kubernetes.io/set-identifier: <ingress-name>-<environment-name>-<colour>
external-dns.alpha.kubernetes.io/aws-weight: "100"
spec:
ingressClassName: default
tls:
- hosts:
- helloworld.rubyapp.cloud-platform.service.justice.gov.uk
secretName: secret-cert
rules:
- host: helloworld.rubyapp.cloud-platform.service.justice.gov.uk
http:
paths:
- path: /
pathType: ImplementationSpecific
backend:
service
name: rubyapp-service
port:
number: 4567
(Note - Please change the ingress-name and environment-name values in the above example, you can get the environment-name value from your namespace label “cloud-platform.justice.gov.uk/environment-name”. The colour should be green for ingress in EKS live cluster)
Use platform-level error page
Some teams want application’s serve their own error’s example 404's, but want to serve cloud platforms custom error page from ingress controller default backend for other error codes like 502,503 and 504, this can be done by using custom-http-errors annotation in your ingress for error codes teams want to serve the cloud platforms custom error page.
Example Ingress file to use platform-level error page for custom-http-errors: “502,503,504”. All other errors except 502,503,504 will be served from the application error page.
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: helloworld-rubyapp-ingress
annotations:
nginx.ingress.kubernetes.io/custom-http-errors: "502,503,504"
external-dns.alpha.kubernetes.io/set-identifier: <ingress-name>-<environment-name>-<colour>
external-dns.alpha.kubernetes.io/aws-weight: "100"
spec:
tls:
- hosts:
- helloworld.rubyapp.cloud-platform.service.justice.gov.uk
secretName: secret-cert
rules:
- host: helloworld.rubyapp.cloud-platform.service.justice.gov.uk
http:
paths:
- path: /
backend:
serviceName: rubyapp-service
servicePort: 4567
Not use platform-level error page
Some teams want application’s to serve all errors. It is possible as there is a fix from Nginx-Ingress to cancel out the global-custom-http-errors set to serve cloud platforms custom error page from default backend at ingress controller, by adding the Ingress annotation and setting an error not used in global-custom-http-errors.
If you have custom-default-backend set up in your namespace, adding the default-backend annotation along with custom-http-errors annotation with unused error code in global-custom-http-errors, will serve custom error page from namespace default-backend service, when the service in the Ingress rule does not have active endpoints.
Example Ingress file to set custom-http-errors: "418" as annotation, which is not used at global-custom-http-errors:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: helloworld-rubyapp-ingress
annotations:
nginx.ingress.kubernetes.io/custom-http-errors: "418"
external-dns.alpha.kubernetes.io/set-identifier: <ingress-name>-<environment-name>-<colour>
external-dns.alpha.kubernetes.io/aws-weight: "100"
spec:
ingressClassName: default
tls:
- hosts:
- helloworld.rubyapp.cloud-platform.service.justice.gov.uk
secretName: secret-cert
rules:
- host: helloworld.rubyapp.cloud-platform.service.justice.gov.uk
http:
paths:
- path: /
pathType: ImplementationSpecific
backend:
service
name: rubyapp-service
port:
number: 4567