GKE Ingress health check configuration

If you set the front end of Google Kubernetes Engine to ingress, a GCP health check will be automatically created and passed. Otherwise, the backend service will not open even if the pod is working properly.

The default health check behavior that is automatically created is to check that you can HTTP GET / and get 200OK. If the backend is implemented to handle this request, it will pass a health check.

If you want to configure other than this, you need to configure it appropriately by operating GCP instead of kubernetes.

The relevant health check can be accessed from the Cloud Console’s Cloud Load Balancing.
There is a health check for each backend service, and you can change settings such as monitoring targets.

Customize health check

You can control GCP health checks by creating a custom resource called BackendConfig CRD.
The reference is a Service annotation.

apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
  name: envoy-beconfig
spec:
  healthCheck:
    checkIntervalSec: 15
    timeoutSec: 15
    healthyThreshold: 1
    unhealthyThreshold: 2
    type: HTTPS
    requestPath: /healthz
    port: 8443
---
apiVersion: v1
kind: Service
metadata:
  name: envoy
  labels:
    name: envoy
  annotations:
    cloud.google.com/backend-config: '{"default": "envoy-beconfig"}'
    cloud.google.com/app-protocols: '{"https":"HTTPS"}'
spec:
  ports:
  - name: https
    port: 443
    targetPort: https
  selector:
    name: envoy

The confusing point is that in a container-native load balancing configuration, the port you specify for spec.headlthCheck.port is the pod’s containerPort, not the Service.

Example for envoy

If you use envoy as backend, you’ll likely need to change your GCP health check settings to skip requests that return 200OK.
By changing the protocol, host, and path, you can set the request that actually works.

  • The default protocol for health check is HTTP, but it does not support HTTP if envoy provides HTTPS service.
  • SNI configuration with envoy requires the appropriate host for the request
  • Not all services where / unconditionally returns 200OK

If you want to return 200 OK on a particular Path, add a route like the following to the routing virtual_hosts and envoy will respond directly.

  virtual_hosts:
  - name: example
    domains:
    - "*"
    routes:
    - match:
        prefix: "/healthz"
      direct_response:
        status: "200"

If you specify a wildcard for the domain, health checks without specifying a host is possible.

⁋ Dec 2, 2021↻ Nov 7, 2024
中馬崇尋
Chuma Takahiro