Packet filtering on kubernetes

NetworkPolicy is a Kubernetes resource that restricts packet transfer between pods.

It can limit the connectivity topology among internal pods and may also allow the implementation of IP-ban through a deny list. While the interface of the NetworkPolicy is standardized, it’s crucial to be aware that interpretation and behavior can vary depending on implementations.

For instance, the implementation of GKE’s NetworkPolicy traditionally adopted Calico and, in clusters with Dataplane V2 enabled, operates with Cilium. Even when using the same resource definitions, the behavior differs between Calico and Cilium.

Considering variations in configuration for each cluster, including cloud providers other than GKE or future migrations, it is necessary to validate your configuration beforehand.

iptables alternative

In traditional bare-metal Linux hosts, packet filtering was commonly implemented with iptables. However, in Kubernetes, NetworkPolicy is employed for similar purposes.

Depending on the implementation, there can be conflicts between Kubernetes’ internal network functionality and iptables. As a result, using tools like iptables or nftables within containers may often be impractical. Introducing traditional tools forcibly could potentially break the container network.

Sample NetworkPolicy

A minimum example for restricting connection IPs using NetworkPolicy is as follows:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: test-network-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      role: db
  policyTypes:
    - Ingress
  ingress:
    - from:
        - ipBlock:
            cidr: 172.17.0.0/16
            except:
              - 172.17.1.0/24

In this way, you can define the target pod and the filter.

A target pod is specified with spec.podSelector, defined along with the generic pod selector rule.
- Since NetworkPolicy is a namespaced resource, metadata.namespace also functions as a filter for the targeted pod.
Use ingress.from to filter the source of the connection and control access.
- ipBlock.cidr allows the connection, while ipBlock.except denies the connection. It may not be immediately obvious, so please be cautious.
- In this example, the determination is based on IP, but it is also possible to control pod-to-pod connections using podSelector.

A less obvious point to note from the definition is that applying this resource results in a behavior where connections other than 172.17.0.0/16 are denied.

You can confirm active NetworkPolicy with the kubectl get networkpolicy command.
If the filter is not functioning, it may be due to an inappropriate pod selector. Also, since errors may not be clear in cases of unintentional definitions, testing the configuration is crucial.
Changes to NetworkPolicy take effect immediately.

Implement IP-ban in GKE Dataplane V2

However, GKE Dataplane V2 allows the implementation of IP-ban from the internet with the following definition:

  ingress:
    - from:
        - ipBlock:
            cidr: 0.0.0.0/0
            except:
              - 172.17.1.0/24

Since the interpretation of ipBlock can vary depending on the implementation, this can be specific to GKE Dataplane V2.

The ipBlock.except must be a subnet internal to ipBlock.cidr, but 0.0.0.0/0 encompasses all IPs.
However, in Dataplane V2, 0.0.0.0/0 behaves as excluding internal pod IPs rather than including “all.” Consequently, applying it casually can result in the disruption of internal connections.

In conclusion, NetworkPolicy in this format operates as expected only for edge containers facing the internet.
Another constraint is that the container’s peer must be an external endpoint’s IP. In other words, IP filtering may not function behind a proxy.

Proxy behavior depends on the implementation, but for GKE, prerequisites may include edge containers created with type: LoadBalancer for Services with externalTrafficPolicy: Local setting, or containers behind a TCP proxy that decodes IP using ProxyProtocol.

In GKE, Services created with type: NodePort are also functioning as expected.

Finally, it’s safer to assume that internal containers cannot be controlled by IP.

⁋ Dec 3, 2023↻ Nov 7, 2024