GitOps and Admission Control for Calico Network Policy

We see different personas within customers deploying 3 types of controls:

This is different from the traditional firewall world, where the security admin is responsible for managing security policies, and the change management window could be several weeks in duration. Adopting that model in kubernetes will simply act against the very principles of enabling the developers. So how can we make policy writing and enforcement simple, yet adhere to organizational processes? The answer lies in simple tooling, GitOps and governance.

Policies have business logic that must be implemented in YAML. The business logic (allow access for service A to service B, Open port 443 inbound on service B, permit access to slack webhook etc.) can easily be written by development team. The challenge is getting the YAML correct, testing it, and making sure that something else is not broken because of the proposed changes.

As a solution, I propose the following:

Your policy blueprint can be at either end of the spectrum. One end is being restrictive, where you lock down the policy updates via a central team. The other end is being permissive, where you have a fully decentralized platform and developers have the freedom to create their own policies. Either way, you are bound to answer questions like: “how do I ensure that the policies are indeed enforced?”, “how do I make sure that the policies are not violating my organization controls?” etc.

In the sections below, we demonstrate an example of decentralized policy pipeline. See the diagram below.

First, you define a standard template for access policy representation, and build tooling for security policy YAML generation from that template. Then instead of writing plain YAML, the application teams define their access policies in a simple, standard format as enforced in that template. This allows the app teams to focus on business logic and not worry about policy syntax or accuracy. Also in most cases, you have more than 1 cluster running and developers do not need to worry about which policy to apply on which cluster.

Then you introduce a set of validation checks in your CI (continuous integration) pipeline. These validation checks represent how you have defined your policy governance. Some examples:

If the PR (pull request) for policy changes passes through the validation checks, it is automatically synced to the specific cluster by the GitOps operator. At this point, you can re-apply the policy governance checks using an admission controller. If you have a single user deploying calico policies via GitOps and no other user is even allowed to create/update/delete network policies, then obviously you do not need any admission control checks. In contrast, if your users can update policies individually, then you need to have governance checks in place via an admission controller.

The steps in an end-to-end policy workflow should be automated as much as possible.

In our end-to-end example, we will do the following:

Let us create a standard template for developers, so they do not need to spend time researching how to create policies. This also makes the template the standard format, so all the policies are written exactly in the same way and are much easier to understand and troubleshoot. A simple template can be as follows:

While calico policies support a rich set of enforcement, you can make it easy by following a standard template. For example, Calico policies support policy enforcement on 3 types of selectors: pods, namespaces, service accounts. You can pick the appropriate selector in your organization and enforce it via the template.

In our example workflow, the policies are submitted to a central repo by corresponding app admins. As each user submits a pull request containing policy YAMLs, this is right step to apply the required checks. These checks are part of your policy governance and depend on the controls specific your organization. Some example checks for calico policies:

WARN — ./policytests/calicopolicy.yaml — You should use networkset for allow-networks-egress. Globalnetworkset is meant to be used for cluster wide resources.

FAIL — ./policytests/calicopolicy.yaml — You are trying to apply a policy cluster-wide and not allowed to do so. Please use networkpolicy for default-deny-cluster

After the PR is merged, the master has the latest version of the policies. This is the source of truth and needs to be applied to the cluster.

To replicate the above end-to-end workflow in your cluster, make sure to have the latest version (3.10) of Calico, as it supports policy for namespace selector.

We started by defining scope (how to enable a decentralized policy workflow) and the challenges (policy writing complexity, governance checks). Then we built an end-to-end policy workflow as an example. This workflow enables app teams to build and maintain their own policies (decentralized), and ensures governance in the CI pipeline. Flux ensures that Git remains as the source of truth for policies. All the pull requests record each and every change to the policies.

Hope you found this useful. In the next post in this series, we will dive into testing considerations for Calico network policies.

Related posts:

As part of our general work with systems and possibilities that are resilient, local, human and humane, we have become involved with a group running a solidarity gardening system (…

Let the mind be in you. What a powerful promise. God is ready to empower us to have victory over sin and death. And all we have to do is to let his mind be in us. To everyone, God is offering the…

Los muros tiemblan cuando las cosas se me salen de control y los suelos se inundan con las lagrimas que no puedo evitar. Dentro de mi existe un punto de esperanza, de lo que me gusta, lo que me hace…