Patterns on JayeX - Cloud Native CI/CD Built On Kubernetes

Version Everything

Mon, 01 Jan 0001 00:00:00 +0000

Always version everything explicitly in git and never use latest or SNAPSHOT images or artifacts!

The reason is you never know when a latest or SNAPSHOT image or artifact will upgrade in any container/pod/process/step in any environment. Its basically random. So not versioning things means random failures.

If you explicitly version things in all repositories things are stable and repeatable.

If you are worried about upgrading versions of things over time then look at either tools like dependabot or the updatebot plugin which we use extensively throughout JayeX to upgrade versions of libraries, images, binaries etc.

Pull Dependent Versions

Mon, 01 Jan 0001 00:00:00 +0000

When you Version Everything you need a way to pull dependent versions into your git repositories.

Using the pull model means that you periodically create Pull Requests on your git repositories to upgrade dependencies.

One solution is to use a tool tool like dependabot.

Or you can run periodic jobs via the updatebot plugin to create pull requests.

e.g. we automatically pull upgrades of chart versions into our Version Stream via this github action

Push Version Changes

Mon, 01 Jan 0001 00:00:00 +0000

When you Version Everything you need a way to pull dependent versions into your git repositories.

Using the push model means that as you create a new versioned release of an artifact (library, binary, image, chart or whatever) you generate the necessary Pull Requests in your downstream repositories to upgrade to use this version.

We do this in JayeX using the updatebot plugin as part of a release pipeline. The go-plugin release pipelines pipelines have a promote-release which uses the updatebot plugin to promote the new version to other git repositories. Here are some examples:

GitOps

Mon, 01 Jan 0001 00:00:00 +0000

The GitOps pattern means you use source code and git repositories for both your applications, your infrastructure and environments too.

In the DevOps space we have been using git to version infrastructure configuration with many tools like terraform and before that tools like Ansible.

With kubernetes you can connect to a cluster and modify it via kubectl apply or helm install`.

However with the GitOps pattern developers don’t modify kubernetes directly; instead they propose changes to a git repository via a Pull Request which when it gets approved and merged causes the kubernetes cluster to be modified via some kind of operator such as the JayeX git operator

Pipelines As Code

Mon, 01 Jan 0001 00:00:00 +0000

Define your Continuous Integration and Continuous Delivery Pipelines as code and keep them in git with your application code.

In JayeX you can see the Source Layout for more detail using standard Tekton Pipelines YAML for defining pipelines in a cloud native self-contained way.

One downside with Pipelines As Code is that you can end up copy/pasting lots of YAML into lots of repositories which can become tricky to both maintain and to keep on recent images and configurations.

Trunk Based Development

Mon, 01 Jan 0001 00:00:00 +0000

This approach is essentially:

use a single branch for releases
work in small batches
use Continuous Integration to frequently merge changes to the main branch

It is described in the Accelerate book.

The idea is to avoid long term feature branches which have a cost in terms of maintenance and being the opposite of Continuous Integration.

For more background see the Trunk Based Development site which has lots of examples and approaches.

Automate Continuous Delivery

Mon, 01 Jan 0001 00:00:00 +0000

As described in the Accelerate book, high performing teams automate their Continuous Delivery.

This lets your team focus on delivering business value for your customers rather than writing pipelines by hand.

If you use JayeX for your environments and to create projects you will have completely automated Continuous Delivery using Pipelines As Code, Preview Environments and Trunk Based Development

Preview Environments

Mon, 01 Jan 0001 00:00:00 +0000

If you are using Trunk Based Development you want to keep the main branch clean and ready to release at all times so you can quickly get important bug fixes out.

But how can you shift left and do more testing on changes before you are ready to merge to the main branch?

The idea of a Preview Environment is to spin up a temporary environment to deploy the code from a Pull Request before the change is merged to get fast feedback from your team on the change and to shift left and get more testing done before you agree to merge the change.

Prefer Cloud Over Kubernetes

Mon, 01 Jan 0001 00:00:00 +0000

You can deploy a database via a helm chart in your kubernetes cluster. Or you can configure your cloud provider to create a managed database offering.

You can deploy, say, vault as helm charts inside your kubernetes cluster. Or you can use your cloud providers secret store solution such as:

Alibaba Cloud KMS Secret Manager
Amazon Secret Manager
Azure Key Vault
GCP Secret Manager

However installing a chart in kubernetes is a very different thing to having a fully managed service which considers things like:

Map Cloud IAM to Kubernetes Service Accounts

Mon, 01 Jan 0001 00:00:00 +0000

Cloud providers make it relatively easy to map kubernetes Service Account resources to cloud IAM roles and accounts which avoids having to expose secrets to access cloud resources which simplifies things and makes your applications more secure.

On AWS use IAM roles for service accounts (IRSA)

On GCP use Workload Identity (WLI)

In both cases this maps cloud IAM roles to kubernetes ServiceAccount resources using annotations.

This means that you don’t have to populate your kubernetes cluster with cluster-admin style cloud IAM secrets - which makes your system more secure and reduces the possibility of accidentally exposing a secret.