TLS and DNS on JayeX - Cloud Native CI/CD Built On Kubernetes

Demo

Mon, 01 Jan 0001 00:00:00 +0000

Demo

The following demo walks you through how to setup TLS and DNS:

Background

Mon, 01 Jan 0001 00:00:00 +0000

To achieve this we will use a couple of open source projects to help enable automated DNS for your applications.

For this guide we are going to assume you own a domain called foo.io which is managed by Google Cloud DNS, if it is not see configure cloud dns to manage a domain. A similar procedure is described for an external registrar, see the Azure section.

A common requirement for domains is to have production services accessed using a parent / apex domain for example:

Azure

Mon, 01 Jan 0001 00:00:00 +0000

DNS zone creation

Create a common resource group dedicated to all your DNS zones:

$ az group create --name rg-dns --location westeurope
{
  "id": "/subscriptions/49721339-fe83-4562-afec-783c3f00c06f/resourceGroups/rg-dns",
  "location": "westeurope",
  "managedBy": null,
  "name": "rg-dns",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}

Create an Azure DNS zone with the name of your domain:

$ az network dns zone create -g rg-dns -n foo.io
{
  "etag": "00000002-0000-0000-e2d9-80ef0df0d601",
  "id": "/subscriptions/49721339-fe83-4562-afec-783c3f00c06f/resourceGroups/rg-dns/providers/Microsoft.Network/dnszones/foo.io",
  "location": "global",
  "maxNumberOfRecordSets": 10000,
  "name": "foo.io",
  "nameServers": [
    "ns1-05.azure-dns.com.",
    "ns2-05.azure-dns.net.",
    "ns3-05.azure-dns.org.",
    "ns4-05.azure-dns.info."
  ],
  "numberOfRecordSets": 2,
  "registrationVirtualNetworks": null,
  "resolutionVirtualNetworks": null,
  "resourceGroup": "rg-dns",
  "tags": {},
  "type": "Microsoft.Network/dnszones",
  "zoneType": "Public"
}

Domain DNS servers configuration

In your registrar admin panel, find the DNS servers section of the domain you want to use and replace the default ones by those from the first step:

Google

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

cluster created using JayeX GCP Terraform getting started
own a domain and have GCP manage it, configure cloud dns to manage a domain
latest JayeX CLI, Infrastructure and Cluster git repository updates upgrade

Cloud Infrastructure

First we will configure the cloud infrastructure requirements:

a GCP Service Account with the dns.admin role, see here for more information
a managed cloud dns zone, see here for more information

To satisfy these requirements go to your infrastructure repository (contains Terraform main.tf) and add to your values.auto.tfvars the following:

Cluster

Mon, 01 Jan 0001 00:00:00 +0000

Next we will configure the cluster requirements:

Install external-dns - Kubernetes controller which watches for new Kubernetes Ingress resources and creates A records in Google Cloud DNS which will propagate globally across the internet
Install cert-manager - Kubernetes controller which watches for requests to ask Let’s Encrypt to issue a new wildcard TLS certificate for your domain and will manage this including renewals

Cert-manager will use the cluster issuer to request a TLS certificate. A Kubernetes secret will be automatically created and contain the TLS cert. The nginx controllers in the nginx namespace will use this secret in the jx namespace for the default SSL certificate which will automatically enable TLS for all applications in your cluster.

Multiple domains

Mon, 01 Jan 0001 00:00:00 +0000

Example: Use different domains for production and staging environments

Add acme chart to helmfiles/jx-production/helmfile.yaml

- chart: jx3/acme
  version: 0.0.19
  condition: jxRequirementsIngressTLS.enabled
  name: acme-jx
  namespace: jx-production
  values:
  - jx-values.yaml
  - acme-values.yaml

Create a file helmfiles/jx-production/acme-values.yaml

issuer:
  enabled: true
  cluster: false

Do the same for other environments, e.g. staging. Make sure the correct namespace is set.

Add ingress and tls configuration to the environments in jx-requirements.yaml

  environments:
    - ingress:
        domain: staging.foo.io
        externalDNS: true
        tls:
          email: admin@foo.com
          enabled: true
          production: true
          secretName: tls-staging-foo-io-p
      key: staging
    - ingress:
        domain: foo.io
        externalDNS: true
        tls:
          email: admin@foo.com
          enabled: true
          production: true
          secretName: tls-foo-io-p
      key: production

Make sure to use the correct secretName, as it’s being generated (production or staging) by the acme chart.

FAQ

Mon, 01 Jan 0001 00:00:00 +0000

What if I have a chartmuseum with charts running using nip.io?

It is best to comment out your JayeX chartmuseum repository and charts from your helmfile until your new domain and ingress is working. Then uncomment and make sure you update the chartmuseum URL to your new one.

What if I use a subdomain with an apex domain in a different GCP project?

When using a subdomain Terraform will create a managed zone in GCP, add the recordsets to your parent / apex domain.