Using Crossplane in GitOps: Common Considerations

Deploying Order

Argo CD Syncwave

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: crossplane-app
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "100"
...
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: olm-app
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "100"
...
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: sealed-secrets-controller
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "100"
...
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: crossplane-ext-app
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "150"

Dependency Resolving by Crossplane

spec:
patches:
- fromObject:
apiVersion: v1
kind: ConfigMap
name: common-settings
namespace: crossplane-system
fieldPath: data.region
toFieldPath: spec.forProvider.region
---
apiVersion: kubernetes.crossplane.io/v1alpha1
kind: Object
metadata:
name: csv-my-logging-stack
annotations:
kubernetes.crossplane.io/managementType: "ObservableAndDeletable"
spec:
references:
- fromObject:
apiVersion: kubernetes.crossplane.io/v1alpha1
kind: Object
name: sub-my-logging-stack
fieldPath: status.atProvider.manifest.status.currentCSV
toFieldPath: spec.forProvider.manifest.metadata.name
forProvider:
manifest:
apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
namespace: operators
providerConfigRef:
name: provider-config-dev

Uninstall in Reversed Order

Using Hooks

Argo CD Hooks

apiVersion: batch/v1
kind: Job
metadata:
name: post-install-job
namespace: argocd
annotations:
argocd.argoproj.io/hook: PostSync
spec:
template:
metadata:
name: post-install-job
spec:
containers:
- name: check-csv
image: quay.io/bitnami/kubectl:latest
command:
- /bin/sh
- -c
- |
current_seconds=0
limit_seconds=$(( $(date +%s) + 600 ))
installing=0

while [ ${current_seconds} -lt ${limit_seconds} ]; do
if [ $(kubectl get csv -n "{{.Values.metadata.namespace}}" -o name | grep -c clusterserviceversion) -eq 0 ] || \
[ $(kubectl get csv -n "{{.Values.metadata.namespace}}" --no-headers | grep -vc Succeeded) -gt 0 ]; then
installing=1
echo "INFO: CSVs still installing."
sleep 1
else
installing=0
break
fi
current_seconds=$(( $(date +%s) ))
done

if [ ${installing} -eq 0 ]; then
echo "INFO: All CSVs are ready."
else
echo "ERROR: CSVs still not ready."
exit 1
fi
restartPolicy: Never
serviceAccountName: argocd-application-controller
backoffLimit: 1

Helm Hooks

Health Check

---
apiVersion: v1
kind: ConfigMap
metadata:
name: argocd-cm
namespace: argocd
labels:
app.kubernetes.io/name: argocd-cm
app.kubernetes.io/part-of: argocd
data:
resource.customizations.health.operators.coreos.com_ClusterServiceVersion: |
hs = {}
hs.status = "Progressing"
hs.message = ""
if obj.status ~= nil then
if obj.status.phase == "Succeeded" then
hs.status = "Healthy"
end
hs.message = obj.status.message
end
return hs
resource.customizations.health.elasticsearch.k8s.elastic.co_Elasticsearch: |
hs = {}
hs.status = "Progressing"
hs.message = ""
if obj.status ~= nil and obj.status.health ~= nil and obj.status.phase ~= nil then
if obj.status.health == "green" and obj.status.phase == "Ready" then
hs.status = "Healthy"
end
hs.message = "health = " .. obj.status.health .. " phase = " .. obj.status.phase
end
return hs
resource.customizations.health.kubernetes.crossplane.io_Object: |
hs = {}
hs.status = "Progressing"
hs.message = ""
if obj.status ~= nil and obj.status.atProvider ~= nil then
kind = obj.spec.forProvider.manifest.kind
res = obj.status.atProvider.manifest
if res ~= nil then
if kind == "ClusterServiceVersion" then
if res.status ~= nil then
if res.status.phase == "Succeeded" then
hs.status = "Healthy"
end
hs.message = res.status.message
end
elseif kind == "Elasticsearch" then
if res.status ~= nil and res.status.health ~= nil and res.status.phase ~= nil then
if res.status.health == "green" and res.status.phase == "Ready" then
hs.status = "Healthy"
end
hs.message = "health = " .. res.status.health .. " phase = " .. res.status.phase
end
end
end
end
return hs

Organizing Configuration

  • Place common configuration that can be applied to all environments in one single place. This may include the common infrastructure configuration, shared applications and their settings that are applicable to all environments. For example, in the demo project, we use config/ folder to store all configuration required to deploy Crossplane and its provider, OLM, Sealed Secret Controller, as well as the Argo CD customization settings and RBAC settings.
  • Place per-environment configuration in separate places and one place for each environment. Some environment may have its unique configuration which can be put in a place represents that specific environment. For example, there can be separate folders for dev, staging, and product environment. In our demo project, we use environments/ folder to host all environment specific configuration, where we place the Crossplane ProviderConfig, the encrypted secret that represents the target cluster kubeconfig credentials, and so on. These are all configuration unique to each cluster.
  • Use branch to track per-release configuration. It is a very common practice for developer to track code changes among different releases using git release branch, especially when you have multiple releases that need to be maintained at the same time. The same rules apply to configuration changes in GitOps. When you have multiple releases to support and the configuration keeps changing from release to release, you can create branch for each release. Of course, it will bring additional effort to merge changes among branches and resolve merge conflicts when needed. The good thing is, all branching and merging practices that you have already been familiar with when dealing with code changes can also be applied to the configuration changes.
  • Host multiple applications manifests in a monolithic repository. This is a very effective way to manage applications in a small project where you put all applications configuration manifests in a single repository. It can be stored in separate folders, one folder for each application.
  • Host multiple applications manifests separately in multiple repositories. This is suitable for a large project in which you may have multiple products, each product has its own set of applications, and maintained by different team. By putting configuration manifests for these applications in separate repositories, you can leverage the sophisticated organization or repository membership and access control capabilities provided by the git infrastructure provider such as GitHub, to manage application deployment for each team differently in a fine grained manner.
  • Use Helm to parameterize deployment manifests and turn into reusable templates. In some cases, you may want your application manifests to be configurable, e.g.: to allow Ops or SREs to choose which version to install, which storage to pick up, or which namespace to apply, etc. Helm as a deployment tool for Kubernetes application is widely used. It can help you extract parameters out of deployment manifest, and turn the manifest into a reusable template. Helm can also be used together with Crossplane, especially when you only use Crossplane providers to provision applications and, do not use its composition. In such case, Crossplane plays very similar role as Kubernetes controller or Operator does with its wide range of providers that extend the scope of what you can manage using GitOps.
  • Use Kustomize to override deployment manifests as a base for a specific environment. In some cases, you may want your application manifests to be customized on a per environment basis. Instead of Helm, this can also be achieved by using Kustomize. You can put the manifests with default configuration in a folder as a base layer, then put environment specific manifest pieces in a separate folder that represents a specific environment to override the base layer. This can also be applied when you use Crossplane, especially when using composition. For example, you can define the default CompositeResourceClaim (XRC) at base layer, then override it at environment specific layer.
  • Use App of Apps pattern when use Argo CD to mange a set of applications. In Argo CD, an Application resource is a unit that deploys a set of manifests. Since an Application is a Kubernetes resource, it can be managed by Argo CD too. Furthermore, an Application resource can manage multiple other Application resources. That is called App of Apps pattern. By using this pattern, you can deploy a set of applications in one go. Also, increasing or decreasing Application resources can be done by adding or removing manifests to git repository instead of operating Argo CD via its Web UI or command line.
├── config
│ ├── argocd-apps
│ ├── infra
│ ├── services
│ └── apps
└── environments
├── base
└── overlays
├── dev
├── staging
└── prod

--

--

--

Life is coding and writing! I am a software engineer who have been in IT field for 10+ years. I would love to write beautiful code and story for people.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Install Python 3.7 and Jupyter Notebooks on Windows, Mac OS, and Linux

Sardes Test Platform — Part I

SQL School — How Marketing and Other Departments Can Save Hours of Work

How to Buy an iPhone 13 Fast Before the Holidays

Ritsec CTF : WEB Challenges Writeup

5 things I wish I knew before adding variable in .env

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
MorningSpace

MorningSpace

Life is coding and writing! I am a software engineer who have been in IT field for 10+ years. I would love to write beautiful code and story for people.

More from Medium

Argo CD Basics— CNCF Roadmap

Using GitOps, Multiple Argo Instances, and Environments with Argo CD at Scale

Demystifying GitOps - Intro

Four Pillars of Kubernetes Fleet Management