Crossplane is a CNCF sandbox project which can extend the Kubernetes API to manage and compose infrastructure. It allows you to assemble infrastructure from multiple vendors and expose as higher level abstraction to consume without having to write any code.
As the second post of my Crossplane journey, in this post, I will share with you what I learned with some opinionated tips as complements to the official Crossplane documents on how to build, publish and install Crossplane package.
About Crossplane Package
Before jump into the details, below is a very quick recap of what Crossplane package is. You can find more detailed information about Crossplane package from Crossplane documentation.
Crossplane packages are opinionated OCI images that contain a stream of YAML that can be parsed by the Crossplane package manager. It can be pushed to and pulled from any OCI-compatible registry such as Docker Hub.
There are two types of packages: provider package and configuration package. Provider extends Crossplane to enable infrastructure resource provisioning. As an example, in my case, I am using provider-kubernetes, which can enable deployment and management of arbitrary Kubernetes resources on clusters. Provider package is used to wrap up and distribute provider so that can be installed and consumed by other people.
On the other hand, configuration package is used to wrap up a set of
Composition definitions in YAML, which can be installed on clusters so that you can leverage the powerful composition capability provided by Crossplane to quickly compose different resources together to provision infrastructure that meets your specific need.
The crossplane.yaml file
No matter it is for provider or configuration, all packages must have a file called
crossplane.yaml in the root directory. For providers, it is typically located at the
package folder in the project root directory. This file contains the package’s metadata which governs how Crossplane will install the package.
In my case, I have both provider and configuration where the configuration depends on the provider which requires it to be installed at first. The provider dependency along with its version constraint is defined in
crossplane.yaml. If the dependency is not installed, Crossplane will install it at the latest version that fits within the provided constraints. As an example, below is the
crossplane.yaml file for my configuration package:
- provider: quay.io/morningspace/provider-kubernetes-amd64
There are couple of things to note:
- When specify the provider as dependency using
spec.dependsOn, you should not append the provider version (i.e. the image tag) after the provider name. This may lead to the missing dependency error when Crossplane tries to install the provider.
- When specify the version constraint, you should strictly follow the semver spec. Otherwise, it may not be able to find the appropriate version even dependency is found. This may lead to the incompatible dependency error.
In the above example, the configuration package depends on a provider where the provider image is pulled from
quay.io/morningspace/provider-kubernetes-amd64. It also defines the version constraint
">=v0.0.0-0which means all versions started from
v0.0.0 including all prerelease versions are qualified to be installed to serve this configuration package.
According to Crossplane documentation, it is recommended to use Crossplane CLI to build Crossplane package, which essentially works as a kubectl plugin. To build a package, you can navigate to the package root directory and choose one of the below commands to execute, depending on which type of package you are going to build:
# To build a provider package
kubectl crossplane build provider# To build a configuration package
kubectl crossplane build configuration
Besides that, if you build package for a provider, you can also use
make. To build source code and other artifacts including the package for host platform, just run below command in project root directory:
To build for all platforms, you can run:
After you successfully build the package locally, you can push the package to the target registry. If you use Crossplane CLI, again, navigate to the package root directory and choose one of the below commands to execute, depending on which type of package you are going to push:
# To push a provider package
kubectl crossplane push provider $ORG_NAME/$PROVIDER_NAME:$PROVIDER_VERSION# To push a configuration package
kubectl crossplane push configuration $ORG_NAME/$CONFIGURATION_NAME:$CONFIGURATION_VERSION
As an example, I want to push my configuration package with version
v0.0.1 to quay.io:
kubectl crossplane push configuration quay.io/morningspace/my-test-configuration:v0.0.1
Besides that, if you publish package for a provider, you can also use
make. To push the package to the registry, just run below command in project root directory:
If you only want to publish package for a specific platform, e.g. linux_amd64, to the registry, you can specify
PLATFORMS before run
make publish. For example:
PLATFORMS=linux_amd64 make publish
Push to Registry other than Docker Hub
By default, most Crossplane providers use Docker Hub as the target registry. In order to push package to other registry such as
quay.io. You got two things to do.
Firstly, you need to modify the provider
crossplane.yaml file, change the
spec.controller.image to point to the right registry. Otherwise, when install the provider, it will still go back to check Docker Hub. Below is an example:
The Crossplane Kubernetes provider enables management of Kubernetes Objects.
Secondly, most providers define
DOCKER_REGISTRY with the default value
crossplane in their
Makefile files. This is required by the provider build scripts to push to the target registry. This also needs to be updated to reflect the new registry. For example:
# DOCKER_REGISTRY = crossplane
DOCKER_REGISTRY = quay.io/morningspace
Now that you have pushed provider and configuration packages to the target registry, you can install them on clusters using Crossplane CLI from anywhere. Choose one of the below commands to execute, depending on which type of package you are going to install:
# To install a provider package
kubectl crossplane install provider $ORG_NAME/$PROVIDER_NAME:$PROVIDER_VERSION# To install a configuration package
kubectl crossplane install configuration $ORG_NAME/$CONFIGURATION_NAME:$CONFIGURATION_VERSION
As an example, I want to install my configuration package with version
v0.0.1 from quay.io:
kubectl crossplane install configuration quay.io/morningspace/my-test-configuration:v0.0.1
If it is installed successfully, you should see the
Composition definitions included in this package are installed on the cluster. It will also install its dependency, the provider package.
To verify the install results, you can run:
kubectl get crossplane
It will list all the
Composition resources, the
ProviderRevision resources, the
ConfigurationRevision resources that are currently available on the cluster.
Pay attention to the column
Configuration resources, as well as the column
ConfigurationRevision resources, they all need to be
TRUE. Otherwise, there must be some errors occurred during the installation. For example:
NAME INSTALLED HEALTHY PACKAGE AGE
provider.pkg.crossplane.io/morningspace-provider-kubernetes-amd64 True True quay.io/morningspace/provider-kubernetes-amd64:v0.0.0-29.ga50151c 6dNAME HEALTHY REVISION IMAGE STATE DEP-FOUND DEP-INSTALLED AGE
providerrevision.pkg.crossplane.io/morningspace-provider-kubernetes-amd64-f098e786458d True 1 quay.io/morningspace/provider-kubernetes-amd64:v0.0.0-29.ga50151c Active 6dNAME INSTALLED HEALTHY PACKAGE AGE
configuration.pkg.crossplane.io/morningspace-my-test-configuration True True quay.io/morningspace/my-test-configuration:v0.0.1 5d1hNAME HEALTHY REVISION IMAGE STATE DEP-FOUND DEP-INSTALLED AGE
configurationrevision.pkg.crossplane.io/morningspace-my-test-configuration-6c6e494f5a22 True 1 quay.io/morningspace/my-test-configuration:v0.0.1 Active 1 1 5d1h
Also, in the above example, the
ConfigurationRevision shows that there is one dependency found (
DEP-FOUNDis 1) and it has been installed successfully (
DEP-INSTALLED is 1).
When there are errors, you can run below command which gives you detailed information of all packages that are being installed:
kubectl get lock -o yaml
To inspect a specific provider or configuration package that is in issue, you can run
kubectl describe against the corresponding resource, e.g. the
ProviderRevision resources for provider, the
ConfigurationRevision resources for configuration. Usually, you should be able to know the error reason by checking the Status and Events field.