Managing multiple software versions
Software projects are often required to maintain multiple versions of the software in parallel in order to support different users of the software with different needs. As far as source code goes, the typical way to maintain multiple versions is by using different Git branches. Using branches in conjunction with Konflux can be somewhat tedious as a separate Component needs to be defined for each branch and a separate Application needs to be defined for each collection of components that need to be tested and released together. In addition multiple other resources such as ImageRepository, IntegrationTestScenario, and ReleasePlan need to be defined for each Component or Application to enable a full CI/CD process.
The Konflux Project Controller seeks to streamline the process of managing multiple versions by introducing the following objects and concepts:
-
A Project is used to describe a major piece of software that can be worked on by multiple teams over an extended period of time. A project may contain one or more development streams.
-
A ProjectDevelopmentStream indicates an independent stream of development. A ProjectDevelopmentStream can contain one or more Applications each containing one or more Components.
-
As described above, starting a new development stream involves creating a large amount of Application, Component, and other resources. The Project Controller helps to streamline that by allowing to create a ProjectDevelopmentStreamTemplate resource that specifies the resources to be created and allows for using variables to customize them. Using a template, many similar development streams can be created quickly.
Using the Project Controller
In the sections below we will describe how to use the custom resources supported by the Project Controller to manage multiple software versions and generate Application, Component, and other resources.
Before you begin
The typical process of onboarding a component to Konflux includes having the Konflux Build Service generate a PR to the component Git repository which includes the Tekton Pipeline-as-Code (PaC) pipeline definition for building that component. As a part of this process various other resources such as the PaC Repository resource and Secret resources for accessing the repository are also created.
When using the Project Controller to create components the process of generating the PR is skipped because in this document we assume the branches being created in the component Git repositories already contain PaC pipeline files. Each Git repository, however, needs to undergo this process at least once, therefore it is recommended to onboard at least one component from each repository from the UI or by other means before attempting to manage components in that repository via the Project Controller.
The rest of this document is going to assume that process is being followed and would, on occasion request that data would be copied from the component that was already on-boarded by other means.
Creating a Project
Create a project resource by applying YAML like the following:
apiVersion: projctl.konflux.dev/v1beta1
kind: Project
metadata:
name: multi-version-konflux-sample
spec:
displayName: "Multi-version demonstration sample project"
description: |
A sample project to demonstrate how to use the projects API.
Creating a ProjectDevelopmentStreamTemplate
To enable quickly creating multiple development streams, we must create a template for them.
apiVersion: projctl.konflux.dev/v1beta1
kind: ProjectDevelopmentStreamTemplate
metadata:
name: multi-version-konflux-sample-template
spec:
project: multi-version-konflux-sample
variables:
- name: version
description: A version number for a new development stream
- name: versionName
description: A K8s-compliant name for the version
defaultValue: "{{hyphenize .version}}"
resources:
- apiVersion: appstudio.redhat.com/v1alpha1
kind: Application
metadata:
annotations:
application.thumbnail: "9"
name: "multi-version-konflux-sample-{{.versionName}}"
spec:
displayName: "multi-version-konflux-sample-{{.versionName}}"
- apiVersion: appstudio.redhat.com/v1alpha1
kind: Component
metadata:
annotations:
build.appstudio.openshift.io/pipeline: '{"name":"docker-build","bundle":"latest"}'
build.appstudio.openshift.io/status: '{"pac":{"state":"enabled","merge-url":"https://github.com/konflux-ci/multi-version-konflux-sample/pull/1","configuration-time":"Wed, 07 Aug 2024 08:59:18 UTC"},"message":"done"}'
name: multi-version-konflux-sample-{{.versionName}}
spec:
application: "multi-version-konflux-sample-{{.versionName}}"
componentName: "multi-version-konflux-sample-{{.versionName}}"
source:
git:
context: ./
dockerfileUrl: Dockerfile
revision: "{{.version}}"
url: https://github.com/konflux-ci/multi-version-konflux-sample.git
- apiVersion: appstudio.redhat.com/v1alpha1
kind: ImageRepository
metadata:
name: imagerepository-for-multi-version-konflux-sample-{{.versionName}}
labels:
appstudio.redhat.com/application: multi-version-konflux-sample-{{.versionName}}
appstudio.redhat.com/component: multi-version-konflux-sample-{{.versionName}}
spec:
image:
name: konflux-samples-tenant/multi-version-konflux-sample-{{.versionName}}
visibility: public
notifications:
- config:
url: 'https://bombino.preprod.api.redhat.com/v1/sbom/quay/push'
event: repo_push
method: webhook
title: SBOM-event-to-Bombino
The resources section for the template may be created by looking at the YAML for existing resources and copying it while removing generated and unnecessary data and adding variable references where needed.
The following kinds of resources are currently supported. Please do not attempt to create other kinds as that would prevent the template from being applied successfully.
apiVersion | kind | Comments |
---|---|---|
appstudio.redhat.com/v1alpha1 |
Application |
|
appstudio.redhat.com/v1alpha1 |
Component |
|
appstudio.redhat.com/v1alpha1 |
ImageRepository |
Every component should have an accompanying |
appstudio.redhat.com/v1beta2 |
IntegrationTestScenario |
Note the |
appstudio.redhat.com/v1alpha1 |
ReleasePlan |
Note: When in doubt - the source of truth about what the YAML for your resources should look like is the version you can find on the Konflux clusters. We try to keep the examples here up to date, but with this being a static document there may be a gap between what you see here and what you should configure in your system. It is recommended that you follow the process of creating the template by copying the YAML of the resources you have and cleaning it as described below, and not by copying the sample you see here.
Here are specific examples for how to clean up and use the YAML for certain resource kinds:
-
For any kind of resource specified in the ProjectDevelopmentStreamTemplate, the
namespace
,creationTimestamp
,generation
,resourceVersion
,uid
,ownerReferences
, andfinalizers
metadata fields should be removed as well as thestatus
section. -
For Application resources the
metadata.name
andspec.displayName
fields should contain variable references. -
For Component resources:
-
The following deprecated annotations should be removed:
-
image.redhat.com/image
-
-
The
spec.containerImage
field should be removed. -
The following fields should probably contain variable references:
-
spec.application
-
spec.componentName
-
source.git.revision
-
-
The
build.appstudio.openshift.io/status
annotation is in place to make the UI present the component’s pipeline as customized. It’s not mandatory if you can ignore the pipeline status being misrepresented in the UI. Note that this annotation’s value needs to contain a reference to a merged PR that added the PaC pipeline files. There is no harm in having multiple components reference the same PR, as long as thepac.component.appstudio.openshift.io/finalizer
finalizer is not added to the component’s finalizers list.
-
-
For ImageRepository resources:
-
The labels referring to the owning component and application should probably contain variable references.
-
To allow for correct ownership configuration between Component and ImageRepository resources, every component that has its built image pushed to the default registry organization that is managed automatically by Konflux needs to have a dedicated ImageRepository resource. This implies each component has its own container image repository.
-
To meet the above requirement the
spec.image.name
field should contain a variable reference.
-
-
For IntegrationTestScenario resources:
-
Make sure you query for the
v1beta2
version of those resources, and that you specify that version in your template. -
The
spec.application
field should probably contain variable references.
-
-
For ReleasePlan resources:
-
The
spec.application
field should probably contain variable references.
-
Some notes about using template variables:
-
You can use the Go text/template syntax to place template variable values into various resource attributes as well as variable default values.
-
You can use the custom
hyphenize
template function to create a value suitable for use in resource names. -
It’s advisable to quote strings that contain variable references and other template syntax elements to prevent the curly braces from being parsed as JSON embedded into YAML.
Creating a ProjectDevelopmentStream
Once the Project and ProjectDevelopmentStreamTemplate resources are in place, we can create ProjectDevelopmentStream resources.
apiVersion: projctl.konflux.dev/v1beta1
kind: ProjectDevelopmentStream
metadata:
name: multi-version-konflux-sample-v1-0-0
spec:
project: multi-version-konflux-sample
template:
name: multi-version-konflux-sample-template
values:
- name: version
value: "v1.0.0"
Creating this ProjectDevelopmentStream resource will cause the resources specified by the referenced ProjectDevelopmentStreamTemplate resource to get created. Since we’ve used the version
template variable in the spec.git.revision
field of the component resources, each component version will use a different branch of the component repository.
When you look at your components in the Konflux UI, you may see notifications that PRs were sent to configure pipelines for them. In addition the source code links in the UI will not work until corresponding branches are actually created in your Git repository. To overcome those issues, create and push a branch with the appropriate name for each new component and then create and merge a PR into it that will cause the push pipeline to run for that branch. For the pipeline to run you need to adjust the Tekton PaC pipeline YAML code as described blow. We recommend that the first PR you send into a branch would include those adjustments.
Branching your component repositories
Beyond creating new Git branches for your components in order to maintain different versions, you must also adjust the .tekton/*.yaml
files within those branches in order to make the pipelines run and target the right components.
In particular the following changes must be made each time a new branch is created in each of the pipeline YAML files:
-
The
pipelinesascode.tekton.dev/on-cel-expression
annotation should be adjusted to specify and filter by the right branch name. For example, for a pull request pipeline that resides in thev1.0.0
branch the annotation value would be:event == "pull_request" && target_branch == "v1.0.0"
For a push pipeline in the same branch the value would be:
event == "push" && target_branch == "v1.0.0"
-
The
appstudio.openshift.io/application
andappstudio.openshift.io/component
labels must be adjusted to specify the right Application and Component respectively. Failing to do this will cause builds of the pipeline to be associated with the wrong application or component. -
The value for the
output-image
parameter should be set to match the value of thespec.image.name
field of theImageRepository
resource that corresponds to the component the pipeline would build.If your template is setup in such a way that the repo branch name would appear in the image name, you can use the
target_branch
PaC variable like so:- name: output-image value: quay.io/redhat-user-workloads/my-tenant/my-app-{{target_branch}}/my-comp-{{target_branch}}:{{revision}}
Please note that the examples in this document do not allow for this setup because the image names contain the hyphenated version value while the branch names contain the unhyphenated value.
Known limitations
The following limitations exist in the current controller implementation and are likely to be resolved in the future.
-
Resource creation order is important. You must first create Project resources followed by ProjectDevelopmentStreamTemplate resources and only then ProjectDevelopmentStream resources.
-
If a ProjectDevelopmentStreamTemplate is modified, resources that were already created using that template do not get updated unless either:
-
The controller gets restarted
-
The ProjectDevelopmentStream resource referring to the template is modified
-
-
If a resource created by a template is modified, the configuration is not aligned back with the template unless either:
-
The controller gets restarted
-
The ProjectDevelopmentStream resource referring to the template is modified
-
-
A ProjectDevelopmentStream that isn’t referring a template may be modified to refer to a template. Similarly, the template ProjectDevelopmentStream it’s referring to may be changed. In both those cases, resources owned by the ProjectDevelopmentStream but not defined by the new template do not get deleted.