OpenShift Pipelines

Overview

At the end of this chapter you should be able to:

  • Use Cluster Tasks in OpenShift

  • Create or Update Cluster Tasks in OpenShift

  • Use OpenShift Pipelines builder GUI to run a Pipeline

If you are not in tutorial chapter folder, then navigate to the folder:

cd $TUTORIAL_HOME/openshift

It is assumed that you have completed the following chapters before trying this chapters exercies:

Demo Application

The Cloud Native Application pipeline will,

  • Git clone using git-clone task, the https://github.com/redhat-scholars/tekton-greeter repository

  • Will use maven task, maven task require two workspaces one for source and other for maven-settings

  • Build the container image using the buildah task and push to the internal container registry

  • Finally deploy the application onto Kubernetes using openshift-client task or use kn task to deploy it as Serverless Knative application.

Project Preparation

All exercises of this chapter will be done a namesapace pipelines-demos, lets create the project if not done earlier and switch to the pipelines-demos namespace:

odc create project
Figure 1. Create Project
odc create project pipelines demos
Figure 2. Project Name and Details

After entering the details click Create to create and navigate to the pipelines-demos project:

odc on pipelines demos
Figure 3. Project pipelines-demos

Since we will be use oc to execute few commands, lets make sure we are in the pipelines-demos project:

oc project pipelines-demos
Now using project "pipelines-demos" on server "https://api.gcp.kameshs.dev:6443".

Prepare Application Deployment

Now we are all set to deploy the application, before hitting the create we need prepare the namespace with few additonal resources as shown in the following section:

Deploy Nexus

To show maven artifact caching and using customized maven settings.xml we will deploy Sonatype Nexus to the cluster:

kubectl apply -n pipelines-demos \
  -f $TUTORIAL_HOME/install/utils/nexus.yaml

Wait for the nexus3 pod to come up before proceeding further with exercises:

watch kubectl -n pipelines-demos get pods,svc

A sucessfully running nexus3 pod should show the following services and pods:

NAME                         READY   STATUS    RESTARTS   AGE
pod/nexus-75fff8bbbd-cmlxs   1/1     Running   0          3m25s

NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
service/nexus        NodePort    10.100.205.22   <none>        8081:31743/TCP   3m25s

In OpenShift we can use routes to expose the service if we need to access it from outside the cluster by:

oc expose svc -n pipelines-demos nexus

Create maven-settings ConfigMap

For us to mount the maven settings.xml, we will create ConfigMap holding the custom maven settings.xml:

kubectl create -n pipelines-demos cm maven-settings \
  --from-file=settings.xml=$TUTORIAL_HOME/workspaces/maven-settings.xml
configmap/maven-settings created

You can check the ConfigMap contents using the command:

kubectl get -n pipelines-demos cm maven-settings -o yaml \
    -o jsonpath='{.data}'

The output of the above command should be same as in maven-settings.xml.

Deploy OpenShift Serverless

As we will be deploying a Knative serverless application as part of the pipeline, that we will be running later in this chapter, lets deploy Knative to the cluster:

OpenShift users can install Knative by following the instructions here for installing OpenShift Serverless.

Follow the instructions for installing the Serverless Operator on the cluster and then follow the instructions for creating Knative Serving.

Check Default Storage Class

Before we create our pipeline ensure that the kubernetes cluster has a default storage class defined:

kubectl get sc

An example default storage class in Google Cloud:

NAME                 PROVISIONER            RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
standard (default)   kubernetes.io/gce-pd   Delete          WaitForFirstConsumer   true                   2d3h
  • In OpenShift cluster the default storage class varies based on the underlying cloud platform. Refer to OpenShift Documentation to know more.

Create PVC

Create the PVC tekton-tutorial-sources, which we will use as part of the exercises in this chapter and the upcoming ones.

kubectl apply -n pipelines-demos -f $TUTORIAL_HOME/workspaces/sources-pvc.yaml

Required Cluster Tasks

The Cloud Native Application pipeline requires the following tasks:

  • Git Clone

  • Maven

  • Buildah

  • Openshift Client

  • Kn client

OpenShift has all these tasks intalled as ClusterTasks as part of the openshift-pipelines install.

You can list the available ClusterTasks using the command:

tkn clustertasks ls

The command should show the following output:

NAME                       DESCRIPTION   AGE
buildah                                  13 hours ago
buildah-v0-11-3                          13 hours ago
git-clone                                13 hours ago
jib-maven                                13 hours ago
kn                                       13 hours ago
maven                                    13 hours ago
openshift-client                         13 hours ago
openshift-client-v0-11-3                 13 hours ago
s2i                                      13 hours ago
s2i-dotnet-3                             13 hours ago
s2i-dotnet-3-v0-11-3                     13 hours ago
s2i-go                                   13 hours ago
s2i-go-v0-11-3                           13 hours ago
s2i-java-11                              13 hours ago
s2i-java-11-v0-11-3                      13 hours ago
s2i-java-8                               13 hours ago
s2i-java-8-v0-11-3                       13 hours ago
s2i-nodejs                               13 hours ago
s2i-nodejs-v0-11-3                       13 hours ago
s2i-perl                                 13 hours ago
s2i-perl-v0-11-3                         13 hours ago
s2i-php                                  13 hours ago
s2i-php-v0-11-3                          13 hours ago
s2i-python-3                             13 hours ago
s2i-python-3-v0-11-3                     13 hours ago
s2i-ruby                                 13 hours ago
s2i-ruby-v0-11-3                         13 hours ago
s2i-v0-11-3                              13 hours ago
tkn                                      13 hours ago

Update Cluster Tasks

As we will need the updated version of kn, maven and buildah clustertasks, run the following command to update them:

Update buildah Cluster Task

oc replace \
  -f https://raw.githubusercontent.com/tektoncd/catalog/master/task/buildah/0.1/buildah.yaml --dry-run=client -oyaml \
  | yq '.kind="ClusterTask"' \
  | oc replace -f -
clustertask.tekton.dev/buildah replaced

Update maven Cluster Task

oc replace \
  -f https://raw.githubusercontent.com/tektoncd/catalog/master/task/maven/0.1/maven.yaml --dry-run=client -oyaml \
  | yq '.kind="ClusterTask"' \
  | oc replace -f -
clustertask.tekton.dev/maven replaced

Update kn Cluster Task

oc replace \
  -f https://raw.githubusercontent.com/tektoncd/catalog/master/task/kn/0.1/kn.yaml --dry-run=client -oyaml \
  | yq '.kind="ClusterTask"' \
  | oc replace -f -
clustertask.tekton.dev/kn replaced

Deploy application from GitHub Repository

Let us use the OpenShift Developer Console(ODC) to deploy the application from GitHub repository, click From Git in the OpenShift Developer Console:

odc deploy app from git

In the Git Repo URL field, enter the tutorials GitHub repository URL:

https://github.com/redhat-scholars/tekton-tutorial-greeter
ocp dc choose git

By default the builder image will choosen to be Java, based on the GitHub repository content. Leave the Builder Image Version to be 11 and update the Name to be greeter:

ocp dc name it
Figure 4. Greeter Application

On the resources you can choose either Deployment or Knative Service. Try choosing Deployment, and should see an option to Add Pipeline with default Pipeline template avaiable for this runtime Java and resource type Deployment, as shown:

ocp dc deploy add pipeline
Figure 5. Java Deployment Pipeline
You Show/Hide Pipeline visualization by toggling Show[Hide] Pipeline visualization

But for this exercise we will deploy the application as Knative Service, so flipping Resources selection to be of type Knative Service will not show any available pipelines:

ocp dc ksvc no pipeline
Figure 6. Java Knative Service

Hit Cancel button to cancel the deployment. We will do a fresh deployment once we deploy the Pipeline template for runtime Java and resource type Knative Service.

Create Pipeline Template for Java and Knative Service

As part of this exercise we will deploy a Pipeline Template, that will allow us to deploy the a Java Knative Service application using ODC.

---
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: cloud-native-app-serverless
  namespace: openshift (1)
  labels:
    app.kubernetes.io/instance: tekton-tutorial-greeter
    app.kubernetes.io/name: tekton-tutorial-greeter
    pipeline.openshift.io/runtime: java (2)
    pipeline.openshift.io/type: knative (3)
spec:
  description: >-
    The Pipeline to build and deploy the Hello World Java App <a href="https://github.com/redhat-scholars/tekton-tutorial-greeter" class="bare">https://github.com/redhat-scholars/tekton-tutorial-greeter</a> as Knative Service.
  params:
    - name: IMAGE_NAME
      description: >-
        The fully qualified image name e.g example.com/tekton-tutorial/greeter
      default: image-registry.openshift-image-registry.svc:5000/tektontutorial/greeter
    - name: APP_PROFILE
      description: >-
        The application profile to use for building the application. In this example it should be either:
          * quarkus
          * springboot
    - name: APP_NAME (4)
      description: The Knative Service name
    - name: GITHUB_REPO_URL
      description: The GitHub Repo of the Java Application
      default: <a href="https://github.com/redhat-scholars/tekton-tutorial-greeter" class="bare">https://github.com/redhat-scholars/tekton-tutorial-greeter</a>
    - name: GITHUB_REPO_REVISION
      description: The GitHub revision to use
      default: master
  workspaces:
    - name: source
    - name: maven-settings
  tasks:
    - name: clone-sources
      taskRef:
        name: git-clone
        kind: ClusterTask
      params:
        - name: url
          value: $(params.GITHUB_REPO_URL)
        - name: revision
          value: $(params.GITHUB_REPO_REVISION)
        # just to do a fresh clone, as we already have the directory cloned
        # to reuse it we can exlude this step
        - name: deleteExisting
          value: 'true'
      workspaces:
        - name: output
          workspace: source
    - name: build-java-test
      taskRef:
        name: maven
        kind: ClusterTask
      runAfter:
        - clone-sources
      params:
        - name: GOALS
          value:
            - '-pl'
            - $(params.APP_PROFILE)
            - -B
            - 'clean'
            - 'test'
      workspaces:
        - name: maven-settings
          workspace: maven-settings
        - name: source
          workspace: source
    - name: build-java-app
      taskRef:
        name: maven
        kind: ClusterTask
      runAfter:
        - build-java-test
      params:
        - name: GOALS
          value:
            - '-pl'
            - $(params.APP_PROFILE)
            - -B
            - '-DskipTests'
            - 'clean'
            - 'package'
      workspaces:
        - name: maven-settings
          workspace: maven-settings
        - name: source
          workspace: source
    - name: build-java-app-image
      taskRef:
        name: buildah
        kind: ClusterTask
      runAfter:
        - build-java-app
      params:
        - name: CONTEXT
          value: '$(params.APP_PROFILE)'
        - name: IMAGE
          value: '$(params.IMAGE_NAME)'
        # needed for OpenShift buildah run as vfs is the default storage driver
        - name: STORAGE_DRIVER
          value: vfs
        # since pushing to internal registry we skip TLS verify
        - name: TLSVERIFY
          value: 'false'
      workspaces:
        - name: source
          workspace: source
    - name: deploy-kn-service
      taskRef:
        name: kn
        kind: ClusterTask
      runAfter:
        - build-java-app-image
      params:
        - name: ARGS
          value:
            - 'service'
            - 'create'
            - '$(params.APP_NAME)'
            - '--force'
            - '--image=$(params.IMAGE_NAME)@$(tasks.build-java-app-image.results.IMAGE_DIGEST)' (5)
1 If you want the Pipeline Template to be visible via the ODC Add Pipeline option. For the Pipeline Template to be available when creating application via ODC, the Pipeline Template need to be deployed in openshift namespace
2 Identifies or tags the Pipeline Template to be used with Java runtime applications
3 The APP_NAME parameter will be automatically set by OpenShift to the Name(Greeter Application) of the application that was set while creating application via ODC
4 Identifies or tags the Pipeline Template to be used with Knative service type applications

Deploy Pipeline Template

Deploy the Pipeline Template cloud-native-app-serverless by:

oc apply -f cloud-native-app-serverless-pipeline.yaml
pipeline.tekton.dev/cloud-native-app-serverless created

When now follow the steps to add a git project via ODC, after choosing resources to be Knative Service, you should see the available pipeline (cloud-native-app-serverless) as shown:

ocp dc add kn pipeline
Figure 7. Java Knative Service Pipeline

Points to Ponder

Now are all set to deploy the Tekton Tutorial Java applications as Knative Service using OpenShift Pipelines. But before creating the application, lets pause for few minutes to see what we have understood so far:

  • What ClusterTasks are available as part of OpenShift Pipelines

  • How to update the ClusterTask with updates/changes

  • How to make Pipeline Template available for runtime and service type combination

Deploy Application with Pipeline

Getting back to ODC, follow the steps to deploy the application from GitHub, but this time you should be able to see a Pipeline Template available for runtime Java and resource type Knative Service. Click Add Pipeline checkbox to use the Pipeline Template that can build and deploy the Java Knative Service:

ocp dc check add kn pipeline
Figure 8. Add Pipeline

Then click the Create button, to create the "greeter" application with pipeline enabled:

odc greeter app created
Figure 9. Greeter Application

As noticed in the above figure the application is yet to be deployed, as no pipelines have run.

View Pipelines

You should now be able to see a pipeline called greeter in ODC, by navigating using sidebar navigation >Pipelines as shown:

odc view greeter pipeline
Figure 10. greeter Pipeline

The Pipeline name is same as the application name greeter, click the greeter Pipeline link, to see more details such as:

  • Visualization - this will be actively showing the Pipeline Task status when its run

  • Labels with runtime and type

  • The list of Tasks that will be run as part of the Pipeline

odc greeter pipeline details
Figure 11. greeter Pipeline details

Run Pipeline

To run the Pipeline, get back to the Pipelines list and click Start from the actions menu:

odc greeter pipeline start
Figure 12. Start greeter Pipeline

The Start pipeline will request the details such as Parameters and workspaces to be set for the Pipeline run, provide the details as:

IMAGE_NAME
image-registry.openshift-image-registry.svc:5000/pipelines-demos/greeter
Update the highlighted path to match your namespace, in case you are not using pipelines-demos for your deployments
APP_PROFILE
quarkus (1)
1 You can also set it to be springboot
APP_NAME
greeter (1)
1 As described earlier the APP_NAME parameter automatically maps to the Application Name

Leave the GITHUB_REPO_URL and GITHUB_REPO_REVISION to be defaults as, https://github.com/redhat-scholars/tekton-tutorial-greeter and master respectively.

Choose the Workspace  Source to be of type PVC and use tekton-tutorial-sources as the value for PVC.

Choose the Workspace  maven-settings to be of type ConfigMap and use maven-settings as the value for ConfigMap.

The completed parameters should look like:

odc greeter pipeline params
Figure 13. greeter Pipeline parameters

Click the button Start to kick start the pipeline. The Pipeline run be automatically show as in the following screenshot:

odc greeter pipeline running
Figure 14. greeter Pipeline run
It will take few minutes for the pipeline to be running and complete. You can watch the status via visualization.

Watch Pipelines

You can watch the pipeline from sidebar Pipelines, and selecting greeter from the list:

odc greeter pipeline running from list
Figure 15. Pipelines list

Application Topology

When you navigate to sidebar menu Toplogy, you should see the greeter application has transitioned from waiting to running state. When you click on the application you should see more details about the application which includes the running greeter Pipeline.

odc greeter topology view
Figure 16. greeter Knative Service Topology

A successful pipeline should have the Java Knative Service deployed and running as shown:

odc view greeter ksvc

Click on the URL in the Routes, to access the application and see the response Meeow!! from Tekton πŸ˜ΊπŸš€.

Points to Ponder

Now you have understood:

  • What ClusterTasks are available as part of OpenShift Pipelines

  • How to update the ClusterTask with updates/changes

  • How to make Pipeline Template available for runtime and service type combination

  • Start the Pipeline from the Pipelines view by configuring the required parameters