Triggers

This chapter build is in progress, expect to change and not working as expected

At the end of this chapter you will be able to :

  • What are Tekton Triggers

  • How to Trigger a Pipeline or Task based on Events

Overview

Until now we have seen how to create Tasks and Pipelines. But whenever it comes to running the Tasks and Pipelines we always relied on doing it manually. In the Chapter we will be expoloring how to do it using Triggers, e.g. say you push a commit in to the source repository and it triggers a PipelineRun to deploy a new application or a TaskRun to push the image to container registry. Lets rock!

If you are not in tutorial chapter folder, then navigate to the folder:

cd $TUTORIAL_HOME/triggers

Install Triggers

In OpenShfit, the OpenShift pipelines operator installs Tekton Pipelines and Tekton Triggers in openshift-pipelines namespace. Its not needed install Tekton Triggers. 
kubectl apply -f https://storage.googleapis.com/tekton-releases/triggers/previous/v0.6.1/release.yaml

Wait for the Tekton Trigger controller and webhook to be ready:

watch kubectl -n tekton-pipelines get pods

A successful Tekton run should have the highlighted Tekton Trigger pods running in the tekton-pipelines:

NAME                                           READY   STATUS    RESTARTS   AGE
tekton-pipelines-controller-849ccccd7f-gc6dp   1/1     Running   1          3d3h
tekton-pipelines-webhook-75bc7666c-9crwq       1/1     Running   1          3d3h
tekton-triggers-controller-697c9b844d-9lz4x    1/1     Running   0          15h
tekton-triggers-webhook-6bcb96f965-gqrbh       1/1     Running   0          15h

Prepare namespace

All exercises of this chapter will be done a namesapace triggers-demo, lets create and switch to the triggers-demo namespace.

kubectl create ns triggers-demo
kubectl config set-context --current --namespace triggers-demo

As part of this exercise we will need to install the following external tasks to be installed in the namespace:

kubectl apply -n triggers-demo \
  -f https://raw.githubusercontent.com/tektoncd/catalog/master/task/git-clone/0.1/git-clone.yaml \
  -f https://raw.githubusercontent.com/tektoncd/catalog/master/task/maven/0.1/maven.yaml \
  -f https://raw.githubusercontent.com/tektoncd/catalog/master/task/buildah/0.1/buildah.yaml \
  -f https://raw.githubusercontent.com/tektoncd/catalog/master/task/kn/0.1/kn.yaml \
  -f $TUTORIAL_HOME/workspaces/list-directory-task.yaml

Check if all the tasks are available:

tkn task ls

The command should show an output like:

NAME               DESCRIPTION              AGE
buildah            Buildah task builds...   11 hours ago
git-clone          These Tasks are Git...   11 hours ago
kn                 This Task performs ...   9 minutes ago
list-directory     Simple directory li...   11 hours ago
maven              This Task can be us...   11 hours ago

Deploy Nexus

To show maven artifact caching and using customized maven settings.xml we will deploy Sonatype Nexus to the cluster:

oc apply -n triggers-demo \
  -f $TUTORIAL_HOME/install/utils/nexus.yaml

Wait for the nexus3 pod to come up before proceeding further with exercises:

watch oc -n triggers-demo get pods,svc

A sucessfully running nexus3 pod should show the following services and pods:

NAME                         READY   STATUS    RESTARTS   AGE
pod/nexus-75fff8bbbd-cmlxs   1/1     Running   0          3m25s

NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
service/nexus        NodePort    10.100.205.22   <none>        8081:31743/TCP   3m25s

The nexus maven repository could be opened using the url:

$(minikube service nexus -n triggers-demo)

In OpenShift,we can use routes to expose the service if we need to access it from outside,by:

oc expose svc -n triggers-demo nexus

Create maven-settings ConfigMap

For us to mount the maven settings.xml, we will create ConfigMap holding the custom maven settings.xml:

oc create -n triggers-demo cm maven-settings \
  --from-file=settings.xml=$TUTORIAL_HOME/workspaces/maven-settings.xml
configmap/maven-settings created

You can check the ConfigMap contents using the command:

oc get -n triggers-demo cm maven-settings -oyaml \
  | yq r - data['settings.xml']

The output of the above command should be same as in maven-settings.xml.

Before we create our pipeline ensure that the kubernetes cluster has a default storage class defined:

Deploy Knative Serving

As we will be deploying a Knative serverless application as part of the pipeline, that we will be running later in this chapter, lets deploy Knative to the cluster:

  • Minikube

  • OpenShift

Do it only once per cluster 
$TUTORIAL_HOME/bin/enable_knative.sh

Wait for the serving deployment to complete without errors.

OpenShift users can install Knative i.e OpenShift Serverless using its Operators.

Check Default Storage Class

oc get sc

  • Kubernetes

  • OpenShift

In minkube cluster it should show an output like:

NAME                 PROVISIONER                RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
standard (default)   k8s.io/minikube-hostpath   Delete          Immediate           false                  9h

An example default storage class in Google Cloud:

NAME                 PROVISIONER            RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
standard (default)   kubernetes.io/gce-pd   Delete          WaitForFirstConsumer   true                   2d3h

  • If you dont have one please check Kubernetes Docs on how to have one configured.

  • In OpenShift cluster the default storage class varies based on the underlying cloud platform. Refer to OpenShift Documentation to know more.

Create PVC

Create the PVC tekton-tutorial-sources, which we will use as part of the exercises in this chapter and the upcoming ones.

oc apply -n triggers-demo -f $TUTORIAL_HOME/workspaces/sources-pvc.yaml

Create Pipeline

As part of the Trigger exercise we will be runing the greeter-app-deploy pipeline, lets deploy the pipeline to be used later:

kubectl apply -n triggers-demo \
  -f $TUTORIAL_HOME/workspaces/greeter-app-deploy.yaml

Verify the available pipelines:

tkn pipeline -n triggers-demo ls
NAME                    AGE            LAST RUN   STARTED   DURATION   STATUS
greeter-app-deploy   1 minute ago   ---        ---       ---        ---

Deploy Gogs

Since we need to trigger a Pipeline with a git push, we will deploy a Git server to the local cluster. This will avoid us the need to expose the service to the public internet and also helps us to test things quickly with our local cluster setup.

Gogs is super quick and easy to install i.e. self-hosted git service. Lets deploy that into the Kubernetes Cluster.

Create Gogs Ingress

Create the gogs ingress so that we can access the application from host machine using http:

export GOGS_HOSTNAME="gogs.$(minikube ip).nip.io"
yq w $TUTORIAL_HOME/install/git/gogs-ingress.yaml \
  spec.virtualhost.fqdn $GOGS_HOSTNAME |\
  kubectl apply -n triggers-demo -f -

Create gogs config file app.ini from the template and add that to gogs ConfigMap:

sed "s/@GOGS_HOSTNAME@/$GOGS_HOSTNAME/g" $TUTORIAL_HOME/install/git/app.ini.template \
  | tee $TUTORIAL_HOME/install/git/app.ini
kubectl apply -n triggers-demo \
  -k $TUTORIAL_HOME/install/git/
We use kustomize that allows us to do some customizations on the Kubenetes manifests before we apply them. Kustomize is built into kubectl and can be used with -k option. 
configmap/gogs-config created
service/gogs-postgresql created
service/gogs created
deployment.apps/gogs-postgresql created
deployment.apps/gogs created
persistentvolumeclaim/gogs-data created
persistentvolumeclaim/pgdata-pvc created
persistentvolumeclaim/pgwal-pvc created

Wait for the gogs-postgresql and gogs pods to come up, watch the status using :

watch kubectl get pods,svc -lapp=gogs

A suceesfull deployment should show:

NAME                                  READY   STATUS    RESTARTS   AGE
pod/gogs-854766b6d7-nsmsp             1/1     Running   0          64s
pod/gogs-postgresql-745dfd5cb-spc46   1/1     Running   0          64s

NAME                      TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
service/gogs              NodePort    10.106.116.15    <none>        3000:31550/TCP   64s
service/gogs-postgresql   ClusterIP   10.105.150.118   <none>        5432/TCP         64s

You can access the gogs service using ingress url:

open http://$GOGS_HOSTNAME
gogs home
Figure 1. Gogs Home

Create ServiceAccount, Roles and Role Bindins

We need to query Kubernetes objects, create Triggers and finally deploy the Knative Service as part of this chapter' exercies. Lets create a Kubernetes Service Account and add the required roles/permissions:

kubectl apply -k $TUTORIAL_HOME/triggers/rbac
serviceaccount/pipeline created
role.rbac.authorization.k8s.io/app-deployer-role created
role.rbac.authorization.k8s.io/tekton-triggers-admin created
role.rbac.authorization.k8s.io/tekton-triggers-createwebhook created
rolebinding.rbac.authorization.k8s.io/app-deployer-binding created
rolebinding.rbac.authorization.k8s.io/tekton-triggers-admin-binding created
rolebinding.rbac.authorization.k8s.io/tekton-triggers-createwebhook-binding created
In OpenShift you might no need the pipeline SA, as the OpenShift Pipelines creates it by default

Initialize Source Repository

To run the exercises of this chapter we need the following setup in Git(gogs):

We will run the following Task to perform the necessary intializations listed above:

kubectl create -n triggers-demo \
  -f gogs-init-taskrun.yaml
taskrun.tekton.dev/init-gogs-qkf84 created

View the task run logs using,

tkn tr logs -f -a $(tkn tr ls -n triggers-demo | awk 'NR==2{print $1}')

A successfull initializations should show an output like:

[init-gogs] Created admin user gogs:gogs
[init-gogs] Created git repo tekton-tutorial-greeter
[init-gogs] Repository Clone url http://gogs.192.168.64.5.nip.io/gogs/https://github.com/redhat-scholars/tekton-tutorial-greeter.git

You should now be able to open the Repository Clone url from the output above and view the respository content.

Trigger Template

TriggerTemplate is responsible to create the Tekton Resources when it receives the Event from the EventListener.

Create Trigger Template

kubectl create -n triggers-demo \
  -f greeter-trigger-template.yaml
triggertemplate.triggers.tekton.dev/tekton-greeter-trigger-template created

List the available TriggerTemplates:

tkn tt -n triggers-demo ls

It should be only one now:

NAME                                 AGE
tekton-greeter-trigger-template   1 minute ago

Trigger Bindings

TriggerBinding is responsible to bind the Event payload with Template Parameters. In this case it will bind the gogs event playload to TriggerTemplate parameters, inputs, outputs and workspaces.

Let us see a sample webhook payload that Git events from Gogs might look like:

Gogs Payload sample
{
  "ref": "refs/heads/master",
  "before": "d84fc84f8d7a1da0b49140f644613168e4af3110",
  "after": "becd943814ef904215c0c6f301434ce8faf31c26",
  "compare_url": "http://gogs.192.168.64.5.nip.io/gogs/tekton-greeter/compare/d84fc84f8d7a1da0b49140f644613168e4af3110...becd943814ef904215c0c6f301434ce8faf31c26",
  "commits": [
    {
      "id": "becd943814ef904215c0c6f301434ce8faf31c26",
      "message": "First Test Trigger\n",
      "url": "http://gogs.192.168.64.5.nip.io/gogs/tekton-greeter/commit/becd943814ef904215c0c6f301434ce8faf31c26",
      "author": {
        "name": "Kamesh Sampath",
        "email": "kamesh.sampath@hotmail.com",
        "username": ""
      },
      "committer": {
        "name": "Kamesh Sampath",
        "email": "kamesh.sampath@hotmail.com",
        "username": ""
      },
      "added": [],
      "removed": [],
      "modified": [
        "src/main/java/com/redhat/developers/GreetingResource.java"
      ],
      "timestamp": "2020-07-29T05:24:44Z"
    }
  ],
  "repository": {
    "id": 5,
    "owner": {
      "id": 1,
      "login": "gogs",
      "full_name": "",
      "email": "admin@gogs.com",
      "avatar_url": "https://secure.gravatar.com/avatar/4d667b96bad9b51aafc728a20ebce918",
      "username": "gogs"
    },
    "name": "tekton-greeter",
    "full_name": "gogs/tekton-greeter",
    "description": "",
    "private": false,
    "fork": false,
    "parent": null,
    "empty": false,
    "mirror": false,
    "size": 323584,
    "html_url": "http://gogs.192.168.64.5.nip.io/gogs/tekton-greeter",
    "ssh_url": "gogs@gogs.192.168.64.5.nip.io:gogs/tekton-greeter.git",
    "clone_url": "http://gogs.192.168.64.5.nip.io/gogs/tekton-greeter.git",
    "website": "",
    "stars_count": 0,
    "forks_count": 0,
    "watchers_count": 1,
    "open_issues_count": 0,
    "default_branch": "master",
    "created_at": "2020-07-28T15:22:20Z",
    "updated_at": "2020-07-28T15:22:24Z"
  },
  "pusher": {
    "id": 1,
    "login": "gogs",
    "full_name": "",
    "email": "admin@gogs.com",
    "avatar_url": "https://secure.gravatar.com/avatar/4d667b96bad9b51aafc728a20ebce918",
    "username": "gogs"
  },
  "sender": {
    "id": 1,
    "login": "gogs",
    "full_name": "",
    "email": "admin@gogs.com",
    "avatar_url": "https://secure.gravatar.com/avatar/4d667b96bad9b51aafc728a20ebce918",
    "username": "gogs"
  }
}

We can use any attribute/data from the JSON payload and bind them as the value in TriggerBindings. For this chapter exercise we are interested in two attributes namely:

  • after: the commit hash after our push is merged into the master

  • repository.clone_url: The Git Repo clone url

The HTTP Body and Headers of the eventt payload are available as body and header JSONPath variables. We can retrieve the values using JSONPath expressions. Check the Event Variable Interpolation for more details.

To retreieve the after and repository.clone_url we will the JSONPath expressions $(body.after) and $(body.repoistory.clone_url) respectively.

Create Trigger Bindings

gogs-triggerbindings.yaml
---
apiVersion: triggers.tekton.dev/v1alpha1
kind: TriggerBinding
metadata:
  name: gogs-triggerbinding
spec:
  params:
    - name: gitrevision
      value: $(body.after)
    - name: gitrepositoryurl
      value: $(body.repository.clone_url)
kubectl create -n triggers-demo -f gogs-triggerbindings.yaml
triggerbinding.triggers.tekton.dev/gogs-triggerbinding created

List the available TriggerBindings:

tkn tb -n triggers-demo ls

It should show an output like:

NAME                  AGE
gogs-triggerbinding   7 seconds ago

Event Listener

Event Listener is primary interace for external sources to send events, that will trigger the creation of Tekton resources defined as part of the TriggerTemplate.

Create Event Listener

kubectl apply -n triggers-demo -f gogs-eventlistener.yaml
eventlistener.triggers.tekton.dev/gogs-webhook created

List the available EventListeners:

tkn el -n triggers-demo ls

It should show an output like:

NAME           AGE
gogs-webhook   4 seconds ago

Each EventListener will have a service named el-<EventListener Name> exposed automatically for sources to send events to Triggers.

kubectl get svc -n triggers-demo -leventlistener=gogs-webhook
NAME              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
el-gogs-webhook   ClusterIP   10.97.199.207    <none>        8080/TCP         51s

Create Event Listener Ingress

If you need the EventListner service to be available outside of the cluster thereby you can use them as part of the webhook, the service el-gogs-webhook needs to be exposed via Ingress:

  • Minikube

  • OpenShift

export EL_WEBHOOK_URL="$(kubectl get svc -n triggers-demo el-gogs-webhook -o yaml \
  | yq r - 'metadata.name').$(minikube ip).nip.io"
export EL_WEBHOOK_LISTENER_PORT="$(kubectl get svc -n triggers-demo el-gogs-webhook -o yaml \
  | yq r - 'spec.ports.(name==http-listener).port')"
yq w eventlistener-ingress.yaml \
  spec.virtualhost.fqdn $EL_WEBHOOK_URL \
  | yq w - 'spec.routes[0].services[0].name' el-gogs-webhook \
  | yq w - 'spec.routes[0].services[0].port' $EL_WEBHOOK_LISTENER_PORT \
  | kubectl apply -n triggers-demo -f  -
httpproxy.projectcontour.io/el-gogs-webhook-ingress

Lets verify if the ingress has been configured correctly:

kubectl get httpproxy -n triggers-demo el-gogs-webhook-ingress -o yaml

The command should the following output (trimmed for brevity):

apiVersion: projectcontour.io/v1
kind: HTTPProxy
metadata:
  name: el-gogs-webhook-ingress
  namespace: triggers-demo
spec:
  routes:
  - services:
    - name: el-gogs-webhook
      port: 8080
  virtualhost:
    fqdn: el-gogs-webhook.192.168.64.5.nip.io
oc expose svc -n triggers-demo el-gogs-webhook --targetPort=http-listener

Configure Webhook

For the GitHub Triggers to work, we need to have the Webhook configured for the Git Repository tekton-greeter to point to the el-gogs-webhook-ingress:

kubectl create -n triggers-demo \
  -f gogs-webhook-taskrun.yaml
taskrun.tekton.dev/webhook-gogs-lwkjt created

View the task run logs using,

tkn tr logs -f -a $(tkn tr ls -n triggers-demo | awk 'NR==2{print $1}')

A successfull initializations should show an output like:

[webhook-gogs] Configured webhook: http://el-gogs-webhook.192.168.64.5.nip.io

As you note now the tekton-greeter Git repository in Gogs is configured with the webhook to send Git events to EventListener.

gogs webhook settings

Triggers in Action

Terminal 1

As we expect the triggers to start a Pipeline, lets open a new terminal and watch the running Pipelines:

watch tkn pr -n triggers-demo ls

You Ctrl+c the watch once you see it running and monitor the logs.

Terminal 2

Open a new terminal and watch the logs the event listener to see the incoming events:

stern -n triggers-demo gogs-webhook

Once the event playload is received from the gogs, you should see output(trimmed for brevity) as shown below:

...
el-gogs-webhook-88bc89db-ld6jz event-listener {"level":"info","logger":"eventlistener","caller":"resources/create.go:93","msg":"Generating resource: kind: &APIResource{Name:pipelineruns,Namespaced:true,Kind:PipelineRun,Verbs:[delete deletecollection get list patch create update watch],ShortNames:[pr prs],SingularName:pipelinerun,Categories:[tekton tekton-pipelines],Group:tekton.dev,Version:v1beta1,StorageVersionHash:4xDTCrDXyFg=,}, name: greeter-app-","knative.dev/controller":"eventlistener"}
el-gogs-webhook-88bc89db-ld6jz event-listener {"level":"info","logger":"eventlistener","caller":"resources/create.go:101","msg":"For event ID \"rrgcn\" creating resource tekton.dev/v1beta1, Resource=pipelineruns","knative.dev/controller":"eventlistener"}
...

Clone and Edit the source

Using your favorite IDE clone the repo tekton-greeter from Gogs using the clone url.

git clone <your gogs tekton-greeter clone url>

Open the tekton-tutorial-greeter in your IDE and edit the Java file:

  • Quarkus

  • SpringBoot

  • src/main/java/com/redhat/developers/GreetingResource.java to update the "Meeow!! from Tekton 😺🚀" to "😺 Tekton Rocks!"

  • src/test/java/com/redhat/developers/GreetingResourceTest.java to match "😺 Tekton Rocks!".

  • src/main/java/com/redhat/developer/demos/GreetingController.java to update the "Meeow!! from Tekton 😺🚀" to "😺 Tekton Rocks!"

  • src/test/java/com/redhat/developer/demos/GreetingControllerTest.java to match "😺 Tekton Rocks!".

Commit and push to see the 😺 Pipeline greeter-app-deploy running 🚀.

Use the username gogs and password gogs when prompted during Git push to repository.

View the pipeline run logs using,

tkn pr logs -f -a $(tkn pr ls -n triggers-demo | awk 'NR==2{print $1}')

Once the Pipeline succeeds, verify the Knative Service greeter:

http --body (kn service describe greeter -o url)/hello

The command should shown output like:

😺 Tekton Rocks!

Expriment with updating the sources to your taste and see the pipeline triggers.

Points to Ponder

To use Tekton Triggers we need:

  • A Kubernetes Service Account with required premissions assigned

  • A TriggerTemplate which defines what Tekton Resources to create for a events

  • A TriggerBinding which allows to extract the data from the event payload and bind them to Tekton Parameters of Task/Pipeline

  • A EventListener that will listen for the events and trigger the executuion of Tekton Resources as part of the Trigger Template

Cleanup

If you no longer going to use the triggers-demo, you can do:

kubectl delete ns triggers-demo