Using Private Repositories and Registries

Currently, there is an incompatibility between Tekton pipelines webhook and Minikube. Please expect no logs when running tkn commands .

At the end of this chapter you will be able to :

Use Tekton Authentication Secrets
How to push linux container image to external registry
How to clone from a private Github respository

Overview

In many practical usecases you might need to pull from private Git repsoitories or might need to push to an external container registry such as Quay.io. In both cases the access requires you to authenticate. With Tekton this is achieved using the Kubernetes Service Account(SA) and Kubernetes Secret.

If you are not in tutorial chapter folder, then navigate to the folder:

cd $TUTORIAL_HOME/private_repos_reg

Let us create a new namespace called workspace-auth-demo and switch to that namespace context.

It is assumed that you have completed Workspaces and reusing the same namespace that was prepared in that chapter.
You have a container registry account with Quay.io or Docker Hub
You have a GitHub account and a private repository

Pulling from Private Source Repository

In this exercise we will see how to pull i.e. clone the source code from the private source reporsitory. For this example we will use GitHub as the remote source repository.

To be able to run this exercise you need:

A remote GitHub Repository, you can pull fork the https://github.com/redhat-scholars/tekton-tutorial-greeter and push that as your private repository
A GitHub Personal Access Token(PAT) to access the remote repository.

Set the requried environment variables to be used when creating the github-pat-secret:

export GITHUB_USERNAME='<your github.com username>'
export TEKTON_TUTORIAL_GITHUB_PAT='<your github.com username personal accesstoken>'

Create Github PAT Secret

Create the Kubernetes secret that can hold your GitHub.com credentials:

kubectl create secret generic github-pat-secret \
    --type=kubernetes.io/basic-auth \
    --from-literal=username=$GITHUB_USERNAME \
    --from-literal=password=$TEKTON_TUTORIAL_GITHUB_PAT

secret/github-pat-secret created

Annotate Secret to be used with GitHub.com

To make Tekton use the Secret github-pat-secret with github.com, we need to annotate the Secret with tekton.dev/git-0=https://github.com.

kubectl annotate -n workspace-auth-demo secret github-pat-secret \
  "tekton.dev/git-0=https://github.com"

secret/github-pat-secret annotated

Verify the github-pat-secret as the right type, annotations and credential values:

kubectl get -n workspace-auth-demo secret github-pat-secret -o yaml

The command should show an output (trimmed for brevity) as shown in the following listing, with your GitHub.com username and PAT.

apiVersion: v1
data:
  password: REDACTED
  username: REDACTED
kind: Secret
metadata:
  annotations:
    tekton.dev/git-0: https://github.com
  name: github-pat-secret
  namespace: workspace-auth-demo
type: kubernetes.io/basic-auth

Create Service Account

Let us create a Kubernetes ServiceAccount that could be used to pull from the private GitHub repository:

kubectl create sa -n workspace-auth-demo github-bot

serviceaccount/github-bot created

Patch Service Account

Now patch the github-bot service account to use the github-pat-secret credentials:

kubectl patch serviceaccount github-bot \
  -p '{"secrets": [{"name": "github-pat-secret"}]}'

serviceaccount/github-bot patched

Lets verify if the service account has the secret added:

kubectl get sa -n workspace-auth-demo github-bot -o yaml

The command should show an output (trimmed for brevity) like:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: github-bot
  namespace: workspace-auth-demo
secrets:
- name: github-pat-secret
- name: github-bot-token-9p2wg

Create Pipeline

Create the git clone pipeline which will clone from the private GitHub repo and simply list the contents:

kubectl apply -n workspace-auth-demo -f secretworld-app-clone.yaml

pipeline.tekton.dev/secretworld-app-clone created

Run Pipeline

tkn
kubectl

tkn pipeline start secretworld-app-clone \
  --namespace=workspace-auth-demo \
  --serviceaccount=github-bot \
  --param private-github-repo-url='https://github.com/redhat-scholars/tekton-secretworld' \
  --workspace name=source,claimName=tekton-tutorial-sources \
  --use-param-defaults \
  --showlog

Ensure that the pipeline run has been set to run with the Service Account github-bot

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  generateName: secretworld-app-clone-
  labels:
    tekton.dev/pipeline: secretworld-app-clone
spec:
  serviceAccountName: github-bot (1)
  pipelineRef:
    name: secretworld-app-clone
  params:
    - name: private-github-repo-url
      value: https://github.com/redhat-scholars/tekton-secretworld
  workspaces:
    - name: source
      persistentVolumeClaim:
        claimName: tekton-tutorial-sources

1	Please make sure you have updated the private GitHub repository to match your private repository

Run the pipeline:

kubectl create -n workspace-auth-demo -f secretworld-app-clone-run.yaml

View the pipeline run logs using,

tkn pr logs -f -a $(tkn pr ls -n workspace-auth-demo | awk 'NR==2{print $1}')

A successful clone from private repo will show the following output (trimmed brevity):

[clone-sources : credential-initializer] {"level":"info","ts":1595922122.9077983,"caller":"creds-init/main.go:44","msg":"Credentials initialized."}

[clone-sources : clone] + CHECKOUT_DIR=/workspace/output/
[clone-sources : clone] + '[[' true '==' true ]]
[clone-sources : clone] + cleandir
[clone-sources : clone] + '[[' -d /workspace/output/ ]]
[clone-sources : clone] + rm -rf /workspace/output//Dockerfile /workspace/output//README.md /workspace/output//k8s /workspace/output//mvnw /workspace/output//mvnw.cmd /workspace/output//pom.xml /workspace/output//src
[clone-sources : clone] + rm -rf /workspace/output//.dockerignore /workspace/output//.git /workspace/output//.gitignore /workspace/output//.mvn
[clone-sources : clone] + rm -rf '/workspace/output//..?*'
[clone-sources : clone] + test -z
[clone-sources : clone] + test -z
[clone-sources : clone] + test -z
[clone-sources : clone] + /ko-app/git-init -url https://github.com/redhat-scholars/tekton-secretworld -revision master -refspec  -path /workspace/output/ '-sslVerify=true' '-submodules=true' -depth 1
[clone-sources : clone] {"level":"info","ts":1595922137.4565356,"caller":"git/git.go:139","msg":"Successfully cloned https://github.com/redhat-scholars/tekton-secretworld @ 5250e1fa185805373e620d1c04a0c48129efd2ee (grafted, HEAD, origin/master) in path /workspace/output/"}
[clone-sources : clone] {"level":"info","ts":1595922137.4990256,"caller":"git/git.go:180","msg":"Successfully initialized and updated submodules in path /workspace/output/"}
[clone-sources : clone] + cd /workspace/output/
[clone-sources : clone] + git+ tr -d '\n'
[clone-sources : clone]  rev-parse HEAD
[clone-sources : clone] + RESULT_SHA=5250e1fa185805373e620d1c04a0c48129efd2ee
[clone-sources : clone] + EXIT_CODE=0
[clone-sources : clone] + '[' 0 '!=' 0 ]
[clone-sources : clone] + echo -n 5250e1fa185805373e620d1c04a0c48129efd2ee

[list-cloned-repo : credential-initializer] {"level":"info","ts":1595922139.7837844,"caller":"creds-init/main.go:44","msg":"Credentials initialized."}


[list-cloned-repo : list-directory] total 44
[list-cloned-repo : list-directory] drwxr-xr-x    4 root     root          4096 Jul 28 07:42 src
[list-cloned-repo : list-directory] -rw-r--r--    1 root     root          4147 Jul 28 07:42 pom.xml
[list-cloned-repo : list-directory] -rwxr-xr-x    1 root     root          6607 Jul 28 07:42 mvnw.cmd
[list-cloned-repo : list-directory] -rwxr-xr-x    1 root     root         10069 Jul 28 07:42 mvnw
[list-cloned-repo : list-directory] drwxr-xr-x    2 root     root          4096 Jul 28 07:42 k8s
[list-cloned-repo : list-directory] -rw-r--r--    1 root     root           111 Jul 28 07:42 README.md
[list-cloned-repo : list-directory] -rw-r--r--    1 root     root            87 Jul 28 07:42 Dockerfile

[list-cloned-repo : show-readme] 🥳 Yay! 🎉
[list-cloned-repo : show-readme]
[list-cloned-repo : show-readme] You have successfully cloned from private GitHub repository. 👏👏
[list-cloned-repo : show-readme]
[list-cloned-repo : show-readme] 😺 Tekton Rocks!! 🚀

Pushing to external registry

To able push to an external container registry, its requried that the Pipline is run with a ServiceAccount that has container registry credentials configured via ServiceAccount secrets. Kubernetes provider a Secret type called docker-registry, that can be used to configure the container registry credentials.

Set the requried environment variables to be used when creating the container-registry-secret:

export CONTAINER_REGISTRY_SERVER='quay.io' (1)
export CONTAINER_REGISTRY_USER='<your registry user>'
export CONTAINER_REGISTRY_PASSWORD='<your registry user password>'

1	The container registry server URL, for Quay.io its quay.io and for DockerHub it is https://index.docker.io/v2/

Create Container Registry Secret

kubectl create secret -n workspace-auth-demo docker-registry container-registry-secret \
  --docker-server=$CONTAINER_REGISTRY_SERVER \
  --docker-username=$CONTAINER_REGISTRY_USER \
  --docker-password=$CONTAINER_REGISTRY_PASSWORD

secret/container-registry-secret created

Create Service Account

kubectl create sa -n workspace-auth-demo build-bot

serviceaccount/build-bot created

Patch Service Account

Now patch the build-bot service account to use the container-registry-secret credentials:

kubectl patch serviceaccount build-bot \
  -p '{"secrets": [{"name": "container-registry-secret"}]}'

serviceaccount/build-bot patched

Lets verify if the service account has the secret added:

kubectl get sa -n workspace-auth-demo build-bot -o yaml

The command should show an output like:

apiVersion: v1
kind: ServiceAccount
metadata:
  creationTimestamp: "2020-07-28T03:34:32Z"
  name: build-bot
  namespace: auth-demo
  resourceVersion: "53879"
  selfLink: /api/v1/namespaces/auth-demo/serviceaccounts/build-bot
  uid: 628067fd-91d1-4cdd-b6a6-88b4f7280ff0
secrets:
- name: container-registry-secret
- name: build-bot-token-8nl2v

Running Tasks with privileged access on OpenShift

Sometimes, Tekton Tasks or ClusterTasks require privileged access to create certain Kubernetes resources. Using a service account and adding it to a security policy is the safe way to run those kind of Tasks or ClusterTasks.

For our example, we should run the following command:

oc adm policy add-scc-to-user anyuid -z build-bot -n workspace-auth-demo

Create Pipeline

Create the build and push app pipeline:

kubectl apply -n workspace-auth-demo \
  -f greeter-app-build.yaml

pipeline.tekton.dev/greeter-app-build created

Run Pipeline

tkn
kubectl

tkn pipeline start greeter-app-build \
  --namespace=workspace-auth-demo \
  --serviceaccount=build-bot \
  --workspace name=maven-settings,config=maven-settings \
  --workspace name=source,claimName=tekton-tutorial-sources \
  --param app-profile='quarkus' \
  --param image-name=$CONTAINER_REGISTRY_SERVER/$CONTAINER_REGISTRY_USER/tekton-tutorial-greeter  \
  --showlog

Ensure that the pipeline run has been set to run with the Service Account build-bot

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  generateName: greeter-app-build-
  labels:
    tekton.dev/pipeline: greeter-app-build
spec:
  serviceAccountName: build-bot
  params:
    - name: app-profile
      value: quarkus
    # enable this for OpenShift
    #    - name: storageDriver
    #      value: vfs
    -
  pipelineRef:
    name: greeter-app-build
  workspaces:
    - name: maven-settings
      configmap:
        name: maven-settings
    - name: source
      persistentVolumeClaim:
        claimName: tekton-tutorial-sources

Run the pipeline:

kubectl create -n workspace-auth-demo -f greeter-app-build-run.yaml

View the pipeline run logs using,

tkn pr logs -f -a $(tkn pr ls -n workspace-auth-demo | awk 'NR==2{print $1}')

A successful build and push will show the following output ( trimmed brevity):

...
[build-java-app-image : push] + buildah --storage-driver=overlay push --tls-verify=true --digestfile /workspace/source/image-digest quay.io/rhdevelopers/tekton-tutorial-greeter docker://quay.io/rhdevelopers/tekton-tutorial-greeter
[build-java-app-image : push] Getting image source signatures
[build-java-app-image : push] Copying blob sha256:90c2e42f948b524cf98005073e0b0aa2065160abf9e8b314976c064e270d92ac
[build-java-app-image : push] Copying blob sha256:869b43e5dca37fa63d84e9bc588613678c4fe6fa2072a72a6ab5487424db2891
[build-java-app-image : push] Copying blob sha256:f9ddbcc4e7954a705b700c35c5e5beceabd86af121a6e561d86437a8512a6be6
[build-java-app-image : push] Copying blob sha256:7b08010864ba4c7ce9dfe1b90244b459b77c0387051659d37454783d10ab1113
[build-java-app-image : push] Copying config sha256:5bd61d725dc47d1f8b7c225d8d52f2730321cadad65988a0de60300a711a2e2b
[build-java-app-image : push] Writing manifest to image destination
[build-java-app-image : push] Copying config sha256:5bd61d725dc47d1f8b7c225d8d52f2730321cadad65988a0de60300a711a2e2b
[build-java-app-image : push] Writing manifest to image destination
[build-java-app-image : push] Storing signatures

[build-java-app-image : digest-to-results] + cat /workspace/source/image-digest
[build-java-app-image : digest-to-results] + tee /tekton/results/IMAGE_DIGEST
[build-java-app-image : digest-to-results] sha256:0e9e267a96f1ea48fe00642cd82ef689e44c7d467d6db3a3a543fa5b42fe53dc

A successful pipeline should have pushed the image to the external container registry. The following screenshot shows the image pushed to rhdevelopers Quay.io container registry repo:

If you noticed the highlighted sha256 from the log above, is same as that of the sha256 listed in the container registry repository.

Points to Ponder

When you need to pull from remote source repository or push to external container registry you need:

A Kubernetes Secret to hold the container registry credentials
A Kubernetes Service Account, with the container registry Secret added to it
To use the Service Account as serviceAccountName in PipelineRuns/TaskRuns
To annotate Secrets to map which Secret to be used with source repository

You can find more details about using Authentication with Tekton here.

Cleanup

Delete the workspace and its resources:

kubectl delete ns workspace-auth-demo