SyncWaves and Hooks

Syncwaves are used in Argo CD to order how manifests are applied to the cluster.

On the other hand resource hooks breaks up the delivery of these manifests in different phases.

Using a combination of syncwaves and resource hooks, you can control how your application rolls out.

This example will take you through the following steps:

Using Syncwaves to order deployment
Exploring Resource Hooks
Using Syncwaves and Hooks together

The sample application that we will deploy is a TODO application with a database and apart from deployment files, syncwaves and resource hooks are used:

Using Sync Waves

A Syncwave is a way to order how Argo CD applies the manifests that are stored in git. All manifests have a wave of zero by default, but you can set these by using the argocd.argoproj.io/sync-wave annotation.

Example:

metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "2"

The wave can also be negative as well.

metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-5"

When Argo CD starts a sync action, the manifest get placed in the following order:

The Phase that they’re in (we’ll cover phases in the next section)
The wave the resource is annotated in (starting from the lowest value to the highest)
By kind (Namespaces first, then services, then deployments, etc …)
By name (ascending order)

Read more about syncwaves on the official documentation site.

Exploring the Manifests

The sample application that we will deploy has the following manifests:

The Namespace with syncwave as -1:

todo-namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: todo
  annotations:
    argocd.argoproj.io/sync-wave: "-1"

The PostgreSQL with syncwave as 0:

Minikube
OpenShift

The PostgreSQL deployment with syncwave as 0:

postgresql-deployment.yaml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgresql
  namespace: todo
  annotations:
    argocd.argoproj.io/sync-wave: "0"
spec:
  selector:
    matchLabels:
      app: postgresql
  template:
    metadata:
      labels:
        app: postgresql
    spec:
      containers:
        - name: postgresql
          image: postgres:12
          imagePullPolicy: Always
          ports:
            - name: tcp
              containerPort: 5432
          env:
            - name: POSTGRES_PASSWORD
              value: admin
            - name: POSTGRES_USER
              value: admin
            - name: POSTGRES_DB
              value: todo

The PostgreSQL Service with syncwave as 0:

postgresql-service.yaml

---
apiVersion: v1
kind: Service
metadata:
  name: postgres
  namespace: todo
  annotations:
    argocd.argoproj.io/sync-wave: "0"
spec:
  selector:
    app: postgresql
  ports:
    - name: pgsql
      port: 5432
      targetPort: 5432

The PostgreSQL deployment with syncwave as 0:

postgres-deployment.yaml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgresql
  namespace: todo
  annotations:
    argocd.argoproj.io/sync-wave: "0"
spec:
  selector:
    matchLabels:
      app: postgresql
  template:
    metadata:
      labels:
        app: postgresql
    spec:
      containers:
        - name: postgresql
          image: quay.io/redhatdemo/openshift-pgsql12-primary:centos7
          imagePullPolicy: Always
          ports:
            - name: tcp
              containerPort: 5432
          env:
            - name: PG_USER_PASSWORD
              value: admin
            - name: PG_USER_NAME
              value: admin
            - name: PG_DATABASE
              value: todo
            - name: PG_NETWORK_MASK
              value: all

The PostgreSQL Service with syncwave as 0:

postgres-service.yaml

---
apiVersion: v1
kind: Service
metadata:
  name: postgresql
  namespace: todo
  annotations:
    argocd.argoproj.io/sync-wave: "0"
spec:
  selector:
    app: postgresql
  ports:
    - name: pgsql
      port: 5432
      targetPort: 5432

The Database table creation with syncwave as 1:

postgres-create-table.yaml

apiVersion: batch/v1
kind: Job
metadata:
  name: todo-table
  namespace: todo
  annotations:
    argocd.argoproj.io/sync-wave: "1"
spec:
  ttlSecondsAfterFinished: 100
  template:
    spec:
      containers:
        - name: postgresql-client
          image: postgres:12
          imagePullPolicy: Always
          env:
            - name: PGPASSWORD
              value: admin
          command: ["psql"]
          args:
            [
              "--host=postgresql",
              "--username=admin",
              "--no-password",
              "--dbname=todo",
              "--command=create table Todo (id bigint not null,completed boolean not null,ordering integer,title varchar(255),url varchar(255),primary key (id));create sequence hibernate_sequence start with 1 increment by 1;",
            ]
      restartPolicy: Never
  backoffLimit: 1

The TODO application deployment with syncwave as 2:

todo-deployment.yaml

---
apiVersion: "v1"
kind: "ServiceAccount"
metadata:
  labels:
    app.kubernetes.io/name: "todo-gitops"
    app.kubernetes.io/version: "1.0.0"
  name: "todo-gitops"
  namespace: todo
  annotations:
    argocd.argoproj.io/sync-wave: "2"
---
apiVersion: "apps/v1"
kind: "Deployment"
metadata:
  labels:
    app.kubernetes.io/name: "todo-gitops"
    app.kubernetes.io/version: "1.0.0"
  name: "todo-gitops"
  namespace: todo
  annotations:
    argocd.argoproj.io/sync-wave: "2"
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: "todo-gitops"
      app.kubernetes.io/version: "1.0.0"
  template:
    metadata:
      labels:
        app.kubernetes.io/name: "todo-gitops"
        app.kubernetes.io/version: "1.0.0"
    spec:
      containers:
      - env:
        - name: "KUBERNETES_NAMESPACE"
          valueFrom:
            fieldRef:
              fieldPath: "metadata.namespace"
        image: "quay.io/rhdevelopers/todo-gitops:1.0.0"
        imagePullPolicy: "Always"
        name: "todo-gitops"
        ports:
        - containerPort: 8080
          name: "http"
          protocol: "TCP"
      serviceAccount: "todo-gitops"

The TODO network:

Minikube
OpenShift

The TODO Service with syncwave as 2:

todo-service.yaml

---
apiVersion: "v1"
kind: "Service"
metadata:
  labels:
    app.kubernetes.io/name: "todo-gitops"
    app.kubernetes.io/version: "1.0.0"
  name: "todo-gitops"
  annotations:
    argocd.argoproj.io/sync-wave: "2"
  namespace: todo
spec:
  ports:
  - name: "http"
    port: 8080
    targetPort: 8080
  selector:
    app.kubernetes.io/name: "todo-gitops"
    app.kubernetes.io/version: "1.0.0"
  type: "NodePort"

The TODO Ingress configuration with syncwave as 3:

todo-ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: todo
  namespace: todo
  annotations:
    argocd.argoproj.io/sync-wave: "3"
spec:
  rules:
    - host: todo.devnation
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: todo-gitops
                port:
                  number: 8080

Add Minikube IP (minikube ip) and the Ingress hostname todo.devnation to your Host file, like /etc/hosts.

Example:

192.168.39.242 bgd.devnation bgdx.devnation todo.devnation

The TODO Service with syncwave as 2:

todo-service.yaml

---
apiVersion: "v1"
kind: "Service"
metadata:
  labels:
    app.kubernetes.io/name: "todo-gitops"
    app.kubernetes.io/version: "1.0.0"
  name: "todo-gitops"
  annotations:
    argocd.argoproj.io/sync-wave: "2"
  namespace: todo
spec:
  ports:
  - name: "http"
    port: 8080
    targetPort: 8080
  selector:
    app.kubernetes.io/name: "todo-gitops"
    app.kubernetes.io/version: "1.0.0"

The TODO Route configuration with syncwave as 3:

todo-route.yaml

apiVersion: route.openshift.io/v1
kind: Route
metadata:
  labels:
    app: todo
  name: todo
  namespace: todo
  annotations:
    argocd.argoproj.io/sync-wave: "3"
spec:
  port:
    targetPort: 8080
  to:
    kind: Service
    name: todo-gitops
    weight: 100

Argo CD will apply the Namespace first (since it’s the lowest value), and make sure it returns a "healthy" status before moving on.

Next, the PostgreSQL Deployment will be applied. After that reports healthy will continue with the rest of resources.

Argo CD won’t apply the next manifest until the previous reports "healthy".

Exploring Resource Hooks

Now that you’re familiar with syncwaves, we can begin exploring applying manifests in phases using resource hooks.

Controlling your sync operation can be further redefined by using hooks. These hooks can run before, during, and after a sync operation. These hooks are:

PreSync - Runs before the sync operation. This can be something like a database backup before a schema change
Sync - Runs after PreSync has successfully ran. This will run alongside your normal manifests.
PostSync - Runs after Sync has ran successfully. This can be something like a Slack message or an email notification.
SyncFail - Runs if the Sync operation as failed. This is also used to send notifications or do other evasive actions.

To enable a sync, annotate the specific object manifest with argocd.argoproj.io/hook with the type of sync you want to use for that resource. For example, if I wanted to use the PreSync hook:

metadata:
  annotations:
    argocd.argoproj.io/hook: PreSync

You can also have the hooks be deleted after a successful/unsuccessful run.

HookSucceeded - The resource will be deleted after it has succeeded.
HookFailed - The resource will be deleted if it has failed.
BeforeHookCreation - The resource will be deleted before a new one is created (when a new sync is triggered).

You can apply these with the argocd.argoproj.io/hook-delete-policy annotation. For example

metadata:
  annotations:
    argocd.argoproj.io/hook: PostSync
    argocd.argoproj.io/hook-delete-policy: HookSucceeded

Since a sync can fail in any phase, you can come to a situation where the application never reports healthy!

Although hooks can be any resource, they are usually Pods and/or Jobs.

To read more about resource hooks, consult the official documentation

Exploring Manifests

Take a look at this PostSync manifest which sends an HTTP request to insert a new TODO item:

todo-insert-data.yaml

apiVersion: batch/v1
kind: Job
metadata:
  name: todo-insert
  annotations:
    argocd.argoproj.io/hook: PostSync (1)
    argocd.argoproj.io/hook-delete-policy: HookSucceeded
spec:
  ttlSecondsAfterFinished: 100
  template:
    spec:
      containers:
        - name: httpie
          image: alpine/httpie:2.4.0
          imagePullPolicy: Always
          command: ["http"]
          args:
            [
              "POST",
              "todo-gitops:8080/api",
              "title=Finish ArgoCD tutorial",
              "--ignore-stdin"
            ]
      restartPolicy: Never
  backoffLimit: 1

1	This means that this Job will run in the `PostSync` phase, after the application of the manifests in the `Sync` phase.

Since I don’t have a deletion policy, this job will "stick around" after completion.

The execution order can be seen in the following diagram:

Deploying The Application

You can see all deployment files by visiting the repo.

Taking a look at this manifest file: todo-application.yaml:

Minikube
OpenShift

todo-application.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: todo-app
  namespace: argocd
spec:
  destination:
    namespace: todo
    server: https://kubernetes.default.svc
  project: default
  source:
    path: apps/todo
    repoURL: https://github.com/redhat-developer-demos/openshift-gitops-examples
    targetRevision: minikube
  syncPolicy:
    automated:
      prune: true
      selfHeal: false
    syncOptions:
    - CreateNamespace=true

It will show that this will deploy the application in the todo namespace.

Create this application:

kubectl apply -f documentation/modules/ROOT/examples/minikube/todo-yaml/todo-application.yaml

todo-application.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: todo-app
  namespace: openshift-gitops
spec:
  destination:
    namespace: todo
    server: https://kubernetes.default.svc
  project: default
  source:
    path: apps/todo
    repoURL: https://github.com/redhat-developer-demos/openshift-gitops-examples
    targetRevision: main
  syncPolicy:
    automated:
      prune: true
      selfHeal: false
    syncOptions:
    - CreateNamespace=true

It will show that this will deploy the application in the todo namespace.

Create this application:

kubectl apply -f documentation/modules/ROOT/examples/todo-yaml/todo-application.yaml

application.argoproj.io/todo-app created

On the Argo CD WebUI, you should see another application appear.

Clicking on this "card" should take you over to the tree view.

Observe the sync process. You will see the order that the resource has been applied, first the namespace creation and last the creation of Route to access the application.

Once the application is fully synced. Take a look at the pods and jobs in the namespace:

kubectl get pods -n todo

You should see that the Job is finished, but still there.

NAME                           READY   STATUS      RESTARTS   AGE
postgresql-599467fd86-cgj9v    1/1     Running     0          32s
todo-gitops-679d88f6f4-v4djp   1/1     Running     0          19s
todo-table-xhddk               0/1     Completed   0          27s

Your application should look like this.

The todo-insert Job is not shown as it was configured to be deleted if succeeded:

argocd.argoproj.io/hook-delete-policy: HookSucceeded