Managing pipeline runs | Pipelines as Code | Red Hat OpenShift Pipelines 1.11

Creating a pipeline run using Pipelines as Code
Running a pipeline run using Pipelines as Code
Monitoring pipeline run status using Pipelines as Code
Cleaning up pipeline run using Pipelines as Code
Using incoming webhook with Pipelines as Code
Additional resources

Using Pipelines as Code, you can create pipelines in your code repository and run these pipelines.

Creating a pipeline run using Pipelines as Code

To run pipelines using Pipelines as Code, you can create pipelines definitions or templates as YAML files in the .tekton/ directory of the repository. You can reference YAML files in other repositories using remote URLs, but pipeline runs are only triggered by events in the repository containing the .tekton/ directory.

The Pipelines as Code resolver bundles the pipeline runs with all tasks as a single pipeline run without external dependencies.

For pipelines, use at least one pipeline run with a spec, or a separated Pipeline object.
For tasks, embed task spec inside a pipeline, or define it separately as a Task object.

Parameterizing commits and URLs

You can specify the parameters of your commit and URL by using dynamic, expandable variables with the {{<var>}} format. Currently, you can use the following variables:

{{repo_owner}}: The repository owner.
{{repo_name}}: The repository name.
{{repo_url}}: The repository full URL.
{{revision}}: Full SHA revision of a commit.
{{sender}}: The username or account id of the sender of the commit.
{{source_branch}}: The branch name where the event originated.
{{target_branch}}: The branch name that the event targets. For push events, it’s the same as the source_branch.
{{pull_request_number}}: The pull or merge request number, defined only for a pull_request event type.
{{git_auth_secret}}: The secret name that is generated automatically with Git provider’s token for checking out private repos.

Matching an event to a pipeline run

You can match different Git provider events with each pipeline by using special annotations on the pipeline run. If there are multiple pipeline runs matching an event, Pipelines as Code runs them in parallel and posts the results to the Git provider as soon a pipeline run finishes.

Matching a pull event to a pipeline run

You can use the following example to match the pipeline-pr-main pipeline with a pull_request event that targets the main branch:

...
  metadata:
    name: pipeline-pr-main
  annotations:
    pipelinesascode.tekton.dev/on-target-branch: "[main]" (1)
    pipelinesascode.tekton.dev/on-event: "[pull_request]"
...

1	You can specify multiple branches by adding comma-separated entries. For example, `"[main, release-nightly]"`. In addition, you can specify the following: Full references to branches such as `"refs/heads/main"` Globs with pattern matching such as `"refs/heads/\"` Tags such as `"refs/tags/1.\"`

Matching a push event to a pipeline run

You can use the following example to match the pipeline-push-on-main pipeline with a push event targeting the refs/heads/main branch:

...
  metadata:
    name: pipeline-push-on-main
  annotations:
    pipelinesascode.tekton.dev/on-target-branch: "[refs/heads/main]" (1)
    pipelinesascode.tekton.dev/on-event: "[push]"
...

You can specifiy multiple branches by adding comma-separated entries. For example, "[main, release-nightly]". In addition, you can specify the following:

Full references to branches such as "refs/heads/main"
Globs with pattern matching such as "refs/heads/\*"
Tags such as "refs/tags/1.\*"

Advanced event matching

Pipelines as Code supports using Common Expression Language (CEL) based filtering for advanced event matching. If you have the pipelinesascode.tekton.dev/on-cel-expression annotation in your pipeline run, Pipelines as Code uses the CEL expression and skips the on-target-branch annotation. Compared to the simple on-target-branch annotation matching, the CEL expressions allow complex filtering and negation.

To use CEL-based filtering with Pipelines as Code, consider the following examples of annotations:

To match a pull_request event targeting the main branch and coming from the wip branch:

...
  pipelinesascode.tekton.dev/on-cel-expression: |
    event == "pull_request" && target_branch == "main" && source_branch == "wip"
...

To run a pipeline only if a path has changed, you can use the .pathChanged suffix function with a glob pattern:
```
...
  pipelinesascode.tekton.dev/on-cel-expression: |
    event == "pull_request" && "docs/\*.md".pathChanged() (1)
...
```
1 Matches all markdown files in the docs directory.

To match all pull requests starting with the title [DOWNSTREAM]:

...
  pipelinesascode.tekton.dev/on-cel-expression: |
    event == "pull_request && event_title.startsWith("[DOWNSTREAM]")
...

To run a pipeline on a pull_request event, but skip the experimental branch:

...
  pipelinesascode.tekton.dev/on-cel-expression: |
    event == "pull_request" && target_branch != experimental"
...

For advanced CEL-based filtering while using Pipelines as Code, you can use the following fields and suffix functions:

event: A push or pull_request event.
target_branch: The target branch.
source_branch: The branch of origin of a pull_request event. For push events, it is same as the target_branch.
event_title: Matches the title of the event, such as the commit title for a push event, and the title of a pull or merge request for a pull_request event. Currently, only GitHub, Gitlab, and Bitbucket Cloud are the supported providers.
.pathChanged: A suffix function to a string. The string can be a glob of a path to check if the path has changed. Currently, only GitHub and Gitlab are supported as providers.

Using the temporary GitHub App token for Github API operations

You can use the temporary installation token generated by Pipelines as Code from GitHub App to access the GitHub API. The token value is stored in the temporary {{git_auth_secret}} dynamic variable generated for private repositories in the git-provider-token key.

For example, to add a comment to a pull request, you can use the github-add-comment task from Tekton Hub using a Pipelines as Code annotation:

...
  pipelinesascode.tekton.dev/task: "github-add-comment"
...

You can then add a task to the tasks section or finally tasks in the pipeline run definition:

[...]
tasks:
  - name:
      taskRef:
        name: github-add-comment
      params:
        - name: REQUEST_URL
          value: "{{ repo_url }}/pull/{{ pull_request_number }}" (1)
        - name: COMMENT_OR_FILE
          value: "Pipelines as Code IS GREAT!"
        - name: GITHUB_TOKEN_secret_NAME
          value: "{{ git_auth_secret }}"
        - name: GITHUB_TOKEN_secret_KEY
          value: "git-provider-token"
...

1	By using the dynamic variables, you can reuse this snippet template for any pull request from any repository.

On GitHub Apps, the generated installation token is available for 8 hours and scoped to the repository from where the events originate unless configured differently on the cluster.

Additional resources

CEL language specification

Running a pipeline run using Pipelines as Code

With default configuration, Pipelines as Code runs any pipeline run in the .tekton/ directory of the default branch of repository, when specified events such as pull request or push occurs on the repository. For example, if a pipeline run on the default branch has the annotation pipelinesascode.tekton.dev/on-event: "[pull_request]", it will run whenever a pull request event occurs.

In the event of a pull request or a merge request, Pipelines as Code also runs pipelines from branches other than the default branch, if the following conditions are met by the author of the pull request:

The author is the owner of the repository.
The author is a collaborator on the repository.
The author is a public member on the organization of the repository.
The pull request author is listed in an OWNER file located in the repository root of the main branch as defined in the GitHub configuration for the repository. Also, the pull request author is added to either approvers or reviewers section. For example, if an author is listed in the approvers section, then a pull request raised by that author starts the pipeline run.

...
  approvers:
    - approved
...

If the pull request author does not meet the requirements, another user who meets the requirements can comment /ok-to-test on the pull request, and start the pipeline run.

Pipeline run execution

A pipeline run always runs in the namespace of the Repository custom resource definition (CRD) associated with the repository that generated the event.

You can observe the execution of your pipeline runs using the tkn pac CLI tool.

To follow the execution of the last pipeline run, use the following example:
```
$ tkn pac logs -n <my-pipeline-ci> -L (1)
```
1 my-pipeline-ci is the namespace for the Repository CRD.

To follow the execution of any pipeline run interactively, use the following example:

$ tkn pac logs -n <my-pipeline-ci> (1)

1	`my-pipeline-ci` is the namespace for the `Repository` CRD. If you need to view a pipeline run other than the last one, you can use the `tkn pac logs` command to select a `PipelineRun` attached to the repository:

If you have configured Pipelines as Code with a GitHub App, Pipelines as Code posts a URL in the Checks tab of the GitHub App. You can click the URL and follow the pipeline execution.

Restarting a pipeline run

You can restart a pipeline run with no events, such as sending a new commit to your branch or raising a pull request. On a GitHub App, go to the Checks tab and click Re-run.

If you target a pull or merge request, use the following comments inside your pull request to restart all or specific pipeline runs:

The /retest comment restarts all pipeline runs.
The /retest <pipelinerun-name> comment restarts a specific pipeline run.
The /cancel comment cancels all pipeline runs.
The /cancel <pipelinerun-name> comment cancels a specific pipeline run.

The results of the comments are visible under the Checks tab of a GitHub App.

Monitoring pipeline run status using Pipelines as Code

Depending on the context and supported tools, you can monitor the status of a pipeline run in different ways.

Status on GitHub Apps

When a pipeline run finishes, the status is added in the Check tabs with limited information on how long each task of your pipeline took, and the output of the tkn pipelinerun describe command.

Log error snippet

When Pipelines as Code detects an error in one of the tasks of a pipeline, a small snippet consisting of the last 3 lines in the task breakdown of the first failed task is displayed.

Pipelines as Code avoids leaking secrets by looking into the pipeline run and replacing secret values with hidden characters. However, Pipelines as Code cannot hide secrets coming from workspaces and envFrom source.

Annotations for log error snippets

In the TektonConfig custom resource, in the pipelinesAsCode.settings spec, you can set the error-detection-from-container-logs parameter to true. In this case, Pipelines as Code detects the errors from the container logs and adds them as annotations on the pull request where the error occurred.

Adding annotations for log error snippets is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

Currently, Pipelines as Code supports only the simple cases where the error looks like makefile or grep output of the following format:

<filename>:<line>:<column>: <error message>

You can customize the regular expression used to detect the errors with the error-detection-simple-regexp parameter. The regular expression uses named groups to give flexibility on how to specify the matching. The groups needed to match are filename, line, and error. You can view the Pipelines as Code config map for the default regular expression.

By default, Pipelines as Code scans only the last 50 lines of the container logs. You can increase this value in the error-detection-max-number-of-lines field or set -1 for an unlimited number of lines. However, such configurations may increase the memory usage of the watcher.

Status for webhook

For webhook, when the event is a pull request, the status is added as a comment on the pull or merge request.

Failures

If a namespace is matched to a Repository custom resource definition (CRD), Pipelines as Code emits its failure log messages in the Kubernetes events inside the namespace.

Status associated with Repository CRD

The last 5 status messages for a pipeline run is stored inside the Repository custom resource.

$ oc get repo -n <pipelines-as-code-ci>

NAME                  URL                                                        NAMESPACE             SUCCEEDED   REASON      STARTTIME   COMPLETIONTIME
pipelines-as-code-ci   https://github.com/openshift-pipelines/pipelines-as-code   pipelines-as-code-ci   True        Succeeded   59m         56m

Using the tkn pac describe command, you can extract the status of the runs associated with your repository and its metadata.

Notifications

Pipelines as Code does not manage notifications. If you need to have notifications, use the finally feature of pipelines.

Additional resources

An example of the git-clone task used for cloning private repositories

Cleaning up pipeline run using Pipelines as Code

There can be many pipeline runs in a user namespace. By setting the max-keep-runs annotation, you can configure Pipelines as Code to retain a limited number of pipeline runs that matches an event. For example:

...
  pipelinesascode.tekton.dev/max-keep-runs: "<max_number>" (1)
...

Pipelines as Code starts cleaning up right after it finishes a successful execution, retaining only the maximum number of pipeline runs configured using the annotation.

Pipelines as Code skips cleaning the running pipelines but cleans up the pipeline runs with an unknown status.
Pipelines as Code skips cleaning a failed pull request.

Using incoming webhook with Pipelines as Code

Using an incoming webhook URL and a shared secret, you can start a pipeline run in a repository.

To use incoming webhooks, specify the following within the spec section of the Repository custom resource definition (CRD):

The incoming webhook URL that Pipelines as Code matches.
The Git provider and the user token. Currently, Pipelines as Code supports github, gitlab, and bitbucket-cloud.

When using incoming webhook URLs in the context of GitHub app, you must specify the token.
The target branches and a secret for the incoming webhook URL.

Example: Repository CRD with incoming webhook

apiVersion: "pipelinesascode.tekton.dev/v1alpha1"
kind: Repository
metadata:
  name: repo
  namespace: ns
spec:
  url: "https://github.com/owner/repo"
  git_provider:
    type: github
    secret:
      name: "owner-token"
  incoming:
    - targets:
      - main
      secret:
        name: repo-incoming-secret
      type: webhook-url

Example: The repo-incoming-secret secret for incoming webhook

apiVersion: v1
kind: secret
metadata:
  name: repo-incoming-secret
  namespace: ns
type: Opaque
stringData:
  secret: <very-secure-shared-secret>

To trigger a pipeline run located in the .tekton directory of a Git repository, use the following command:

$ curl -X POST 'https://control.pac.url/incoming?secret=very-secure-shared-secret&repository=repo&branch=main&pipelinerun=target_pipelinerun'

Pipelines as Code matches the incoming URL and treats it as a push event. However, Pipelines as Code does not report status of the pipeline runs triggered by this command.

To get a report or a notification, add it directly with a finally task to your pipeline. Alternatively, you can inspect the Repository CRD with the tkn pac CLI tool.