Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions config/302-pac-configmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,23 @@ data:
# Allow fetching remote tasks
remote-tasks: "true"

# Optional comma-separated host allowlist for remote annotation URLs. Bare
# hosts and https:// hosts allow HTTPS fetches. HTTP fetches are disabled
# unless the host is explicitly listed with http://.
# Wildcards such as "*.example.com" match host suffixes; broad patterns such
# as "*.com" are allowed but risky.
remote-tasks-url-allowlist: ""

Comment thread
chmouel marked this conversation as resolved.
# Optional comma-separated CIDRs to block in addition to built-in unsafe
# ranges such as loopback, link-local, private, multicast, and shared address
# space. These built-in checks block typical Kubernetes service targets when
# they resolve to private cluster IPs. Use this to block additional
# cluster-specific service and pod CIDRs.
remote-tasks-url-blocked-cidrs: ""

# Maximum response size, in bytes, for remote annotation fetches.
remote-tasks-url-max-response-size: "1048576"
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like exposing low level stuff as configuration but if for whatever reason someone has a task/prun bigger than 1mb to fetch then we let them fetch it


# Using the URL of the Tekton dashboard, Pipelines-as-Code generates a URL to the
# PipelineRun on the Tekton dashboard
tekton-dashboard-url: ""
Expand Down
30 changes: 30 additions & 0 deletions docs/content/docs/api/configmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,33 @@ remote-tasks: "true"

{{< /param >}}

{{< param name="remote-tasks-url-allowlist" type="string" default="" id="param-remote-tasks-url-allowlist" >}}
Restricts remote annotation URLs to a comma-separated list of hosts. Leave empty to allow HTTPS fetches from any public host after IP-range safety checks. Bare hosts and `https://` entries allow HTTPS; plaintext HTTP requires an explicit `http://` entry for the host. Wildcards such as `*.example.com` match host suffixes; broad patterns such as `*.com` are allowed but risky and should be used only when the operator intends that scope.

```yaml
remote-tasks-url-allowlist: "raw.githubusercontent.com,https://gitlab.example.com,http://internal.example.com"
```

{{< /param >}}

{{< param name="remote-tasks-url-blocked-cidrs" type="string" default="" id="param-remote-tasks-url-blocked-cidrs" >}}
Adds comma-separated CIDRs that remote annotation URLs must not reach. Pipelines-as-Code always blocks loopback, link-local, private, multicast, unspecified, and shared address space. These built-in checks block typical Kubernetes service targets when they resolve to private cluster IPs; use this setting for any additional cluster-specific pod or service CIDRs.

```yaml
remote-tasks-url-blocked-cidrs: "10.128.0.0/14,172.30.0.0/16"
```

{{< /param >}}

{{< param name="remote-tasks-url-max-response-size" type="integer" default="1048576" id="param-remote-tasks-url-max-response-size" >}}
Maximum response size, in bytes, for remote annotation fetches.

```yaml
remote-tasks-url-max-response-size: "1048576"
```

{{< /param >}}

### Dashboard Integration

{{< param name="tekton-dashboard-url" type="string" id="param-tekton-dashboard-url" >}}
Expand Down Expand Up @@ -364,6 +391,9 @@ data:
hub-url: "https://artifacthub.io/api/v1"
hub-catalog-type: "artifacthub"
remote-tasks: "true"
remote-tasks-url-allowlist: ""
remote-tasks-url-blocked-cidrs: ""
remote-tasks-url-max-response-size: "1048576"

tekton-dashboard-url: "https://tekton.example.com"

Expand Down
2 changes: 1 addition & 1 deletion docs/content/docs/guides/creating-pipelines/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ weight: 1

This page covers how to write PipelineRun definitions that Pipelines-as-Code picks up from your `.tekton/` directory. Use it when you need to define CI/CD pipelines triggered by Git events.

Pipelines-as-Code follows the standard Tekton template format as closely as possible. You write your templates as `.yaml` files in the `.tekton/` directory at the top level of your repository, and Pipelines-as-Code runs them. You can reference YAML files in other repositories using [remote HTTP URLs]({{< relref "/docs/guides/pipeline-resolution#remote-http-url" >}}), but PipelineRuns only trigger from events in the repository that contains the `.tekton/` directory.
Pipelines-as-Code follows the standard Tekton template format as closely as possible. You write your templates as `.yaml` files in the `.tekton/` directory at the top level of your repository, and Pipelines-as-Code runs them. You can reference YAML files in other repositories using [remote URLs]({{< relref "/docs/guides/pipeline-resolution#remote-url" >}}), but PipelineRuns only trigger from events in the repository that contains the `.tekton/` directory.

Using its [resolver]({{< relref "/docs/guides/pipeline-resolution" >}}), Pipelines-as-Code bundles each PipelineRun with all its referenced Tasks into a single self-contained PipelineRun with no external dependencies.

Expand Down
18 changes: 10 additions & 8 deletions docs/content/docs/guides/pipeline-resolution/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Resolver
weight: 3
---

This page explains how the Pipelines-as-Code resolver processes your `.tekton/` directory and assembles self-contained PipelineRuns. It covers the overall resolution process and the `pipelinesascode.tekton.dev/task` annotation for fetching tasks from Artifact Hub, HTTP URLs, or your repository. For the `pipelinesascode.tekton.dev/pipeline` annotation that references remote Pipelines, see [Remote Pipelines]({{< relref "remote-pipelines" >}}).
This page explains how the Pipelines-as-Code resolver processes your `.tekton/` directory and assembles self-contained PipelineRuns. It covers the overall resolution process and the `pipelinesascode.tekton.dev/task` annotation for fetching tasks from Artifact Hub, remote URLs, or your repository. For the `pipelinesascode.tekton.dev/pipeline` annotation that references remote Pipelines, see [Remote Pipelines]({{< relref "remote-pipelines" >}}).

The resolver exists to solve a practical problem: Tekton PipelineRuns can reference external Tasks and Pipelines by name, but those references must be available on the cluster at runtime. Rather than requiring you to pre-install every Task, Pipelines-as-Code resolves all references at submission time and embeds everything into a single PipelineRun. This ensures your pipeline is fully self-contained and portable.

Expand Down Expand Up @@ -36,7 +36,7 @@ If you need to test your PipelineRun locally before sending it in a pull request

## Remote task annotations

Remote task annotations let you pull Task and Pipeline definitions from external sources -- such as Artifact Hub, HTTP URLs, or other repositories -- without committing them to your `.tekton/` directory. Pipelines-as-Code fetches and inlines them during resolution.
Remote task annotations let you pull Task and Pipeline definitions from external sources -- such as Artifact Hub, remote URLs, or other repositories -- without committing them to your `.tekton/` directory. Pipelines-as-Code fetches and inlines them during resolution.

If the resolver finds a PipelineRun referencing a remote task or Pipeline through an annotation, it automatically fetches and inlines the resource.

Expand All @@ -57,7 +57,7 @@ pipelinesascode.tekton.dev/task: "[git-clone, pylint]"
### Hub support for tasks

{{< callout type="warning" >}}
**Deprecated**: Tekton Hub integration (the `tektonhub` catalog type) is deprecated and will be removed in a future release. If you are using a self-hosted Tekton Hub instance, please migrate to [Artifact Hub](https://artifacthub.io) or fetch tasks directly from a [remote URL](#remote-http-url) or [git repository](#tasks-inside-the-repository). The default Artifact Hub integration is unaffected.
**Deprecated**: Tekton Hub integration (the `tektonhub` catalog type) is deprecated and will be removed in a future release. If you are using a self-hosted Tekton Hub instance, please migrate to [Artifact Hub](https://artifacthub.io) or fetch tasks directly from a [remote URL](#remote-url) or [git repository](#tasks-inside-the-repository). The default Artifact Hub integration is unaffected.
{{< /callout >}}

[Artifact Hub](https://artifacthub.io/packages/search?kind=7&kind=11) is a public registry where the Tekton community publishes reusable Tasks and Pipelines. When you reference a task by name alone, Pipelines-as-Code fetches it from Artifact Hub by default.
Expand Down Expand Up @@ -108,15 +108,17 @@ There is no fallback between different hubs. If Pipelines-as-Code does not find

There is no support for custom hubs from the CLI when using the `tkn pac resolve` command.

### Remote HTTP URL
### Remote URL

If the annotation value starts with `http://` or `https://`, Pipelines-as-Code fetches the task directly from that remote URL:
If the annotation value starts with `https://`, Pipelines-as-Code fetches the task directly after applying the remote URL safety checks.

Plaintext `http://` URLs are disabled by default. A cluster operator can opt in for a specific host by adding an `http://` entry to `remote-tasks-url-allowlist`.

```yaml
pipelinesascode.tekton.dev/task: "[https://remote.url/task.yaml]"
```

### Remote HTTP URL from a private repository
### Remote URLs from a private repository

If you use the GitHub or GitLab provider and the remote task URL uses the same host as the Repository CR, Pipelines-as-Code uses the provided token to fetch the URL through the GitHub or GitLab API. This lets you reference tasks from private repositories without exposing credentials.

Expand All @@ -126,11 +128,11 @@ When you use the GitHub provider and your repository URL looks like this:

<https://github.com/organization/repository>

and the remote HTTP URL is a GitHub "blob" URL:
and the remote URL is a GitHub "blob" URL:

<https://github.com/organization/repository/blob/mainbranch/path/file>

If the remote HTTP URL has a slash (`/`) in the branch name, you need to URL-encode it with the `%2F` character:
If the remote URL has a slash (`/`) in the branch name, you need to URL-encode it with the `%2F` character:

<https://github.com/organization/repository/blob/feature%2Fmainbranch/path/file>

Expand Down
53 changes: 45 additions & 8 deletions pkg/matcher/annotation_tasks_install.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
"github.com/openshift-pipelines/pipelines-as-code/pkg/hub"
hubtypes "github.com/openshift-pipelines/pipelines-as-code/pkg/hub/vars"
"github.com/openshift-pipelines/pipelines-as-code/pkg/params"
"github.com/openshift-pipelines/pipelines-as-code/pkg/params/clients"
"github.com/openshift-pipelines/pipelines-as-code/pkg/params/info"
"github.com/openshift-pipelines/pipelines-as-code/pkg/params/settings"
"github.com/openshift-pipelines/pipelines-as-code/pkg/provider"
Expand All @@ -34,6 +35,19 @@ type RemoteTasks struct {
DeprecatedHubResources []string // tracks resources resolved from deprecated tektonhub catalogs
}

func (rt RemoteTasks) remoteResourceFetchOptions() clients.RemoteResourceFetchOptions {
if rt.Run == nil || rt.Run.Info.Pac == nil {
return clients.RemoteResourceFetchOptions{
MaxResponseBytes: clients.DefaultRemoteResourceMaxResponseBytes,
}
}
return clients.RemoteResourceFetchOptions{
AllowedHosts: rt.Run.Info.Pac.RemoteTasksURLAllowlist,
BlockedCIDRs: rt.Run.Info.Pac.RemoteTasksURLBlockedCIDRs,
MaxResponseBytes: int64(rt.Run.Info.Pac.RemoteTasksURLMaxResponseSize),
}
}

// nolint: dupl
func (rt *RemoteTasks) convertToPipeline(ctx context.Context, uri, data string) (*tektonv1.Pipeline, error) {
decoder := k8scheme.Codecs.UniversalDeserializer()
Expand Down Expand Up @@ -101,20 +115,43 @@ func (rt *RemoteTasks) convertTotask(ctx context.Context, uri, data string) (*te

func (rt *RemoteTasks) getRemote(ctx context.Context, uri string, fromHub bool, kind string) (string, error) {
rt.Logger.Debugf("getRemote: uri=%s kind=%s fromHub=%t", uri, kind, fromHub)
if fetchedFromURIFromProvider, task, err := rt.ProviderInterface.GetTaskURI(ctx, rt.Event, uri); fetchedFromURIFromProvider {
rt.Logger.Debugf("getRemote: fetched %s via provider hook for uri=%s", kind, uri)
return task, err
}
fetchOptions := rt.remoteResourceFetchOptions()
loweredURI := strings.ToLower(uri)

switch {
case strings.HasPrefix(uri, "https://"), strings.HasPrefix(uri, "http://"): // if it starts with http(s)://, it is a remote resource
if strings.HasPrefix(loweredURI, "http://") || strings.HasPrefix(loweredURI, "https://") {
if err := clients.ValidateRemoteResourceProviderURL(uri, fetchOptions); err != nil {
return "", err
}
if fetchedFromURIFromProvider, task, err := rt.ProviderInterface.GetTaskURI(ctx, rt.Event, uri); fetchedFromURIFromProvider {
rt.Logger.Debugf("getRemote: fetched %s via provider hook for uri=%s", kind, uri)
if err != nil {
return task, err
}
Comment on lines +127 to +129
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if err != nil {
return task, err
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if fetchedFromURIFromProvider is true, then there is no chance of err being non-nil

if err := clients.CheckRemoteResourceResponseSize(int64(len(task)), fetchOptions); err != nil {
return "", err
}
return task, nil
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
}
} else if err != nil {
return task, err
}

rt.Logger.Debugf("getRemote: fetching %s from http(s) url", kind)
data, err := rt.Run.Clients.GetURL(ctx, uri)
data, err := rt.Run.Clients.GetRemoteResourceURL(ctx, uri, fetchOptions)
if err != nil {
return "", err
}
rt.Logger.Infof("successfully fetched %s from remote HTTPS URL", uri)
rt.Logger.Infof("successfully fetched %s from remote URL", uri)
return string(data), nil
}
if fetchedFromURIFromProvider, task, err := rt.ProviderInterface.GetTaskURI(ctx, rt.Event, uri); fetchedFromURIFromProvider {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

rt.Logger.Debugf("getRemote: fetched %s via provider hook for uri=%s", kind, uri)
if err != nil {
return task, err
}
if err := clients.CheckRemoteResourceResponseSize(int64(len(task)), fetchOptions); err != nil {
return "", err
}
return task, nil
}
Comment on lines +121 to +152
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not create the policy as part of remoteResourceFetchOptions?
It is a small overhead but the same newRemoteResourcePolicy call is being made multiple times by the clients.Func methods.


switch {
case fromHub && strings.Contains(uri, "://"): // if it contains ://, it is a remote custom catalog
split := strings.Split(uri, "://")
catalogID := split[0]
Expand Down
Loading
Loading