diff --git a/README.md b/README.md index 7c9c040..a181252 100644 --- a/README.md +++ b/README.md @@ -8,14 +8,14 @@ Version-controlled specifications for API Details are located in the `docs/api` This repository is currently under development. Documentation and examples are being added as they are created. -| Task | Status | -| ---------------------------------------------------------------------------------------- | ------ | -| Example CRS Azure Kubernetes Service (AKS) Terraform | ✅ | -| Competition API Spec v1.4.0 | ✅ | -| CRS API Spec v1.4.0 | ✅ | -| Telemetry Spec v1.0 | ✅ | -| <[Challenge Generation script](https://github.com/aixcc-finals/generate-challenge-task)> | ✅ | -| End-to-end example webservice that implements specs | ✅ | +| Task | Version | Status | +| ------------------------------------------------------------------ | ------- | ------ | +| Example CRS Azure Kubernetes Service (AKS) Terraform | N/A | ✅ | +| Competition API Spec | v1.4.0 | ✅ | +| CRS API Spec | v1.4.0 | ✅ | +| Telemetry Spec | v1.0 | ✅ | +| <[Challenge Generation script](generate-challenge-task/README.md)> | N/A | ✅ | +| End-to-end example webservice that implements specs | N/A | ✅ | ## Contributing diff --git a/docs/README.md b/docs/README.md index f49b50c..9465ba6 100644 --- a/docs/README.md +++ b/docs/README.md @@ -41,8 +41,6 @@ Welcome to the documentation page for the AIxCC Final Competition. - [Improving the structure of OTEL data](telemetry/telemetry_best_practices.md#improving-the-structure-of-otel-data) - [Verifying telemetry](telemetry/telemetry_best_practices.md#verifying-telemetry) - [Source File Language Determination](source_language_determination) -- [Tailnet-accessible hosted Competitor Test Server](tailscale-hosted-competitor-test-server/hosted_competitor_test_server.md#tailnet-accessible-hosted-competitor-test-server) -- [Tailscale Management Tools](tailscale-hosted-competitor-test-server/tailscale_management_tools.md#tailscale-management-tools) - [Round Information](round_info/README.md#round-information) - [Exhibition Round 1](round_info/exhibition-round-1.md#exhibition-round-1) diff --git a/docs/api/README.md b/docs/api/README.md index 1f1c51d..ddb3576 100644 --- a/docs/api/README.md +++ b/docs/api/README.md @@ -71,102 +71,6 @@ CRS->>API: Create Bundle "accepted" CRS->>API: Modify Bundle ``` -### Request Integration Test Task - -After deploying your CRS, you now have the ability to task your CRS with a simple “[integration test](https://github.com/aixcc-finals/integration-test/tree/challenges/integration-test-delta-01)” challenge (delta scan) to -make sure your CRS and Telemetry are working as expected prior to the opening of the Round. - -This endpoint is available from now until 5 minutes prior to the opening of a Round. It is accessible from the internet and from inside the tailnet. `duration_secs` is an optional parameter, and defaults to 4 hours. -Provide your Competition API credentials to kick off the task. This will send a task to the Round's expected CRS hostname (e.g. `team-moniker-exhibition2` for exhibition2) on the tailnet. - -```mermaid -sequenceDiagram - accTitle: Smoke Test Workflow - - User->>API: /v1/request/delta - API->>CRS: Hardcoded integration-test task - CRS->>API: POV/Patch/Bundle/Sarif Submission -``` - -Here is an example curl you may use to trigger the integration testing task. - -```bash -curl -u 11111111-1111-1111-1111-111111111111:pY8rLk7FvQ2hZm9GwUx3Ej5BnTcV4So0 -X 'POST' 'https://api.aixcc.tech/v1/request/delta/' -H 'Content-Type: application/json' -d '{"duration_secs": 3600 }' -``` - -### Request Arbitrary Exhibition 3 Task - -We have updated the production Competition API /v1/request/ endpoint to include all exhibition3 challenges. Exercising this endpoint should look the same as exercising the /v1/request/delta/ endpoint, but -instead of putting delta in the URL path, you can input the challenge name. - -For example: - -```bash -curl -u : -X 'POST' 'https://api.aixcc.tech/v1/request/ex3-tk-full-01/' --json '{"duration_secs":43200}' -``` - -You may, of course, still use /v1/request/delta/, and you will get the same integration test challenge as you would have before Exhibition 3. In order to get a list of the challenges that are available, you may use -the /v1/request/list/ endpoint: - -```bash -curl -u : -X 'GET' 'https://api.aixcc.tech/v1/request/list/' -``` - -Some teams have noted that the hosted competitor test servers were having issues scaling to respond to the number of submissions. Unfortunately, this is a limit. The official competition API, however, is built to -handle hundreds of POVs at the same time. So if you are running into any issues with the competitor test server, we highly recommend you use this new endpoint. We will still also still be updating the competitor test -servers as we have been before when updates are made to evaluation scripts. - -Use the following URL/hostname configurations in order to use this new requesting feature: - -**CRS API URL**: `https://-final.tail7e9b4c.ts.net` - -**Competition API URL**: `https://api.tail7e9b4c.ts.net` - -Here's a reference for which challenge names correspond to which repos: - -- **Apache Commons Compress** - - ex3-cc-delta-02 - - ex3-cc-delta-03 - - ex3-cc-full-01 -- **FreeRDP** - - ex3-fp-delta-01 - - ex3-fp-full-01 -- **Integration Test** (There should be no significant differences between "ex3-integration-test-delta-01" and "delta") - - ex3-integration-test-delta-01 - - ex3-integration-test-unharnessed-delta-01 - - delta -- **libpng** - - ex3-lp-delta-01 -- **libxml2** - - ex3-lx-delta-01 - - ex3-lx-delta-02 -- **sqlite3** - - ex3-sq-delta-01 - - ex3-sq-delta-02 - - ex3-sq-delta-03 - - ex3-sq-full-01 -- **Apache Tika** - - ex3-tk-full-01 - - ex3-tk-delta-02 - - ex3-tk-delta-03 - - ex3-tk-delta-04 - - ex3-tk-delta-05 -- **Apache Zookeeper** - - ex3-zk-delta-01 - - ex3-zk-delta-02 - - ex3-zk-full-01 -- **Curl** - - ex3-cu-full-01 - - ex3-cu-delta-01 -- **libexif** - - ex3-ex-delta-01 -- **libpostal** - - ex3-libpostal-full-01 -- **s2n-tls** - - ex3-s2n_tls-full-01 -- **IPF** - - ex3-ipf-full-01 - ## CRS API Task Statuses The CRS API has a status endpoint which provides a summary of tasks by status, among other things. The statuses which tasks go through are documented in the state diagram below. @@ -200,7 +104,7 @@ To run the generator using docker or podman: ```bash docker run --rm -v $PWD:/local openapitools/openapi-generator-cli generate \ - -i /local/competition-swagger-v0.1.json \ + -i /local/competition-swagger.json \ -g lang \ -o /local/out ``` @@ -208,7 +112,7 @@ docker run --rm -v $PWD:/local openapitools/openapi-generator-cli generate \ - `-v $PWD:/local` mounts the current working directory into the `/local` directory in the container. A different host path could be provided for `$PWD`. All paths used in the following steps would be relative to the new path instead of `$PWD`. - All arguments after `openapitools/openapi-generator-cli` are passed to the generator CLI inside the container -- `-i /local/competition-swagger-v0.1.json` is the path relative to the current working directory of the swagger file. On the host, `competition-swagger-v-0.1.json` is located in `$PWD`. If you want to change the file or +- `-i /local/competition-swagger-v0.1.json` is the path relative to the current working directory of the swagger file. On the host, `competition-swagger.json` is located in `$PWD`. If you want to change the file or path it must be a descendant of `$PWD`. - `-g lang` is used to specify the generator to run. If you pass an invalid value it will list all of the options. - `-o /local/out` is the path relative to the current working directory to output the generated code. @@ -247,7 +151,7 @@ If you do not wish to integrate the UI directly into the CRS system it is still python -m http.server ``` -- Change the path to the swagger file at the top of your page to the desired file. For example, `/competition-swagger-v0.1.json` if you placed a swagger spec called `competition-swagger-v0.1.json` at - `swagger-ui/dist/competition-swagger-v0.1.json`. +- Change the path to the swagger file at the top of your page to the desired file. For example, `/competition-swagger.json` if you placed a swagger spec called `competition-swagger.json` at + `swagger-ui/dist/competition-swagger.json`. - This will only allow viewing the documentation. Experimenting with the API endpoints requires a running API server serving the spec file. If the spec is served from the API server and it supports CORS, it is possible to specify the full URL to the spec in the box. This will allow you to experiment with the API using the `Try it Out` button. diff --git a/docs/tailscale-hosted-competitor-test-server/hosted_competitor_test_server.md b/docs/tailscale-hosted-competitor-test-server/hosted_competitor_test_server.md deleted file mode 100644 index 9751fd1..0000000 --- a/docs/tailscale-hosted-competitor-test-server/hosted_competitor_test_server.md +++ /dev/null @@ -1,68 +0,0 @@ -# Tailnet-accessible hosted Competitor Test Server - -The organizers have set up a test competition API server for each team that is reachable from the internet, but tasks your CRS hosted on the tailnet. Telemetry from these runs get forwarded to your team’s telemetry -server. - -It is intended that competitors may use this to task their CRS with challenges from prior rounds that are available in the aixcc-finals org. There are three changes to your CRS deployment that are required in order for -you to use this server. - -1. Change the URL of the Competition API to `https://test--api.tail7e9b4c.ts.net` -2. If you are using Kubernetes Tailscale Operator, add another Service for this test URL so that it can be accessible from your cluster. See - [proxies.yaml](https://github.com/aixcc-finals/example-crs-architecture/blob/main/example-crs-architecture/k8s/base/tailscale-connections/proxies.yaml) for context. -3. Append `-testing` to your CRS hostname, for example `team-moniker-testing`. This is the URL that is configured by default. - -If you want to use a different CRS URL **for the tailscale-hosted competitor test server**, use the following: - -Update CRS hostname used by the hosted competitor testing server - -```bash -curl -u 11111111-1111-1111-1111-111111111111:pY8rLk7FvQ2hZm9GwUx3Ej5BnTcV4So0 -X PATCH https://.tasker.aixcc.tech/crs/url/ -H 'Content-Type: application/json' -d '{"hostname":"team-moniker-testing-1"}' -``` - -Get CRS hostname used by the hosted competitor testing server - -```bash -curl -u 11111111-1111-1111-1111-111111111111:pY8rLk7FvQ2hZm9GwUx3Ej5BnTcV4So0 https://.tasker.aixcc.tech/crs/url/ -``` - -Kick off tasks with the server - -```bash -# Integration Test -curl -u 11111111-1111-1111-1111-111111111111:pY8rLk7FvQ2hZm9GwUx3Ej5BnTcV4So0 -X 'POST' 'https://.tasker.aixcc.tech/v1/request/delta/' -# LibPNG -curl -u 11111111-1111-1111-1111-111111111111:pY8rLk7FvQ2hZm9GwUx3Ej5BnTcV4So0 -X 'POST' 'https://.tasker.aixcc.tech/webhook/trigger_task' -H 'Content-Type: application/json' -d '{ - "challenge_repo_url": "git@github.com:aixcc-finals/example-libpng.git", - "challenge_repo_base_ref": "0cc367aaeaac3f888f255cee5d394968996f736e", - "challenge_repo_head_ref": "fdacd5a1dcff42175117d674b0fda9f8a005ae88", - "fuzz_tooling_url": "https://github.com/aixcc-finals/oss-fuzz-aixcc.git", - "fuzz_tooling_ref": "d5fbd68fca66e6fa4f05899170d24e572b01853d", - "fuzz_tooling_project_name": "libpng", - "duration": 3600 -}' -# Zookeeper -curl -u 11111111-1111-1111-1111-111111111111:pY8rLk7FvQ2hZm9GwUx3Ej5BnTcV4So0 -X 'POST' 'https://.tasker.aixcc.tech/webhook/trigger_task' -H 'Content-Type: application/json' -d '{ - "challenge_repo_url": "git@github.com:aixcc-finals/afc-zookeeper.git", - "challenge_repo_base_ref": "d19cef9ca254a4c1461490ed8b82ffccfa57461d", - "challenge_repo_head_ref": "5ee4f185d0431cc88f365ce779aa04a87fe7690f", - "fuzz_tooling_url": "https://github.com/aixcc-finals/oss-fuzz-aixcc.git", - "fuzz_tooling_ref": "challenge-state/zk-ex1-delta-01", - "fuzz_tooling_project_name": "zookeeper", - "duration": 3600 -}' -# Libxml2 -curl -u 11111111-1111-1111-1111-111111111111:pY8rLk7FvQ2hZm9GwUx3Ej5BnTcV4So0 -X 'POST' 'https://.tasker.aixcc.tech/webhook/trigger_task' -H 'Content-Type: application/json' -d '{ - "challenge_repo_url": "git@github.com:aixcc-finals/afc-libxml2.git", - "challenge_repo_base_ref": "792cc4a1462d4a969d9d38bd80a52d2e4f7bd137", - "challenge_repo_head_ref": "9d1cb67c31933ee5ae3ee458940f7dbeb2fde8b8", - "fuzz_tooling_url": "https://github.com/aixcc-finals/oss-fuzz-aixcc.git", - "fuzz_tooling_ref": "challenge-state/lx-ex1-delta-01", - "fuzz_tooling_project_name": "libxml2", - "duration": 3600 -}' - -``` - -## IMPORTANT NOTE - -If you are using this test server, don’t forget to change your Competition API URL back to `https://api.tail7e9b4c.ts.net` prior to the start of the round. diff --git a/docs/tailscale-hosted-competitor-test-server/tailscale_management_tools.md b/docs/tailscale-hosted-competitor-test-server/tailscale_management_tools.md deleted file mode 100644 index fb3c48a..0000000 --- a/docs/tailscale-hosted-competitor-test-server/tailscale_management_tools.md +++ /dev/null @@ -1,44 +0,0 @@ -# Tailscale Management Tools - -Each team has access to a set of APIs to manage their Tailscale devices. You can use this tool to list devices, delete devices, and update hostnames of tailscale devices. Each endpoint uses the Competition API -credentials for your team. - -List Devices - -Lists devices on the tailnet for your team. - -```bash -curl -u 11111111-1111-1111-1111-111111111111:pY8rLk7FvQ2hZm9GwUx3Ej5BnTcV4So0 https://.tasker.aixcc.tech/tailscale/device/ -``` - -Show Device - -Shows information about a single device by name - -```bash -curl -u 11111111-1111-1111-1111-111111111111:pY8rLk7FvQ2hZm9GwUx3Ej5BnTcV4So0 https://.tasker.aixcc.tech/tailscale/device/team-moniker-foo -``` - -Rename Device - -Change the hostname of a device on the tailnet - -```bash -curl -u 11111111-1111-1111-1111-111111111111:pY8rLk7FvQ2hZm9GwUx3Ej5BnTcV4So0 -X PATCH https://.tasker.aixcc.tech/tailscale/device/team-moniker-foo -H 'Content-Type: application/json' -d '{"hostname":"team-moniker-exhibition2"}' -``` - -Delete Device - -Deletes a specific device on the tailnet - -```bash -curl -u 11111111-1111-1111-1111-111111111111:pY8rLk7FvQ2hZm9GwUx3Ej5BnTcV4So0 -X DELETE https://.tasker.aixcc.tech/tailscale/device/team-moniker-foo -``` - -Delete all Devices - -**WARNING** This deletes all devices on your team’s tailnet. - -```bash -curl -u 11111111-1111-1111-1111-111111111111:pY8rLk7FvQ2hZm9GwUx3Ej5BnTcV4So0 -X DELETE https://.tasker.aixcc.tech/tailscale/device/ -``` diff --git a/example-crs-architecture/README.md b/example-crs-architecture/README.md index 287a95a..6a5510b 100644 --- a/example-crs-architecture/README.md +++ b/example-crs-architecture/README.md @@ -234,7 +234,7 @@ The deployment of the AKS cluster and its resources are performed by the `Makefi ## Deploy - Log into your Azure tenant with `az login --tenant aixcc.tech` -- Clone this repository if needed: `git clone git@github.com:aixcc-finals/example-crs-architecture.git /` +- Clone this repository if needed: `git clone git@github.com:AIxCyberChallenge/example-crs-architecture.git /` - Make required changes to `backend.tf` - Make any wanted changes to `main.tf`, `outputs.tf`, `providers.tf`, and `variables.tf` - Update `./env` with accurate values for each variable diff --git a/example-crs-webservice/README.md b/example-crs-webservice/README.md index aadf03c..67cc184 100644 --- a/example-crs-webservice/README.md +++ b/example-crs-webservice/README.md @@ -20,7 +20,7 @@ In other words, competitors are expected to develop a web server for receiving t | Competition | <----> | CRS API Web Server | ``` -The [example-crs-architecture](https://github.com/aixcc-finals/example-crs-architecture/tree/main/docs/api) repository provides the swagger specification files which may be used to autogenerate code for both the http server stub as well as the http client for sending submissions to the scoreboard. +The [example-crs-architecture](https://github.com/AIxCyberChallenge/example-crs-architecture/tree/main/docs/api) repository provides the swagger specification files which may be used to autogenerate code for both the http server stub as well as the http client for sending submissions to the scoreboard. - `competition-swagger-*.json` - description of the competition HTTP API - `crs-swagger-*.json` - description of the CRS HTTP API @@ -45,7 +45,7 @@ Instead of installing the dependencies on your local machine, you may also use t The `example-competition-api` provides a mock implementation of the competition API which will validate input formats and provide example outputs. -The complete instructions for running the `example-competition-server` are available under [example-competition-server/](https://github.com/aixcc-finals/example-crs-architecture/tree/main/example-competition-server). +The complete instructions for running the `example-competition-server` are available under [example-competition-server/](https://github.com/AIxCyberChallenge/example-crs-architecture/tree/main/example-competition-server). All endpoints use HTTP Basic authentication. Use `11111111-1111-1111-1111-111111111111` and `secret` as the credentials. @@ -78,7 +78,7 @@ $ uvicorn server:app --reload --port 1324 --log-config=../../log-conf.yaml --env The web server will be accessible at `http://localhost:1324` by default. -The [generate-challenge-task](https://github.com/aixcc-finals/generate-challenge-task) script may be used to construct an HTTP request that the task server expects. +The [generate-challenge-task](https://github.com/AIxCyberChallenge/example-crs-architecture/tree/main/generate-challenge-task) script may be used to construct an HTTP request that the task server expects. Alternatively, the `example-competition-server` may be used to generate and send tasks to the task server. @@ -105,7 +105,7 @@ Detailed instructions for autogeneration of the code as well as manually added c **Competition API Client** -The client code for interacting with the competition scoreboard was autogenerated using `openapi-generator` following the provided instructions in the example-crs-architecture [Readme](https://github.com/aixcc-finals/example-crs-architecture/blob/main/docs/api/README.md#openapi-generator). +The client code for interacting with the competition scoreboard was autogenerated using `openapi-generator` following the provided instructions in the example-crs-architecture [Readme](https://github.com/AIxCyberChallenge/example-crs-architecture/blob/main/docs/api/README.md#openapi-generator). From the `example-crs-architecture`, repository run the following command using the `competition-swagger-*.json` specification file to autogenerate the Competition API client code: @@ -156,7 +156,7 @@ The `Enum` classes must also contain `str` and all `UUID` types can be changed t CRS HTTP endpoints are required to have Basic Authentication. The autogenerated server code has been modified to include an initial implementation of this based on the fastapi [documentation](https://fastapi.tiangolo.com/advanced/security/http-basic-auth/). -To ensure the task server implementation is correct, the [generate-challenge-task](https://github.com/aixcc-finals/generate-challenge-task) script may be used to construct an HTTP request that a task server expects. +To ensure the task server implementation is correct, the [generate-challenge-task](https://github.com/AIxCyberChallenge/example-crs-architecture/tree/main/generate-challenge-task) script may be used to construct an HTTP request that a task server expects. #### Troubleshooting diff --git a/generate-challenge-task/.gitignore b/generate-challenge-task/.gitignore new file mode 100644 index 0000000..37053ea --- /dev/null +++ b/generate-challenge-task/.gitignore @@ -0,0 +1,11 @@ +# Devenv +.devenv* +devenv.local.nix + +# direnv +.direnv + +# pre-commit +.pre-commit-config.yaml + +.env diff --git a/generate-challenge-task/README.md b/generate-challenge-task/README.md new file mode 100644 index 0000000..9fc480f --- /dev/null +++ b/generate-challenge-task/README.md @@ -0,0 +1,113 @@ +# generate-challenge-task + +This bash script will convert an OSS-Fuzz-compatible repository into the Challenge Task format. + +It does not modify the source repository. In particular, it does not add challenges. We recommend using an exemplar repository if you want need your CRS to have a guaranteed vuln to find. + +## Getting Started + +### Step 1: Dependencies + +Ensure the following are all installed to your working environment: + +- [azure-cli](https://learn.microsoft.com/en-us/cli/azure/) +- [jq](https://jqlang.github.io/jq/) +- Git +- tar +- curl + +### Step 2: Git setup + +Make sure that before you run the script, you have access to clone the target repository and the oss-fuzz tooling. + +### Step 3: Azure Setup + +- Set up an Azure Storage Account. +- Set up a Blob Container within above Storage Account. +- Get your account key and connection string. These are both accessible in the Storage Account menu by navigating to your Storage Account -> Security + Networking in the sidebar -> Access keys. + +### Step 4: Setting Environment Variables + +The `example.env` file demonstrates all environment variables that need to be populated. +They are the secrets used to access your CRS and the Azure Storage Container. +Set the environment variables in your shell or copy `example.env` to `.env` in your working directory. This file must be in the working directory to be detected. + +Set the following variables in `.env`: + +- CONTAINER_NAME + +Name of the Azure Blob Storage Container that you created. + +- STORAGE_ACCOUNT + +Name of the Azure Storage Account that you created. + +- STORAGE_KEY and CONNECTION_STRING + +Use previously-mentioned values from step 3. Select and copy the key field and the connection string field to these variables from one of the keys in the Azure portal UI. + +- CRS_API_KEY_ID and CRS_API_TOKEN + +HTTP Basic credentials to access your CRS's API. + +## Running the Script + +### Usage + +```bash +Usage: ./generate-challenge-task.sh [options] + -v Enable verbose output mode + -c URL to the CRS to send challenge task. + -x Execute the crs task challenge against the CRS URL in -c on script finish. If argument is not supplied, the curl will be written to a local file. + -t Target github repo to scan. Format: path to the remote that will work for git clone + -r Git ref to generate diff for delta mode. + -b Git ref to act as base for the diff. Default value is main. Often useful on a repo where the central branch is called master + -o OSS-Fuzz repo to generate oss-fuzz tooling package. Default https://github.com/google/oss-fuzz.git + -p Project folder name inside OSS-Fuzz corresponding to this repo (for example, for https://github.com/apache/commons-compress, this is apache-commons-compress) + -l Instead of uploading to Azure storage, leave tars on local filesystem. Placeholders for sas_urls will be inserted into curl + -H Set harnesses_included in task JSON to false. If not set, harnesses_included defaults to true +``` + +### Usage Examples + +`./generate-challenge-task.sh -t -p -c ` + +Above will run the script with limited output (no -v), producing a full scan Challenge Task, +and will write the curl command to the local file `task_crs.sh` since no `-x` was provided. +Note: The format of the repository should be one that works in `git clone ` e.g. `https://github.com//.git` + +`./generate-challenge-task.sh -t -p -c -x` + +Above will run the script as previous example, but it will attempt to execute the curl for the Challenge Task. +against the CRS URL provided instead of writing it to task_crs.sh + +`./generate-challenge-task.sh -t -p -c -r mybranch|tag|commit_hash` + +Above will run the script in delta scan mode, which generates a diff between main and the ref +(commit hash, tag or branch name) provided in the -r argument. The curl will be written to task_crs.sh and will contain 3 sources instead of 2 because of the included PR diff tarball. + +`./generate-challenge-task.sh -t -p -c -r mybranch|tag|commit_hash -v -b mybranch|tag|commit_hash` + +Above will do the same thing as prior, but with verbose output, and the base branch to diff against will be an arbitrary ref instead of main, due to the included -b flag. + +`./generate-challenge-task.sh -t -p -c -r mybranch|tag|commit_hash -v -o ` + +Above will additionally tar and upload and reference in resultant curl a custom oss-fuzz tooling repository. + +`./generate-challenge-task.sh -t -p -c -r mybranch|tag|commit_hash -l -o ` + +Above will in lieu of uploading the tars to Azure Storage, will store them locally in a directory called `repo-tars` next to the script. The curl that results will not be usable since the tar paths in the curl will be +replaced with placeholders. + +`./generate-challenge-task.sh -c https://my.crs.crsdomain.com -t https://github.com/isc-projects/bind9.git -p bind9 -r bind-9.20 -v -b bind-9.18` + +This is an example script invocation with real-life values. +Note: bind9 was chosen as an arbitrary example and should not be interpreted to signal anything about the competition. + +### Notes on types of acceptable refs + +The following refs have been successfully utilized to create diffs: + +- Full commit hashes +- Tags +- Branch names diff --git a/generate-challenge-task/devenv.lock b/generate-challenge-task/devenv.lock new file mode 100644 index 0000000..bc5abd2 --- /dev/null +++ b/generate-challenge-task/devenv.lock @@ -0,0 +1,116 @@ +{ + "nodes": { + "devenv": { + "locked": { + "dir": "src/modules", + "lastModified": 1733477728, + "owner": "cachix", + "repo": "devenv", + "rev": "db06c0931e187619146cd8e22c3428ae1c3807e1", + "type": "github" + }, + "original": { + "dir": "src/modules", + "owner": "cachix", + "repo": "devenv", + "type": "github" + } + }, + "flake-compat": { + "flake": false, + "locked": { + "lastModified": 1733328505, + "owner": "edolstra", + "repo": "flake-compat", + "rev": "ff81ac966bb2cae68946d5ed5fc4994f96d0ffec", + "type": "github" + }, + "original": { + "owner": "edolstra", + "repo": "flake-compat", + "type": "github" + } + }, + "gitignore": { + "inputs": { + "nixpkgs": [ + "pre-commit-hooks", + "nixpkgs" + ] + }, + "locked": { + "lastModified": 1709087332, + "owner": "hercules-ci", + "repo": "gitignore.nix", + "rev": "637db329424fd7e46cf4185293b9cc8c88c95394", + "type": "github" + }, + "original": { + "owner": "hercules-ci", + "repo": "gitignore.nix", + "type": "github" + } + }, + "nixpkgs": { + "locked": { + "lastModified": 1733477122, + "owner": "cachix", + "repo": "devenv-nixpkgs", + "rev": "7bd9e84d0452f6d2e63b6e6da29fe73fac951857", + "type": "github" + }, + "original": { + "owner": "cachix", + "ref": "rolling", + "repo": "devenv-nixpkgs", + "type": "github" + } + }, + "nixpkgs-stable": { + "locked": { + "lastModified": 1733220138, + "owner": "NixOS", + "repo": "nixpkgs", + "rev": "bcb68885668cccec12276bbb379f8f2557aa06ce", + "type": "github" + }, + "original": { + "owner": "NixOS", + "ref": "nixos-24.05", + "repo": "nixpkgs", + "type": "github" + } + }, + "pre-commit-hooks": { + "inputs": { + "flake-compat": "flake-compat", + "gitignore": "gitignore", + "nixpkgs": [ + "nixpkgs" + ], + "nixpkgs-stable": "nixpkgs-stable" + }, + "locked": { + "lastModified": 1733318908, + "owner": "cachix", + "repo": "pre-commit-hooks.nix", + "rev": "6f4e2a2112050951a314d2733a994fbab94864c6", + "type": "github" + }, + "original": { + "owner": "cachix", + "repo": "pre-commit-hooks.nix", + "type": "github" + } + }, + "root": { + "inputs": { + "devenv": "devenv", + "nixpkgs": "nixpkgs", + "pre-commit-hooks": "pre-commit-hooks" + } + } + }, + "root": "root", + "version": 7 +} diff --git a/generate-challenge-task/devenv.nix b/generate-challenge-task/devenv.nix new file mode 100644 index 0000000..ff90bec --- /dev/null +++ b/generate-challenge-task/devenv.nix @@ -0,0 +1,51 @@ +{ pkgs, lib, config, inputs, ... }: + +{ + # https://devenv.sh/basics/ + env.GREET = "devenv"; + + # https://devenv.sh/packages/ + packages = [ + pkgs.git + pkgs.azure-cli + pkgs.jq + pkgs.gnutar + pkgs.curl + ]; + + # https://devenv.sh/languages/ + # languages.rust.enable = true; + + # https://devenv.sh/processes/ + # processes.cargo-watch.exec = "cargo-watch"; + + # https://devenv.sh/services/ + # services.postgres.enable = true; + + # https://devenv.sh/scripts/ + scripts.hello.exec = '' + echo hello from $GREET + ''; + + enterShell = '' + hello + git --version + ''; + + # https://devenv.sh/tasks/ + # tasks = { + # "myproj:setup".exec = "mytool build"; + # "devenv:enterShell".after = [ "myproj:setup" ]; + # }; + + # https://devenv.sh/tests/ + enterTest = '' + echo "Running tests" + git --version | grep --color=auto "${pkgs.git.version}" + ''; + + # https://devenv.sh/pre-commit-hooks/ + # pre-commit.hooks.shellcheck.enable = true; + + # See full reference at https://devenv.sh/reference/options/ +} diff --git a/generate-challenge-task/devenv.yaml b/generate-challenge-task/devenv.yaml new file mode 100644 index 0000000..fefdd49 --- /dev/null +++ b/generate-challenge-task/devenv.yaml @@ -0,0 +1,14 @@ +# yaml-language-server: $schema=https://devenv.sh/devenv.schema.json +inputs: + nixpkgs: + url: github:cachix/devenv-nixpkgs/rolling +# If you're using non-OSS software, you can set allowUnfree to true. +# allowUnfree: true + +# If you're willing to use a package that's vulnerable +# permittedInsecurePackages: +# - "openssl-1.1.1w" + +# If you have more than one devenv you can merge them +#imports: +# - ./backend diff --git a/generate-challenge-task/example.env b/generate-challenge-task/example.env new file mode 100644 index 0000000..ae8a2d5 --- /dev/null +++ b/generate-challenge-task/example.env @@ -0,0 +1,10 @@ +# Set the following variables to required values +# Do not commit this file to github + +# Make certain to single-quote the CONNECTION_STRING value because of the possibility of special characters. +CONNECTION_STRING='hello there' +CONTAINER_NAME=testname +CRS_API_KEY_ID=username +CRS_API_TOKEN=password +STORAGE_ACCOUNT=testacct +STORAGE_KEY=fake diff --git a/generate-challenge-task/generate-challenge-task.sh b/generate-challenge-task/generate-challenge-task.sh new file mode 100755 index 0000000..eae8a75 --- /dev/null +++ b/generate-challenge-task/generate-challenge-task.sh @@ -0,0 +1,421 @@ +#!/bin/bash +# +# Generate a crs/task command by tarring up required repos and issuing crs/task spec curl + +# default configs + +set -e -o pipefail + +SCRIPT_HOME="$PWD" +ROOT_WORK_DIR="$(mktemp -d)" +TARS_DIR="$ROOT_WORK_DIR/repo-tars" +REPO_WORK_DIR="$ROOT_WORK_DIR/work" +OSS_FUZZ_REPO="https://github.com/google/oss-fuzz.git" +VERBOSE=0 +LOCAL_TARS=0 +HARNESSES_INCLUDED="true" +BASE_REF="main" + +usage() { + echo "Usage: $0 [options]" + echo " -v Enable verbose output mode" + echo " -c URL to the CRS to send challenge task." + echo " -x Execute the crs task challenge against the CRS URL in -c on script finish. If argument is not supplied, the curl will be written to a local file." + echo " -t Target github repo to scan. Format: path to the remote that will work for git clone" + echo " -r Git ref to generate diff for delta mode." + echo " -b Git ref to act as base for the diff. Default value is main. Often useful on a repo where the central branch is called master" + echo " -o OSS-Fuzz repo to generate OSS-Fuzz tooling package. Default https://github.com/google/oss-fuzz.git" + echo " -p Project folder name inside OSS-Fuzz corresponding to this repo (for example, for https://github.com/apache/commons-compress, this is apache-commons-compress)" + echo " -l Instead of uploading to Azure storage, leave tars on local filesystem. Placeholders for sas_urls will be inserted into curl" + echo " -H Set harnesses_included in task JSON to false. If not set, harnesses_included defaults to true" + exit 1 +} + +parseargs() { + while getopts ":hvxlHc:t:r:b:o:p:" opt; do + case $opt in + v) VERBOSE=1 ;; + h) usage ;; + x) EXECUTE_CURL=1 ;; + l) LOCAL_TARS=1 ;; + H) HARNESSES_INCLUDED="false" ;; + c) CRS_URL=$OPTARG ;; + t) + TARGET_REPO=$OPTARG + REPO_FOLDER="$(echo "$TARGET_REPO" | sed 's|[^/]*/||g' | sed 's|.git||g')" + ;; + r) REF=$OPTARG ;; + b) BASE_REF=$OPTARG ;; + o) OSS_FUZZ_REPO=$OPTARG ;; + p) OSS_FUZZ_PROJECT_NAME=$OPTARG ;; + \?) + echo "Invalid option: -$OPTARG" + usage + ;; + :) + echo "Option -$OPTARG requires an argument" + usage + ;; + esac + done + if [ -z "$CRS_URL" ]; then + echo "Ensure -c (CRS URL) is set" + usage + fi + if [ -z "$TARGET_REPO" ]; then + echo "Ensure -t (target repo) is set" + usage + fi + if [ -z "$OSS_FUZZ_PROJECT_NAME" ]; then + echo "Ensure -p (OSS-Fuzz project name for this repo) is set" + usage + fi +} + +getenvvars() { + if [ -f ./.env ]; then + # shellcheck disable=SC1091 + source ./.env + fi + if [ $VERBOSE == 1 ]; then echo "Check for required environment variables"; fi + if [ "${LOCAL_TARS:-0}" != 1 ]; then + for envvar in CONTAINER_NAME STORAGE_ACCOUNT STORAGE_KEY CONNECTION_STRING; do + if [ -z "${!envvar}" ]; then + echo "Ensure $envvar variable is set." + exit 2 + fi + done + fi + for envvar in CRS_API_KEY_ID CRS_API_TOKEN; do + if [ -z "${!envvar}" ]; then + echo "Ensure $envvar variable is set." + exit 2 + fi + done +} + +checkdeps() { + local required_version installed_version + required_version="2.58.0" + if [ $VERBOSE == 1 ]; then echo "Checking dependencies..."; fi + if ! (which az >/dev/null 2>&1); then + echo "azure-cli not found and is a required dependency. Install azcli >= $required_version" + exit 3 + fi + installed_version=$(az version -o yaml | grep "azure-cli:" | sed 's/azure-cli: //') + if [ "$(printf '%s\n' "$required_version" "$installed_version" | sort -V | head -n 1)" != $required_version ]; then + echo "azcli version found: $installed_version is less than required: $required_version. Please ugrade." + exit 3 + fi + if ! (which git >/dev/null 2>&1); then + echo "git required, please install git." + exit 3 + fi + if ! (which jq >/dev/null 2>&1); then + echo "jq required, please install jq." + exit 3 + fi + if ! (which tar >/dev/null 2>&1); then + echo "tar required, please install tar." + exit 3 + fi + if ! (which curl >/dev/null 2>&1); then + echo "curl required, please install curl." + exit 3 + fi +} + +repotar() { + local repo=$1 + if [ -n "$2" ]; then + local diff_ref=$2 + fi + local dirname + dirname=$(echo "$repo" | xargs -I {} basename {} .git) + if [ $VERBOSE == 1 ]; then echo "Check out $repo"; fi + mkdir -p "${REPO_WORK_DIR}" + cd "${REPO_WORK_DIR}" || exit 5 + if ! (git clone "$repo") | tail -n +1 >/dev/null 2>&1; then + echo "git clone of $repo failed. Verify you have access/permissions set up to clone this repo" + exit 4 + fi + cd "$dirname" || exit 5 + # get the pr diff + if [ -n "$diff_ref" ]; then + if [ $VERBOSE == 1 ]; then echo "Diff detected: Tarring diff: $BASE_REF vs $diff_ref alongside repo"; fi + local base_dir="$ROOT_WORK_DIR/base-checkout" + local diff_dir="$ROOT_WORK_DIR/diff-checkout" + mkdir -p "$base_dir" + mkdir -p "$diff_dir" + cp -R "${REPO_WORK_DIR}/$dirname" "$base_dir" + cp -R "${REPO_WORK_DIR}/$dirname" "$diff_dir" + cd "$base_dir/$dirname" + git checkout "$BASE_REF" + rm -rf .git + rm -rf .github + rm -rf .aixcc + cd "$diff_dir/$dirname" + git checkout "$diff_ref" + rm -rf .git + rm -rf .github + rm -rf .aixcc + cd "${ROOT_WORK_DIR}" + set +e #Need to unset this for the diff since diff returns 1 if a difference exists and breaks script + git diff --no-index "$base_dir" "$diff_dir" | + sed 's|a'"$base_dir/$dirname"'|a|g' | # remove base_dir and diff_dir from any a/b refs + sed 's|b'"$diff_dir/$dirname"'|b|g' | + sed 's|b'"$base_dir/$dirname"'|b|g' | + sed 's|a'"$diff_dir/$dirname"'|a|g' | + sed 's| '"$base_dir/$dirname"'/| |g' | # remove any remaining instances, without a or b, and remove the leading slash + sed 's| '"$diff_dir/$dirname"'/| |g' >ref.diff + set -e + mkdir diff && mv -- ref.diff diff + # shellcheck disable=SC2001 + tar czf "$TARS_DIR/diff-$(echo "$diff_ref" | sed 's|/|-|g').tar.gz" diff + fi + if [[ "$dirname" != *"oss-fuzz"* ]]; then + cd "${REPO_WORK_DIR}/$dirname" + git checkout "$BASE_REF" + fi + if [ $VERBOSE == 1 ]; then echo "Remove .git for $repo"; fi + rm -rf .git + if [ $VERBOSE == 1 ]; then echo "Remove .github for $repo"; fi + rm -rf .github + if [ $VERBOSE == 1 ]; then echo "Remove .aixcc for $repo"; fi + rm -rf .aixcc + if [ $VERBOSE == 1 ]; then echo "Tarring $repo"; fi + if [[ "$dirname" == *"oss-fuzz"* ]]; then + cd .. + mv "$dirname" fuzz-tooling + mkdir fuzz-tooling-wrapper + mv fuzz-tooling fuzz-tooling-wrapper + cd fuzz-tooling-wrapper + tar czf "$TARS_DIR/$dirname.tar.gz" fuzz-tooling + else + cd .. + mkdir "$dirname-wrapper" + mv "$dirname" "$dirname-wrapper" + cd "$dirname-wrapper" + tar czf "$TARS_DIR/$dirname.tar.gz" "$dirname" + fi +} + +uploadtar() { + local tar=$1 + if [ $VERBOSE == 1 ]; then echo "Upload tar $tar"; fi + + #check tar type for pathing in Az storage + if [[ "$tar" == *"oss-fuzz"* ]]; then + OSS_FUZZ_SHA256="$(sha256sum "$tar" | cut -d' ' -f1)" + OSS_FUZZ_TAR_NAME="$OSS_FUZZ_SHA256.tar.gz" + if [ $VERBOSE == 1 ]; then echo "Uploading oss-fuzz tooling tar as $OSS_FUZZ_TAR_NAME"; fi + az storage blob upload \ + --container-name "$CONTAINER_NAME" \ + --account-name "$STORAGE_ACCOUNT" \ + --file "$tar" \ + --name "$OSS_FUZZ_TAR_NAME" \ + --sas-token "$STORAGE_KEY" \ + --overwrite + if [ $VERBOSE == 1 ]; then echo "Get SAS URL for uploaded oss-fuzz tar $OSS_FUZZ_TAR_NAME"; fi + OSS_FUZZ_SAS_URL=$( + az storage blob generate-sas \ + --account-name "$STORAGE_ACCOUNT" \ + --container-name "$CONTAINER_NAME" \ + --name "$OSS_FUZZ_TAR_NAME" \ + --permissions r \ + --expiry "$(date -u -d "4 hours" +"%Y-%m-%dT%H:%M:%SZ")" \ + --output tsv \ + --connection-string "$CONNECTION_STRING" \ + --full-uri + ) + elif [[ "$tar" == *"diff"*"tar.gz"* ]]; then + DIFF_SHA256="$(sha256sum "$tar" | cut -d' ' -f1)" + DIFF_TAR_NAME="$DIFF_SHA256.tar.gz" + if [ $VERBOSE == 1 ]; then echo "Uploading diff tar as $DIFF_TAR_NAME"; fi + az storage blob upload \ + --container-name "$CONTAINER_NAME" \ + --account-name "$STORAGE_ACCOUNT" \ + --file "$tar" \ + --name "$DIFF_TAR_NAME" \ + --sas-token "$STORAGE_KEY" \ + --overwrite + if [ $VERBOSE == 1 ]; then echo "Get SAS URL for uploaded diff tar $DIFF_TAR_NAME"; fi + DIFF_SAS_URL=$( + az storage blob generate-sas \ + --account-name "$STORAGE_ACCOUNT" \ + --container-name "$CONTAINER_NAME" \ + --name "$DIFF_TAR_NAME" \ + --permissions r \ + --expiry "$(date -u -d "4 hours" +"%Y-%m-%dT%H:%M:%SZ")" \ + --output tsv \ + --connection-string "$CONNECTION_STRING" \ + --full-uri + ) + else + REPO_SHA256="$(sha256sum "$tar" | cut -d' ' -f1)" + REPO_TAR_NAME="$REPO_SHA256.tar.gz" + if [ $VERBOSE == 1 ]; then echo "Uploading target repo tar as $REPO_TAR_NAME"; fi + az storage blob upload \ + --container-name "$CONTAINER_NAME" \ + --account-name "$STORAGE_ACCOUNT" \ + --file "$tar" \ + --name "$REPO_TAR_NAME" \ + --sas-token "$STORAGE_KEY" \ + --overwrite + if [ $VERBOSE == 1 ]; then echo "Get SAS URL for uploaded target repo tar $REPO_TAR_NAME"; fi + REPO_SAS_URL=$( + az storage blob generate-sas \ + --account-name "$STORAGE_ACCOUNT" \ + --container-name "$CONTAINER_NAME" \ + --name "$REPO_TAR_NAME" \ + --permissions r \ + --expiry "$(date -u -d "4 hours" +"%Y-%m-%dT%H:%M:%SZ")" \ + --output tsv \ + --connection-string "$CONNECTION_STRING" \ + --full-uri + ) + fi +} + +generatecurl() { + local focus_repo=$1 + local project_name=$2 + local target_sas_url=$3 + local oss_fuzz_sas_url=$4 + local msgid + msgid="$(uuidgen)" + local taskid + taskid="$(uuidgen)" + local currtime="$(($(date +%s) * 1000))" + # shellcheck disable=SC2004 + local duetime="$(($currtime + 14400000))" + local harnesses_included=$HARNESSES_INCLUDED + if [ -n "${5+x}" ]; then + local diff_sas_url=$5 + local payload="{ \ + \"message_id\": \"$msgid\",\ + \"message_time\": $currtime,\ + \"tasks\": [{\ + \"task_id\": \"$taskid\",\ + \"type\": \"delta\",\ + \"metadata\": {\ + \"round.id\": \"local-dev\",\ + \"task.id\": \"$taskid\"\ + },\ + \"deadline\": $duetime,\ + \"focus\": \"$focus_repo\",\ + \"harnesses_included\": $harnesses_included,\ + \"project_name\": \"$project_name\",\ + \"source\": [{\ + \"type\": \"repo\",\ + \"url\": \"$target_sas_url\",\ + \"sha256\": \"$REPO_SHA256\" \ + },\ + {\ + \"type\": \"fuzz-tooling\",\ + \"url\": \"$oss_fuzz_sas_url\",\ + \"sha256\": \"$OSS_FUZZ_SHA256\"\ + }, + { + \"type\": \"diff\", + \"url\": \"$diff_sas_url\",\ + \"sha256\": \"$DIFF_SHA256\" + }\ + ]\ + }\ + ]\ + }" + else + local payload="{ \ + \"message_id\": \"$msgid\",\ + \"message_time\": $currtime,\ + \"tasks\": [{\ + \"task_id\": \"$taskid\",\ + \"type\": \"full\",\ + \"deadline\": $duetime,\ + \"focus\": \"$focus_repo\",\ + \"harnesses_included\": $harnesses_included,\ + \"metadata\": {\ + \"round.id\": \"local-dev\",\ + \"task.id\": \"$taskid\"\ + },\ + \"project_name\": \"$project_name\",\ + \"source\": [{\ + \"type\": \"repo\",\ + \"url\": \"$target_sas_url\",\ + \"sha256\": \"$REPO_SHA256\" \ + },\ + {\ + \"type\": \"fuzz-tooling\",\ + \"url\": \"$oss_fuzz_sas_url\",\ + \"sha256\": \"$OSS_FUZZ_SHA256\"\ + }]\ + }\ + ]\ + }" + fi + CURL_CMD="curl -s -X POST \"$CRS_URL/v1/task/\" -H \"Content-Type: application/json\" \ + --user \"$CRS_API_KEY_ID\":\"$CRS_API_TOKEN\" -d '$(echo "$payload" | jq -c)'" +} + +sendcurl() { + echo "Execute Curl Command at $CRS_URL" + eval "$1" +} + +cleanup() { + if [ $VERBOSE == 1 ]; then echo "Remove temp dir $ROOT_WORK_DIR"; fi + rm -rf "$ROOT_WORK_DIR" +} + +main() { + parseargs "$@" + getenvvars + checkdeps + + if [ "$VERBOSE" == 1 ]; then echo "Working dir: $ROOT_WORK_DIR"; fi + # Start doing stuff + mkdir -p "$TARS_DIR" + for repo in "$TARGET_REPO" "$OSS_FUZZ_REPO"; do + if [ -n "${REF+x}" ] && [ "$TARGET_REPO" == "$repo" ]; then + repotar "$repo" "$REF" + else + repotar "$repo" + fi + done + if [ "$LOCAL_TARS" != 1 ]; then + for tar in "$TARS_DIR"/*; do + if [ -f "$tar" ]; then + uploadtar "$tar" + fi + done + else + cp -R "$TARS_DIR" "$SCRIPT_HOME" + fi + if [ -n "${REF+x}" ] && [ "$LOCAL_TARS" != 1 ]; then + if [ $VERBOSE == 1 ]; then echo "Generating curl for a delta scan..."; fi + generatecurl "$REPO_FOLDER" "$OSS_FUZZ_PROJECT_NAME" "$REPO_SAS_URL" "$OSS_FUZZ_SAS_URL" "$DIFF_SAS_URL" + elif [ -n "${REF+x}" ] && [ "$LOCAL_TARS" == 1 ]; then + if [ $VERBOSE == 1 ]; then echo "Generating curl for a delta scan with placeholders due to -l..."; fi + generatecurl "$REPO_FOLDER" "$OSS_FUZZ_PROJECT_NAME" "repo-placeholder" "oss-fuzz-placeholder" "diff-placeholder" + elif [ "$LOCAL_TARS" == 1 ]; then + if [ $VERBOSE == 1 ]; then echo "Generating curl for a full scan with placeholders due to -l..."; fi + generatecurl "$REPO_FOLDER" "$OSS_FUZZ_PROJECT_NAME" "repo-placeholder" "oss-fuzz-placeholder" + else + if [ $VERBOSE == 1 ]; then echo "Generating curl for full scan, no delta/PR..."; fi + generatecurl "$REPO_FOLDER" "$OSS_FUZZ_PROJECT_NAME" "$REPO_SAS_URL" "$OSS_FUZZ_SAS_URL" + fi + if [ $VERBOSE == 1 ]; then echo "DEBUG: CURL_CMD: $CURL_CMD"; fi + if [ "$EXECUTE_CURL" == 1 ]; then + sendcurl "$CURL_CMD" + else + cd "$SCRIPT_HOME" + if [ $VERBOSE == 1 ]; then echo "Output curl command to task_crs.sh in $SCRIPT_HOME"; fi + echo "#!/bin/bash" >task_crs.sh + echo "$CURL_CMD" >>task_crs.sh + chmod +x task_crs.sh + fi + cleanup +} + +main "$@"