Improve spec and compliance testing

.github/workflows/deploy.yaml

@@ -5,23 +5,14 @@ on:
       - main
   workflow_dispatch:
 
-concurrency:
-  group: "pages"
-  cancel-in-progress: false
-
 jobs:
   deploy:
     name: Deploy
     runs-on: ubuntu-latest
     container:
       image: node:24-alpine
     permissions:
-      contents: read
-      pages: write
-      id-token: write
-    environment:
-      name: github-pages
-      url: ${{ steps.deployment.outputs.page_url }}
+      contents: write
     steps:
       - name: Checkout
         uses: actions/checkout@v4
@@ -32,12 +23,8 @@ jobs:
       - name: Build
         run: |
           yarn build
-      - name: Configure Pages
-        uses: actions/configure-pages@v5
-      - name: Upload artifact
-        uses: actions/upload-pages-artifact@v3
+      - name: Deploy to gh-pages
+        uses: peaceiris/actions-gh-pages@v4
         with:
-          path: 'dist'
-      - name: Deploy to GitHub Pages
-        id: deployment
-        uses: actions/deploy-pages@v4
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: ./dist

.gitignore

@@ -43,3 +43,5 @@ test/sdk-integration/go/bin/
 dist
 node_modules
 out
+tmp
+.tmp

README.md

@@ -64,30 +64,51 @@ Runs the `bin/create-version.js` script, which is used to create a new versioned
 
 **Compliance Testing**
 
-`yarn test:compliance https://api.example.com dist/latest/openapi.yaml`
+`yarn test:compliance`
 
-Executes the `bin/test-compliance.js` script to run compliance tests against the OpenAPI specification. This ensures that the specification is valid and conforms to the OpenAPI standard. It uses [`wiretap`](https://github.com/pb33f/wiretap) that acts like a proxy server to intercept requests and validate them against the OpenAPI spec.
+Runs a comprehensive compliance test suite against a live Cloud Foundry API. This script uses `wiretap` to proxy requests to the API, validating them against the OpenAPI specification in real-time. It clones the `capi-bara-tests` repository and executes its test suite against the proxied API.
+
+> :warning: Unfortunally wiretap seems to be quite instable it cannot properly handle multiform-data requests and sometimes just crashes with memory erros. Its currently as good as it is and helped a lot making the spec compliant.
+
+**Prerequisites:**
+
+- Go must be installed and available in your `PATH`.
+- The following environment variables must be set:
+  - `CF_API_URL`: The URL of the Cloud Foundry API.
+  - `CF_APPS_DOMAIN`: The application domain for your Cloud Foundry instance.
+  - `CF_ADMIN_USER`: The username for an admin user.
+  - `CF_ADMIN_PASSWORD`: The password for the admin user.
+
+**Optional:**
+
+- `THREADS`: The number of parallel test nodes to run (default: 6).
+
+The script will generate a `wiretap-report.json` file in the `out` directory, which contains a detailed report of the API interactions and any compliance issues found.
 
 ```mermaid
 flowchart TD
-      A[CF CLI] -->|HTTP Request| B[Wiretap http://localhost:9090]
-      B -->|HTTP Request| C[Real API Server]
+      A[CAPI BARA Tests] -->|HTTP Request| B[Wiretap Proxy]
+      B -->|HTTP Request| C[CF API Server]
       C -->|HTTP Response| B
       B -->|HTTP Response| A
-
-      A2[CATS Testsuite] -->|HTTP Request| B2[Wiretap http://localhost:9090]
-      B2 -->|HTTP Request| C2[Real API Server]
-      C2 -->|HTTP Response| B2
-      B2 -->|HTTP Response| A2
 ```
 
-Hereby `Wiretap` acts as a proxy server that intercepts HTTP requests and responses between the CF CLI or CATS testsuite and the real API server. It validates the requests and responses against the OpenAPI specification, ensuring compliance.
+**Contract Testing (Mock Server)**
+
+`yarn test:mockserver <spec-file>`
+
+Executes the `bin/test-mockserver.js` script to run contract tests against a running server (either a live API or a mock server). This script uses `wiretap` to validate that the server's responses conform to the OpenAPI specification.
 
-**Mock Server**
+**Example:**
+
+```bash
+yarn test:mockserver http://localhost:4010 dist/latest/openapi.yaml
+```
 
-`yarn test:mockserver`
+This is useful for:
 
-Runs the `bin/test-mockserver.js` script to start a mock server based on the OpenAPI specification. This is useful for testing API clients and integrations without needing a live CAPI environment.
+- Validating a mock server's implementation against the OpenAPI spec.
+- Quickly checking a live API for compliance without running the full `capi-bara-tests` suite.
 
 ## Contributing
 

apis/cf/latest/components/parameters/CreatedAts.yaml

@@ -4,5 +4,5 @@ required: false
 schema:
   type: string
 description: |
-  Timestamp to filter by. When filtering on equality, several comma-delimited timestamps may be passed.
+  Timestamp to filter by. When filtering on equality, several comma-delimited timestamps may be passed. Also supports filtering with [relational operators](#relational-operators).
 example: "2021-01-01T00:00:00Z"

apis/cf/latest/components/parameters/Guid.yaml

@@ -4,4 +4,4 @@ required: true
 schema:
   type: string
   format: uuid
-description: The GUID of the resource
+description: The unique identifier for the resource

apis/cf/latest/components/parameters/LabelSelector.yaml

@@ -1,6 +1,6 @@
 name: label_selector
 in: query
-description: Selectors allow users to filter and group API resources by the labels applied to them.
+description: A query string containing a list of [label selector](#labels-and-selectors) requirements
 required: false
 schema:
   type: string

apis/cf/latest/components/parameters/OrderBy.yaml

@@ -4,5 +4,5 @@ required: false
 schema:
   type: string
 description: |
-  Order results by a specific field. Prepend with - to sort descending.
+  Value to sort by. Defaults to ascending; prepend with `-` to sort descending.
 example: "created_at"

apis/cf/latest/components/parameters/UpdatedAts.yaml

@@ -4,5 +4,5 @@ required: false
 schema:
   type: string
 description: |
-  Timestamp to filter by. When filtering on equality, several comma-delimited timestamps may be passed.
+  Timestamp to filter by. When filtering on equality, several comma-delimited timestamps may be passed. Also supports filtering with [relational operators](#relational-operators).
 example: "2021-01-01T00:00:00Z"

apis/cf/latest/components/requestBodies/AppCreateRequestBody.yaml

@@ -15,11 +15,13 @@ content:
             type: string
         lifecycle:
           $ref: '../schemas/Lifecycle.yaml'
+          description: Provides the lifecycle object for the application
         relationships:
           type: object
           properties:
             space:
               $ref: '../schemas/RelationshipToOne.yaml'
+              description: A relationship to a space
           required:
             - space
         metadata:

apis/cf/latest/components/requestBodies/AppUpdateRequestBody.yaml

@@ -15,6 +15,7 @@ content:
             type: string
         lifecycle:
           $ref: '../schemas/Lifecycle.yaml'
+          description: Lifecycle to be used when updating the app; note `data` is a required field in lifecycle if lifecycle is updated. `type` may NOT be changed from its current value.
         metadata:
           $ref: '../schemas/Metadata.yaml'
       description: Request schema for updating an app

apis/cf/latest/components/requestBodies/BuildCreateRequestBody.yaml

@@ -16,6 +16,7 @@ content:
           description: App package to stage
         lifecycle:
           $ref: '../schemas/Lifecycle.yaml'
+          description: Lifecycle information for a build
         metadata:
           $ref: '../schemas/Metadata.yaml'
       required:

apis/cf/latest/components/requestBodies/BuildpackCreateRequestBody.yaml

@@ -10,25 +10,21 @@ content:
           description: The name of the buildpack
         stack:
           type: string
-          description: The stack the buildpack is compatible with
+          description: The name of the stack that the buildpack will use
         position:
           type: integer
-          description: The position of the buildpack in the order of execution
-        state:
+          description: The order in which the buildpacks are checked during buildpack auto-detection
+        lifecycle:
           type: string
-          description: The state of the buildpack
-          enum:
-            - AWAITING_UPLOAD
-            - READY
+          description: The version of buildpack the buildpack will use. `buildpack` indicates [Classic Buildpacks](https://docs.cloudfoundry.org/buildpacks/classic.html). `cnb` indicates [Cloud Native Buildpacks](https://docs.cloudfoundry.org/buildpacks/cnb/)
         enabled:
           type: boolean
-          description: Whether the buildpack is enabled
+          description: Whether or not the buildpack will be used for staging
         locked:
           type: boolean
-          description: Whether the buildpack is locked
-        filename:
-          type: string
-          description: The filename of the buildpack
+          description: Whether or not the buildpack is locked to prevent updating the bits
+        metadata:
+          $ref: '../schemas/Metadata.yaml'
       required:
         - name
       description: Request schema for creating a buildpack

apis/cf/latest/components/requestBodies/BuildpackUpdateRequestBody.yaml

@@ -10,16 +10,16 @@ content:
           description: The name of the buildpack
         position:
           type: integer
-          description: The position of the buildpack in the order of execution
+          description: The order in which the buildpacks are checked during buildpack auto-detection
         enabled:
           type: boolean
-          description: Whether the buildpack is enabled
+          description: Whether or not the buildpack will be used for staging
         locked:
           type: boolean
-          description: Whether the buildpack is locked
+          description: Whether or not the buildpack is locked to prevent updating the bits
         stack:
           type: string
-          description: The stack the buildpack is compatible with
+          description: The name of the stack that the buildpack will use
         metadata:
           $ref: '../schemas/Metadata.yaml'
       description: Request schema for updating a buildpack

apis/cf/latest/components/requestBodies/DomainCreateRequestBody.yaml

@@ -4,6 +4,8 @@ content:
   application/json:
     schema:
       type: object
+      required:
+        - name
       properties:
         name:
           type: string
@@ -17,16 +19,15 @@ content:
             guid:
               type: string
               format: uuid
-          description: The router group the domain will be associated with
+          description: "The desired router group guid.  _note: creates a `tcp` domain; cannot be used when `internal` is set to `true` or domain is scoped to an org_\""
         relationships:
           type: object
           properties:
             organization:
               $ref: '../schemas/RelationshipToOne.yaml'
+              description: A relationship to the organization the domain will be scoped to;  _note cannot be used when `internal` is set to `true` or domain is associated with a router group_
             shared_organizations:
               $ref: '../schemas/RelationshipToMany.yaml'
+              description: A relationship to organizations the domain will be shared with  _Note cannot be used without an organization relationship_
         metadata:
           $ref: '../schemas/Metadata.yaml'
-      required:
-        - name
-      description: Request schema for creating a domain

apis/cf/latest/components/requestBodies/DropletCreateRequestBody.yaml

@@ -6,7 +6,13 @@ content:
       type: object
       properties:
         relationships:
-          $ref: '../schemas/Relationships.yaml'
+          type: object
+          properties:
+            app:
+              $ref: '../schemas/RelationshipToOne.yaml'
+              description: App to create droplet for
+          required:
+            - app
         process_types:
           type: object
           description: Process names and start commands for the droplet

apis/cf/latest/components/requestBodies/IsolationSegmentCreate.yaml

@@ -6,7 +6,7 @@ content:
       properties:
         name:
           type: string
-          description: The name of the isolation segment
+          description: Name of the isolation segment; isolation segment names must be unique across the entire system, and case is ignored when checking for uniqueness
         metadata:
           $ref: '../schemas/Metadata.yaml'
       required:

apis/cf/latest/components/requestBodies/OrganizationQuotaCreateRequestBody.yaml

@@ -62,6 +62,7 @@ content:
           properties:
             organizations:
               $ref: '../schemas/RelationshipToMany.yaml'
+              description: A relationship to the organizations where the quota is applied
         metadata:
           $ref: '../schemas/Metadata.yaml'
       required:

apis/cf/latest/components/requestBodies/PackageCreateRequestBody.yaml

@@ -28,14 +28,8 @@ content:
           type: object
           properties:
             app:
-              type: object
-              properties:
-                data:
-                  type: object
-                  properties:
-                    guid:
-                      type: string
-                      format: uuid
+              $ref: '../schemas/RelationshipToOne.yaml'
+              description: A relationship to an app
           required:
             - app
         metadata:

apis/cf/latest/components/requestBodies/PackageUpdateRequestBody.yaml

@@ -7,4 +7,10 @@ content:
       properties:
         metadata:
           $ref: '../schemas/Metadata.yaml'
+        username:
+          type: string
+          description: The username for the image’s registry. Only possible for Docker package.
+        password:
+          type: string
+          description: The password for the image’s registry. Only possible for Docker package.
       description: Request schema for updating a package

apis/cf/latest/components/requestBodies/PackageUploadRequestBody.yaml

@@ -8,8 +8,7 @@ content:
         bits:
           type: string
           format: binary
-          description: A gzipped tar file that contains the package bits
         resources:
-          type: string
-          description: Fingerprints of the application bits that have previously been pushed to Cloud Foundry
-      description: Request schema for uploading package bits
+          type: array
+          items:
+            $ref: '../schemas/ResourceMatch.yaml'

apis/cf/latest/components/requestBodies/ProcessUpdateRequestBody.yaml

@@ -10,8 +10,10 @@ content:
           description: The command used to start the process; use null to revert to the buildpack-detected or procfile-provided start command
         health_check:
           $ref: '../schemas/HealthCheck.yaml'
+          description: The health check to perform on the process
         readiness_health_check:
           $ref: '../schemas/HealthCheck.yaml'
+          description: The readiness health check to perform on the process
         metadata:
           $ref: '../schemas/Metadata.yaml'
       description: Request schema for updating a process

apis/cf/latest/components/requestBodies/RoleCreate.yaml

@@ -6,7 +6,7 @@ content:
       properties:
         type:
           type: string
-          description: Role type
+          description: Role to create
           enum:
             - organization_user
             - organization_auditor
@@ -21,7 +21,10 @@ content:
           properties:
             user:
               $ref: '../schemas/RelationshipToOne.yaml'
+              description: A relationship to a user; the user can be defined by either a `guid` or, if the `set_roles_by_username` [feature_flag](#list-of-feature-flags) is enabled, a `username` (with the option of including an `origin` to disambiguate it)
             organization:
               $ref: '../schemas/RelationshipToOne.yaml'
+              description: A relationship to an organization; required only when creating an organization role
             space:
               $ref: '../schemas/RelationshipToOne.yaml'
+              description: A relationship to a space; required only when creating a space role

apis/cf/latest/components/requestBodies/RouteCreateRequestBody.yaml

@@ -6,16 +6,21 @@ content:
       properties:
         host:
           type: string
+          description: The host component for the route; not compatible with routes specifying the `tcp` protocol
         path:
           type: string
+          description: The path component for the route; should begin with a `/` and not compatible with routes specifying the `tcp` protocol
         port:
           type: integer
+          description: The port the route will listen on; only compatible with routes leveraging a domain that supports the `tcp` protocol. For `tcp` domains, a port will be randomly assigned if not specified
         relationships:
           type: object
           properties:
             domain:
               $ref: '../schemas/RelationshipToOne.yaml'
+              description: A relationship to the domain of the route
             space:
               $ref: '../schemas/RelationshipToOne.yaml'
+              description: A relationship to the space containing the route; routes can only be mapped to destinations in that space
       required:
         - relationships

apis/cf/latest/components/requestBodies/RouteDestinations.yaml

@@ -8,3 +8,4 @@ content:
           type: array
           items:
             $ref: '../schemas/RouteDestination.yaml'
+          description: List of destinations to add to route; destinations without `process.type` specified will get process type `"web"` by default