Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions docs/dev/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,13 +85,13 @@ NVCF supports four workload types on the compute plane:

Each component ships an `AGENTS.md` with detailed internals, data flows, and API contracts:

- [`src/compute-plane-services/nvca/AGENTS.md`](../../src/compute-plane-services/nvca/AGENTS.md)
- [`src/compute-plane-services/ess-agent/AGENTS.md`](../../src/compute-plane-services/ess-agent/AGENTS.md)
- [`src/compute-plane-services/byoo-otel-collector/AGENTS.md`](../../src/compute-plane-services/byoo-otel-collector/AGENTS.md)
- [`src/invocation-plane-services/http-invocation/AGENTS.md`](../../src/invocation-plane-services/http-invocation/AGENTS.md)
- [`src/invocation-plane-services/grpc-proxy/AGENTS.md`](../../src/invocation-plane-services/grpc-proxy/AGENTS.md)
- [`src/invocation-plane-services/llm-api-gateway/AGENTS.md`](../../src/invocation-plane-services/llm-api-gateway/AGENTS.md)
- [`src/invocation-plane-services/ratelimiter/AGENTS.md`](../../src/invocation-plane-services/ratelimiter/AGENTS.md)
- [`src/control-plane-services/nats-auth-callout/AGENTS.md`](../../src/control-plane-services/nats-auth-callout/AGENTS.md)
- [`src/control-plane-services/function-autoscaler/AGENTS.md`](../../src/control-plane-services/function-autoscaler/AGENTS.md)
- [`examples/AGENTS.md`](../../examples/AGENTS.md)
- [src/compute-plane-services/nvca/AGENTS.md](https://github.com/NVIDIA/nvcf/blob/main/src/compute-plane-services/nvca/AGENTS.md)
- [src/compute-plane-services/ess-agent/AGENTS.md](https://github.com/NVIDIA/nvcf/blob/main/src/compute-plane-services/ess-agent/AGENTS.md)
- [src/compute-plane-services/byoo-otel-collector/AGENTS.md](https://github.com/NVIDIA/nvcf/blob/main/src/compute-plane-services/byoo-otel-collector/AGENTS.md)
- [src/invocation-plane-services/http-invocation/AGENTS.md](https://github.com/NVIDIA/nvcf/blob/main/src/invocation-plane-services/http-invocation/AGENTS.md)
- [src/invocation-plane-services/grpc-proxy/AGENTS.md](https://github.com/NVIDIA/nvcf/blob/main/src/invocation-plane-services/grpc-proxy/AGENTS.md)
- [src/invocation-plane-services/llm-api-gateway/AGENTS.md](https://github.com/NVIDIA/nvcf/blob/main/src/invocation-plane-services/llm-api-gateway/AGENTS.md)
- [src/invocation-plane-services/ratelimiter/AGENTS.md](https://github.com/NVIDIA/nvcf/blob/main/src/invocation-plane-services/ratelimiter/AGENTS.md)
- [src/control-plane-services/nats-auth-callout/AGENTS.md](https://github.com/NVIDIA/nvcf/blob/main/src/control-plane-services/nats-auth-callout/AGENTS.md)
- [src/control-plane-services/function-autoscaler/AGENTS.md](https://github.com/NVIDIA/nvcf/blob/main/src/control-plane-services/function-autoscaler/AGENTS.md)
- [examples/AGENTS.md](https://github.com/NVIDIA/nvcf/blob/main/examples/AGENTS.md)
10 changes: 5 additions & 5 deletions docs/dev/grpc-load-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,13 @@ The `grpc_echo_sample` from the same repository shares the
</Note>

The source, build instructions, and registry push examples are in the
[nv-cloud-function-helpers](https://github.com/NVIDIA/nv-cloud-function-helpers/tree/main/examples/function_samples/load_tester_supreme)
[NVCF examples](https://github.com/NVIDIA/nvcf/tree/main/examples/function-samples/load-tester-supreme)
repository. Build and push the image to whichever container registry your
cluster has credentials for:

```bash
git clone https://github.com/NVIDIA/nv-cloud-function-helpers.git
cd nv-cloud-function-helpers/examples/function_samples/load_tester_supreme
git clone https://github.com/NVIDIA/nvcf.git
cd nvcf/examples/function-samples/load-tester-supreme

# Build
docker build --platform linux/amd64 -t load_tester_supreme .
Expand Down Expand Up @@ -115,8 +115,8 @@ The CLI saves the function and version IDs automatically. Run
## Clone the load test scripts

```bash
git clone https://github.com/NVIDIA/nv-cloud-function-helpers.git
cd nv-cloud-function-helpers/examples/load-tests
git clone https://github.com/NVIDIA/nvcf.git
cd nvcf/examples/load-tests
```

## Install k6
Expand Down
10 changes: 5 additions & 5 deletions docs/dev/http-load-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,13 +29,13 @@ for high-throughput benchmarking and includes:
- Built-in OpenTelemetry tracing

The source, build instructions, and registry push examples are in the
[nv-cloud-function-helpers](https://github.com/NVIDIA/nv-cloud-function-helpers/tree/main/examples/function_samples/load_tester_supreme)
[NVCF examples](https://github.com/NVIDIA/nvcf/tree/main/examples/function-samples/load-tester-supreme)
repository. Build and push the image to whichever container registry your
cluster has credentials for:

```bash
git clone https://github.com/NVIDIA/nv-cloud-function-helpers.git
cd nv-cloud-function-helpers/examples/function_samples/load_tester_supreme
git clone https://github.com/NVIDIA/nvcf.git
cd nvcf/examples/function-samples/load-tester-supreme

# Build
docker build --platform linux/amd64 -t load_tester_supreme .
Expand Down Expand Up @@ -100,8 +100,8 @@ k3d, Docker Desktop) it is typically `localhost` or `127.0.0.1`.
## Clone the load test scripts

```bash
git clone https://github.com/NVIDIA/nv-cloud-function-helpers.git
cd nv-cloud-function-helpers/examples/load-tests
git clone https://github.com/NVIDIA/nvcf.git
cd nvcf/examples/load-tests
```

## Install k6
Expand Down
8 changes: 4 additions & 4 deletions docs/dev/http-soak-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,13 @@ for high-throughput benchmarking and includes:
- Built-in OpenTelemetry tracing

The source, build instructions, and registry push examples are in the
[nv-cloud-function-helpers](https://github.com/NVIDIA/nv-cloud-function-helpers/tree/main/examples/function_samples/load_tester_supreme)
[NVCF examples](https://github.com/NVIDIA/nvcf/tree/main/examples/function-samples/load-tester-supreme)
repository. Build and push the image to whichever container registry your
cluster has credentials for:

```bash
git clone https://github.com/NVIDIA/nv-cloud-function-helpers.git
cd nv-cloud-function-helpers/examples/function_samples/load_tester_supreme
git clone https://github.com/NVIDIA/nvcf.git
cd nvcf/examples/function-samples/load-tester-supreme

# Build (multi-arch)
docker buildx build --platform linux/amd64,linux/arm64 -t load_tester_supreme .
Expand Down Expand Up @@ -162,7 +162,7 @@ you want a steady, predictable load.

<Note>
The latest version of this script is maintained in the
[nv-cloud-function-helpers](https://github.com/NVIDIA/nv-cloud-function-helpers)
[NVCF examples](https://github.com/NVIDIA/nvcf/tree/main/examples)
repository.

</Note>
Expand Down
2 changes: 1 addition & 1 deletion docs/ngc-managed/cluster-management/monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ The cluster agent and operator emit Prometheus-style metrics. The following metr
| nvca_cluster_group | The NVCA cluster group |
| nvca_version | The NVCA version |

Cluster maintainers can scrape the available metrics. See a full example of how to do this with an OpenTelemetry Collector in the cluster [here](https://github.com/NVIDIA/nv-cloud-function-helpers/tree/main/examples/cluster_monitoring_sample).
Cluster maintainers can scrape the available metrics. See a full example of how to do this with an OpenTelemetry Collector in the cluster [cluster monitoring example](https://github.com/NVIDIA/nvcf/tree/main/examples/cluster-monitoring-sample).

Use the following examples of a PodMonitor for NVCA Operator and ServiceMonitor for NVCA for reference:

Expand Down
2 changes: 1 addition & 1 deletion docs/user/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -344,7 +344,7 @@ Or use the `--debug` flag or `NVCF_DEBUG=true` environment variable per-command.
```

<Note>
For immediate testing, you can use `load_tester_supreme` from `nvcf-onprem` (see [self-hosted-artifact-manifest](./manifest.md)), which supports the `{"message": "hello world"}` request body above. For more function samples, see the [nv-cloud-function-helpers](https://github.com/NVIDIA/nv-cloud-function-helpers) repository and [function-creation](./function-creation.md) for function creation documentation.
For immediate testing, you can use `load_tester_supreme` from `nvcf-onprem` (see [self-hosted-artifact-manifest](./manifest.md)), which supports the `{"message": "hello world"}` request body above. For more function samples, see the [NVCF examples](https://github.com/NVIDIA/nvcf/tree/main/examples) repository and [function-creation](./function-creation.md) for function creation documentation.

</Note>

Expand Down
2 changes: 1 addition & 1 deletion docs/user/cluster-management/monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ The cluster agent and operator emit Prometheus-style metrics. The following metr
| nvca_cluster_group | The NVCA cluster group |
| nvca_version | The NVCA version |

Cluster maintainers can scrape the available metrics. See a full example of how to do this with an OpenTelemetry Collector in the cluster [here](https://github.com/NVIDIA/nv-cloud-function-helpers/tree/main/examples/cluster_monitoring_sample).
Cluster maintainers can scrape the available metrics. See a full example of how to do this with an OpenTelemetry Collector in the cluster [cluster monitoring example](https://github.com/NVIDIA/nvcf/tree/main/examples/cluster-monitoring-sample).

Use the following examples of a PodMonitor for NVCA Operator and ServiceMonitor for NVCA for reference:

Expand Down
10 changes: 5 additions & 5 deletions docs/user/container-functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ Container-based functions require building and pushing a Cloud Functions compati

## Resources

- Example containers can be found [in the examples repository](https://github.com/NVIDIA/nv-cloud-function-helpers/tree/main/examples).
- Example containers can be found [in the examples directory](https://github.com/NVIDIA/nvcf/tree/main/examples).

- The repository also contains [helper functions](https://github.com/NVIDIA/nv-cloud-function-helpers/blob/main/helper_library/nv_cloud_function_helpers/nvcf_container/helpers.py) that are useful when authoring your container, including:
- The repository also contains [helper functions](https://github.com/NVIDIA/nvcf/blob/main/src/libraries/python/nv-cloud-function-helpers/nv_cloud_function_helpers/nvcf_container/helpers.py) that are useful when authoring your container, including:

- Helpers that parse Cloud Functions-specific parameters on invocation
- Helpers that can be used to instrument your container with Cloud Functions compatible logs
Expand Down Expand Up @@ -43,7 +43,7 @@ Cloud Functions also expects the following directories in the container to remai

## Composing a FastAPI Container

It's possible to use any container with Cloud Functions as long as it implements a server with the above endpoints. The below is an example of a FastAPI-based container compatible with Cloud Functions. Clone the [FastAPI echo example](https://github.com/NVIDIA/nv-cloud-function-helpers/tree/main/examples/function_samples/fastapi_echo_sample).
It's possible to use any container with Cloud Functions as long as it implements a server with the above endpoints. The below is an example of a FastAPI-based container compatible with Cloud Functions. Clone the [FastAPI echo example](https://github.com/NVIDIA/nvcf/tree/main/examples/function-samples/fastapi-echo-sample).

### Create the "requirements.txt" File

Expand Down Expand Up @@ -289,7 +289,7 @@ Create the function via the NVCF API. In this example, we defined the inference

### Additional Examples

See more examples of containers that are Cloud Functions compatible [in the function samples directory](https://github.com/NVIDIA/nv-cloud-function-helpers/tree/main/examples/function_samples/).
See more examples of containers that are Cloud Functions compatible [in the function samples directory](https://github.com/NVIDIA/nvcf/tree/main/examples/function-samples/).

## Creating gRPC-based Functions

Expand All @@ -300,7 +300,7 @@ Cloud Functions supports function invocation via gRPC. During function creation,
- The function container must implement a gRPC port, endpoint and health check. The health check is expected to be served by the gRPC inference port, there is no need to define a separate health endpoint path.

- See [gRPC health checking](https://grpc.io/docs/guides/health-checking/).
- See an [example container](https://github.com/NVIDIA/nv-cloud-function-helpers/blob/main/examples/function_samples/grpc_echo_sample/grpc_echo_server.py) with a gRPC server that is Cloud Functions compatible.
- See an [example container](https://github.com/NVIDIA/nvcf/blob/main/examples/function-samples/grpc-echo-sample/grpc_echo_server.py) with a gRPC server that is Cloud Functions compatible.

### gRPC Function Creation via API

Expand Down
2 changes: 1 addition & 1 deletion docs/user/function-creation.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ For LLM functions, see [LLM Gateway](./llm-gateway.md#function-configuration) fo

The following is a reference of available variables via the headers of the invocation message (auto-populated by Cloud Functions), accessible within the container.

For examples of how to extract and use some of these variables, see [NVCF Container Helper Functions](https://github.com/NVIDIA/nv-cloud-function-helpers/tree/main).
For examples of how to extract and use some of these variables, see [NVCF Container Helper Functions](https://github.com/NVIDIA/nvcf/tree/main/src/libraries/python/nv-cloud-function-helpers).

| Name | Description |
| ---------------------------- | ------------------------------------------------------- |
Expand Down
4 changes: 2 additions & 2 deletions docs/user/helm-functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ All Pod specs in your helm chart will be updated with pull secrets at runtime, s

**Example Creation via API**

Please see our [sample helm chart used](https://github.com/NVIDIA/nv-cloud-function-helpers/tree/main/examples/function_samples/helmchart_samples/inference_test_sample) in this example for reference.
Please see our [sample helm chart used](https://github.com/NVIDIA/nvcf/tree/main/examples/function-samples/helmchart-samples/inference-test-sample) in this example for reference.

Below is an example function creation API call creating a helm-based function:

Expand Down Expand Up @@ -72,7 +72,7 @@ For gRPC-based functions, set `"inferenceURL" : "/gRPC"`. This signals to Cloud
To create a multi-node helm deployment, you need to use the following format for the `instanceType`:
`<CSP>.GPU.<GPU_NAME>_<number of gpus per node>x[.x<number of nodes>]`. For example, `DGXC.GPU.L40S_1x` is a single L40S instance while `ON-PREM.GPU.B200_8x.x2` is two full nodes of 8-way B200.

A sample helm chart for a multi-node deployment can be found [in the multi-node helm example](https://github.com/NVIDIA/nv-cloud-function-helpers/tree/main/examples/function_samples/helmchart_samples/multi_node_helm_function_test/).
A sample helm chart for a multi-node deployment can be found [in the multi-node helm example](https://github.com/NVIDIA/nvcf/tree/main/examples/function-samples/helmchart-samples/multi-node-helm-function-test/).

## Limitations

Expand Down
6 changes: 3 additions & 3 deletions docs/user/helmfile-installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@ Environment configuration files define how NVCF is deployed in your specific env
Set `HELMFILE_ENV` to your environment name and copy the base configuration.
The filename must match `HELMFILE_ENV` because Helmfile uses it to select the
environment file. The template below shows the values to configure for Amazon
EKS ([cp-env-eks-example.yaml](samples/configs/cp-env-eks-example.yaml)).
EKS ([cp-env-eks-example.yaml](https://raw.githubusercontent.com/NVIDIA/nvcf/main/docs/user/samples/configs/cp-env-eks-example.yaml)).

```bash
cd path/to/nvcf-self-managed-stack
Expand Down Expand Up @@ -586,7 +586,7 @@ These credentials will then be used for function deployments. Note that if the r
Copy the secrets template using the same `HELMFILE_ENV` value from Step 2. The
filename must match `HELMFILE_ENV` because Helmfile loads the corresponding
secrets file. The example below shows the required structure
([example-secrets.yaml](samples/configs/cp-example-secrets.yaml)). You must
([example-secrets.yaml](https://raw.githubusercontent.com/NVIDIA/nvcf/main/docs/user/samples/configs/cp-example-secrets.yaml)). You must
replace all instances of `REPLACE_WITH_BASE64_DOCKER_CREDENTIAL` with your
actual base64-encoded registry credentials.

Expand Down Expand Up @@ -731,7 +731,7 @@ registry.
If your `image` registry is private and your cluster nodes do not have built-in credential helpers, you must create Kubernetes `docker-registry` secrets in each NVCF namespace and configure the helmfile to reference them.

1. Create the pull secret in each NVCF namespace
([create-nvcr-pull-secrets.sh](samples/scripts/create-nvcr-pull-secrets.sh)):
([create-nvcr-pull-secrets.sh](https://raw.githubusercontent.com/NVIDIA/nvcf/main/docs/user/samples/scripts/create-nvcr-pull-secrets.sh)):

```bash
export NGC_API_KEY="<your-ngc-api-key>"
Expand Down
2 changes: 1 addition & 1 deletion docs/user/observability.md
Original file line number Diff line number Diff line change
Expand Up @@ -295,7 +295,7 @@ Dashboards are provided in native Grafana JSON format for [file-provisioning](ht
Load dashboards into Grafana by placing them in `/etc/grafana/provisioning/dashboards/` on startup.

Published dashboards will be available in the
[nv-cloud-function-helpers](https://github.com/NVIDIA/nv-cloud-function-helpers) public GitHub repository.
[NVCF examples](https://github.com/NVIDIA/nvcf/tree/main/examples) public GitHub repository.

## Troubleshooting

Expand Down
2 changes: 1 addition & 1 deletion docs/user/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -1090,7 +1090,7 @@ echo "=============================================="

```
[force-cleanup-nvcf.sh](samples/scripts/force-cleanup-nvcf.sh)
[force-cleanup-nvcf.sh](https://raw.githubusercontent.com/NVIDIA/nvcf/main/docs/user/samples/scripts/force-cleanup-nvcf.sh)
**Usage:**
Expand Down
2 changes: 1 addition & 1 deletion docs/v0.5/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -274,7 +274,7 @@ Or use the `--debug` flag or `NVCF_DEBUG=true` environment variable per-command.
```

<Note>
For immediate testing, you can use `load_tester_supreme` from `nvcf-onprem` (see [self-hosted-artifact-manifest](./manifest.md)), which supports the `{"message": "hello world"}` request body above. For more function samples, see the [nv-cloud-function-helpers](https://github.com/NVIDIA/nv-cloud-function-helpers) repository and [function-creation](./function-creation.md) for function creation documentation.
For immediate testing, you can use `load_tester_supreme` from `nvcf-onprem` (see [self-hosted-artifact-manifest](./manifest.md)), which supports the `{"message": "hello world"}` request body above. For more function samples, see the [NVCF examples](https://github.com/NVIDIA/nvcf/tree/main/examples) repository and [function-creation](./function-creation.md) for function creation documentation.

</Note>

Expand Down
2 changes: 1 addition & 1 deletion docs/v0.5/cluster-management/monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ The cluster agent and operator emit Prometheus-style metrics. The following metr
| nvca_cluster_group | The NVCA cluster group |
| nvca_version | The NVCA version |

Cluster maintainers can scrape the available metrics. See a full example of how to do this with an OpenTelemetry Collector in the cluster [here](https://github.com/NVIDIA/nv-cloud-function-helpers/tree/main/examples/cluster_monitoring_sample).
Cluster maintainers can scrape the available metrics. See a full example of how to do this with an OpenTelemetry Collector in the cluster [cluster monitoring example](https://github.com/NVIDIA/nvcf/tree/main/examples/cluster-monitoring-sample).

Use the following examples of a PodMonitor for NVCA Operator and ServiceMonitor for NVCA for reference:

Expand Down
Loading
Loading