diff --git a/docs/geneva/index.mdx b/docs/geneva/index.mdx index 3e296b1..88d5659 100644 --- a/docs/geneva/index.mdx +++ b/docs/geneva/index.mdx @@ -37,6 +37,9 @@ as columns in a Lance dataset. Adding a feature is straightforward: Register the UDF as a virtual column using `Table.add_columns()`. + (Optional) Configure where the UDF will run: locally, on a Ray cluster, or on a Kubernetes cluster with KubeRay (see [Contexts](/geneva/jobs/contexts)). + + Trigger a `backfill` operation (see [Backfilling](/geneva/jobs/backfilling/)). @@ -52,4 +55,4 @@ Visit the following pages to learn more about featuring engineering in LanceDB E - **Overview**: [What is Feature Engineering?](/geneva/overview/) - **UDFs**: [Using UDFs](/geneva/udfs/udfs) · [Blob helpers](/geneva/udfs/blobs/) · [Error handling](/geneva/udfs/error_handling) · [Advanced configuration](/geneva/udfs/advanced-configuration) - **Jobs**: [Backfilling](/geneva/jobs/backfilling/) · [Startup optimizations](/geneva/jobs/startup/) · [Materialized views](/geneva/jobs/materialized-views/) · [Execution contexts](/geneva/jobs/contexts/) · [Geneva console](/geneva/jobs/console) · [Performance](/geneva/jobs/performance/) -- **Deployment**: [Deployment overview](/geneva/deployment/) · [Helm deployment](/geneva/deployment/helm/) · [Troubleshooting](/geneva/deployment/troubleshooting/) \ No newline at end of file +- **Deployment**: [Deployment overview](/geneva/deployment/) · [Helm deployment](/geneva/deployment/helm/) · [Troubleshooting](/geneva/deployment/troubleshooting/) diff --git a/docs/geneva/jobs/contexts.mdx b/docs/geneva/jobs/contexts.mdx index e9f8189..e680d3f 100644 --- a/docs/geneva/jobs/contexts.mdx +++ b/docs/geneva/jobs/contexts.mdx @@ -1,7 +1,7 @@ --- title: Execution Contexts sidebarTitle: Contexts -description: Learn how Geneva automatically packages and deploys your Python execution environment to worker nodes for distributed execution. +description: Learn how to set up your cluster for distributed execution. icon: circle-nodes --- @@ -9,9 +9,7 @@ icon: circle-nodes The APIs on this page require Geneva **v0.10.0** or later. -Geneva automatically packages and deploys your Python execution environment to its worker nodes. This ensures that distributed execution occurs in the same environment and dependencies as your prototype. - -We currently support one processing backend: **Ray**. There are 3 ways to connect to a Ray cluster: +Geneva currently supports one processing backend: **Ray**. There are 3 ways to connect to a Ray cluster: 1. Local Ray 2. KubeRay: create a cluster on demand in your Kubernetes cluster.