diff --git a/datasets/cli.mdx b/datasets/cli.mdx index 2392d92..2df3906 100644 --- a/datasets/cli.mdx +++ b/datasets/cli.mdx @@ -3,7 +3,23 @@ sidebarTitle: CLI title: Laminar datasets CLI --- -The `lmnr datasets` command is used to manage datasets in Laminar. For full CLI documentation visit [here](/platform/cli). +The `lmnr-cli dataset` command manages datasets in Laminar from your terminal: list them, create new ones from local files, and push or pull datapoints. It ships in the standalone [`lmnr-cli`](/platform/cli) npm package, so you do not need the `@lmnr-ai/lmnr` SDK installed to use it. + + +`lmnr-cli` replaces the dataset commands that used to ship with the `@lmnr-ai/lmnr` SDK (`lmnr datasets ...`). The old SDK-bundled CLI is deprecated; use `lmnr-cli dataset` instead. + + +## Install + +```bash +# Run directly with npx (no install) +npx lmnr-cli@latest dataset list + +# Or install globally +npm install -g lmnr-cli +``` + +The rest of this page assumes a global install (`lmnr-cli dataset ...`). With `npx`, prefix each command with `npx lmnr-cli@latest`. ## Usage @@ -46,32 +62,23 @@ data,target - + -```bash -export LMNR_PROJECT_API_KEY= -``` - -Alternatively, you can set it in the `.env` file in the same directory where you run the CLI. +The CLI authenticates as you, the signed-in user. Run `lmnr-cli setup` once in your project: it logs you in (opens a browser to authorize the CLI) and links the current directory to a project by writing `.lmnr/project.json`. Every dataset command then runs against that linked project. ```bash -echo "\nLMNR_PROJECT_API_KEY=" >> .env +lmnr-cli setup ``` -Or, you can also pass the `--project-api-key` flag to the global datasets command, e.g. +Alternatively, target a project explicitly with `--project-id` on any command: - - -```bash -npx lmnr datasets --project-api-key "" list -``` - - ```bash -lmnr datasets --project-api-key "" list +lmnr-cli dataset --project-id list ``` - - + + +Unlike the old SDK-bundled CLI, `lmnr-cli` does not use a project API key. See [Authenticate](/platform/cli#authenticate) and [Directory-scoped projects](/platform/cli#directory-scoped-projects) for details. + @@ -82,18 +89,9 @@ The datapoints are saved to a new file in order to: - Store datasets in the Laminar format. In particular, datapoint id is crucial for versioning ([Learn more](/datasets/introduction#versioning)). - Not overwrite existing files. - - ```bash -npx lmnr datasets create my-cli-dataset data.json -o my-cli-dataset.json +lmnr-cli dataset create my-cli-dataset data.json -o my-cli-dataset.json ``` - - -```bash -lmnr datasets create my-cli-dataset data.json -o my-cli-dataset.json -``` - - @@ -111,18 +109,9 @@ This is because the push operation only pushes new datapoint (versions) to the d Push the changes to Laminar. - - -```bash -npx lmnr datasets push -n my-cli-dataset my-cli-dataset.json -``` - - ```bash -lmnr datasets push -n my-cli-dataset my-cli-dataset.json +lmnr-cli dataset push my-cli-dataset.json -n my-cli-dataset ``` - - This will push the changes to the dataset in Laminar. @@ -130,18 +119,9 @@ This will push the changes to the dataset in Laminar. If you need to update the local dataset with the latest changes from Laminar, you can pull the changes. - - ```bash -npx lmnr datasets pull -n my-cli-dataset my-cli-dataset.json +lmnr-cli dataset pull my-cli-dataset.json -n my-cli-dataset ``` - - -```bash -lmnr datasets pull -n my-cli-dataset my-cli-dataset.json -``` - - This will pull the changes from the dataset in Laminar to the local file `my-cli-dataset.json`. @@ -155,67 +135,30 @@ This will overwrite the contents of the current file `my-cli-dataset.json`. ### Working on an existing dataset - - -```bash -export LMNR_PROJECT_API_KEY= -``` - -Alternatively, you can set it in the `.env` file in the same directory where you run the CLI. + -```bash -echo "\nLMNR_PROJECT_API_KEY=" >> .env -``` +Run `lmnr-cli setup` once to log in and link the current directory to your project (see [above](#creating-a-new-dataset-and-iterating-on-it)): -Or, you can also pass the `--project-api-key` flag to the global datasets command, e.g. - - - ```bash -npx lmnr datasets --project-api-key "" list +lmnr-cli setup ``` - - -```bash -lmnr datasets --project-api-key "" list -``` - - List all datasets and select the one you want to work on. - - -```bash -npx lmnr datasets list -``` - - ```bash -lmnr datasets list +lmnr-cli dataset list ``` - - Pull the data from Laminar to a local file. - - ```bash -npx lmnr datasets pull -n my-dataset my-dataset.json +lmnr-cli dataset pull my-dataset.json -n my-dataset ``` - - -```bash -lmnr datasets pull -n my-dataset my-dataset.json -``` - - This will pull the changes from the dataset in Laminar to the local file `my-dataset.json`. @@ -239,18 +182,9 @@ This is because the push operation only pushes new datapoint (versions) to the d Push the changes to Laminar. - - -```bash -npx lmnr datasets push -n my-dataset my-dataset.json -``` - - ```bash -lmnr datasets push -n my-dataset my-dataset.json +lmnr-cli dataset push my-dataset.json -n my-dataset ``` - - This will push the changes to the dataset in Laminar. @@ -258,35 +192,33 @@ This will push the changes to the dataset in Laminar. ### Setting the CLI to call a local Laminar instance -Global `datasets` command has optional arguments: +The `dataset` command has optional arguments for pointing at a self-hosted instance: -- `--base-url`: The base URL of the Laminar instance. Do NOT include port here. Default is `https://api.lmnr.ai`. -- `--port`: The HTTP port of the Laminar instance. Default is 443. For local self-hosted Laminar, use 8000. -- `--project-api-key`: The API key of the project. If not provided, reads from `LMNR_PROJECT_API_KEY` environment variable. +- `--base-url`: The base URL of the Laminar instance. Do NOT include port here. Default is `https://api.lmnr.ai` (or the `LMNR_BASE_URL` env variable). +- `--port`: The HTTP port of the Laminar instance. Default is 443 (or the `LMNR_HTTP_PORT` env variable). For local self-hosted Laminar, use 8000. +- `--project-id`: The id of the project to target. If not provided, resolves from the nearest `.lmnr/project.json`. -## Reference - - - ```bash -npx lmnr datasets [command] +lmnr-cli dataset --base-url http://localhost --port 8000 list ``` - - + +See [Self-hosting](/platform/cli#self-hosting) for the full self-hosted setup. + +## Reference + ```bash -lmnr datasets [command] +lmnr-cli dataset [command] ``` - - ## General options -These are useful if you want to call a local Laminar instance. +These are shared by every `dataset` subcommand. ``` - --project-api-key Project API key. If not provided, reads from LMNR_PROJECT_API_KEY env variable - --base-url Base URL for the Laminar API. Defaults to https://api.lmnr.ai or LMNR_BASE_URL env variable - --port Port for the Laminar API. Defaults to 443 + --project-id Target project id. Defaults to the linked .lmnr/project.json + --base-url Base URL for the Laminar API. Defaults to https://api.lmnr.ai or LMNR_BASE_URL env variable + --port Port for the Laminar API. Defaults to 443 or LMNR_HTTP_PORT env variable + --json Output structured JSON to stdout ``` ## Commands @@ -295,35 +227,17 @@ These are useful if you want to call a local Laminar instance. List all datasets. - - ```bash -npx lmnr datasets list +lmnr-cli dataset list ``` - - -```bash -lmnr datasets list -``` - - ### Create a new dataset Create a dataset from input files. - - -```bash -npx lmnr datasets create [options] -``` - - ```bash -lmnr datasets create [options] +lmnr-cli dataset create [options] ``` - - ``` Arguments: @@ -331,7 +245,7 @@ Arguments: paths Paths to files or directories containing data to push Options: - -o, --output-file Path to save the pulled data + -o, --output-file Path to save the pulled data (required) --output-format Output format (json, csv, jsonl). Inferred from file extension if not provided -r, --recursive Recursively read files in directories (default: false) --batch-size Batch size for pushing/pulling data (default: 100) @@ -341,18 +255,9 @@ Options: Push datapoints to an existing dataset from a file or files. - - ```bash -npx lmnr datasets push -n [options] +lmnr-cli dataset push [options] ``` - - -```bash -lmnr datasets push -n [options] -``` - - ``` Arguments: @@ -369,18 +274,9 @@ Options: Pull datapoints from a dataset to a file or print them to the console. - - -```bash -npx lmnr datasets pull [options] [output-path] -``` - - ```bash -lmnr datasets pull [options] [output-path] +lmnr-cli dataset pull [options] [output-path] ``` - - ``` Arguments: @@ -393,5 +289,5 @@ Options: --batch-size Batch size for pulling data (default: 100) --limit Limit number of datapoints to pull --offset Offset for pagination (default: 0) - -h, --help display help for command -``` \ No newline at end of file +``` +