DuckDB S3 Dashboard

This project is a web-based dashboard for interacting with DuckDB and S3. It allows users to query data stored in S3 using DuckDB's powerful SQL engine.

Features

• DuckDB Integration: Leverages DuckDB's WebAssembly build for in-browser SQL queries.
• Log Search Dashboard: Designed for searching and analyzing log data.
• Parquet Conversion: Requires log data to be converted to merged.parquet format and uploaded to S3.

Prerequisites

• Node.js and Yarn installed on your system.
• AWS credentials with access to the required S3 buckets.

Setup

1. Clone the repository:

   git clone <repository-url>
   cd duckdb-wasm

2. Create a .env file with your AWS credentials:

   cat <<EOF > .env
   VITE_AWS_ACCESS_KEY_ID=<Your AWS Access Key ID>
   VITE_AWS_SECRET_ACCESS_KEY=<Your AWS Secret Access Key>
   VITE_AWS_SESSION_TOKEN=<Your AWS Session Token (if applicable)>
   EOF

3. Install dependencies:

   yarn install

4. Start the development server:

   yarn dev

Project Structure

• src/services/duckdbService.ts: Handles DuckDB initialization and query execution.
• src/services/s3Service.ts: Fetches Parquet files from S3.
• src/Dashboard.tsx: Main dashboard component.
• vite.config.ts: Vite configuration file.

Scripts

• yarn dev: Starts the development server.
• yarn build: Builds the project for production.
• yarn preview: Previews the production build.

Dependencies

• @duckdb/duckdb-wasm: DuckDB's WebAssembly build.
• @aws-sdk/client-s3: AWS SDK for S3 operations.
• react and react-dom: React library for building the UI.

Additional Notes

• All source code in this project was generated by Copilot Agent.

License

This project is licensed under the MIT License.