Allowing Parquet Output to be visualized and readed.

[romer8]

This pull request introduces significant improvements and refactoring to the data caching and table management logic in the data stream feature. The core enhancements include a unified OPFS cache system supporting both Arrow and Parquet files, streamlined cache key and table naming conventions, and a more robust DuckDB integration for file-backed tables. Additionally, it removes debug logging and simplifies function signatures for clarity and maintainability.

Major improvements to OPFS cache and table management:

• Added support for storing and loading both Arrow and Parquet files in the cache, with unified functions such as saveDataToCache, loadFromCache, and createTableFromOPFS. This enables flexible handling of different data formats and simplifies downstream code. [1] [2] [3]
• Implemented helper functions for file and table naming, file type detection, and file statistics retrieval, ensuring consistent naming and easier file management. [1] [2]
• Improved DuckDB integration by registering file handles for Parquet files and inserting Arrow tables directly, reducing memory usage and supporting browser file system access.

Refactoring and simplification:

• Refactored all cache key usages to consistently use safe, encoded names, and updated all relevant code to use the new naming scheme. This includes changes to getCacheKey, removal of file extensions in table names, and updates to all query logic to use table names derived from cache keys. [1] [2] [3] [4] [5] [6] [7] [8] [9]
• Updated the VPU data loading pipeline to use the new caching and table creation logic, removing direct Arrow buffer handling and simplifying the workflow.

General code cleanup:

• Removed unnecessary debug logging and console statements throughout the codebase for cleaner output and improved performance. [1] [2] [3] [4] [5]
• Minor improvements to DuckDB client instantiation and connection handling for reliability. [1] [2] [3]

These changes collectively make the data stream feature more robust, easier to maintain, and ready for future expansion to new data formats and workflows.