Skip to content

feat: query hot flowBuffer and cold S3 Parquet for raw flows - #1605

Draft
jpinsonneau wants to merge 2 commits into
netobserv:mainfrom
jpinsonneau:s3-parquet
Draft

jpinsonneau wants to merge 2 commits into
netobserv:mainfrom
jpinsonneau:s3-parquet

Conversation

@jpinsonneau

@jpinsonneau jpinsonneau commented Jul 20, 2026

Copy link
Copy Markdown
Member

Summary

  • Console config: flowBuffer (FLP Service URL) and s3 / consolePlugin.s3 for cold Parquet (legacy s3Query still accepted).
  • Backend: FLP client to /api/flowbuffer/flows (port 9200) and S3 Hive Parquet querier for GetFlows / export.
  • Frontend: buffer-only mode when Loki is off; query warnings for partial/hot-tier results; Overview continues to use Prometheus.

Hub PR (operator): netobserv/netobserv-operator#2845

Test plan

  • go test ./pkg/flp/... ./pkg/s3query/... ./pkg/handler/...
  • Unit / frontend tests for capabilities and query options
  • With operator Kind setup: GetFlows via FLP buffer and S3; buffer-only UI; Overview metrics from Prom
  • Confirm ConfigMap emits s3 (not only legacy s3Query)

jpinsonneau and others added 2 commits July 20, 2026 14:30
Add minio-go, parquet-go, and related deps for cold Parquet queries
from object storage.

Co-authored-by: Cursor <cursoragent@cursor.com>
Route GetFlows through FLP /api/flowbuffer/flows and Hive-layout S3
Parquet (consolePlugin.s3), with buffer-only UI when Loki is off and
warnings for partial results.

Co-authored-by: Cursor <cursoragent@cursor.com>
@openshift-ci

openshift-ci Bot commented Jul 20, 2026

Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 61b1fb11-dbc5-4022-85b2-185bfb3409c2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@openshift-ci

openshift-ci Bot commented Jul 20, 2026

Copy link
Copy Markdown

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci

openshift-ci Bot commented Jul 20, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign jotak for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant