Skip to content

[BUG] Update spark.rapids.sql.reader.multithreaded.combine.sizeBytes recommendation to default #2119

Description

@parthosa

Currently, the AutoTuner recommends 10m for spark.rapids.sql.reader.multithreaded.combine.sizeBytes as compared to the plugin default. We should update the heuristic to:

  1. Introduce a rule for updating the metric
  2. Disable recommendation (so that plugin default gets used)

Ref: https://nvidia.github.io/cudf-spark/docs/additional-functionality/advanced_configs.html

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions