Conversation
Signed-off-by: Partho Sarthi <psarthi@nvidia.com>
Signed-off-by: Partho Sarthi <psarthi@nvidia.com>
The PR appears safe to merge, with a non-blocking request to add regression coverage for the new alias-preservation behavior. Findings
|
4 tasks
Signed-off-by: Partho Sarthi <psarthi@nvidia.com>
Signed-off-by: Partho Sarthi <psarthi@nvidia.com>
Signed-off-by: Partho Sarthi <psarthi@nvidia.com>
Comment on lines
+86
to
+116
| def get_sql_func_aliases(sql_func_value): | ||
| """ | ||
| Return SQL function aliases from a semicolon-separated SQL Func cell. | ||
| """ | ||
| if pd.isna(sql_func_value): | ||
| return set() | ||
| sql_func_str = str(sql_func_value).strip() | ||
| if sql_func_str == "" or sql_func_str.lower() in ("none", "nan"): | ||
| return set() | ||
| return { | ||
| alias.strip() for alias in sql_func_str.split(";") | ||
| if alias.strip() and alias.strip() not in SupportLevel.__members__ | ||
| } | ||
|
|
||
|
|
||
| def should_preserve_tools_string(column_name, tools_value, plugin_value): | ||
| """ | ||
| Preserve tools-side expression aliases when plugin generated metadata is less informative. | ||
| SQL function aliases are parser lookup keys in tools, and the generated plugin CSV can omit | ||
| aliases for newer Spark versions even though older-version aliases should still be recognized. | ||
| Preserve only when the plugin aliases are blank or a strict subset of the tools aliases, so an | ||
| intentional plugin alias rename is accepted instead of being hidden by a string-length heuristic. | ||
| """ | ||
| if column_name != "SQL Func": | ||
| return False | ||
|
|
||
| tools_aliases = get_sql_func_aliases(tools_value) | ||
| plugin_aliases = get_sql_func_aliases(plugin_value) | ||
| if not tools_aliases: | ||
| return False | ||
| return not plugin_aliases or plugin_aliases < tools_aliases |
There was a problem hiding this comment.
The new alias normalization and preservation logic has no committed automated coverage. Focused tests should cover blank and NaN-like values, support-level sentinels, strict subsets, disjoint renames, and the final emitted CSV value; otherwise, a later sync change could silently remove parser lookup aliases.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the repo-local supported-ops-sync skill workflow and sync-helper script updates split out from #2095.
This PR documents the boundary between deterministic automation and support validation: the CSV sync mechanics are automated, but support promotion remains evidence-gated for parser behavior, plugin code paths, Spark versions, datasource formats, Spark configs, and event-log coverage.
The new
TNEWoverride config entries stay in #2095 because they are part of that concrete supported-ops data sync.Changes
NS,S, andCO.Automation Boundary
TNEW, update score files, and generate a report.Testing