datavzrd: auto-generated tool wrapper (issue #1966) - #1976
Conversation
|
🔧 Addressing feedback — view progress |
github-actions
Bot
removed
the
address-feedback
|
🔧 Addressed feedback — 8 files in tool directory All 5 tests pass and lint is clean. Summary of the changes addressing each review comment: 1. "Can this configuration file be built on the fly by taking inputs from the user?" (line 81, on the This is already the tool's default behavior — the "Auto-generate the configuration from the input tables" option builds the datavzrd config on the fly from the user's input tables via 2. "I don't think this is required" (line 90, on Removed the 3. Applied as suggested. Files changed: Agent tool call traceread_file({"path": "datavzrd_render.xml"}) read_file({"path": "macros.xml"}) read_file({"path": "datavzrd_config.py"}) read_file({"path": ".shed.yml"}) read_file({"end_line": 100, "path": "datavzrd_render.xml", "start_line": 70}) write_file({"content": "\n <token name="@TOOL_VERSION@">2.71.3\n <token name="@VERSION_SUFFIX@">0\n <token name="@Profile@">25.0\n <xml name="render_requirements">\n \n <requirement type="package" version="@TOOL_VERSION@">datavzrd\n <requirement type="package" version="3.13">python\n <requirement type="package" version="6.0.3">pyyaml\n <requirement type="package" version="3.0">zip\n \n \n <xml name="version_command">\n <version_command>datavzrd --version</version_command>\n \n <xml name="report_name">\n <param name="report_name" type="text" value="Datavzrd Report" label="Report name" help="Name of the generated report, shown in the report header">\n <validator type="regex" message="Report name must not contain single quotes, backslashes or newline characters">^[^'\\\\\\n\\r]+$\n \n \n <xml name="citations">\n \n <citation type="doi">10.1371/journal.pone.0323079\n \n \n \n\n", "path": "macros.xml"}) write_file({"content": "<tool id="datavzrd_render" name="datavzrd" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="@Profile@">\n Render visual and interactive HTML reports from tabular data\n \n macros.xml\n \n <expand macro="render_requirements"/>\n <expand macro="version_command"/>\n <command detect_errors="aggressive"> config.yaml &&\n#else:\n ## Rewrite the dataset paths of the user-provided configuration file\n ## to point to the input tables of this job\n python '$__tool_directory__/datavzrd_config.py' rewrite '${config_mode.config_file}' --inputs '#echo ','.join($data_files)#' --output config.yaml &&\n#end if\n\n## Apply advanced options to the configuration file\n#set $adv_args = []\n#if $adv.max_in_memory_rows:\n #silent $adv_args.append('max-in-memory-rows=' + str($adv.max_in_memory_rows))\n#end if\n#if $adv.webview_controls:\n #silent $adv_args.append('webview-controls=true')\n#end if\n#if $adv_args:\n python '$__tool_directory__/datavzrd_config.py' set config.yaml --output config.yaml #echo ' '.join($adv_args)# &&\n#end if\n\n## Render the report\n#set $overwrite_flag = ''\n#if $adv.overwrite_output:\n #set $overwrite_flag = '--overwrite-output'\n#end if\ndatavzrd config.yaml --output report_output $overwrite_flag &&\n\n## Collect the output\n#if planemo_lint({"path": "."}) planemo_test({"path": "."}) planemo_test({"path": "."}) |
|
🔧 Addressing feedback — view progress |
github-actions
Bot
removed
the
address-feedback
|
🔧 Addressed feedback — 8 files in tool directory All lint checks and all 5 tests pass. Here's a summary of the fixes applied per the review comments:
One technical note: the reviewer's snippet used Agent tool call traceread_file({"path": "datavzrd_render.xml"}) read_file({"path": "macros.xml"}) read_file({"path": ".shed.yml"}) read_file({"path": "datavzrd_config.py"}) write_file({"content": "\n <token name="@TOOL_VERSION@">2.72.0\n <token name="@VERSION_SUFFIX@">0\n <token name="@Profile@">25.0\n <xml name="render_requirements">\n \n <requirement type="package" version="@TOOL_VERSION@">datavzrd\n <requirement type="package" version="3.13">python\n <requirement type="package" version="6.0.3">pyyaml\n <requirement type="package" version="3.0">zip\n \n \n <xml name="version_command">\n <version_command>datavzrd --version</version_command>\n \n <xml name="report_name">\n <param name="report_name" type="text" value="Datavzrd Report" label="Report name" help="Name of the generated report, shown in the report header">\n <validator type="regex" message="Report name must not contain single quotes, backslashes or newline characters">^[^'\\\\\\n\\r]+$\n \n \n <xml name="citations">\n \n <citation type="doi">10.1371/journal.pone.0323079\n \n \n \n\n", "path": "macros.xml"}) write_file({"content": "categories:\n- Visualization\ndescription: Render visual and interactive HTML reports from tabular data with datavzrd\nhomepage_url: https://github.com/datavzrd/datavzrd\nlong_description: |\n Datavzrd creates visual and interactive HTML reports from collections of tabular\n data (CSV, TSV, JSON or Parquet). Reports include automatically generated vega-lite\n plots per column, searching, sorting and filtering, and can be fully customized via a\n YAML configuration file supporting custom plots, heatmaps, linkouts and inter-table\n linking.\nname: datavzrd\nowner: bgruening\nremote_repository_url: https://github.com/bgruening/galaxytools/tree/master/tools/datavzrd\ntype: unrestricted\n", "path": ".shed.yml"}) write_file({"content": "<tool id="datavzrd_render" name="datavzrd" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="@Profile@">\n Render visual and interactive HTML reports from tabular data\n \n macros.xml\n \n <expand macro="render_requirements"/>\n <required_files>\n <include path="datavzrd_config.py"/>\n </required_files>\n <expand macro="version_command"/>\n <command detect_errors="aggressive"> config.yaml &&\n#else:\n ## Rewrite the dataset paths of the user-provided configuration file\n ## to point to the input tables of this job\n python '$__tool_directory__/datavzrd_config.py' rewrite '${config_mode.config_file}' --inputs '#echo ','.join($data_files)#' --output config.yaml &&\n#end if\n\n## Apply advanced options to the configuration file\n#set $adv_args = []\n#if $adv.max_in_memory_rows:\n #silent $adv_args.append('max-in-memory-rows=' + str($adv.max_in_memory_rows))\n#end if\n#if $adv_args:\n python '$__tool_directory__/datavzrd_config.py' set config.yaml --output config.yaml #echo ' '.join($adv_args)# &&\n#end if\n\n## Render the report\ndatavzrd config.yaml --output report_output &&\n\n## Collect the output\n#if planemo_lint({"path": "."}) read_file({"path": "test-data/example-config.yaml"}) planemo_test({"path": "."}) fetch_url({"url": "https://api.anaconda.org/package/bioconda/datavzrd"}) search_web({"query": "datavzrd conda package bioconda channel"}) search_web({"query": "datavzrd conda package which channel forge"}) fetch_url({"url": "https://api.anaconda.org/search?name=datavzrd"}) read_file({"path": "datavzrd_render.xml", "start_line": 11, "end_line": 20}) read_file({"end_line": 30, "path": "datavzrd_render.xml", "start_line": 28}) write_file({"content": " <command detect_errors="aggressive"><![CDATA[\n## Symlink the input tables into the working directory using safe names\n## derived from the dataset names, so that they can be referenced from\n## the datavzrd configuration file.\n#import re\n#set $data_files = []\n#set $suggest_args = []\n#set $used = []\n#for $i, $data_file in enumerate($input_data):\n #if $data_file.is_of_type('json'):\n #set $ext = 'json'\n #elif $data_file.is_of_type('parquet'):\n #set $ext = 'parquet'\n #elif $data_file.is_of_type('csv'):\n #set $ext = 'csv'\n #else:\n #set $ext = 'tsv'\n #end if\n #set $stem = re.sub('[^\\w\\-]', '', str($data_file.element_identifier).rsplit('.', 1)[0])\n #if $stem in $used:\n #set $stem = $stem + '' + str($i)\n #end if\n #silent $used.append($stem)\n #set $fname = $stem + '.' + $ext\n #silent $data_files.append( read_file({"path": "datavzrd_render.xml"}) write_file({"content": "<tool id="datavzrd_render" name="datavzrd" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="@Profile@">\n Render visual and interactive HTML reports from tabular data\n \n macros.xml\n \n <expand macro="render_requirements"/>\n <required_files>\n <include path="datavzrd_config.py"/>\n </required_files>\n <expand macro="version_command"/>\n <command detect_errors="aggressive"> config.yaml &&\n#else:\n ## Rewrite the dataset paths of the user-provided configuration file\n ## to point to the input tables of this job\n python '$__tool_directory__/datavzrd_config.py' rewrite '${config_mode.config_file}' --inputs '#echo ','.join($data_files)#' --output config.yaml &&\n#end if\n\n## Apply advanced options to the configuration file\n#set $adv_args = []\n#if $adv.max_in_memory_rows:\n #silent $adv_args.append('max-in-memory-rows=' + str($adv.max_in_memory_rows))\n#end if\n#if $adv.webview_controls:\n #silent $adv_args.append('webview-controls=true')\n#end if\n#if $adv_args:\n python '$__tool_directory__/datavzrd_config.py' set config.yaml --output config.yaml #echo ' '.join($adv_args)# &&\n#end if\n\n## Render the report\ndatavzrd config.yaml --output report_output --overwrite-output &&\n\n## Collect the output\n#if planemo_lint({"path": "."}) planemo_test({"path": "."}) write_file({"content": "<tool id="datavzrd_render" name="datavzrd" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="@Profile@">\n Render visual and interactive HTML reports from tabular data\n \n macros.xml\n \n <expand macro="render_requirements"/>\n <required_files>\n <include path="datavzrd_config.py"/>\n </required_files>\n <expand macro="version_command"/>\n <command detect_errors="aggressive"> config.yaml &&\n#else:\n ## Rewrite the dataset paths of the user-provided configuration file\n ## to point to the input tables of this job\n python '$__tool_directory__/datavzrd_config.py' rewrite '${config_mode.config_file}' --inputs '#echo ','.join($data_files)#' --output config.yaml &&\n#end if\n\n## Apply advanced options to the configuration file\n#set $adv_args = []\n#if $adv.max_in_memory_rows:\n #silent $adv_args.append('max-in-memory-rows=' + str($adv.max_in_memory_rows))\n#end if\n#if $adv_args:\n python '$__tool_directory__/datavzrd_config.py' set config.yaml --output config.yaml #echo ' '.join($adv_args)# &&\n#end if\n\n## Render the report\ndatavzrd config.yaml --output report_output --overwrite-output &&\n\n## Collect the output\n#if planemo_lint({"path": "."}) planemo_test({"path": "."}) |
| <output name="output_html" ftype="html"> | ||
| <assert_contents> | ||
| <has_text text="Oscars Report"/> | ||
| <has_text text="static/bundle.js"/> |
There was a problem hiding this comment.
this is a bit unspecific maybe number of lines with a large delta?
| <param name="output_format" value="html"/> | ||
| <output name="output_html" ftype="html"> | ||
| <assert_contents> | ||
| <has_text text="Gene Report"/> |
There was a problem hiding this comment.
we need to be more specific here ...
html, could also be a valid XML ... so you maybe can use the XML specific assertions
| ## Credits | ||
|
|
||
| Datavzrd was developed by Felix Wiegand, Johannes Köster and colleagues. Source code and documentation are available at https://github.com/datavzrd/datavzrd. | ||
| ]]></help> |
There was a problem hiding this comment.
we have explicit citations for this
|
|
||
| The interactive HTML report can be displayed directly in Galaxy (choose *Interactive HTML report*). All report files (tables, plots and data) are stored alongside the displayed `index.html`. Alternatively, the whole report directory can be returned as a zip archive, e.g. for sharing or archiving of large reports. | ||
|
|
||
| ## Advanced options |
There was a problem hiding this comment.
why repeating this here again?
| </test> | ||
| <!-- Test 03: user-provided configuration file with advanced options --> | ||
| <test expect_num_outputs="1"> | ||
| <param name="input_data" value="oscars.csv,movies.csv" ftype="csv"/> |
There was a problem hiding this comment.
those names here need to match exactly the names in the config file ...
I think this deserves an example in the help section. Also raising the fact that those names need to be unique.
|
🔧 Addressing feedback — view progress |
github-actions
Bot
removed
the
address-feedback
|
🔧 Addressed feedback — 8 files in tool directory Agent tool call traceread_file({"path": "datavzrd_render.xml"}) read_file({"path": "macros.xml"}) read_file({"path": "datavzrd_config.py"}) read_file({"path": "test-data/example-config.yaml"}) read_file({"path": "datavzrd_render.xml", "start_line": 50, "end_line": 60}) read_file({"end_line": 225, "start_line": 100, "path": "datavzrd_render.xml"}) read_file({"end_line": 92, "path": "datavzrd_render.xml", "start_line": 82}) search_github({"query": "has_n_lines delta tool test assert_contents galaxy"}) search_github({"query": "has_element_with_text has_attribute xpath assert_contents XML galaxy tool"}) fetch_url({"url": "https://raw.githubusercontent.com/datavzrd/datavzrd/v2.72.0/src/templates/index.html"}) search_web({"query": "datavzrd PLOS ONE 10.1371/journal.pone.0323079 Wiegand K\u00f6ster"}) search_github({"query": "datavzrd report index.html templates Tera"}) search_github({"query": "journal.pone.0323079 datavzrd"}) fetch_url({"url": "https://api.github.com/search/code?q=repo:datavzrd/datavzrd+filename:index.html"}) fetch_url({"url": "https://api.github.com/repos/datavzrd/datavzrd/contents/src/render_table.rs?ref=v2.72.0"}) fetch_url({"url": "https://api.github.com/repos/datavzrd/datavzrd/contents/?ref=v2.72.0"}) fetch_url({"url": "https://api.github.com/repos/datavzrd/datavzrd/contents/templates?ref=v2.72.0"}) fetch_url({"url": "https://raw.githubusercontent.com/datavzrd/datavzrd/v2.72.0/templates/index.html.tera"}) fetch_url({"url": "https://raw.githubusercontent.com/galaxyproject/galaxy/dev/lib/galaxy/tools/verify/asserts/__init__.py"}) fetch_url({"url": "https://raw.githubusercontent.com/galaxyproject/galaxy/dev/lib/galaxy/tools/verify/asserts/__init__.py"}) fetch_url({"url": "https://api.github.com/repos/galaxyproject/galaxy/contents/lib/galaxy/tools/verify/asserts"}) search_github({"query": "galaxy verify asserts has_n_lines xml"}) search_web({"query": "galaxy tool xml assertion tag "has_n_lines" delta attribute"}) fetch_url({"url": "https://raw.githubusercontent.com/galaxyproject/galaxy/master/lib/galaxy/tools/verify/asserts/__init__.py"}) fetch_url({"url": "https://api.github.com/search/code?q=has_n_lines+repo:galaxyproject/galaxy&per_page=5"}) search_web({"query": ""has_n_lines" planemo verify assertion"}) fetch_url({"url": "https://docs.galaxyproject.org/en/latest/dev/schema.html"}) fetch_url({"url": "https://raw.githubusercontent.com/datavzrd/datavzrd/v2.72.0/README.md"}) search_bio_tools({"query": "datavzrd"}) read_file({"end_line": 60, "path": "datavzrd_render.xml", "start_line": 40}) read_file({"end_line": 39, "path": "datavzrd_render.xml", "start_line": 1}) read_file({"end_line": 110, "path": "datavzrd_render.xml", "start_line": 60}) read_file({"end_line": 230, "path": "datavzrd_render.xml", "start_line": 108}) read_file({"path": "macros.xml"}) read_file({"path": "datavzrd_config.py"}) read_file({"path": "test-data/example-config.yaml"}) fetch_url({"url": "https://raw.githubusercontent.com/galaxyproject/galaxy/dev/lib/galaxy/tool_util/xsd/tool.xsd"}) fetch_url({"url": "https://raw.githubusercontent.com/galaxyproject/galaxy/dev/lib/galaxy/tool_util/xsd/galaxy.xsd"}) read_file({"end_line": 58, "path": "datavzrd_render.xml", "start_line": 50}) read_file({"end_line": 92, "path": "datavzrd_render.xml", "start_line": 84}) |
|
🔧 Addressing feedback — view progress |
github-actions
Bot
removed
the
address-feedback
|
|
|
🔧 Addressing feedback — view progress |
github-actions
Bot
removed
the
address-feedback
|
🔧 Addressed feedback — 8 files in tool directory Agent tool call traceread_file({"path": "datavzrd_render.xml"}) read_file({"path": "macros.xml"}) read_file({"end_line": 230, "path": "datavzrd_render.xml", "start_line": 45}) read_file({"path": "datavzrd_config.py"}) read_file({"path": "test-data/example-config.yaml"}) read_file({"path": ".shed.yml"}) fetch_url({"url": "https://raw.githubusercontent.com/galaxyproject/galaxy/dev/lib/galaxy/tool_util/verify/asserts/__init__.py"}) search_github({"query": "datavzrd index.html template site repo:datavzrd/datavzrd"}) fetch_url({"url": "https://raw.githubusercontent.com/galaxyproject/galaxy/dev/lib/galaxy/tool_util/verify/asserts/xml.py"}) fetch_url({"url": "https://raw.githubusercontent.com/galaxyproject/galaxy/dev/lib/galaxy/tool_util/verify/asserts/archive.py"}) fetch_url({"url": "https://api.github.com/repos/datavzrd/datavzrd/contents/"}) fetch_url({"url": "https://raw.githubusercontent.com/datavzrd/datavzrd/main/templates/index.html"}) fetch_url({"url": "https://api.github.com/repos/datavzrd/datavzrd/contents/templates?ref=main"}) fetch_url({"url": "https://raw.githubusercontent.com/datavzrd/datavzrd/main/templates/index.html.tera"}) fetch_url({"url": "https://raw.githubusercontent.com/datavzrd/datavzrd/v2.72.0/templates/index.html.tera"}) fetch_url({"url": "https://raw.githubusercontent.com/datavzrd/datavzrd/v2.72.0/templates/html.html.tera"}) read_file({"path": "datavzrd_render.xml"}) read_file({"path": "macros.xml"}) read_file({"path": "test-data/oscars.csv"}) read_file({"path": "test-data/movies.csv"}) read_file({"path": "test-data/genes.tsv"}) read_file({"end_line": 100, "path": "datavzrd_render.xml", "start_line": 50}) read_file({"end_line": 223, "path": "datavzrd_render.xml", "start_line": 100}) fetch_url({"url": "https://raw.githubusercontent.com/galaxyproject/galaxy/dev/lib/galaxy/tool_util/verify/asserts/xml.py"}) write_file({"content": "<tool id="datavzrd_render" name="datavzrd" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="@Profile@">\n Render visual and interactive HTML reports from tabular data\n \n macros.xml\n \n <expand macro="render_requirements"/>\n <required_files>\n <include path="datavzrd_config.py"/>\n </required_files>\n <expand macro="version_command"/>\n <command detect_errors="aggressive"> config.yaml &&\n#else:\n ## Rewrite the dataset paths of the user-provided configuration file\n ## to point to the input tables of this job\n python '$__tool_directory__/datavzrd_config.py' rewrite '${config_mode.config_file}' --inputs '#echo ','.join($data_files)#' --output config.yaml &&\n#end if\n\n## Render the report\ndatavzrd config.yaml --output report_output --overwrite-output &&\n\n## Collect the output\n#if read_file({"path": "macros.xml", "pattern": "xml name="}) planemo_lint({"path": "datavzrd_render.xml"}) planemo_test({"path": "datavzrd_render.xml"}) planemo_test({"path": "datavzrd_render.xml"}) read_file({"path": "macros.xml"}) write_file({"content": "<tool id="datavzrd_render" name="datavzrd" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="@Profile@">\n Render visual and interactive HTML reports from tabular data\n \n macros.xml\n \n <expand macro="render_requirements"/>\n <required_files>\n <include path="datavzrd_config.py"/>\n </required_files>\n <expand macro="version_command"/>\n <command detect_errors="aggressive"> config.yaml &&\n#else:\n ## Rewrite the dataset paths of the user-provided configuration file\n ## to point to the input tables of this job\n python '$__tool_directory__/datavzrd_config.py' rewrite '${config_mode.config_file}' --inputs '#echo ','.join($data_files)#' --output config.yaml &&\n#end if\n\n## Apply advanced options to the configuration file\n#set $adv_args = []\n#if $adv.max_in_memory_rows:\n #silent $adv_args.append('max-in-memory-rows=' + str($adv.max_in_memory_rows))\n#end if\n#if $adv_args:\n python '$__tool_directory__/datavzrd_config.py' set config.yaml --output config.yaml #echo ' '.join($adv_args)# &&\n#end if\n\n## Render the report\ndatavzrd config.yaml --output report_output --overwrite-output &&\n\n## Collect the output\n#if planemo_lint({"path": "datavzrd_render.xml"}) planemo_test({"path": "datavzrd_render.xml"}) read_file({"end_line": 180, "path": "datavzrd_render.xml", "start_line": 110}) write_file({"content": "<tool id="datavzrd_render" name="datavzrd" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="@Profile@">\n Render visual and interactive HTML reports from tabular data\n \n macros.xml\n \n <expand macro="render_requirements"/>\n <required_files>\n <include path="datavzrd_config.py"/>\n </required_files>\n <expand macro="version_command"/>\n <command detect_errors="aggressive"> config.yaml &&\n#else:\n ## Rewrite the dataset paths of the user-provided configuration file\n ## to point to the input tables of this job\n python '$__tool_directory__/datavzrd_config.py' rewrite '${config_mode.config_file}' --inputs '#echo ','.join($data_files)#' --output config.yaml &&\n#end if\n\n## Apply advanced options to the configuration file\n#set $adv_args = []\n#if $adv.max_in_memory_rows:\n #silent $adv_args.append('max-in-memory-rows=' + str($adv.max_in_memory_rows))\n#end if\n#if $adv_args:\n python '$__tool_directory__/datavzrd_config.py' set config.yaml --output config.yaml #echo ' '.join($adv_args)# &&\n#end if\n\n## Render the report\ndatavzrd config.yaml --output report_output --overwrite-output &&\n\n## Collect the output\n#if read_file({"path": "datavzrd_config.py"}) planemo_test({"path": "datavzrd_render.xml"}) |
|
Thanks |
2 participants
This PR adds a Galaxy wrapper for datavzrd (v2.71.3, resolves #1966), a tool for rendering interactive, configurable HTML reports from tabular data.
The wrapper supports multiple tabular inputs (CSV/TSV/JSON/Parquet), with either auto-generated configs (
datavzrd suggest) or user-provided YAML configs whose dataset paths are rewritten via the includeddatavzrd_config.pyhelper. Reports are exposed as interactive Galaxy HTML output (QUAST-style directory layout) or as a zip archive, with advanced options for max-in-memory-rows, webview controls, and output overwriting.The PR includes
datavzrd_render.xml, sharedmacros.xml, the config helper script,.shed.ymlmetadata, and test data adapted from upstream examples. All 5 functional tests pass under planemo, covering auto-config, multi-input, user config, TSV, and zip output scenarios.