Skip to content

Conversation

@amir9667
Copy link

@amir9667 amir9667 commented Nov 26, 2025

This pull request introduces definition and integration of class-level sources and targets for the affected dataset modules.
references to cfg have been removed and replaced with static, structured access through DatasetSources and DatasetTargets.

Key Changes

1- Added fully structured sources and targets attributes to each updated dataset class.
2-Replaced all cfg[...] usages with direct references to
ClassName.sources.files[...],
ClassName.sources.tables[...], or
ClassName.targets[...].

🧑‍💻 Contributor Checklist

Before requesting a review, make sure you've completed all of the following:

  • All tests pass locally or via CI
  • (for more information on local test, check tox in the Contributing section)
  • (CI tests are automatically executed when creating a PR, you can see the results of the checks below)
  • Workflow has run at least once in Test mode
  • (optional if no dataset changes are involved)
  • Relevant documentation is updated (API, new features, etc.)
  • Dataset-versions are updated when existing datasets are adjusted.
  • Added a note to CHANGELOG.rst about the changes
  • Added yourself to AUTHORS.rst

Optional:

  • Changes have been tested in Everything mode
  • Extend the checklist for reviewers: Which aspects should be reviewed in particular?

🔍 Reviewer Checklist

During your review, please check the following:

  • Is the code clean, readable, and efficient? Are there any oddities or obvious inefficiencies?
  • Does the code work as expected? (should already be verified by contributor)
  • Do all tests pass? (see CI results)
  • Is the documentation complete and up to date?
  • Is CHANGELOG.rst updated accordingly?
  • Is all necessary metadata complete and correct?
    - [ ] If metadata is pending: Is there an appropriate issue filed?

ClaraBuettner and others added 30 commits July 8, 2025 15:07
…executed

The version is set after the execution of all tasks
You successfully defined the sources and targets inside the HeavyDutyTransport class.

You updated the download_hgv_data function to use the new class attributes instead of the old config variable.

You correctly kept the DATASET_CFG variable at the top of the file, as it's still needed for other parameters and constants.

FOR INDIVIDUAL TRAVEL

This commit refactors several dataset modules to standardize how input sources and output targets are defined, making the code more modular and easier to maintain.

Key changes include:

Replaced External Config Calls: Removed direct calls to config.datasets() for accessing source and target paths and tables. This reduces reliance on scattered YAML definitions.

Centralized Dependencies in Classes: Defined all data dependencies directly within each dataset's main Python class using the DatasetSources and DatasetTargets objects.

Separated Files and Tables: Following team feedback, inputs in DatasetSources were correctly categorized into .files (for CSVs) and .tables (for database tables) for better clarity.

Updated Function Logic: All functions were updated to use the new class attributes (e.g., ClassName.sources.files['key']) instead of the old cfg dictionary.

Improved Modularity: Updated function calls to pass the entire sources object where necessary, making the functions more independent and reusable.

Affected datasets:

power_plants

substation_extraction

chp

heavy_duty_transport

motorized_individual_travel

FOR CHARGING INFRASTRUCTURE
Key changes include:

Defined sources and targets: Moved all input (URLs, tables) and output (files, tables) definitions into the MITChargingInfrastructure class using DatasetSources and DatasetTargets. This makes the dataset's dependencies explicit and self-contained.

Added Dependencies from Docstring: In addition to the tracbev URL, source tables for mv_grid_districts and buildings were added based on the class docstring's Dependencies section.

Removed Global Config: Deleted the global DATASET_CFG variable to eliminate reliance on the external YAML configuration for sources and targets.

Updated Data Handling: Modified the get_tracbev_data function to use the new class attributes (MITChargingInfrastructure.sources.urls and MITChargingInfrastructure.targets.files) for all data access.
amir9667 and others added 30 commits December 30, 2025 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants