feat(Dataset): add fill_gaps method using utilsforecast - #31
Open
a11to1n3 wants to merge 3 commits into
Open
Conversation
moobeck
reviewed
Dec 11, 2025
moobeck
left a comment
moobeck
left a comment
Collaborator
There was a problem hiding this comment.
I’d probably just use the fill_gaps method from utilsforecast ( from utilsforecast.preprocessing import fill_gaps). It works directly with Polars anyway.
I’d also sort the data before applying the backfill. Have you actually tested this function? Just from reading the code, I’m not really sure how well it handles the index, for example.
Adds fill_gaps() method to Dataset class that:
- Uses utilsforecast.preprocessing.fill_gaps to identify missing timestamps
- Fills target column with 0 (default)
- Fills covariate columns with backward fill (bfill)
- Marks filled rows with not_for_sale=1.0
Features:
- Auto-detects week start day for weekly datasets
- Supports daily, weekly, monthly, quarterly, yearly frequencies
- Preserves original data while adding missing timestamps
Example:
filled_dataset = dataset.fill_gaps()
a11to1n3
force-pushed
the
gap-filling-utilsforecast
branch
from
December 11, 2025 12:50
65c9df4 to
bb09a3a
Compare
…ap-filling-utilsforecast
- Filter pharmacy2 data by IDs from pharmacy_filtered_ids.csv (594 series) - Shift all time series to align to common end date 2020-05-12 - Split processed data into Pharmacy2_features.parquet and Pharmacy2_targets.parquet - Simplify load_pharmacy2 to only perform gap filling at runtime - Add fill_time_series_gaps utility function - Fix test_dataset_operations.py to handle numeric encoded IDs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds fill_gaps() method to Dataset class that:
Features:
Example:
filled_dataset = dataset.fill_gaps()