[WIP] Serialize/deserialize with libcudf pack/unpack#5025
Closed
shwina wants to merge 114 commits into
Closed
Conversation
And fix for pack: was using moved-from data. Forgot to add first metadata element which stands in for the table
Because that is also expensive to copy so we need to enforce taking ownership
…erialize-with-pack-unpack
…erialize-with-pack-unpack
…cudf into fea-deprecate-nvstrings
Make documentation uniform for stream and memory resource params in cpp
…erialize-with-pack-unpack
[REVIEW] Deprecate nvstrings
[REVIEW] Fix DEBUG compilation failure due to `fixed_point.hpp`
Remove unused native deps from java library
…erialize-with-pack-unpack
As some serialization strategies may rely on some or all frames on host, add logic to `Serializable` to track which frames are on host or device and ensure they are handled appropriately.
Member
|
FWIW I think PR ( #5309 ) should make the needed changes to |
Member
|
PR ( #5309 ) was merged into |
harrism
added
2 - In Progress
Contributor
Author
|
@VibhuJawa tested this PR and reported that it actually slowed things down for some of his work. Are we interested in pursuing this further? cc: @kkraus14 @devavret @jakirkham @jrhemstad |
Contributor
|
This could be handy if/when we have cudf configuration. Use/don't use packing could be a switchable option. |
Member
|
Yeah that is what we are discussing in issue ( #5311 ). |
Collaborator
|
@shwina going to close this for now since it's stalled. We can reopen if desired. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Still WIP. Depends on #4941 and #5309