What is counted in the "Bytes transferred"? #302
|
I have loaded the same collection multiple times to determine which number of threads would be the most efficient to use. The number of "Bytes transferred" has been a different value almost every time. Which bytes are being included in that value please? |
Replies: 3 comments 6 replies
|
@collinss-jpl can you provide some insight here? |
|
DUM derives the size of each file using the st_size value returned from a call to os.stat. The total bytes transferred tally is updated here for each successfully transferred file. If files are skipped or failed, their sizes are not included in the transfer total, so this might explain the discrepancies between multiple runs of the same archive. |
|
All the runs I am comparing reported that all the files were uploaded with 0 skipped, failed, and unprocessed. |
|
Did some testing on my end and I can reproduce the issue. Will issue a bug fix and a new release that should correct the issue tomorrow. |
|
DUM v2.2.3 is now available from PyPI, which adds a fix for the inaccurate bytes transferred value. You can upgrade to v2.2.3 by running the following within your existing Python environment for DUM:
|
|
@jordanpadams |
|
@plawton-umd what version of Python are you running? you may need to upgrade to Python 3.12 or 3.13 in order to get the new version |
DUM v2.2.3 is now available from PyPI, which adds a fix for the inaccurate bytes transferred value.
You can upgrade to v2.2.3 by running the following within your existing Python environment for DUM:
pip install --upgrade pds-data-upload-manager