Skip to content

Default behaviour for write_map is to use lossy compression with floating-point numbers #3

@ziotom78

Description

@ziotom78

The default value for the compress parameter to write_map is True, meaning that the code will use one of the compression algorithms specified by the FITS standard. Since these algorithms are lossy when applied to maps containing floating-point values, the code will silently introduce an error in the data written to disk, as the following example shows:

import numpy
import qubic.io

FILE_NAME = '/tmp/compressed_map.fits'

INPUT_PIXELS = numpy.random.normal(size=12 * 64 * 64)
qubic.io.write_map(FILE_NAME, INPUT_PIXELS, dtype=numpy.float64)
OUTPUT_PIXELS = qubic.io.read_map(FILE_NAME)

DIFF = OUTPUT_PIXELS - INPUT_PIXELS
print('Stddev of the difference: {0:.3f}'
      .format(numpy.std(DIFF)))

# Result:
# Stddev of the difference: 0.018

(Note that the code uses float64 instead of the default float32 type when calling write_map.) Setting compress=False in the above call to qubic.io.write_map makes the difference between INPUT_PIXELS and OUTPUT_PIXELS to vanish.

Since this behaviour might be surprising for people not versed with the FITS standard, I am suggesting a couple of workarounds:

  1. Make compress default to False, and specify in the docstring for write_map that using True with floating-point maps will introduce an error in the pixel values (currently, the docstring does not mention the compress parameter anywhere but in the declaration);
  2. Make the code emit a warning when compress is True and the datatype of the parameter map is a floating-point value.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions