Skip to content

Float precision errors #65

@n1k0

Description

@n1k0

Hello, thanks for the useful library. I'm having an issue with a mass expressed in grams or kilograms:

> import Mass
> Mass.grams 18 |> Mass.inGrams
18.000000000000004 : Float
> Mass.grams 18 == Mass.kilograms 0.018
False : Bool
> Mass.grams 17 == Mass.kilograms 0.017
True : Bool

This is because the grams function uses kilograms (0.001 * numGrams) instead of kilograms (numGrams / 1000) which would solve the issue.

Am I understanding things wrong? Or would you like me to craft a PR?

Thank you

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions