add DECIMAL type support as primary key

[gpalanisamyYopp]

No description provided.

[gemini-code-assist]

Summary of Changes

Hello @gpalanisamyYopp, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the sourcedb-to-spanner pipeline by adding robust support for MySQL DECIMAL data types, especially when they serve as primary keys. The changes ensure that DECIMAL columns are accurately identified, their values are precisely managed during range-based splitting, and the entire migration process for such tables is thoroughly verified through new test cases. This improves the reliability and completeness of data migration for a broader range of MySQL schema configurations.

Highlights

• DECIMAL Type Support: Introduced comprehensive support for MySQL's DECIMAL data type, particularly when used as a primary key, ensuring it is correctly recognized and handled throughout the data pipeline.
• Precision in BigDecimal Splitting: Enhanced the splitBigDecimal method to maintain precision during range splitting operations by correctly handling the scale of BigDecimal values when converting to and from BigInteger.
• New IndexType: Added a new DECIMAL enum value to IndexType to explicitly categorize decimal primary keys.
• Expanded Test Coverage: Included new unit and integration tests to validate the correct discovery, splitting, and migration of tables with DECIMAL primary keys from MySQL to Spanner.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[VardhanThigle]

I am not sure if we have FLOAT type checked in yet.
Or should I hold on this reivew till FLOAT is merged in

[rossd-bitquill]

This PR has some overlap with the float PR. It would likely be simpler to start with the float PR, and then we will rebase this PR on top of that one.

[VardhanThigle]

Do we really need to couple this with stepSize or could we handle the scale as is?

My worry here is any rounding error in this exponentiation.

[rossd-bitquill]

The scale is explicitly required to calculate the StepSize, as the StepSize is used to calculate the smallest possible difference between two values (later on in the Splitter) and then check for inequality of two values by comparing the difference to the smallest possible difference between two values.

I don't see a way to decouple these two concepts, but I also don't expect to have rounding issues given we are using a BigDecimal and a Java built-in method to perform the exponentiation. As I understand the BigDecimal type, it is explicitly designed to handle such calculations without loss of precision.

[nmemond]

@VardhanThigle I think I see what you mean, and I expect working with the scale directly would actually simplify things a bit (I'm thinking it should allow us to use setScale and then unscaledValue to get a corresponding BigInteger, and then split with that). I'm going to work on swapping out the decimal step size in favour of just the scale itself.

[nmemond]

I would also say, I personally think it makes more sense for the SourceColumnIndexInfo class to have just the numeric scale, and then this gets used in various ways depending on the column type (since at the end of the day, the idea of "scale" is the same for decimal, double, and float, we just make use of it in different ways). But that can be a later improvement/refactoring.

[VardhanThigle]

Could we please fix the build?

[nmemond]

@VardhanThigle I unfortunately don't have permissions to push to the fork and the developer that owns this is out of office for the rest of the year. I've re-created the PR but up-to-date with main and with build and issues fixed in #3119. Please close out this PR.