Skip to content

fix: OPTIC-683: Detect and convert large integers automatically in API responses before parsing as JSON to retain precision #6111

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jul 22, 2024

Conversation

bmartel
Copy link
Contributor

@bmartel bmartel commented Jul 19, 2024

PR fulfills these requirements

  • Commit message(s) and PR title follows the format [fix|feat|ci|chore|doc]: TICKET-ID: Short description of change made ex. fix: DEV-XXXX: Removed inconsistent code usage causing intermittent errors
  • Tests for the changes have been added/updated (for bug fixes/features)
  • Docs have been added/updated (for bug fixes/features)
  • Best efforts were made to ensure docs/code are concise and coherent (checked for spelling/grammatical errors, commented out code, debug logs etc.)
  • Self-reviewed and ran all changes on a local instance (for bug fixes/features)

Change has impacts in these area(s)

(check all that apply)

  • Product design
  • Backend (Database)
  • Backend (API)
  • Frontend

Describe the reason for change

When data within JSONB fields of the database would store large integers from other processes, when it came time to displaying these visually it would be done with loss of precision. The JS support for large integers is not very good being it only actually stores floats and has only 32bit support. Therefore to fix the display issues of these, I found that we were already performing a 2 stage response with all api calls from datamanager, therefore I added a method to replace large integers that are present in the textual data as the reviver function to JSON.parse running. This solves the issue and ultimately ensures we do not modify unknowingly data stored on the server while still retaining the visual accuracy for consumption in the UI.

Before
image

After
image

What alternative approaches were there?

  • Support BigInt in JS.
    • Problem here is it doesn't actually solve the original problem.
    • Can be explicitly supported in addition to this solution but this requires that all other Number values be converted to BigInt to be able to perform mathematical operations on them.
    • Many caveats to cover which ultimately the solution we need is to be as low risk to data corruption as possible.

What feature flags were used to cover this change?

Don't have one yet, will entertain one if we feel it necessary.

Does this PR introduce a breaking change?

(check only one)

  • Yes, and covered entirely by feature flag(s)
  • Yes, and covered partially by feature flag(s)
  • No
  • Not sure (briefly explain the situation below)

What level of testing was included in the change?

(check all that apply)

  • e2e
  • integration
  • unit

Which logical domain(s) does this change affect?

DataManager API proxy.

…I responses before parsing as JSON to retain precision.
@github-actions github-actions bot added the fix label Jul 19, 2024
Copy link

netlify bot commented Jul 19, 2024

Deploy Preview for label-studio-docs-new-theme canceled.

Name Link
🔨 Latest commit d757726
🔍 Latest deploy log https://app.netlify.com/sites/label-studio-docs-new-theme/deploys/669e67589157c5000877e541

Copy link

netlify bot commented Jul 19, 2024

Deploy Preview for heartex-docs canceled.

Name Link
🔨 Latest commit d757726
🔍 Latest deploy log https://app.netlify.com/sites/heartex-docs/deploys/669e67583d458c00083ad584

bmartel added 3 commits July 19, 2024 13:27
…one to perform a targeting deserialization such as this.
… NodeJS implementation not providing context third parameter in tests.
@yyassi-heartex
Copy link
Contributor

yyassi-heartex commented Jul 19, 2024

/selenium test

'/selenium' is an unknown pull-request command.
See '/help'

@bmartel bmartel enabled auto-merge (squash) July 22, 2024 14:10
@bmartel bmartel merged commit f323e6a into develop Jul 22, 2024
31 checks passed
@hlomzik hlomzik deleted the fb-optic-683 branch July 22, 2024 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants