Do not know how to serialize a bigint

TypeError: Do not know how to serialize a BigInt #816

Comments

kakserpom commented Dec 26, 2021

The text was updated successfully, but these errors were encountered:

sid405 commented Dec 28, 2021

Hey @kakserpom, this was fixed in Prisma 3.7.0. If you update your Prisma version (or your electron app), this error should go away. Let me know if that’s not the case, and I’ll reopen!

kakserpom commented Dec 28, 2021

sid405 commented Jan 6, 2022

Hey, could you try out the latest Prisma CLI and let me know if this is fixed on there?

I haven’t released a new version of the Electron app yet, but I will do that with the next release (this Tuesday). If you could confirm that this is fixed for you on the CLI, then it’ll be fixed in the Electron app as well (they use the same code).

aetheryx commented Jan 11, 2022 •

@madebysid I am still seeing this using the Prisma CLI on Prisma v3.8.0:

sid405 commented Jan 11, 2022

Hmm, I can’t seem to reproduce this, what are you doing exactly that causes this?

Here’s what I did:

aetheryx commented Jan 11, 2022 •

sid405 commented Jan 11, 2022

Ah, I see it now as well, weird. It only seems to happen when the ID is a BigInt. Nevertheless, I’ll look into this, thanks for the repro!

thojansen commented Apr 22, 2022

I ran into the same issue and created my own prisma studio launch script by using the studio-server. It almost works, but I now get a Value must be a valid BigInt error. So I guess simply adding the toJSON and fromJSON will not do the trick completely.

I think it would be awesome if you could open source prisma studio at some point.

kyranet commented May 18, 2022

Ah, I see it now as well, weird. It only seems to happen when the ID is a BigInt. Nevertheless, I’ll look into this, thanks for the repro!

Not entirely true, consider this schema:

The studio is able to render them just fine:

However, this one schema does not:

And will result on a non-recoverable error:

Consider mapping BigInt to String for JSON stringify #146

Comments

Eccenux commented Apr 30, 2018

Description

Current implementation (in Chrome) throws on BigInt strinigify:

I think it would make sens if a string mapping would be provided (similar to Date mapping provided with date.toISOString() method).

Example

This is an example and that currently works:

It produces a valid(!) JSON:

A parser could then be provided to recognize and parse BigInt if required.

The text was updated successfully, but these errors were encountered:

bakkot commented Apr 30, 2018

See discussion in #24.

I don’t think outputting a string is a good default behavior here, since it would not (and cannot) parse back to a BigInt. You can of course trivially make stringify do the thing you want already, as you’ve pointed out.

Eccenux commented Apr 30, 2018

Oh. Didn’t realize implementing toJSON on BigInt is already supported. Never mind then :-). And thanks.

kaizhu256 commented Jun 7, 2018

@bakkot, there’s prior art for allowing JSON.stringify (to string mapping), but not JSON.parse with Date-objects.

figuring out how to idiot-proof serializing and passing-around 64bit integers to-and-fro
browser server databases
is a big enough industry-painpoint in my mind, that BigInt should try to address it in some form.

JavaScript BigInt and JSON

Published: January 04, 2019 • javascript, java

In my previous blog post, I took a closer look at the new numeric type BigInt in JavaScript

Problem: JSON

I wrote in my previous blog post that JSON does not support BigInt. If you try to serialize and deserialize an object with BigInt values in it, the methods JSON.stringify() and JSON.parse() throw errors:

Fortunately, there is a way to work around this problem by converting BigInt to string and back. I already showed you a JavaScript solution for this in my previous blog post. I will show you this solution again a bit further below, but first, we look at the problems that a BigInt data type tries to solve.

Problem: Maximum values

In this example, we use a Spring Boot RestController with a GET handler that creates a POJO with three fields and sends it back to the caller as JSON.

The Payload POJO consists of three fields. All data types can store potentially larger values than the JavaScript number type can hold.

From JavaScript, we call the service with the Fetch API.

We get back this response.

This is not a bug. It’s just how the number data type is implemented in JavaScript. See the specification for more information.

If we call this endpoint from JavaScript, we get back the following response:

There are similar problems when sending objects from a browser to the back end.

From JavaScript, we send an object with a number that is bigger than Number.MAX_SAFE_INTEGER

On the server, the POST handler prints out this information.

Same error as before. 2305843009213693951 can’t be represented precisely in the number data type. JSON.stringify() therefore writes 2305843009213694000 into the JSON and sends that over the wire to the back end.

The JSON.stringify() method handles this case by writing null into the JSON.

Solution: Sending BigInt as a string in JSON

As a solution for the missing BigInt support in JSON, we convert the BigInts to strings and then convert them back to a number type. We do this in JavaScript and in the Spring Boot back end application.

Server: Serialization

To customize the serialization process in Jackson, we have to write subclasses of the com.fasterxml.jackson.databind.JsonSerializer class and override the serialize method.

Next, we need to customize the Jackson JSON parser. In Spring Boot, we can do this in two ways. Either globally or locally on the field level.

We also annotate this class with @Component to denote this class as a Spring-managed bean. Spring Boot automatically picks up this class and configures the ObjectMapper instance used for JSON serialization.

When we call our two endpoints from the previous section, we get back these JSON responses.

With this setup, we get the following JSON responses. Notice that value1 is serialized as a regular number because we didn’t add a serializer to that field.

Client: Deserialization

After setting up the server, we now receive strings in the form «2147483647n» that signify a BigInt value. If we call JSON.parse() on this JSON, we get back a string, but we want an object with a BigInt property.

The JSON.parse() method supports an optional second parameter called reviver. This is a function that is called for each key/value pair. Here we check if the value is a string, contains just numbers, and the letter n at the end. If that is the case, convert it to a BigInt.

Unfortunately, the json() method of the Fetch response object does not support the reviver argument, so we can’t use this pattern.

Client: Serialization

As we have seen in the first section, serializing an object with JSON.stringify() throws an error if it contains properties with BigInt values. Fortunately, this method also supports a second optional parameter called replacer. A function that is called for each key/value pair. In this replacer function, we check the data type, and when it’s a BigInt, convert it to a string and append the letter ‘n’.

The following code sends a POST request to our Spring Boot backend and passes the replacer function to the JSON.stringify() method.

This is the JSON the browser sends to the server.

Server: Deserialization

We may customize the deserialization process with the Jackson library by creating subclasses of the com.fasterxml.jackson.databind.JsonDeserializer class. I wrote a deserializer for BigInteger and Long and long for this example.

In these two implementations, I check for null and for empty strings. In these two cases, the deserializer returns null in all other cases it removes the last letter (‘n’) and tries to convert the string to a BigInteger, respectively, a Long.

You can configure the deserializers, like the serializers, globally or locally on a field-by-field basis.

Globally, you register the deserializers with this.addDeserializer() in a module class.

Alternatively, specify the deserializer with the @JsonDeserialize annotation on the field or the set method.

You get these values when the client sends the POST from the previous section. Notice the correct value for prime

jest-worker fails with «Do not know how to serialize a BigInt» in messageParent.ts instead of showing the actual assertion error message when there are 2 or more test suite files executing in parallel #11617

Comments

klesun commented Jun 28, 2021 •

Link to repl or repo (highly encouraged)

🐛 Bug Report

When you have multiple test suite files executing in parallel, and some of the tests are asserting values with bigint numbers in them and the assertion fails, instead of showing this assertion error message, jest-worker fails itself on attempt to serialise this value during the call to the:

To Reproduce

Steps to reproduce the behavior:

Expected behavior

You should have seen the assertion error informing you that expect(1n).toEqual(2n); expectation failed

Actual behavior

But instead you get following output due to an internal jest-worker error:

envinfo

There is likely a number of ways how to better address this problem.

The text was updated successfully, but these errors were encountered:

klesun commented Jun 28, 2021 •

@SimenB, since you changed this part of jest-worker not so long ago, maybe you could, please, give us an advice here for the discussion to move in the right direction?

klesun commented Jun 28, 2021

kunal-kushwaha commented Jun 28, 2021

sauravhiremath commented Jun 28, 2021

Will look into this tomorrow!

SHaTRO commented Sep 5, 2021

The choice to use JSON stock JSON serialization moving forward is not helpful. While Jest has been doing better and better with comparison support of native BigInt, the reporting of assertion failures (for me, OOB) still consistently reports an inability to serialize the bigint. With this configuration (OOB) of Jest, polyfills to fix this problem are inconsistent at best.

There are new serializers that can be configured and dropped in for the one way (or even two-way if you configure them correctly) representations for error reports, for example: json-bigint

Reversible BigInt Serialization

I recently had to do a bit of serialized JSON exchange between a client and server, and initially had some problems extending data with the new BigInt type, because the default JSON.stringify doesn’t like it, throwing this error:

After a little digging I found that it is very easy to implement your own version of JSON.stringify, one that takes care of BigInt values:

However, this creates irreversible JSON, i.e. if I pass it into JSON.parse in the client, then for each original BigInt I will get a string with a number in it, which even if I parse further, I cannot tell if it really was a BigInt.

It is a well-known problem, if you look around, there are discussions and some specification drafts suggesting a solution for this in the future.

But I ended up with a very simple solution that’s shown below, which works perfectly today, without having to wait for a better day…

Or alternatively, you can provide your own custom implementation for the officially supported toJSON type serializer for BigInt :

Above we simply append n at the end of each number, which makes it look like BigInt in JavaScript, and we can parse such format easily with RegEx.

And in order de-serialize such JSON, we can use the following parser:

It works the same as JSON.parse, while also returning BigInt for every string that complies with our 123n format.

And if you are concerned that such format as 123n may suddenly get into conflict with some random string, there are ways to avoid it…

The easiest one is to obfuscate the format a bit, like instead of 123n format, use something more verbose, like 123#bigint or similar, as long as serializer and de-serializer use the same format, it will work perfectly fine.

And to make it completely error-proof, you can count the number of serialized BigInt values, pass the count into the client, which then will count number of pattern matches, compare it to the counter, and throw error, if there is a discrepancy. But for most cases this is probably an overkill.