[Go] Marshaling `int64` or `unit64` arrays to JSON can cause loss of data

[erezrokah]

Describe the bug, including details regarding any error messages, version, and platform.

When marshaling array.Int64 or array.Uint64 to JSON the original int64 or unit64 values can be lost.
The cause is that the values are casted to float64 before marshaling, and that type is not big enough to hold all possible int64 or unit64 values.

Relevant code:

arrow/go/arrow/array/numeric.gen.go

Line 104 in 4ec231b

vals[i] = float64(a.values[i]) // prevent uint8 from being seen as binary data

arrow/go/arrow/array/numeric.gen.go

Line 199 in 4ec231b

vals[i] = float64(a.values[i]) // prevent uint8 from being seen as binary data

Adding the following tests to numeric_test.go result in failures:

func TestInt64MarshalJSON(t *testing.T) {
	pool := memory.NewCheckedAllocator(memory.NewGoAllocator())
	defer pool.AssertSize(t, 0)

	var (
		vs = []int64{-5474557666971701248}
	)

	b := array.NewInt64Builder(pool)
	defer b.Release()

	for _, v := range vs {
		b.Append(v)
	}

	arr := b.NewArray().(*array.Int64)
	defer arr.Release()

	jsonBytes, err := json.Marshal(arr)
	if err != nil {
		t.Fatal(err)
	}
	got := string(jsonBytes)
	want := `[-5474557666971701248]`
	if got != want {
		t.Fatalf("got=%s, want=%s", got, want)
	}
}

func TestUInt64MarshalJSON(t *testing.T) {
	pool := memory.NewCheckedAllocator(memory.NewGoAllocator())
	defer pool.AssertSize(t, 0)

	var (
		vs = []uint64{14697929703826477056}
	)

	b := array.NewUint64Builder(pool)
	defer b.Release()

	for _, v := range vs {
		b.Append(v)
	}

	arr := b.NewArray().(*array.Uint64)
	defer arr.Release()

	jsonBytes, err := json.Marshal(arr)
	if err != nil {
		t.Fatal(err)
	}
	got := string(jsonBytes)
	want := `[14697929703826477056]`
	if got != want {
		t.Fatalf("got=%s, want=%s", got, want)
	}
}

A possible solution is to only do the casting to float64 when the array is of unit8 like the comment says, or skip the casting for int64 and unit64 arrays

Component(s)

Go

[erezrokah]

take