Skip to content

Added support for BIGINT UNSIGNED type in MySQL and MariaDB DBTypes #1735

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
AndreiKingsley wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Added support for BIGINT UNSIGNED type in MySQL and MariaDB DBTypes #1735

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
fb75764
add support for `BIGINT UNSIGNED` type in MySQL and MariaDB DBTypes
AndreiKingsley Mar 10, 2026
c3742a9
add tests for JDBC type mappings in MySQL and MariaDB
AndreiKingsley Mar 11, 2026
16e112e
add tests for `BIGINT UNSIGNED` in mysqlTest
AndreiKingsley Mar 12, 2026
8a0c1f8
MariaDB JDBC test fix SMALLINT
AndreiKingsley Mar 12, 2026
4072587
add tests for `BIGINT UNSIGNED` in MariaDB JDBC test
AndreiKingsley Mar 12, 2026
8867e07
Merge branch 'master' into fix_jdbc_types
AndreiKingsley Mar 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions dataframe-jdbc/src/main/kotlin/org/jetbrains/kotlinx/dataframe/io/db/MariaDb.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package org.jetbrains.kotlinx.dataframe.io.db

import java.math.BigInteger
import java.sql.ResultSet
import kotlin.reflect.KType
import kotlin.reflect.full.withNullability
Expand Down Expand Up @@ -34,6 +35,9 @@ public object MariaDb : DbType("mariadb") {
if (tableColumnMetadata.sqlTypeName == "SMALLINT" && tableColumnMetadata.javaClassName == "java.lang.Short") {
return typeOf<Short>().withNullability(tableColumnMetadata.isNullable)
}
if (tableColumnMetadata.sqlTypeName == "BIGINT UNSIGNED") {
return typeOf<BigInteger>().withNullability(tableColumnMetadata.isNullable)
}
return super.getExpectedJdbcType(tableColumnMetadata)
}

Expand Down
4 changes: 4 additions & 0 deletions dataframe-jdbc/src/main/kotlin/org/jetbrains/kotlinx/dataframe/io/db/MySql.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package org.jetbrains.kotlinx.dataframe.io.db

import java.math.BigInteger
import java.sql.ResultSet
import java.util.Locale
import kotlin.reflect.KType
Expand All @@ -20,6 +21,9 @@ public object MySql : DbType("mysql") {
if (tableColumnMetadata.sqlTypeName == "INT UNSIGNED") {
return typeOf<Long>().withNullability(tableColumnMetadata.isNullable)
}
if (tableColumnMetadata.sqlTypeName == "BIGINT UNSIGNED") {
return typeOf<BigInteger>().withNullability(tableColumnMetadata.isNullable)
}
return super.getExpectedJdbcType(tableColumnMetadata)
}

Expand Down
75 changes: 75 additions & 0 deletions dataframe-jdbc/src/test/kotlin/org/jetbrains/kotlinx/dataframe/io/db/jdbcTypesTest.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
@file:Suppress("ClassName")

package org.jetbrains.kotlinx.dataframe.io.db

import io.kotest.matchers.shouldBe
import org.junit.Test
import org.junit.experimental.runners.Enclosed
import org.junit.runner.RunWith
import java.math.BigInteger
import kotlin.reflect.KType
import kotlin.reflect.typeOf

// TODO: complete and enhance (#1736)
@RunWith(Enclosed::class)
class JdbcTypesTest {

abstract class ColumnType(
val sqlTypeName: String,
val jdbcType: Int,
val javaClassName: String,
val expectedKotlinType: KType,
) {
fun mockkColMetaData() =
Copy link
Copy Markdown
Collaborator

@Jolanrensen Jolanrensen Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since when does "mock" have 2 k's? ;P

Copy link
Copy Markdown
Collaborator

@Jolanrensen Jolanrensen Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh and "metadata" is one word, so it would become: "mockColMetadata()"

Copy link
Copy Markdown
Collaborator Author

@AndreiKingsley AndreiKingsley Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because I often use MockK variables usually called mockk.. 😆

TableColumnMetadata(
"name",
sqlTypeName,
jdbcType,
10,
javaClassName,
false,
Copy link
Copy Markdown
Collaborator

@Jolanrensen Jolanrensen Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm by setting this to false always, you might not catch the cases where someone forgets to take care about nullability (which is an easy mistake to make). Maybe we should test both isNullable and !isNullable. This could also be done in a next pr of course

)
}

class MariaDBTypes {

object BIGINT_UNSIGNED : ColumnType(
"BIGINT UNSIGNED",
Copy link
Copy Markdown
Collaborator

@Jolanrensen Jolanrensen Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend using named arguments. Makes refactoring easier :)

20,
"java.math.BigInteger",
typeOf<BigInteger>(),
)

val types: List<ColumnType> = listOf(
BIGINT_UNSIGNED,
)

@Test
fun `all MariaDB SQL types should match expected type`() {
types.forEach { type ->
MariaDb.getExpectedJdbcType(type.mockkColMetaData()) shouldBe type.expectedKotlinType
}
}
}

class MySqlDBTypes {

object BIGINT_UNSIGNED : ColumnType(
"BIGINT UNSIGNED",
20,
Copy link
Copy Markdown
Collaborator

@Jolanrensen Jolanrensen Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd use Sql.Types for this

Copy link
Copy Markdown
Collaborator

@Jolanrensen Jolanrensen Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, it doesn't exist?... interesting

Copy link
Copy Markdown
Collaborator Author

@AndreiKingsley AndreiKingsley Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, it's a MySQL/MariaDB specific type.

Copy link
Copy Markdown
Collaborator

@Jolanrensen Jolanrensen Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

classic

"java.math.BigInteger",
typeOf<BigInteger>(),
)

val types: List<ColumnType> = listOf(
BIGINT_UNSIGNED,
)

@Test
fun `all MariaDB SQL types should match expected type`() {
types.forEach { type ->
MySql.getExpectedJdbcType(type.mockkColMetaData()) shouldBe type.expectedKotlinType
}
}
}
}
30 changes: 22 additions & 8 deletions dataframe-jdbc/src/test/kotlin/org/jetbrains/kotlinx/dataframe/io/local/mariadbTest.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import org.junit.BeforeClass
import org.junit.Ignore
import org.junit.Test
import java.math.BigDecimal
import java.math.BigInteger
import java.sql.Blob
import java.sql.Connection
import java.sql.DriverManager
Expand Down Expand Up @@ -64,6 +65,7 @@ interface Table1MariaDb {
val longtextCol: String
val enumCol: String
val setCol: Char
val bigintUnsignedCol: BigInteger
val jsonCol: String
}

Expand Down Expand Up @@ -100,6 +102,7 @@ interface Table2MariaDb {
val longtextCol: String?
val enumCol: String?
val setCol: Char?
val bigintUnsignedCol: BigInteger?
val jsonCol: String?
}

Expand Down Expand Up @@ -176,6 +179,7 @@ class MariadbTest {
longtextCol LONGTEXT NOT NULL,
enumCol ENUM('Value1', 'Value2', 'Value3') NOT NULL,
setCol SET('Option1', 'Option2', 'Option3') NOT NULL,
bigintUnsignedCol BIGINT UNSIGNED NOT NULL,
jsonCol JSON NOT NULL
CHECK (JSON_VALID(jsonCol))
)
Expand Down Expand Up @@ -215,7 +219,8 @@ class MariadbTest {
mediumtextCol MEDIUMTEXT,
longtextCol LONGTEXT,
enumCol ENUM('Value1', 'Value2', 'Value3'),
setCol SET('Option1', 'Option2', 'Option3')
setCol SET('Option1', 'Option2', 'Option3'),
bigintUnsignedCol BIGINT UNSIGNED
)
"""
connection.createStatement().execute(createTableQuery2.trimIndent())
Expand All @@ -227,8 +232,8 @@ class MariadbTest {
bitCol, tinyintCol, smallintCol, mediumintCol, mediumintUnsignedCol, integerCol, intCol,
integerUnsignedCol, bigintCol, floatCol, doubleCol, decimalCol, dateCol, datetimeCol, timestampCol,
timeCol, yearCol, varcharCol, charCol, binaryCol, varbinaryCol, tinyblobCol, blobCol,
mediumblobCol, longblobCol, textCol, mediumtextCol, longtextCol, enumCol, setCol, jsonCol
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
mediumblobCol, longblobCol, textCol, mediumtextCol, longtextCol, enumCol, setCol, bigintUnsignedCol, jsonCol
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""".trimIndent()

@Language("SQL")
Expand All @@ -238,8 +243,8 @@ class MariadbTest {
bitCol, tinyintCol, smallintCol, mediumintCol, mediumintUnsignedCol, integerCol, intCol,
integerUnsignedCol, bigintCol, floatCol, doubleCol, decimalCol, dateCol, datetimeCol, timestampCol,
timeCol, yearCol, varcharCol, charCol, binaryCol, varbinaryCol, tinyblobCol, blobCol,
mediumblobCol, longblobCol, textCol, mediumtextCol, longtextCol, enumCol, setCol
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
mediumblobCol, longblobCol, textCol, mediumtextCol, longtextCol, enumCol, setCol, bigintUnsignedCol
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""".trimIndent()

connection.prepareStatement(insertData1).use { st ->
Expand Down Expand Up @@ -275,7 +280,8 @@ class MariadbTest {
st.setString(28, "longtextValue$i")
st.setString(29, "Value$i")
st.setString(30, "Option$i")
st.setString(31, JSON_STRING)
st.setObject(31, BigInteger.valueOf((i * 1000).toLong()))
st.setString(32, JSON_STRING)

st.executeUpdate()
}
Expand Down Expand Up @@ -314,6 +320,7 @@ class MariadbTest {
st.setString(28, "longtextValue$i")
st.setString(29, "Value$i")
st.setString(30, "Option$i")
st.setObject(31, BigInteger.valueOf((i * 2000).toLong()))
st.executeUpdate()
}
}
Expand Down Expand Up @@ -396,7 +403,8 @@ class MariadbTest {
table1Df.filter { "integerCol"<Int>() > 100 }.rowsCount() shouldBe 2
table1Df[0][11] shouldBe 10.0
table1Df[0][26] shouldBe "textValue1"
table1Df[0][31] shouldBe JSON_STRING // TODO: https://github.com/Kotlin/dataframe/issues/462
table1Df[0][31] shouldBe BigInteger.valueOf(1000L)
table1Df[0][32] shouldBe JSON_STRING // TODO: https://github.com/Kotlin/dataframe/issues/462

val table2Df = dataframes[1].cast<Table2MariaDb>()

Expand All @@ -418,7 +426,7 @@ class MariadbTest {
result[0][1] shouldBe 1

val result1 = df1.select("smallintCol")
.add("smallintCol2") { "smallintCol"<Int?>() }
.add("smallintCol2") { "smallintCol"<Short?>() }

result1[0][1] shouldBe 10

Expand All @@ -442,6 +450,11 @@ class MariadbTest {

result5[0][1] shouldBe 100

val result5a = df1.select("bigintUnsignedCol")
.add("bigintUnsignedCol2") { "bigintUnsignedCol"<BigInteger>() }

result5a[0][1] shouldBe BigInteger.valueOf(1000)

val result6 = df1.select("floatCol")
.add("floatCol2") { "floatCol"<Float>() }

Expand All @@ -465,6 +478,7 @@ class MariadbTest {
schema.columns["mediumintUnsignedCol"]!!.type shouldBe typeOf<Int>()
schema.columns["integerUnsignedCol"]!!.type shouldBe typeOf<Long>()
schema.columns["bigintCol"]!!.type shouldBe typeOf<Long>()
schema.columns["bigintUnsignedCol"]!!.type shouldBe typeOf<BigInteger>()
schema.columns["floatCol"]!!.type shouldBe typeOf<Float>()
schema.columns["doubleCol"]!!.type shouldBe typeOf<Double>()
schema.columns["decimalCol"]!!.type shouldBe typeOf<BigDecimal>()
Expand Down
25 changes: 19 additions & 6 deletions dataframe-jdbc/src/test/kotlin/org/jetbrains/kotlinx/dataframe/io/local/mysqlTest.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import org.junit.BeforeClass
import org.junit.Ignore
import org.junit.Test
import java.math.BigDecimal
import java.math.BigInteger
import java.sql.Connection
import java.sql.DriverManager
import java.sql.SQLException
Expand Down Expand Up @@ -65,6 +66,7 @@ interface Table1MySql {
val longtextCol: String
val enumCol: String
val setCol: Char
val bigintUnsignedCol: BigInteger
}

@DataSchema
Expand Down Expand Up @@ -100,6 +102,7 @@ interface Table2MySql {
val longtextCol: String?
val enumCol: String?
val setCol: Char?
val bigintUnsignedCol: BigInteger?
val jsonCol: String?
}

Expand Down Expand Up @@ -171,6 +174,7 @@ class MySqlTest {
longtextCol LONGTEXT NOT NULL,
enumCol ENUM('Value1', 'Value2', 'Value3') NOT NULL,
setCol SET('Option1', 'Option2', 'Option3') NOT NULL,
bigintUnsignedCol BIGINT UNSIGNED NOT NULL,
location GEOMETRY,
data JSON
CHECK (JSON_VALID(data))
Expand Down Expand Up @@ -213,6 +217,7 @@ class MySqlTest {
longtextCol LONGTEXT,
enumCol ENUM('Value1', 'Value2', 'Value3'),
setCol SET('Option1', 'Option2', 'Option3'),
bigintUnsignedCol BIGINT UNSIGNED,
location GEOMETRY,
data JSON
CHECK (JSON_VALID(data))
Expand All @@ -228,8 +233,8 @@ class MySqlTest {
bitCol, tinyintCol, smallintCol, mediumintCol, mediumintUnsignedCol, integerCol, intCol,
integerUnsignedCol, bigintCol, floatCol, doubleCol, decimalCol, dateCol, datetimeCol, timestampCol,
timeCol, yearCol, varcharCol, charCol, binaryCol, varbinaryCol, tinyblobCol, blobCol,
mediumblobCol, longblobCol, textCol, mediumtextCol, longtextCol, enumCol, setCol, location, data
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ST_GeomFromText('POINT(1 1)'), ?)
mediumblobCol, longblobCol, textCol, mediumtextCol, longtextCol, enumCol, setCol, bigintUnsignedCol, location, data
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ST_GeomFromText('POINT(1 1)'), ?)
""".trimIndent()

@Language("SQL")
Expand All @@ -239,8 +244,8 @@ class MySqlTest {
bitCol, tinyintCol, smallintCol, mediumintCol, mediumintUnsignedCol, integerCol, intCol,
integerUnsignedCol, bigintCol, floatCol, doubleCol, decimalCol, dateCol, datetimeCol, timestampCol,
timeCol, yearCol, varcharCol, charCol, binaryCol, varbinaryCol, tinyblobCol, blobCol,
mediumblobCol, longblobCol, textCol, mediumtextCol, longtextCol, enumCol, setCol, location, data
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ST_GeomFromText('POINT(1 1)'), ?)
mediumblobCol, longblobCol, textCol, mediumtextCol, longtextCol, enumCol, setCol, bigintUnsignedCol, location, data
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ST_GeomFromText('POINT(1 1)'), ?)
""".trimIndent()

connection.prepareStatement(insertData1).use { st ->
Expand Down Expand Up @@ -276,7 +281,8 @@ class MySqlTest {
st.setString(28, "longtextValue$i")
st.setString(29, "Value$i")
st.setString(30, "Option$i")
st.setString(31, "{\"key\": \"value\"}")
st.setObject(31, BigInteger.valueOf((i * 1000).toLong()))
st.setString(32, "{\"key\": \"value\"}")
st.executeUpdate()
}
}
Expand Down Expand Up @@ -314,7 +320,8 @@ class MySqlTest {
st.setString(28, "longtextValue$i")
st.setString(29, "Value$i")
st.setString(30, "Option$i")
st.setString(31, "{\"key\": \"value\"}")
st.setObject(31, BigInteger.valueOf((i * 2000).toLong()))
st.setString(32, "{\"key\": \"value\"}")
st.executeUpdate()
}
}
Expand Down Expand Up @@ -441,6 +448,11 @@ class MySqlTest {

result5[0][1] shouldBe 100

val result5a = df1.select("bigintUnsignedCol")
.add("bigintUnsignedCol2") { "bigintUnsignedCol"<BigInteger>() }

result5a[0][1] shouldBe BigInteger.valueOf(1000)

val result6 = df1.select("floatCol")
.add("floatCol2") { "floatCol"<Float>() }

Expand All @@ -464,6 +476,7 @@ class MySqlTest {
schema.columns["mediumintUnsignedCol"]!!.type shouldBe typeOf<Int>()
schema.columns["integerUnsignedCol"]!!.type shouldBe typeOf<Long>()
schema.columns["bigintCol"]!!.type shouldBe typeOf<Long>()
schema.columns["bigintUnsignedCol"]!!.type shouldBe typeOf<BigInteger>()
schema.columns["floatCol"]!!.type shouldBe typeOf<Float>()
schema.columns["doubleCol"]!!.type shouldBe typeOf<Double>()
schema.columns["decimalCol"]!!.type shouldBe typeOf<BigDecimal>()
Expand Down
Loading