Add schema caching and manual config for inserts#256
Add schema caching and manual config for inserts#256alex-clickhouse wants to merge 6 commits intomainfrom
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Pull request overview
This PR adds schema-resolution optimizations for ClickHouseClient.InsertBinaryAsync by introducing a reusable SchemaResolver with two new InsertOptions knobs: supplying column types explicitly (skips schema probing entirely) and caching a table schema per client instance (avoids repeated probe round-trips). It also updates docs and examples, and adds tests to validate the new behavior.
Changes:
- Add
InsertOptions.ColumnTypes(manual schema) andInsertOptions.UseSchemaCache(per-client schema caching) forInsertBinaryAsync. - Refactor schema discovery/probing logic into a new internal
SchemaResolver. - Add an example + NUnit tests, and document the feature in release notes/changelog.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| examples/README.md | Adds a link to the new schema optimization insert example. |
| examples/Program.cs | Runs the new SchemaOptimization example in the example runner. |
| examples/Insert/Insert_008_SchemaOptimization.cs | Demonstrates ColumnTypes and UseSchemaCache usage for InsertBinaryAsync. |
| ClickHouse.Driver/Utility/SchemaResolver.cs | New schema resolution component with manual schema + cache strategies. |
| ClickHouse.Driver/InsertOptions.cs | Adds ColumnTypes and UseSchemaCache options and propagates them through WithQueryId. |
| ClickHouse.Driver/ClickHouseClient.cs | Replaces inline schema probing with SchemaResolver.ResolveAsync. |
| ClickHouse.Driver.Tests/InsertBinarySchemaTests.cs | Adds tests for manual schema and schema cache behavior. |
| RELEASENOTES.md | Documents the new insert schema options for v1.1.0. |
| CHANGELOG.md | Mirrors release note entries for the new insert schema options for v1.1.0. |
You can also share your feedback on Copilot code review. Take the survey.
There was a problem hiding this comment.
Pull request overview
Adds insert-schema optimization capabilities to ClickHouseClient.InsertBinaryAsync by introducing a dedicated schema resolution component, plus new InsertOptions knobs to either provide schema manually or cache resolved schema across inserts. Includes new example usage and automated tests to validate query-count behavior and correctness.
Changes:
- Introduce
SchemaResolverto centralize insert schema discovery, manual schema usage, and optional per-client schema caching. - Add
InsertOptions.ColumnTypes(manual schema) andInsertOptions.UseSchemaCache(cached schema) and wire them intoInsertBinaryAsync. - Add an examples entry + new insert example, and add a comprehensive NUnit test suite covering the new behavior.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| examples/README.md | Adds the new schema optimization insert example to the examples index. |
| examples/Program.cs | Runs the new schema optimization example as part of the insert examples sequence. |
| examples/Insert/Insert_008_SchemaOptimization.cs | Demonstrates using ColumnTypes and UseSchemaCache with InsertBinaryAsync. |
| RELEASENOTES.md | Documents the new insert schema optimization options for the upcoming release. |
| CHANGELOG.md | Mirrors release notes for the new schema optimization options. |
| ClickHouse.Driver/Utility/SchemaResolver.cs | New internal component implementing schema resolution + caching strategy. |
| ClickHouse.Driver/InsertOptions.cs | Adds new public insert options (ColumnTypes, UseSchemaCache) and clones them in WithQueryId. |
| ClickHouse.Driver/ClickHouseClient.cs | Integrates SchemaResolver into InsertBinaryAsync and removes inline schema probing helper. |
| ClickHouse.Driver.Tests/InsertBinarySchemaTests.cs | Adds tests ensuring schema probing is skipped/cached as expected and data round-trips correctly. |
You can also share your feedback on Copilot code review. Take the survey.
Adds two options for avoiding the schema query for inserts:
Manual schema takes priority over auto-discovered, if it's provided.
The latter option also requires the column list to be provided, because we don't know the ordering otherwise. But I don't think anyone relies on the db ordering of columns anyway (and they shouldn't), so I don't think it matters that much.
All this schema stuff is moved to its own class.
Also adds a new example showing how to use this functionality.