E2E POC for python telemetry for connect logs#581

Merged

saishreeeee merged 269 commits intotelemetryfrom

PECOBLR-457

Jun 10, 2025

This pull request is big! We're only showing the most recent 250 commits

Commits on Jun 4, 2025

[ES-402013] Close cursors before closing connection (#38)

Jesse
authored and
saishreeeee
committed
Bump version to 2.0.5 and improve CHANGELOG (#40)

Jesse
authored and
saishreeeee
committed
fix dco issue

moderakh
authored and
saishreeeee
committed
fix dco issue

moderakh
authored and
saishreeeee
committed
dco tunning

moderakh
authored and
saishreeeee
committed
dco tunning

moderakh
authored and
saishreeeee
committed
Github workflows: run checks on pull requests from forks (#47)

Jesse
authored and
saishreeeee
committed
OAuth implementation (#15)

moderakh
authored and
saishreeeee
committed
Automate deploys to Pypi (#48)

Jesse
authored and
saishreeeee
committed
[PECO-205] Add functional examples (#52)

Jesse
authored and
saishreeeee
committed
Bump version to 2.1.0 (#54)

Jesse
authored and
saishreeeee
committed
[SC-110400] Enabling compression in Python SQL Connector (#49)

authored and
saishreeeee
committed
Add tests for parameter sanitisation / escaping (#46)

Jesse
authored and
saishreeeee
committed
Bump thrift dependency to 0.16.0 (#65)

Jesse
authored and
saishreeeee
committed
Bump version to 2.2.0 (#66)

Jesse
authored and
saishreeeee
committed
Support Python 3.11 (#60)

Jesse
authored and
saishreeeee
committed
Bump version to 2.2.1 (#70)

Jesse
authored and
saishreeeee
committed
Add none check on _oauth_persistence in DatabricksOAuthProvider (#71)

jackyhu-db
authored and
saishreeeee
committed
Support custom oauth client id and redirect port (#75)

jackyhu-db
authored and
saishreeeee
committed
Bump version to 2.2.2 (#76)

jackyhu-db
authored and
saishreeeee
committed
Merge staging ingestion into main (#78)

Jesse
authored and
saishreeeee
committed
Bump version to 2.3.0 and update changelog (#80)

Jesse
authored and
saishreeeee
committed
Add pkgutil-style for the package (#84)

lu-wang-dl
authored and
saishreeeee
committed
Add SQLAlchemy Dialect (#57)

Jesse
authored and
saishreeeee
committed
Bump to version 2.4.0(#89)

Jesse
authored and
saishreeeee
committed
Fix syntax in examples in root readme. (#92)

shea-parkes
authored and
saishreeeee
committed
Less strict numpy and pyarrow dependencies (#90)

authored and
saishreeeee
committed
Update example in docstring so query output is valid Spark SQL (#95)

Jesse
authored and
saishreeeee
committed
Bump version to 2.4.1 (#96)

Jesse
authored and
saishreeeee
committed
Update CODEOWNERS (#97)

moderakh
authored and
saishreeeee
committed
Add Andre to CODEOWNERS (#98)

yunbodeng-db
authored and
saishreeeee
committed
Add external auth provider + example (#101)

authored and
saishreeeee
committed
Retry on connection timeout (#103)

andrefurlan-db
authored and
saishreeeee
committed
[PECO-244] Make http proxies work (#81)

Jesse
authored and
saishreeeee
committed
Bump to version 2.5.0 (#104)

Jesse
authored and
saishreeeee
committed
Fix changelog release date for version 2.5.0

Jesse
authored and
saishreeeee
committed
Relax sqlalchemy requirement (#113)

Jesse
authored and
saishreeeee
committed
Update to version 2.5.1 (#114)

Jesse
authored and
saishreeeee
committed
Fix SQLAlchemy timestamp converter + docs (#117)

Jesse
authored and
saishreeeee
committed
Relax pandas and alembic requirements (#119)

Jesse
authored and
saishreeeee
committed
Bump to version 2.5.2 (#118)

Jesse
authored and
saishreeeee
committed
Use urllib3 for thrift transport + reuse http connections (#131)

Jesse
authored and
saishreeeee
committed
Default socket timeout to 15 min (#137)

mattdeekay
authored and
saishreeeee
committed
Bump version to 2.6.0 (#139)

Jesse
authored and
saishreeeee
committed
Fix: some thrift RPCs failed with BadStatusLine (#141)

Jesse
authored and
saishreeeee
committed
Bump version to 2.6.1 (#142)

Jesse
authored and
saishreeeee
committed
[ES-706907] Retry GetOperationStatus for http errors (#145)

Jesse
authored and
saishreeeee
committed
Bump version to 2.6.2 (#147)

Jesse
authored and
saishreeeee
committed
[PECO-626] Support OAuth flow for Databricks Azure (#86)

jackyhu-db
authored and
saishreeeee
committed
Use a separate logger for unsafe thrift responses (#153)

Jesse
authored and
saishreeeee
committed
Improve e2e test development ergonomics (#155)

Jesse
authored and
saishreeeee
committed
Don't raise exception when closing a stale Thrift session (#159)

Jesse
authored and
saishreeeee
committed
Bump to version 2.7.0 (#161)

Jesse
authored and
saishreeeee
committed
Cloud Fetch download handler (#127)

mattdeekay
authored and
saishreeeee
committed
Cloud Fetch download manager (#146)

mattdeekay
authored and
saishreeeee
committed
Cloud fetch queue and integration (#151)

mattdeekay
authored and
saishreeeee
committed
Cloud Fetch e2e tests (#154)

mattdeekay
authored and
saishreeeee
committed
Update changelog for cloudfetch (#172)

mattdeekay
authored and
saishreeeee
committed
Improve sqlalchemy backward compatibility with 1.3.24 (#173)

Jesse
authored and
saishreeeee
committed
OAuth: don't override auth headers with contents of .netrc file (#122)

Jesse
authored and
saishreeeee
committed
Fix proxy connection pool creation (#158)

authored and
saishreeeee
committed
Relax pandas dependency constraint to allow ^2.0.0 (#164)

authored and
saishreeeee
committed
Use hex string version of operation ID instead of bytes (#170)

Jesse
authored and
saishreeeee
committed
SQLAlchemy: fix has_table so it honours schema= argument (#174)

Jesse
authored and
saishreeeee
committed
Fix socket timeout test (#144)

authored and
saishreeeee
committed
Disable non_native_boolean_check_constraint (#120)

authored and
saishreeeee
committed
Remove unused import for SQLAlchemy 2 compatibility (#128)

authored and
saishreeeee
committed
Bump version to 2.8.0 (#178)

Jesse
authored and
saishreeeee
committed
Fix typo in python README quick start example (#186)

authored and
saishreeeee
committed
Configure autospec for mocked Client objects (#188)

Jesse
authored and
saishreeeee
committed
Use urllib3 for retries (#182)

Jesse
authored and
saishreeeee
committed
Bump version to 2.9.0 (#189)

Jesse
authored and
saishreeeee
committed
Explicitly add urllib3 dependency (#191)

authored and
saishreeeee
committed
Bump to 2.9.1 (#195)

Jesse
authored and
saishreeeee
committed
Make backwards compatible with urllib3~=1.0 (#197)

Jesse
authored and
saishreeeee
committed
Convenience improvements to v3 retry logic (#199)

Jesse
authored and
saishreeeee
committed
Bump version to 2.9.2 (#201)

Jesse
authored and
saishreeeee
committed
Github Actions Fix: poetry install fails for python 3.7 tests (#208)

Jesse
authored and
saishreeeee
committed
Make backwards compatible with urllib3~=1.0 [Follow up #197] (#206)

Jesse
authored and
saishreeeee
committed
Bump version to 2.9.3 (#209)

Jesse
authored and
saishreeeee
committed
Add note to sqlalchemy example: IDENTITY isn't supported yet (#212)

Jesse
authored and
saishreeeee
committed
[PECO-1029] Updated thrift compiler version (#216)

nithinkdb
authored and
saishreeeee
committed
[PECO-1055] Updated thrift defs to allow Tsparkparameters (#220)

nithinkdb
authored and
saishreeeee
committed
Update changelog to indicate that 2.9.1 and 2.9.2 have been yanked. (#222)

Jesse
authored and
saishreeeee
committed
Fix changelog typo: _enable_v3_retries (#225)

Jesse
authored and
saishreeeee
committed
Introduce SQLAlchemy reusable dialog tests (#125)

unj1m
authored and
saishreeeee
committed
[PECO-1026] Add Parameterized Query support to Python (#217)

nithinkdb
authored and
saishreeeee
committed
Parameterized queries: Add e2e tests for inference (#227)

Jesse
authored and
saishreeeee
committed
[PECO-1109] Parameterized Query: add suport for inferring decimal types (#228)

Jesse
authored and
saishreeeee
committed
SQLAlchemy 2: reorganise dialect files into a single directory (#231)

Jesse
authored and
saishreeeee
committed
[PECO-1083] Updated thrift files and added check for protocol version (#229)

authored and
saishreeeee
committed
[PECO-840] Port staging ingestion behaviour to new UC Volumes (#235)

Jesse
authored and
saishreeeee
committed
Query parameters: implement support for binding NoneType parameters (#233)

Jesse
authored and
saishreeeee
committed
SQLAlchemy 2: Bump dependency version and update e2e tests for existing behaviour (#236)

Jesse
authored and
saishreeeee
committed
Revert "[PECO-1083] Updated thrift files and added check for protocol version" (#237)

Jesse
authored and
saishreeeee
committed
SQLAlchemy 2: add type compilation for all CamelCase types (#238)

Jesse
authored and
saishreeeee
committed
SQLAlchemy 2: add type compilation for uppercase types (#240)

Jesse
authored and
saishreeeee
committed
SQLAlchemy 2: Stop skipping all type tests (#242)

Jesse
authored and
saishreeeee
committed
[PECO-1134] v3 Retries: allow users to bound the number of redirects to follow (#244)

Jesse
authored and
saishreeeee
committed
Parameters: Add type inference for BIGINT and TINYINT types (#246)

Jesse
authored and
saishreeeee
committed
SQLAlchemy 2: Stop skipping some non-type tests (#247)

Jesse
authored and
saishreeeee
committed
SQLAlchemy 2: implement and refactor schema reflection methods (#249)

Jesse
authored and
saishreeeee
committed
Add GovCloud domain into AWS domains (#252)

jackyhu-db
authored and
saishreeeee
committed
SQLAlchemy 2: Refactor __init__.py into base.py (#250)

Jesse
authored and
saishreeeee
committed
SQLAlchemy 2: Finish implementing all of ComponentReflectionTest (#251)

Jesse
authored and
saishreeeee
committed
SQLAlchemy 2: Finish marking all tests in the suite (#253)

Jesse
authored and
saishreeeee
committed
SQLAlchemy 2: Finish organising compliance test suite (#256)

Jesse
authored and
saishreeeee
committed
SQLAlchemy 2: Fix failing mypy checks from development (#257)

Jesse
authored and
saishreeeee
committed
Enable cloud fetch by default (#258)

Jesse
authored and
saishreeeee
committed
[PECO-1137] Reintroduce protocol checking to Python test fw (#248)

authored and
saishreeeee
committed
sqla2 clean-up: make sqlalchemy optional and don't mangle the user-agent (#264)

Jesse
authored and
saishreeeee
committed
SQLAlchemy 2: Add support for TINYINT (#265)

Jesse
authored and
saishreeeee
committed
Add OAuth M2M example (#266)

jackyhu-db
authored and
saishreeeee
committed
Native Parameters: reintroduce INLINE approach with tests (#267)

Jesse
authored and
saishreeeee
committed
Document behaviour of executemany (#213)

martinitus
authored and
saishreeeee
committed
SQLAlchemy 2: Expose TIMESTAMP and TIMESTAMP_NTZ types to users (#268)

Jesse
authored and
saishreeeee
committed
Drop Python 3.7 as a supported version (#270)

Jesse
authored and
saishreeeee
committed
GH Workflows: remove Python 3.7 from the matrix for _all_ workflows (#274)

Jesse
authored and
saishreeeee
committed
Add README and updated example for SQLAlchemy usage (#273)

Jesse
authored and
saishreeeee
committed
Rewrite native parameter implementation with docs and tests (#281)

Jesse
authored and
saishreeeee
committed
Enable v3 retries by default (#282)

Jesse
authored and
saishreeeee
committed
security: bump pyarrow dependency to 14.0.1 (#284)

Jesse
authored and
saishreeeee
committed
Bump package version to 3.0.0 (#285)

Jesse
authored and
saishreeeee
committed
Fix docstring about default parameter approach (#287)

Falydoor
authored and
saishreeeee
committed
[PECO-1286] Add tests for complex types in query results (#293)

Jesse
authored and
saishreeeee
committed
sqlalchemy: fix deprecation warning for dbapi classmethod (#294)

Jesse
authored and
saishreeeee
committed
[PECO-1297] sqlalchemy: fix: can't read columns for tables containing a TIMESTAMP_NTZ column (#296)

Jesse
authored and
saishreeeee
committed
Prepared 3.0.1 release (#297)

Jesse
authored and
saishreeeee
committed
Make contents of `__init__.py` equal across projects (#304)

authored and
saishreeeee
committed
Fix URI construction in ThriftBackend (#303)

authored and
saishreeeee
committed
[sqlalchemy] Add table and column comment support (#329)

Jesse
authored and
saishreeeee
committed
Pin pandas and urllib3 versions to fix runtime issues in dbt-databricks (#330)

authored and
saishreeeee
committed
SQLAlchemy: TINYINT types didn't reflect properly (#315)

TimTheinAtTabs
authored and
saishreeeee
committed
[PECO-1435] Restore `tests.py` to the test suite (#331)

Jesse
authored and
saishreeeee
committed
Bump to version 3.0.2 (#335)

Jesse
authored and
saishreeeee
committed
Update some outdated OAuth comments (#339)

authored and
saishreeeee
committed
Redact the URL query parameters from the urllib3.connectionpool logs (#341)

mkazia-db
authored and
saishreeeee
committed
Bump to version 3.0.3 (#344)

jackyhu-db
authored and
saishreeeee
committed
[PECO-1411] Support Databricks OAuth on GCP (#338)

authored and
saishreeeee
committed
[PECO-1414] Support Databricks native OAuth in Azure (#351)

jackyhu-db
authored and
saishreeeee
committed
Prep for Test Automation (#352)

benc-db
authored and
saishreeeee
committed
Update code owners (#345)

yunbodeng-db
authored and
saishreeeee
committed
Reverting retry behavior on 429s/503s to how it worked in 2.9.3 (#349)

benc-db
authored and
saishreeeee
committed
Bump to version 3.1.0 (#358)

jackyhu-db
authored and
saishreeeee
committed
[PECO-1440] Expose current query id on cursor object (#364)

kravets-levko
authored and
saishreeeee
committed
Add a default for retry after (#371)

benc-db
authored and
saishreeeee
committed
Fix boolean literals (#357)

aholyoke
authored and
saishreeeee
committed
Don't retry network requests that fail with code 403 (#373)

Jesse
authored and
saishreeeee
committed
Bump to 3.1.1 (#374)

benc-db
authored and
saishreeeee
committed
Fix cookie setting (#379)

benc-db
authored and
saishreeeee
committed
Fixing a couple type problems: how I would address most of #381 (#382)

wyattscarpenter
authored and
saishreeeee
committed
fix the return types of the classes' __enter__ functions (#384)

wyattscarpenter
authored and
saishreeeee
committed
Add Kravets Levko to codeowners (#386)

kravets-levko
authored and
saishreeeee
committed
Prepare for 3.1.2 (#387)

benc-db
authored and
saishreeeee
committed
Update the proxy authentication (#354)

amir-haroun
authored and
saishreeeee
committed
Fix failing tests (#392)

kravets-levko
authored and
saishreeeee
committed
Relax `pyarrow` pin (#389)

dhirschfeld
authored and
saishreeeee
committed
Fix log error in oauth.py (#269)

authored and
saishreeeee
committed
Enable `delta.feature.allowColumnDefaults` for all tables (#343)

authored and
saishreeeee
committed
Fix SQLAlchemy tests (#393)

kravets-levko
authored and
saishreeeee
committed
Add more debug logging for CloudFetch (#395)

kravets-levko
authored and
saishreeeee
committed
Update Thrift package (#397)

m1n0
authored and
saishreeeee
committed
Prepare release 3.2.0 (#396)

kravets-levko
authored and
saishreeeee
committed
move py.typed to correct places (#403)

authored and
saishreeeee
committed
Upgrade mypy (#406)

wyattscarpenter
authored and
saishreeeee
committed
Do not retry failing requests with status code 401 (#408)

Hodnebo
authored and
saishreeeee
committed
[PECO-1715] Remove username/password (BasicAuth) auth option (#409)

jackyhu-db
authored and
saishreeeee
committed
[PECO-1751] Refactor CloudFetch downloader: handle files sequentially (#405)

kravets-levko
authored and
saishreeeee
committed
Fix CloudFetch retry policy to be compatible with all `urllib3` versions we support (#412)

kravets-levko
authored and
saishreeeee
committed
Disable SSL verification for CloudFetch links (#414)

kravets-levko
authored and
saishreeeee
committed
Prepare relese 3.3.0 (#415)

kravets-levko
authored and
saishreeeee
committed
Fix pandas 2.2.2 support (#416)

authored and
saishreeeee
committed
[PECO-1801] Make OAuth as the default authenticator if no authentication setting is provided (#419)

jackyhu-db
authored and
saishreeeee
committed
[PECO-1857] Use SSL options with HTTPS connection pool (#425)

kravets-levko
authored and
saishreeeee
committed
Prepare release v3.4.0 (#430)

kravets-levko
authored and
saishreeeee
committed
[PECO-1926] Create a non pyarrow flow to handle small results for the column set (#440)

jprakash-db
authored and
saishreeeee
committed
[PECO-1961] On non-retryable error, ensure PySQL includes useful information in error (#447)

shivam2680
authored and
saishreeeee
committed
Reformatted all the files using black (#448)

jprakash-db
authored and
saishreeeee
committed
Prepare release v3.5.0 (#457)

jackyhu-db
authored and
saishreeeee
committed
[PECO-2051] Add custom auth headers into cloud fetch request (#460)

jackyhu-db
authored and
saishreeeee
committed
Prepare release 3.6.0 (#461)

jackyhu-db
authored and
saishreeeee
committed
[ PECO - 1768 ] PySQL: adjust HTTP retry logic to align with Go and Nodejs drivers (#467)

jprakash-db
authored and
saishreeeee
committed
[ PECO-2065 ] Create the async execution flow for the PySQL Connector (#463)

jprakash-db
authored and
saishreeeee
committed
Fix for check_types github action failing (#472)

jprakash-db
authored and
saishreeeee
committed
Remove upper caps on dependencies (#452)

arredond
authored and
saishreeeee
committed
Updated the doc to specify native parameters in PUT operation is not supported from >=3.x connector (#477)

jprakash-db
authored and
saishreeeee
committed
Incorrect rows in inline fetch result (#479)

jprakash-db
authored and
saishreeeee
committed
Bumped up to version 3.7.0 (#482)

jprakash-db
authored and
saishreeeee
committed
PySQL Connector split into connector and sqlalchemy (#444)

authored and
saishreeeee
committed
Removed CI CD for python3.8 (#490)

jprakash-db
authored and
saishreeeee
committed
Added CI CD upto python 3.12 (#491)

jprakash-db
authored and
saishreeeee
committed
Merging changes from v3.7.1 release (#488)

jprakash-db
authored and
saishreeeee
committed
Bumped up to version 4.0.0 (#493)

jprakash-db
authored and
saishreeeee
committed
Updated action's version (#455)

newwingbird
authored and
saishreeeee
committed
Support Python 3.13 and update deps (#510)

authored and
saishreeeee
committed
Improve debugging + fix PR review template (#514)

samikshya-db
authored and
saishreeeee
committed
Forward porting all changes into 4.x.x. uptil v3.7.3 (#529)

jprakash-db
authored and
saishreeeee
committed
Updated the CODEOWNERS (#531)

jprakash-db
authored and
saishreeeee
committed
Add version check for urllib3 in backoff calculation (#526)

shivam2680
authored and
saishreeeee
committed
[ES-1372353] make user_agent_header part of public API (#530)

shivam2680
authored and
saishreeeee
committed
Updates runner used to run DCO check to use databricks-protected-runner (#521)

madhav-db
authored and
saishreeeee
committed
Support multiple timestamp formats in non arrow flow (#533)

jprakash-db
authored and
saishreeeee
committed
prepare release for v4.0.1 (#534)

shivam2680
authored and
saishreeeee
committed
Relaxed bound for python-dateutil (#538)

jprakash-db
authored and
saishreeeee
committed
Bumped up the version for 4.0.2 (#539)

jprakash-db
authored and
saishreeeee
committed
Added example for async execute query (#537)

jprakash-db
authored and
saishreeeee
committed
Added urllib3 version check (#547)

jprakash-db
authored and
saishreeeee
committed
Bump version to 4.0.3 (#549)

jprakash-db
authored and
saishreeeee
committed
Cleanup fields as they might be deprecated/removed/change in the future (#553)

vikrantpuppala
authored and
saishreeeee
committed
Refactor decimal conversion in PyArrow tables to use direct casting (#544)

jayantsing-db
authored and
saishreeeee
committed
[PECOBLR-361] convert column table to arrow if arrow present (#551)

shivam2680
authored and
saishreeeee
committed
Update CODEOWNERS (#562)

jprakash-db
authored and
saishreeeee
committed
Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554)

madhav-db
authored and
saishreeeee
committed
PECOBLR-86 improve logging on python driver (#556)
saishreeeee
committed
Update github actions run conditions (#569)

jprakash-db
authored and
saishreeeee
committed
New Complex type test table + Github Action changes (#575)

jprakash-db
authored and
saishreeeee
committed
Added classes required for telemetry (#572)

committed
telemetry client and logs
saishreeeee
committed
fixed error args
saishreeeee
committed
fixed type of client_app_name
saishreeeee
committed
formatting
saishreeeee
committed
TelemetryManager
saishreeeee
committed
fixed statement_id in latency_logger
saishreeeee
committed
updated latency_logger
saishreeeee
committed
only added initial telemetry logs
saishreeeee
committed
removed error logs logic
saishreeeee
committed
removed user logging for telemetry errors
saishreeeee
committed
removed no-op telemetry client
saishreeeee
committed
changed order of args in telemetry client
saishreeeee
committed
shifted thread pool executor to telemetry manager
saishreeeee
committed
formatting
saishreeeee
committed
Revert "shifted thread pool executor to telemetry manager"
saishreeeee
committed
telemetry client changes
saishreeeee
committed
Merge remote-tracking branch 'origin/telemetry' into PECOBLR-457
saishreeeee
committed
fixed spelling error
saishreeeee
committed

Commits on Jun 5, 2025

Commits on Jun 8, 2025

removed discovery_url
saishreeeee
committed

E2E POC for python telemetry for connect logs#581saishreeeee merged 269 commits intotelemetrydatabricks/databricks-sql-python:telemetryfrom PECOBLR-457databricks/databricks-sql-python:PECOBLR-457Copy head branch name to clipboard

Commits

Commits on Jun 4, 2025

E2E POC for python telemetry for connect logs#581
saishreeeee merged 269 commits intotelemetryfrom
PECOBLR-457