Extend inequal join by lgbo-ustc · Pull Request #28 · bigo-sg/ClickHouse

lgbo-ustc · 2024-05-23T07:48:22Z

Changelog category (leave one):

New Feature
Improvement
Performance Improvement
Backward Incompatible Change
Build/Testing/Packaging Improvement
Documentation (changelog entry is not required)
Bug Fix (user-visible misbehavior in an official stable release)
CI Fix or Improvement (changelog entry is not required)
Not for changelog (changelog entry is not required)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

...

Documentation entry for user-facing changes

Documentation is written (mandatory for new features)

Information about CI checks: https://clickhouse.com/docs/en/development/continuous-integration/

CI Settings

NOTE: If your merge the PR with modified CI you MUST KNOW what you are doing
NOTE: Checked options will be applied if set before CI RunConfig/PrepareRunConfig step

Run these jobs only (required builds will be added automatically):

Deny these jobs:

Extra options:

do not test (only style check)
disable merge-commit (no merge from master before tests)
disable CI cache (job reuse)

Only specified batches in multi-batch jobs:

1
2
3
4

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

Improve the dashboard

…n-in-settings-processlist Fix race condition between ProcessList and Settings

…tor-locks Fix locking inside TimerDescriptor

…-ubsan Fix strange code in HostResolvePool

Enable `text_log` by default

Fix a typo

Fix test retries

Sometimes startup can take ~90 seconds [1]: 2024-08-03 23:11:38.756067+00:00 [info] <0.9.0> Time to start RabbitMQ: 94651980 us Unlike normally ~10 [2]: 2024-08-04 00:33:07.016137+00:00 [info] <0.9.0> Time to start RabbitMQ: 10082489 us [1]: https://s3.amazonaws.com/clickhouse-test-reports/67737/b4e3bbcb82158bea4f5db1d9f5c28cfb741d1d51/integration_tests__asan__old_analyzer__[4_6].html [2]: https://s3.amazonaws.com/clickhouse-test-reports/66671/2f00c962711e13ca00af324366421fe4593b4ce6/integration_tests__asan__old_analyzer__[4_6].html I've tried locally, the difference is very small, 3135665 us (+management) vs (2740747 us), but still something, and who knows how it works under pressure. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

Update zlib-ng from 2.0.2 to 2.1.7

Update documentation of `toFloat32/64` and variants

run 01171 test in parallel

…g_fix_in_changleog CI: Changelog: Critical Bug Fix to Bug Fix

…rt-58927-add-test-47892 Revert "Revert "Add a test for ClickHouse#47892""

Introduce `no-flaky-check` tag

…eout CI: Strict job timeout 1.5h for tests, 2h for builds

…tainer Do not try to create azure container if not needed

enable parallel_view_processing in perf tests

…-image [RFC] Do not pass logs for keeper explicitly in the image to allow overriding

…or-async-insert-scatter-partition more logs to debug logical error from async inserts

…_index added tests for page index in parquet files

…cret-fix Fix cluster() for inter-server secret (preserve initial user as before)

…ary bridge cleanup <details> <summary>stacktrace</summary> #5 DB::ContextSharedMutex::lockSharedImpl (this=this@entry=0x7ffff5b96040) at /src/ch/clickhouse/src/Interpreters/Context.cpp:1036 watch = {start_ns = 98678735300141, stop_ns = 0, clock_type = 4, is_running = true} increment = {what = <optimized out>, amount = 1} #6 0x00005555699352d6 in DB::SharedMutexHelper<DB::ContextSharedMutex, DB::SharedMutex>::lock_shared (this=0x7ffff5b96040) at /src/ch/clickhouse/src/Common/SharedMutexHelper.h:70 #7 DB::SharedLockGuard<DB::ContextSharedMutex>::SharedLockGuard (mutex_=..., this=<optimized out>) at /src/ch/clickhouse/src/Common/SharedLockGuard.h:19 #8 DB::ContextSharedPart::getConfigRef (this=0x7ffff5b96000) at /src/ch/clickhouse/src/Interpreters/Context.cpp:760 #9 DB::Context::getConfigRef (this=<optimized out>) at /src/ch/clickhouse/src/Interpreters/Context.cpp:1590 #10 0x000055556450941c in DB::ProxyConfigurationResolverProvider::get (request_protocol=DB::ProxyConfiguration::Protocol::HTTP) at /src/ch/clickhouse/src/Common/ProxyConfigurationResolverProvider.cpp:161 #11 0x00005555669ea75f in DB::BuilderRWBufferFromHTTP::create (this=0x7fffffffa848, credentials_=...) at /src/ch/clickhouse/src/IO/ReadWriteBufferFromHTTP.cpp:814 #12 0x000055556cea0035 in DB::ExternalDictionaryLibraryBridgeHelper::bridgeHandShake (this=0x7ffd60c4ca18) at /src/ch/clickhouse/src/BridgeHelper/ExternalDictionaryLibraryBridgeHelper.cpp:79 #13 0x000055556cea1f34 in DB::ExternalDictionaryLibraryBridgeHelper::removeLibrary (this=0x7ffff5b96044) at /src/ch/clickhouse/src/BridgeHelper/ExternalDictionaryLibraryBridgeHelper.cpp:186 #14 0x0000555566cb0248 in DB::LibraryDictionarySource::~LibraryDictionarySource (this=0x7ffd60c78318) at /src/ch/clickhouse/src/Dictionaries/LibraryDictionarySource.cpp:70 #15 0x000055556708d775 in std::__1::__shared_count::__release_shared[abi:ne190107]() (this=0x7ffd60c78300) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:156 #16 std::__1::__shared_weak_count::__release_shared[abi:ne190107]() (this=0x7ffd60c78300) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:185 #17 std::__1::shared_ptr<DB::IDictionarySource>::~shared_ptr[abi:ne190107]() (this=0x7ffd60c605b0) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:668 #18 DB::HashedDictionary<(DB::DictionaryKeyType)0, false, false>::~HashedDictionary (this=0x7ffd60c60398) at /src/ch/clickhouse/src/Dictionaries/HashedDictionary.h:378 #19 0x0000555569a9c477 in std::__1::__shared_count::__release_shared[abi:ne190107]() (this=0x7ffd60c60380) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:156 #20 std::__1::__shared_weak_count::__release_shared[abi:ne190107]() (this=0x7ffd60c60380) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:185 #21 std::__1::shared_ptr<DB::IExternalLoadable>::~shared_ptr[abi:ne190107]() (this=0x7ffff4832560) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:668 #22 DB::ExternalLoader::LoadingDispatcher::Info::~Info (this=0x7ffff4832548) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:719 #23 std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>::~pair (this=0x7ffff4832530, this@entry=0x7ffd60c60390) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__utility/pair.h:65 #24 __destroy_at<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>, 0> (__loc=__loc@entry=0x7ffff4832530) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/construct_at.h:67 #25 0x0000555569a9c628 in destroy<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>, void, 0> (__p=0x7ffff5b96044, __p@entry=0x7ffff4832530) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:339 #28 std::__1::unique_ptr<std::__1::__hash_node<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, DB::ExternalLoader::LoadingDispatcher::Info>, void*>, std::__1::__hash_node_destructor<std::__1::allocator<std::__1::__hash_node<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, DB::ExternalLoader::LoadingDispatcher::Info>, void*> > > >::~unique_ptr[abi:ne190107]() (this=<optimized out>) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:261 ClickHouse#31 DB::ExternalLoader::LoadingDispatcher::setConfiguration (this=this@entry=0x7ffecad4c300, new_configs=...) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:495 ClickHouse#32 0x0000555569a9116d in DB::ExternalLoader::reloadConfig (this=0x7ffecb3fd740, repository_name=...) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:1533 ClickHouse#33 DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0::operator()() const (this=<optimized out>) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:1334 ClickHouse#34 std::__1::__invoke[abi:ne190107]<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&>(DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&) (__f=...) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__type_traits/invoke.h:149 ClickHouse#35 std::__1::__invoke_void_return_wrapper<void, true>::__call[abi:ne190107]<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&>(DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&) (__args=...) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__type_traits/invoke.h:224 ClickHouse#36 std::__1::__function::__default_alloc_func<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0, void ()>::operator()[abi:ne190107]() (this=<optimized out>) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:210 ClickHouse#37 std::__1::__function::__policy_invoker<void ()>::__call_impl[abi:ne190107]<std::__1::__function::__default_alloc_func<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0, void ()> ClickHouse#38 0x0000555569966aaa in std::__1::__function::__policy_func<void ()>::operator()[abi:ne190107]() const (this=0x7ffff5b96044) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:716 ClickHouse#39 std::__1::function<void()>::operator() (this=0x7ffff5b96044) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:989 ClickHouse#40 BasicScopeGuard<std::__1::function<void()> >::invoke (this=0x7ffff5b96044) at /src/ch/clickhouse/base/base/../base/scope_guard.h:101 ClickHouse#41 BasicScopeGuard<std::__1::function<void()> >::reset (this=0x7ffff5b96044) at /src/ch/clickhouse/base/base/../base/scope_guard.h:63 ClickHouse#42 DB::ContextSharedPart::shutdown (this=0x7ffff5b96000) at /src/ch/clickhouse/src/Interpreters/Context.cpp:909 ClickHouse#43 0x00005555644cf5cb in DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3::operator()() const (this=this@entry=0x7fffffffc870) at /src/ch/clickhouse/programs/server/Server.cpp:1257 ClickHouse#44 0x00005555644b4439 in BasicScopeGuard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3>::invoke (this=0x7fffffffc870) at /src/ch/clickhouse/base/base/../base/scope_guard.h:101 ClickHouse#45 BasicScopeGuard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3>::~BasicScopeGuard (this=0x7fffffffc870) at /src/ch/clickhouse/base/base/../base/scope_guard.h:50 ClickHouse#46 DB::Server::main (this=0x7fffffffe4c8) at /src/ch/clickhouse/programs/server/Server.cpp:2788 ClickHouse#47 0x0000555572cbbee9 in Poco::Util::Application::run (this=0x7fffffffe4c8) at /src/ch/clickhouse/base/poco/Util/src/Application.cpp:315 ClickHouse#48 0x000055556449c390 in DB::Server::run (this=0x7fffffffe4c8) at /src/ch/clickhouse/programs/server/Server.cpp:610 ClickHouse#49 0x0000555564499aec in mainEntryClickHouseServer (argc=6, argv=0x7ffff67ecdc0) at /src/ch/clickhouse/programs/server/Server.cpp:397 ClickHouse#50 0x000055555dc994c2 in main (argc_=<optimized out>, argv_=<optimized out>) at /src/ch/clickhouse/programs/main.cpp:340 </details>

lgbo-ustc force-pushed the extend_inequal_join branch 4 times, most recently from 2e723fd to 4b08b8c Compare May 24, 2024 00:56

lgbo-ustc force-pushed the extend_inequal_join branch 7 times, most recently from 1b214db to 7cfeb3a Compare July 10, 2024 02:16

lgbo-ustc force-pushed the extend_inequal_join branch from 7cfeb3a to 6917c8c Compare July 11, 2024 07:10

lgbo-ustc force-pushed the extend_inequal_join branch from 6917c8c to b77793b Compare July 22, 2024 02:10

lgbo-ustc force-pushed the extend_inequal_join branch from 02ba414 to 7e1b789 Compare July 30, 2024 01:15

azat and others added 16 commits August 3, 2024 23:51

Support true UUID type for MongoDB engine

2f00c96

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

Merge pull request ClickHouse#67701 from ClickHouse/improve-dashboard

aa3b459

Improve the dashboard

Merge pull request ClickHouse#67645 from ClickHouse/fix-race-conditio…

4e3202d

…n-in-settings-processlist Fix race condition between ProcessList and Settings

Merge branch 'master' into fix-log-and-exception-messages-formatting

6b300ab

Fix tests

a051eb2

Merge pull request ClickHouse#67702 from ClickHouse/fix-timer-descrip…

b8050cc

…tor-locks Fix locking inside TimerDescriptor

Merge branch 'master' into add-assert-into-timer-descriptor

3bea8b4

Merge branch 'master' into fix-json-recursion

d37a456

Merge with master

185b6a5

Merge pull request ClickHouse#67727 from ClickHouse/fix-host-resolver…

e768649

…-ubsan Fix strange code in HostResolvePool

Merge pull request ClickHouse#67428 from ClickHouse/enable-text-log

6a20f77

Enable `text_log` by default

Merge pull request ClickHouse#67729 from ClickHouse/fix-typo

3efa208

Fix a typo

Merge branch 'ClickHouse:master' into aliasFeature

a31e2a4

Merge pull request ClickHouse#67738 from azat/tests/fix-retries

c6c4b31

Fix test retries

tests: increase timeout for RabbitMQ startup

264be9c

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

rschu1ze and others added 12 commits August 6, 2024 19:23

Merge pull request ClickHouse#64489 from ClickHouse/zup

26b6225

Update zlib-ng from 2.0.2 to 2.1.7

Merge pull request ClickHouse#67908 from Blargian/docs_toFloat

915e6f6

Update documentation of `toFloat32/64` and variants

Merge pull request ClickHouse#67470 from ClickHouse/chesema-01171-2

2ccfc8c

run 01171 test in parallel

Merge pull request ClickHouse#67919 from ClickHouse/ci_no_critical_bu…

a15fb8b

…g_fix_in_changleog CI: Changelog: Critical Bug Fix to Bug Fix

Merge pull request ClickHouse#67877 from ClickHouse/revert-58990-reve…

1eaccbe

…rt-58927-add-test-47892 Revert "Revert "Add a test for ClickHouse#47892""

Merge pull request ClickHouse#67755 from ClickHouse/no-flaky-check

82a95d0

Introduce `no-flaky-check` tag

Merge pull request ClickHouse#67934 from ClickHouse/ci_strict_job_tim…

6529752

…eout CI: Strict job timeout 1.5h for tests, 2h for builds

Merge pull request ClickHouse#67896 from CurtizJ/fix-azure-create-con…

ec83156

…tainer Do not try to create azure container if not needed

Merge pull request ClickHouse#67565 from ClickHouse/chesema-perf-matview

2d998b7

enable parallel_view_processing in perf tests

Merge pull request ClickHouse#65564 from azat/override-logs-in-keeper…

e3ead8a

…-image [RFC] Do not pass logs for keeper explicitly in the image to allow overriding

Merge pull request ClickHouse#67928 from hanfei1991/hanfei/more-log-f…

6fba2f1

…or-async-insert-scatter-partition more logs to debug logical error from async inserts

Merge pull request ClickHouse#67596 from ClickHouse/test_parquet_page…

0aed98f

…_index added tests for page index in parquet files

lgbo-ustc force-pushed the extend_inequal_join branch 2 times, most recently from 9c230d4 to 5c65c73 Compare August 7, 2024 04:22

pufit and others added 6 commits August 7, 2024 04:58

Merge pull request ClickHouse#66364 from azat/cluster-inter-server-se…

3b642e7

…cret-fix Fix cluster() for inter-server secret (preserve initial user as before)

any/anti/semi join support mixed join conditions

deb58b4

instance template classes

413834d

reduce the size of HashJoin.cpp.o

377eed2

update doc

bd4d648

update

58b7ac2

lgbo-ustc force-pushed the extend_inequal_join branch 2 times, most recently from bf36a35 to 3327685 Compare August 7, 2024 09:14

fix crash

8426e0d

lgbo-ustc force-pushed the extend_inequal_join branch 3 times, most recently from a1130f1 to ce446d6 Compare August 8, 2024 09:38

fixed

248da03

lgbo-ustc force-pushed the extend_inequal_join branch from ce446d6 to 248da03 Compare August 9, 2024 01:51

zhanglistar merged commit e6e06af into master Sep 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extend inequal join#28

Extend inequal join#28
zhanglistar merged 2459 commits intomasterfrom
extend_inequal_join

lgbo-ustc commented May 23, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

lgbo-ustc commented May 23, 2024

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Documentation entry for user-facing changes

Run these jobs only (required builds will be added automatically):

Deny these jobs:

Extra options:

Only specified batches in multi-batch jobs:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants