Workload read without consumer and metrics fixes by niksaveliev · Pull Request #1792 · ydb-platform/ydb

niksaveliev · 2024-02-12T07:17:10Z

Changelog entry

Changelog category

Not for changelog (changelog entry is not required)

Additional information

...

github-actions · 2024-02-12T07:17:39Z

⚪ 2024-02-12 07:17:39 UTC Pre-commit check for e3a492a has started.
⚪ 2024-02-12 07:17:40 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-12 07:19:51 UTC Build successful.
⚪ 2024-02-12 07:20:02 UTC Tests are running...
🔴 2024-02-12 08:35:40 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
67327	56391	0	5	10892	39

github-actions · 2024-02-12T07:17:49Z

⚪ 2024-02-12 07:17:49 UTC Pre-commit check for e3a492a has started.
⚪ 2024-02-12 07:17:50 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-12 07:19:54 UTC Build successful.
⚪ 2024-02-12 07:20:03 UTC Tests are running...
🔴 2024-02-12 08:58:54 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14739	14542	0	13	137	47

github-actions · 2024-02-12T10:31:59Z

⚪ 2024-02-12 10:31:59 UTC Pre-commit check for ecf65aa has started.
⚪ 2024-02-12 10:32:02 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-12 10:34:38 UTC Build successful.
⚪ 2024-02-12 10:34:46 UTC Tests are running...
🔴 2024-02-12 12:16:44 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14740	14551	0	19	137	33

github-actions · 2024-02-12T10:32:07Z

⚪ 2024-02-12 10:32:07 UTC Pre-commit check for ecf65aa has started.
⚪ 2024-02-12 10:32:10 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-12 10:34:42 UTC Build successful.
⚪ 2024-02-12 10:34:52 UTC Tests are running...
🔴 2024-02-12 12:20:00 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
67425	56494	0	4	10892	35

github-actions · 2024-02-13T12:47:51Z

⚪ 2024-02-13 12:47:51 UTC Pre-commit check for c18370b has started.
⚪ 2024-02-13 12:47:52 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-13 12:49:56 UTC Build successful.
⚪ 2024-02-13 12:50:08 UTC Tests are running...
🔴 2024-02-13 14:25:23 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14761	14563	0	15	135	48

github-actions · 2024-02-13T12:48:03Z

⚪ 2024-02-13 12:48:02 UTC Pre-commit check for c18370b has started.
⚪ 2024-02-13 12:48:04 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-13 12:50:09 UTC Build successful.
⚪ 2024-02-13 12:50:19 UTC Tests are running...
🔴 2024-02-13 14:03:35 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
67470	56450	0	3	10972	45

alexnick88 · 2024-02-14T06:27:07Z

ydb/core/persqueue/read_balancer.cpp

        }
    }
-
+    Consumers[CLIENTID_WITHOUT_CONSUMER];


Read without consumer doesn't use read balancer. Why do we need this line?

alexnick88 · 2024-02-14T06:28:54Z

ydb/core/persqueue/ut/resources/counters_topics.html

                        name=topic.reserve.used_bytes: 0
                        name=topic.storage_bytes: 0

+                        consumer=$without_consumer:


This is not what we want to see. No label consumer is possible solution. But some sensors are useless in reading without consumer. Sensor topic.partition.committed_end_to_end_lag_milliseconds_max is undefined for example.

github-actions · 2024-02-15T06:58:33Z

⚪ 2024-02-15 06:58:33 UTC Pre-commit check for 177476e has started.
⚪ 2024-02-15 06:58:34 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-15 07:03:21 UTC Build successful.
⚪ 2024-02-15 07:03:30 UTC Tests are running...
🔴 2024-02-15 08:34:33 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
67530	56498	0	6	10974	52

github-actions · 2024-02-15T06:58:57Z

⚪ 2024-02-15 06:58:57 UTC Pre-commit check for 177476e has started.
⚪ 2024-02-15 06:58:59 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-15 07:03:57 UTC Build successful.
⚪ 2024-02-15 07:04:10 UTC Tests are running...
🔴 2024-02-15 08:39:50 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14774	14586	0	14	128	46

github-actions · 2024-02-20T14:23:15Z

⚪ 2024-02-20 14:23:15 UTC Pre-commit check for 313438d has started.
⚪ 2024-02-20 14:23:16 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-20 14:40:36 UTC Build successful.
⚪ 2024-02-20 14:40:47 UTC Tests are running...
🔴 2024-02-20 16:12:43 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
67709	56713	0	16	10945	35

github-actions · 2024-02-20T14:23:42Z

⚪ 2024-02-20 14:23:42 UTC Pre-commit check for 313438d has started.
⚪ 2024-02-20 14:23:44 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-20 14:44:06 UTC Build successful.
⚪ 2024-02-20 14:44:21 UTC Tests are running...
🔴 2024-02-20 16:28:52 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14692	14471	0	27	153	41

github-actions · 2024-02-21T15:28:19Z

⚪ 2024-02-21 15:28:18 UTC Pre-commit check for f7ef27e has started.
⚪ 2024-02-21 15:28:20 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-21 15:32:43 UTC Build successful.
⚪ 2024-02-21 15:32:54 UTC Tests are running...
🔴 2024-02-21 17:06:31 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
67906	56912	0	4	10951	39

github-actions · 2024-02-21T15:33:49Z

⚪ 2024-02-21 15:33:49 UTC Pre-commit check for f7ef27e has started.
⚪ 2024-02-21 15:33:50 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-21 15:39:08 UTC Build successful.
⚪ 2024-02-21 15:39:18 UTC Tests are running...
🔴 2024-02-21 17:20:42 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14828	14615	0	31	139	43

github-actions · 2024-02-28T05:17:46Z

⚪ 2024-02-28 05:17:45 UTC Pre-commit check for dade8a1 has started.
⚪ 2024-02-28 05:17:48 UTC Build linux-x86_64-release-cmake14 is running...
🟢 2024-02-28 05:47:25 UTC Build successful.

github-actions · 2024-02-28T05:17:51Z

⚪ 2024-02-28 05:17:51 UTC Pre-commit check for dade8a1 has started.
⚪ 2024-02-28 05:17:53 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-28 05:24:10 UTC Build successful.
⚪ 2024-02-28 05:24:23 UTC Tests are running...
🔴 2024-02-28 07:02:59 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14876	14751	0	26	77	22

github-actions · 2024-02-28T05:17:53Z

⚪ 2024-02-28 05:17:52 UTC Pre-commit check for dade8a1 has started.
⚪ 2024-02-28 05:17:55 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-28 05:22:48 UTC Build successful.
⚪ 2024-02-28 05:23:03 UTC Tests are running...
🔴 2024-02-28 06:57:05 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
67982	57068	0	11	10883	20

github-actions · 2024-02-28T09:58:53Z

⚪ 2024-02-28 09:58:52 UTC Pre-commit check for eb7cc11 has started.
⚪ 2024-02-28 09:58:54 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-28 10:12:54 UTC Build successful.
⚪ 2024-02-28 10:13:07 UTC Tests are running...
🔴 2024-02-28 12:00:36 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14879	14735	0	27	96	21

github-actions · 2024-02-28T12:56:36Z

⚪ 2024-02-28 12:56:36 UTC Pre-commit check for 1f4857f has started.
⚪ 2024-02-28 12:56:37 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-28 13:00:49 UTC Build successful.
⚪ 2024-02-28 13:00:59 UTC Tests are running...
🔴 2024-02-28 14:31:09 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
68013	57092	0	7	10888	26

github-actions · 2024-02-28T12:58:14Z

⚪ 2024-02-28 12:58:14 UTC Pre-commit check for 1f4857f has started.
⚪ 2024-02-28 12:58:17 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-28 13:03:08 UTC Build successful.
⚪ 2024-02-28 13:03:21 UTC Tests are running...
🔴 2024-02-28 14:42:14 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14871	14737	0	31	78	25

github-actions · 2024-02-28T12:58:38Z

⚪ 2024-02-28 12:58:38 UTC Pre-commit check for 1f4857f has started.
⚪ 2024-02-28 12:58:42 UTC Build linux-x86_64-release-cmake14 is running...
🟢 2024-02-28 13:03:02 UTC Build successful.

github-actions · 2024-02-29T07:50:16Z

⚪ 2024-02-29 07:50:15 UTC Pre-commit check for 7042e11 has started.
⚪ 2024-02-29 07:50:17 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-29 07:54:55 UTC Build successful.
⚪ 2024-02-29 07:55:05 UTC Tests are running...
🔴 2024-02-29 09:31:25 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14884	14753	0	19	86	26

github-actions · 2024-02-29T07:52:42Z

⚪ 2024-02-29 07:52:42 UTC Pre-commit check for 7042e11 has started.
⚪ 2024-02-29 07:52:45 UTC Build linux-x86_64-release-cmake14 is running...
🟢 2024-02-29 07:57:16 UTC Build successful.

github-actions · 2024-02-29T07:52:50Z

⚪ 2024-02-29 07:52:49 UTC Pre-commit check for 7042e11 has started.
⚪ 2024-02-29 07:52:52 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-29 07:57:17 UTC Build successful.
⚪ 2024-02-29 07:57:31 UTC Tests are running...
🔴 2024-02-29 09:27:19 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
68019	57100	0	6	10890	23

github-actions bot added the new-feature label Feb 12, 2024

github-actions bot added the not-for-changelog label Feb 12, 2024

niksaveliev removed the new-feature label Feb 12, 2024

niksaveliev marked this pull request as ready for review February 14, 2024 04:58

alexnick88 reviewed Feb 14, 2024

View reviewed changes

niksaveliev force-pushed the read-without-consumer-workload-and-counters-fixes branch from e81f997 to 8ff4db8 Compare February 20, 2024 14:20

niksaveliev added 6 commits February 28, 2024 05:12

Workload read without consumer and metrics fixes

e552138

Fix ut

8578a10

Fix counters ut

c4e7ef9

Move no consumer metrics to separate sensors

c147bcf

Fix

4f24717

fix

39df411

niksaveliev force-pushed the read-without-consumer-workload-and-counters-fixes branch from 9958322 to 39df411 Compare February 28, 2024 05:14

Fix uts

d948fbd

FIx

61c74ce

FIx ut

3721c57

alexnick88 approved these changes Feb 29, 2024

View reviewed changes

niksaveliev merged commit dcce157 into ydb-platform:main Feb 29, 2024

serbel324 mentioned this pull request Mar 1, 2024

Mute OperationLog.ConcurrentWrites test #2381

Merged

This was referenced Mar 4, 2024

YDB 647 add media type field #2405

Closed

YDB-647 add media type field #2441

Merged

GrigoriyPA mentioned this pull request Mar 7, 2024

YQ-2737 add TOKEN auth for YT datasource #2561

Merged

This was referenced Mar 8, 2024

HC made storage issue RED when no bsc reply #2578

Merged

time difference lines2 #2599

Closed

This was referenced Mar 11, 2024

Mute TImportTests.CancelUponCreatingTableShouldSucceed #2609

Merged

Fix TImportTests.CancelUponCreatingTableShouldSucceed #2636

Merged

This was referenced Mar 13, 2024

YQ-2957 fix error in s3 stream read actor #2702

Merged

Fixed error due to empty token #2966

Merged

This was referenced Mar 20, 2024

Test alter temp table #3011

Merged

Support sequences backup with current value #3023

Closed

Support sequences backup with current value #3089

Merged

YDBDOCS-632: Documentation of temporary tables #3137

Merged

GrigoriyPA mentioned this pull request Mar 26, 2024

YQ-2738 add integration tests for YT in KQP #3150

Merged

shnikd mentioned this pull request Mar 26, 2024

Test temp indexes #3163

Merged

shnikd mentioned this pull request Apr 4, 2024

Support create sequence in pg parser #3465

Merged

Conversation

niksaveliev commented Feb 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog entry

Changelog category

Additional information

Uh oh!

github-actions bot commented Feb 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexnick88 Feb 14, 2024

Choose a reason for hiding this comment

Uh oh!

alexnick88 Feb 14, 2024

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

niksaveliev commented Feb 12, 2024 •

edited

Loading

github-actions bot commented Feb 12, 2024 •

edited

Loading

github-actions bot commented Feb 12, 2024 •

edited

Loading

github-actions bot commented Feb 12, 2024 •

edited

Loading

github-actions bot commented Feb 12, 2024 •

edited

Loading

github-actions bot commented Feb 13, 2024 •

edited

Loading

github-actions bot commented Feb 13, 2024 •

edited

Loading

github-actions bot commented Feb 15, 2024 •

edited

Loading

github-actions bot commented Feb 15, 2024 •

edited

Loading

github-actions bot commented Feb 20, 2024 •

edited

Loading

github-actions bot commented Feb 20, 2024 •

edited

Loading

github-actions bot commented Feb 21, 2024 •

edited

Loading

github-actions bot commented Feb 21, 2024 •

edited

Loading

github-actions bot commented Feb 28, 2024 •

edited

Loading

github-actions bot commented Feb 28, 2024 •

edited

Loading

github-actions bot commented Feb 28, 2024 •

edited

Loading

github-actions bot commented Feb 28, 2024 •

edited

Loading

github-actions bot commented Feb 28, 2024 •

edited

Loading

github-actions bot commented Feb 28, 2024 •

edited

Loading

github-actions bot commented Feb 28, 2024 •

edited

Loading

github-actions bot commented Feb 29, 2024 •

edited

Loading

github-actions bot commented Feb 29, 2024 •

edited

Loading

github-actions bot commented Feb 29, 2024 •

edited

Loading