Skip to content

YQ-3722 add simdjson parser into RD#10204

Merged
GrigoriyPA merged 4 commits intoydb-platform:mainfrom
GrigoriyPA:YQ-3722-add-simdjson-parser-into-RD
Oct 9, 2024
Merged

YQ-3722 add simdjson parser into RD#10204
GrigoriyPA merged 4 commits intoydb-platform:mainfrom
GrigoriyPA:YQ-3722-add-simdjson-parser-into-RD

Conversation

@GrigoriyPA
Copy link
Copy Markdown
Collaborator

@GrigoriyPA GrigoriyPA commented Oct 8, 2024 •
edited
Loading

Changelog entry

Added simdjson parser into RD

Changelog category

  • Performance improvement

Additional information

@GrigoriyPA GrigoriyPA requested a review from a team as a code owner October 8, 2024 10:19
@github-actions
Copy link
Copy Markdown

github-actions bot commented Oct 8, 2024 •
edited
Loading

⚪ 2024-10-08 10:22:48 UTC Pre-commit check linux-x86_64-release-asan for 3f1e08d has started.
⚪ 2024-10-08 10:22:58 UTC Artifacts will be uploaded here
⚪ 2024-10-08 10:25:55 UTC ya make is running...
🔴 2024-10-08 10:31:38 UTC Build failed, see the logs.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Oct 8, 2024 •
edited
Loading

⚪ 2024-10-08 10:24:29 UTC Pre-commit check linux-x86_64-relwithdebinfo for 3f1e08d has started.
⚪ 2024-10-08 10:24:40 UTC Artifacts will be uploaded here
⚪ 2024-10-08 10:27:33 UTC ya make is running...
🔴 2024-10-08 10:39:56 UTC Build failed, see the logs.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Oct 8, 2024 •
edited
Loading

⚪ 2024-10-08 11:03:56 UTC Pre-commit check linux-x86_64-relwithdebinfo for 3500381 has started.
⚪ 2024-10-08 11:04:07 UTC Artifacts will be uploaded here
⚪ 2024-10-08 11:07:06 UTC ya make is running...
🟡 2024-10-08 12:26:57 UTC Some tests failed, follow the links below. Going to retry failed tests...

Details

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
41287 35721 0 1 5460 105

⚪ 2024-10-08 12:30:33 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-10-08 12:41:48 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
105 (only retried tests) 4 0 0 1 100

🟢 2024-10-08 12:41:54 UTC Build successful.
🟡 2024-10-08 12:42:13 UTC ydbd size 2.8 GiB changed* by +579.2 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 8b39e36 merge: 3500381 diff diff %
ydbd size 3 008 719 720 Bytes 3 009 312 784 Bytes +579.2 KiB +0.020%
ydbd stripped size 477 379 640 Bytes 477 554 168 Bytes +170.4 KiB +0.037%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link
Copy Markdown

github-actions bot commented Oct 8, 2024 •
edited
Loading

⚪ 2024-10-08 11:04:06 UTC Pre-commit check linux-x86_64-release-asan for 3500381 has started.
⚪ 2024-10-08 11:04:17 UTC Artifacts will be uploaded here
⚪ 2024-10-08 11:07:14 UTC ya make is running...
🟡 2024-10-08 12:56:01 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11578 11354 0 64 54 106

🟢 2024-10-08 12:57:03 UTC Build successful.
🟡 2024-10-08 12:57:36 UTC ydbd size 5.7 GiB changed* by +990.9 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 8b39e36 merge: 3500381 diff diff %
ydbd size 6 097 996 744 Bytes 6 099 011 472 Bytes +990.9 KiB +0.017%
ydbd stripped size 1 520 324 752 Bytes 1 520 792 048 Bytes +456.3 KiB +0.031%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link
Copy Markdown

github-actions bot commented Oct 8, 2024 •
edited
Loading

⚪ 2024-10-08 13:37:17 UTC Pre-commit check linux-x86_64-relwithdebinfo for bee4648 has started.
⚪ 2024-10-08 13:37:27 UTC Artifacts will be uploaded here
⚪ 2024-10-08 13:40:27 UTC ya make is running...
⚫ 2024-10-08 13:58:54 UTC Check cancelled

@github-actions
Copy link
Copy Markdown

github-actions bot commented Oct 8, 2024 •
edited
Loading

⚪ 2024-10-08 13:39:43 UTC Pre-commit check linux-x86_64-release-asan for bee4648 has started.
⚪ 2024-10-08 13:39:55 UTC Artifacts will be uploaded here
⚪ 2024-10-08 13:42:49 UTC ya make is running...
⚫ 2024-10-08 13:59:00 UTC Check cancelled

@github-actions
Copy link
Copy Markdown

github-actions bot commented Oct 8, 2024 •
edited
Loading

⚪ 2024-10-08 14:00:37 UTC Pre-commit check linux-x86_64-relwithdebinfo for f62080e has started.
⚪ 2024-10-08 14:03:02 UTC Artifacts will be uploaded here
⚪ 2024-10-08 14:06:33 UTC ya make is running...
🟡 2024-10-08 15:02:55 UTC Some tests failed, follow the links below. Going to retry failed tests...

Details

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
41287 35707 0 4 5462 114

⚪ 2024-10-08 15:06:45 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-10-08 15:17:56 UTC Some tests failed, follow the links below. Going to retry failed tests...

Details

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
118 (only retried tests) 15 0 1 0 102

⚪ 2024-10-08 15:18:05 UTC ya make is running... (failed tests rerun, try 3)
🟢 2024-10-08 15:29:28 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
103 (only retried tests) 1 0 0 0 102

🟢 2024-10-08 15:29:36 UTC Build successful.
🟡 2024-10-08 15:30:02 UTC ydbd size 2.8 GiB changed* by +543.0 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: b8ac272 merge: f62080e diff diff %
ydbd size 3 008 719 776 Bytes 3 009 275 856 Bytes +543.0 KiB +0.018%
ydbd stripped size 477 379 640 Bytes 477 551 224 Bytes +167.6 KiB +0.036%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link
Copy Markdown

github-actions bot commented Oct 8, 2024 •
edited
Loading

⚪ 2024-10-08 14:03:06 UTC Pre-commit check linux-x86_64-release-asan for f62080e has started.
⚪ 2024-10-08 14:03:17 UTC Artifacts will be uploaded here
⚪ 2024-10-08 14:06:10 UTC ya make is running...
🟡 2024-10-08 15:37:58 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11603 11325 0 97 72 109

🟢 2024-10-08 15:38:57 UTC Build successful.
🟡 2024-10-08 15:39:29 UTC ydbd size 5.7 GiB changed* by +936.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: b8ac272 merge: f62080e diff diff %
ydbd size 6 097 996 944 Bytes 6 098 956 080 Bytes +936.7 KiB +0.016%
ydbd stripped size 1 520 324 880 Bytes 1 520 778 288 Bytes +442.8 KiB +0.030%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

kardymonds
kardymonds reviewed Oct 9, 2024
View reviewed changes
Comment thread ydb/core/fq/libs/row_dispatcher/topic_session.cpp
Comment thread ydb/core/fq/libs/row_dispatcher/json_parser.cpp
@GrigoriyPA GrigoriyPA requested a review from kardymonds October 9, 2024 07:50
@GrigoriyPA GrigoriyPA merged commit 8fe290e into ydb-platform:main Oct 9, 2024
@GrigoriyPA GrigoriyPA deleted the YQ-3722-add-simdjson-parser-into-RD branch October 9, 2024 09:05
kardymonds pushed a commit to kardymonds/ydb that referenced this pull request Oct 16, 2024
@GrigoriyPA @kardymonds
YQ-3722 add simdjson parser into RD (ydb-platform#10204)
a0656e0
This was referenced Oct 21, 2024
Merged
Merged
@GrigoriyPA GrigoriyPA mentioned this pull request Oct 23, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kardymonds kardymonds kardymonds approved these changes

@uzhastik uzhastik Awaiting requested review from uzhastik

Assignees

No one assigned

Labels

performance

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@GrigoriyPA @kardymonds