Skip to content

Conversation

@jasnell
Copy link
Member

@jasnell jasnell commented Jun 30, 2025

As a first step to porting portions of the pino structured logger into the runtime, this commit ports the SonicBoom module to the fs module as FastUtf8Stream. Sonicboom is a dependency of pino.

This is a faithful port of the SonicBoom module with some modern updates, such as converting to a Class and using Symbol.dispose. The bulk of the implementation is unchanged from the original.

Refs: #49296 (comment)

/cc @mcollina @ronag @kibertoad @jsumners @mmarchini @feugy

@jasnell jasnell requested a review from mcollina June 30, 2025 02:49
@jasnell jasnell marked this pull request as draft June 30, 2025 02:49
@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/security-wg
  • @nodejs/streams

@nodejs-github-bot nodejs-github-bot added fs Issues and PRs related to the fs subsystem / file system. needs-ci PRs that need a full CI run. stream Issues and PRs related to the stream subsystem. tools Issues and PRs related to the tools directory. labels Jun 30, 2025
@jasnell jasnell added the semver-minor PRs that contain new features and should be released in the next minor version. label Jun 30, 2025
@codecov
Copy link

codecov bot commented Jun 30, 2025

Codecov Report

❌ Patch coverage is 74.48870% with 237 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.96%. Comparing base (cd1a90d) to head (a5d35b0).
⚠️ Report is 209 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/streams/fast-utf8-stream.js 74.21% 235 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #58897      +/-   ##
==========================================
- Coverage   90.06%   89.96%   -0.11%     
==========================================
  Files         648      649       +1     
  Lines      191078   192007     +929     
  Branches    37454    37631     +177     
==========================================
+ Hits       172101   172742     +641     
- Misses      11598    11885     +287     
- Partials     7379     7380       +1     
Files with missing lines Coverage Δ
lib/fs.js 98.18% <100.00%> (+<0.01%) ⬆️
lib/internal/streams/fast-utf8-stream.js 74.21% <74.21%> (ø)

... and 30 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

huge +1. This would be beneficial for other loggers too.

@jasnell jasnell force-pushed the jasnell/port-sonicboom branch from 41f2c92 to 5631e21 Compare June 30, 2025 13:12
@jasnell
Copy link
Member Author

jasnell commented Jun 30, 2025

Tests updated. @mcollina ... So far I have not ported any of the tests that require proxyquire. I'm not sure what's the best way to port that behavior yet. I think, however, that I'd like to get this landed as experimental first and then incrementally improve the test coverage.

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of the worst bugs we fixed are being tests by the proxyquire-based tests (those are error conditions that are impossible to replicate without mocking). They are the battle scars, and they are important to keep around / being ported.

I'm ok for landing this as-is, but can you create a tracking issue for all the tests that you skipped so that they can be ported around?

@jasnell
Copy link
Member Author

jasnell commented Jun 30, 2025

Absolutely. It's really all of the tests that use proxyquire. We can port those by monkey patching the fs APIs. Not as clean but doable. Will just take time

@jasnell jasnell force-pushed the jasnell/port-sonicboom branch 3 times, most recently from bb11c97 to e03efc9 Compare July 4, 2025 16:37
@jasnell jasnell marked this pull request as ready for review July 4, 2025 16:37
@nodejs-github-bot

This comment was marked as outdated.

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@jasnell jasnell force-pushed the jasnell/port-sonicboom branch from e03efc9 to da471d7 Compare July 4, 2025 18:25
@jasnell jasnell force-pushed the jasnell/port-sonicboom branch from da471d7 to f5136bd Compare July 5, 2025 19:40
@nodejs-github-bot

This comment was marked as outdated.

@jasnell jasnell force-pushed the jasnell/port-sonicboom branch from f5136bd to 583dcfb Compare July 5, 2025 23:01
@jasnell
Copy link
Member Author

jasnell commented Jul 5, 2025

@mertcanaltin ... looks like the tests here are hanging and failing consistently. Looks like switching away from using node:test and restructuring the tests helps clear it. Will need to work through refactoring a bit more.

@jasnell jasnell marked this pull request as draft July 5, 2025 23:02
@mertcanaltin
Copy link
Member

mertcanaltin commented Jul 6, 2025

@mertcanaltin ... looks like the tests here are hanging and failing consistently. Looks like switching away from using node:test and restructuring the tests helps clear it. Will need to work through refactoring a bit more.

I would love to help you with this, I'll try an arrangement

@aduh95 aduh95 added the backport-requested-v22.x PRs awaiting manual backport to the v22.x-staging branch. label Aug 26, 2025
@aduh95
Copy link
Contributor

aduh95 commented Aug 26, 2025

Tests are failing on Windows with this change on v22.x-staging, we would need a manual backport

@joyeecheung
Copy link
Member

parallel/test-fastutf8stream-flush-mocks and parallel/test-fastutf8stream-sync have recently failed 14 PRs out of the recent 100 CI runs: #59638

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-requested-v22.x PRs awaiting manual backport to the v22.x-staging branch. fs Issues and PRs related to the fs subsystem / file system. needs-ci PRs that need a full CI run. semver-minor PRs that contain new features and should be released in the next minor version. stream Issues and PRs related to the stream subsystem. tools Issues and PRs related to the tools directory.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants