Skip to content

benchmark: add isUtf8 and isAscii bench #54740

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
RafaelGSS wants to merge 4 commits into from
Closed

benchmark: add isUtf8 and isAscii bench #54740

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a376478
benchmark: add buffer.isUtf8 bench
RafaelGSS Sep 3, 2024
79ee593
benchmark: add buffer.isAscii benchmark
RafaelGSS Sep 3, 2024
f76a126
fixup! benchmark: add buffer.isAscii benchmark
RafaelGSS Sep 3, 2024
a90a7ab
fixup! benchmark: add buffer.isUtf8 bench
RafaelGSS Sep 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions benchmark/buffers/buffer-isascii.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
'use strict';

const common = require('../common.js');
const buffer = require('node:buffer');
const assert = require('node:assert');

const bench = common.createBenchmark(main, {
n: [2e7],
length: ['short', 'long'],
input: ['hello world']
});


function main({ n, input }) {
const normalizedInput = input === 'short' ? input : input.repeat(200);
const encoder = new TextEncoder();
const buff = encoder.encode(normalizedInput);
bench.start();
for (let i = 0; i < n; ++i) {
assert.ok(buffer.isAscii(buff));
}
bench.end(n);
}
23 changes: 23 additions & 0 deletions benchmark/buffers/buffer-isutf8.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
'use strict';

const common = require('../common.js');
const buffer = require('node:buffer');
const assert = require('node:assert');

const bench = common.createBenchmark(main, {
n: [2e7],
length: ['short', 'long'],
input: ['regular string', 'ğ'],
Copy link
Member

@lpinca lpinca Sep 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might make sense to use something like this https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt

Copy link
Member Author

@RafaelGSS RafaelGSS Sep 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that was the idea of the second input ğ. I took it from the tests. But, I can change it.

Copy link
Member

@lpinca lpinca Sep 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but with ğ we only test a string made of 2 bytes characters.

Copy link
Member Author

@RafaelGSS RafaelGSS Sep 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe ∀x∈ℝ: ⌈x⌉ = −⌊−x⌋ should make it better?

Copy link
Member

@lpinca lpinca Sep 3, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think so. Using a string made of mixed 1, 2, 3, and 4 bytes would make the benchmark unrealistic but more accurate. See also simdutf/simdutf#81.

Copy link
Member

@anonrig anonrig Sep 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ps: Any Turkish song lyrics would fit into the utf8 string category.

});


function main({ n, input }) {
const normalizedInput = input === 'short' ? input : input.repeat(300);
const encoder = new TextEncoder();
const buff = encoder.encode(normalizedInput);
bench.start();
for (let i = 0; i < n; ++i) {
assert.ok(buffer.isUtf8(buff));
}
bench.end(n);
}