Skip to content

feat: Add option for native js implementation of jq#2233

Closed
W-Floyd wants to merge 1 commit intogchq:masterfrom
W-Floyd:feat/jq-implementation
Closed

feat: Add option for native js implementation of jq#2233
W-Floyd wants to merge 1 commit intogchq:masterfrom
W-Floyd:feat/jq-implementation

Conversation

@W-Floyd
Copy link
Contributor

@W-Floyd W-Floyd commented Mar 9, 2026

Testing solutions to #2232

Performance with WASM jq can be slow for some reason (I should also investigate).

For example, running length on 512KB.json takes ~12000ms using WASM.

image

Using native JS, the same operation takes 5ms

image

Note that using a smaller input takes less time. E.g 64KB.json takes ~250ms with WASM, but 2ms with native JS.

We should not replace the WASM version yet, as the JS version is not at full parity.

@W-Floyd
Copy link
Contributor Author

W-Floyd commented Mar 9, 2026

Running length.

jq-wasm 1.1.0-jq-1.8.1:

Data Firefox Chromium
512KB ~12500ms ~4300ms
64KB ~300ms ~110ms

jq-web 0.5.1

Data Firefox Chromium
512KB ~55ms ~55ms
64KB ~35ms ~35ms

Looks like an issue with the change to jq-wasm, or my integration thereof.

@W-Floyd
Copy link
Contributor Author

W-Floyd commented Mar 9, 2026

Note that the native JS implementation is still much faster. Passing 5MB.json through length, native JS ran in ~22ms, jq-web v0.5.1 ran in ~220ms

@C85297
Copy link
Member

C85297 commented Mar 10, 2026

I think we should make a decision to either use WASM or JS based JQ rather than providing a choice to users. The functionality of JQ would presumably be the same either way, so I can't see it being a useful a choice for users. If we give them the choice, we will then have to maintain a more complicated operation with two different dependencies.
What do you think @GCHQDeveloper581 ?

@GCHQDeveloper581
Copy link
Contributor

I concur that we only want a single jq implementation, and based on #2232 and the results above it looks like that should be jq-web

@W-Floyd - you stated above "We should not replace the WASM version yet, as the JS version is not at full parity" - could you expand on that at all? What's missing/different in the jq-web version?

@W-Floyd
Copy link
Contributor Author

W-Floyd commented Mar 11, 2026

Sure, per https://github.com/mwh/jqjs:

jqjs supports most of the core jq language features, but lacks some functions and some of the advanced functionality. It also uses JavaScript strings as backing, so does not have jq proper's Unicode support.

It also includes a list of language features implemented/not yet implemented. For most use cases I'm guessing this will work, but it may cause a regression for more advanced queries.

@W-Floyd W-Floyd marked this pull request as draft March 13, 2026 16:23
@GCHQDeveloper581
Copy link
Contributor

Closing as, for now, we have reverted to jq-web instead.

However thanks for your effort on this and the original #2223 - it's much appreciated even if we eventually took a different route (partly influenced by the results of your research).

@W-Floyd
Copy link
Contributor Author

W-Floyd commented Mar 21, 2026

No problem, thanks for getting things reverted! 😅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants