Skip to content

jsondocck: replace JSONPath with jq as json query language. #142479

Closed

Description

@aDotInTheVoid

Impetus: #142327 (comment)

Currently, jsondocck uses JSONPath to query the output in assertions. Unfortunately, it's not a particularly flexible query language. In particular, it can't feed the result of one query to index an object. We'd really like this, so we can run a query to get an ID, then look up the item with that ID.

Using jq, we can do this. It has an implementation in rust: https://docs.rs/jaq-core/2.2.0/jaq_core/index.html

How

In jsondoc, this place would need to be changed to call into jaq instead.

// FIXME: Make this failible, so jsonpath syntax error has line number.
pub fn select(&self, path: &str) -> Vec<&Value> {
jsonpath_rust::query::js_path_vals(path, &self.value).unwrap()
}

When doing so, the contents of Cache::variables should be provided, so they can be used in selectors.

Test Migration

I think the longest part of this will be migrating the tests from jsonpath to jq. It might be possible to hack up jsondocck to call jsonpath_rust::parser::parse_json_path on all the JSONPath queries, and then convert the JSONPath ast to a equivalent jq query.

I'm not sure if this would be more or less effort than "just" doing it manually. But it'd be easier to review the resulting change this way, and it'd probably be more interesting.

Activity

  1. added
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Jun 13, 2025
  2. added
    A-testsuiteArea: The testsuite used to check the correctness of rustc
    T-rustdocRelevant to the rustdoc team, which will review and decide on the PR/issue.
    E-mentorCall for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion.
    and removed
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Jun 13, 2025
  3. aDotInTheVoid commented on Jun 13, 2025

    @aDotInTheVoid
    MemberAuthor

    @fluiderson, you expressed interest in working on this (#142327 (comment)).

    Do you still want to? If so, please @ rustbot claim (without the space) to assign it to you. I can't, due to github permissions reasons.

  4. fmease commented on Jun 14, 2025

    @fmease
  5. self-assigned this
    on Jun 14, 2025
  6. added
    C-enhancementCategory: An issue proposing an enhancement or a PR with one.
    on Jun 15, 2025
  7. fluiderson commented on Jun 15, 2025

    @fluiderson
    Contributor

    @rustbot claim

  8. aDotInTheVoid commented on Jun 16, 2025

    @aDotInTheVoid
    MemberAuthor

    and then convert the JSONPath ast to a equivalent jq query.

    https://github.com/jqlang/jq/wiki/For-JSONPath-users is probably a useful start here

  9. aDotInTheVoid commented on Jun 16, 2025

    @aDotInTheVoid
    MemberAuthor

    Why not something like jq?

    See #t-rustdoc > jsondocck @ 💬. The disussion then was entirely about the availibility of rust impls, and not at all about the expressiveness of the query language.

  10. nnethercote commented on Jun 24, 2025

    @nnethercote
    Contributor

    This is also needed for #142945, and the follow-up that will intern strings.

  11. aDotInTheVoid commented on May 22, 2026

    @aDotInTheVoid
    MemberAuthor

    This was discussed at the all hands: rust-lang/all-hands-2026#72

    Key Ideas:

    • Use has to work around null issues
    • Allow JsonPath and JQ querys to live alongside eachother, to enable an incremental migration
      • Remove JsonPath eventually
  12. kohsine commented on Jun 16, 2026

    @kohsine
    Contributor

    I'm interested in taking this on if no one else is working on it.

  13. aDotInTheVoid commented on Jun 17, 2026

    @aDotInTheVoid
    MemberAuthor

    No one's working on it, go for it! rust-lang/all-hands-2026#72 (comment) is highly relevant, and useful.

    The most important advice is that rewriting all the test assertions is going to be a lot of work. Don't do that until you've got validation that the approach in jsondocck is correct, or you might waste a load of effort.

  14. lolbinarycat commented on Jul 10, 2026

    @lolbinarycat
    Contributor

    The directives have been implemented, but they still need to be documented in rustc-dev-guide next to the other jsondocck directives. one thing to note is how set and jq_set share a namespace.

  15. added a commit that references this issue on Jul 11, 2026
    5b0d5c2
  16. added a commit that references this issue on Jul 11, 2026
    7107318
  17. added a commit that references this issue on Jul 12, 2026
    e967cad
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

A-rustdoc-jsonArea: Rustdoc JSON backendA-testsuiteArea: The testsuite used to check the correctness of rustcC-enhancementCategory: An issue proposing an enhancement or a PR with one.E-mentorCall for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion.T-rustdocRelevant to the rustdoc team, which will review and decide on the PR/issue.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions