Open
Conversation
mattmenefee
force-pushed
the
elasticsearch-9-support
branch
from
e9bc5d7 to
dbd4a3a
Compare
Contributor
Author
mattmenefee
force-pushed
the
elasticsearch-9-support
branch
from
dbd4a3a to
210c1f9
Compare
mattmenefee
force-pushed
the
elasticsearch-9-support
branch
2 times, most recently
from
c09dd71 to
98955eb
Compare
bbatsov
reviewed
Mar 18, 2026
Member
bbatsov
left a comment
bbatsov
left a comment
There was a problem hiding this comment.
Thanks for tackling ES 9 support! We'll review this properly after #1008 and #1010 have been updated and merged, since this builds on top of both.
One thing I'd suggest splitting out now: the Sidekiq testing API changes (require 'sidekiq/testing' -> Sidekiq.testing!(:fake), Sidekiq::Testing.inline! -> Sidekiq.testing!(:inline)) are unrelated to ES 9 and would be easier to review and merge as a separate PR.
The previous implementation parsed the text-format `cat.indices` response by splitting on whitespace and taking column index [2]. This is brittle — column order can change between ES versions. Switch to `format: 'json'` and extract the `index` key from each entry. Also filter out system indices (prefixed with '.') to avoid deleting ES-internal indices like .security or .kibana.
The previous CI setup launched ES via `docker compose up` with a `sleep 15` wait, which was fragile and slow. Promote ES to a proper GitHub Actions `services:` block with a built-in health check (`wait_for_status=yellow`) for reliable startup detection. - Add `concurrency` group to cancel in-progress runs on PR updates - Add `timeout-minutes` to both jobs (15 test, 10 rubocop) - Add `permissions: contents: read` for least-privilege - Rename test job from `ruby-3` to `test` (Ruby 4.0 is in the matrix) - Switch docker-compose image to `docker.elastic.co` registry and parameterize the version via `ES_VERSION` env var
Elasticsearch 9.x introduced breaking changes to the elasticsearch-ruby
client gem that prevent Chewy from working with ES 9 clusters:
1. The scroll API now requires scroll_id nested under a body: key
({scroll: '1m', body: {scroll_id: '...'}}) rather than as a
top-level parameter. Without this, scroll requests raise an
ArgumentError in elasticsearch-ruby 9.x.
2. API responses are now wrapped in Elasticsearch::API::Response objects
instead of returning raw hashes/arrays. Code that calls .map or
other Enumerable methods directly on responses (e.g. cat.indices)
must now call .body first to access the underlying data.
3. The '_type' field was removed from search responses in ES 7+ and
the elasticsearch-ruby 9.x client no longer includes it. All
references to '_type' have been removed: mock response helpers,
EVERFIELDS, wrapper accessors, and documentation examples.
4. The elasticsearch Ruby gem at version 9.x sends a compatible-with=9
header that ES 8.x servers reject. CI matrix entries that test
against ES 8.x must pin the gem to ~> 8.14.
Changes:
- Widen elasticsearch gem dependency from < 9.0 to < 10.0
- Restructure perform_scroll to nest scroll_id under body:, which is
compatible with both elasticsearch-ruby 8.x and 9.x
- Access .body on cat.indices response in the drop_indices test helper
- Remove '_type' from Minitest and RSpec mock response helpers,
EVERFIELDS, Index::Wrapper accessors, and README examples
- Remove obsolete '_parent' from EVERFIELDS
- Add elasticsearch version matrix to CI (9.3.1 default with 8.15.0
spot-checks) using dynamic service container images
- Pin elasticsearch gem to ~> 8.14 in CI for ES 8.x matrix entries
- Add Ruby 3.4 / Rails 8.0 / ES 8.15.0 CI matrix entry
- Add scroll body format assertion in scrolling specs
- Expand migration guide with code examples, gem pinning guidance,
corrected URLs, and more detailed upgrade steps
- Consolidate CHANGELOG entries into single item with sub-bullets
- Default docker-compose ES image to 9.3.1
- Bump version to 9.0.0
mattmenefee
force-pushed
the
elasticsearch-9-support
branch
from
98955eb to
c87f74d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
elasticsearchgem dependency from< 9.0to< 10.0to support ES 9.x clustersscroll_idunderbody:, matching the elasticsearch-ruby 9.x client interface.bodyaccessor oncat.indicesresponse for ES 9.x gem compatibility_type: mock response helpers (Chewy::Minitest::HelpersandChewy::Rspec::Helpers),EVERFIELDS,Index::Wrapperaccessors, and README examples_parentfromEVERFIELDSelasticsearchgem to~> 8.14in CI for ES 8.x matrix entries (the 9.x gem sends acompatible-with=9header that ES 8.x rejects)Breaking changes
search_query.chewynotification payload for scroll requests now nestsscroll_idunderbody:(i.e.{scroll: '1m', body: {scroll_id: '...'}}instead of{scroll: '1m', scroll_id: '...'}). Update any application code that subscribes to scroll notification payloads.'_type' => '_doc'has been removed from mock response helpers inChewy::Minitest::HelpersandChewy::Rspec::Helpers. Remove any test assertions expecting_typein mock responses.Test plan
search_query.chewynotification payload matches new formatmock_elasticsearch_responsehelpers produce valid responses without_type