Repository navigation
fix(s3, oc): ListAsync returns every page; document provider behaviour - #116
Merged
Merged
Conversation
S3 made one ListObjectsV2 call and Oracle one ListObjects call, so any prefix holding more than a page of objects was silently truncated, while the Azure and Google providers already returned the full listing. Follow the continuation token (S3) and NextStartWith (Oracle) until the listing is complete. Adds a SeaweedFS test that writes 1,050 objects and lists them all; it fails on the previous code.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
samerzughul
approved these changes
Nov 19, 2026
This was referenced Nov 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix
S3 and Oracle
ListAsyncreturned only the first page of results (1,000 keys); Azure and Google already returned everything, so the same call behaved differently per provider and callers listing large prefixes were silently truncated.NextContinuationTokenuntilIsTruncatedis false.ListObjects.NextStartWithuntil null.S3CompatibleStoreTests.List_returns_every_object_beyond_the_first_page(SeaweedFS via Testcontainers): writes 1,050 objects and expects all of them. Fails onmain, passes here; all 11 S3 tests pass locally. The Oracle change has no test (the OC tests need live credentials).Docs
docs/provider-behavior.md: method-by-method, provider-by-provider behaviour read from the code (write settings honoured,RemoteBlobfields, buffering, listing, metadata keys, delete results, URL shapes, signed-URL cost, lifetimes).docs/s3-compatible-stores.md: client settings and why, the synchronous startup calls inAddS3CloudFiles, Amazon/Spaces/MinIO/SeaweedFS differences.docs/evolving-the-contract.md: rules for changing the shared interface (new capabilities as separate interfaces, asICloudFilesLifecycleis), ranked roadmap, out of scope.CHANGELOG.md(8.1.2 → 10.0.4), README docs index and corrections (metadata keys are not the same everywhere), CLAUDE.md corrections (GetSignedUrlis implemented on every provider;OpenWriteis S3-only).Findings recorded in the docs but not fixed here: Google
OpenWritesigns a GET URL and PUTs to it; Google ignoresMetadata/Publicon writes and buffers reads in memory; S3WriteTextAsyncdropsMetadata;DeleteAsyncdisagrees on missing keys.