Skip to content

fix(s3): adjust SQS bucket notification timestamp format#10032

Merged
bpandola merged 1 commit into
getmoto:masterfrom
milas:milas/bucket-timestamps
May 23, 2026
Merged

fix(s3): adjust SQS bucket notification timestamp format#10032
bpandola merged 1 commit into
getmoto:masterfrom
milas:milas/bucket-timestamps

Conversation

@milas
Copy link
Copy Markdown
Contributor

@milas milas commented May 20, 2026

Fix S3 bucket notifications for SQS messages to have timestamps in the exactly format expected by AWS.

This is mentioned in the AWS docs1 (already linked in the source):

"eventTime":"The time, in ISO-8601 format (for example, 1970-01-01T00:00:00.000Z) when Amazon S3 finished processing the request"

In moto terms, this is our friend iso_8601_datetime_with_milliseconds

Copy link
Copy Markdown
Collaborator

@bpandola bpandola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@milas Was there a sequence of API calls resulting in an error that led you to this PR? If so, it'd be good to capture that in a test.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 22, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.22%. Comparing base (ef42e0e) to head (6cb06c4).
⚠️ Report is 12 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10032      +/-   ##
==========================================
+ Coverage   93.21%   93.22%   +0.01%     
==========================================
  Files        1328     1329       +1     
  Lines      120736   120999     +263     
==========================================
+ Hits       112541   112799     +258     
- Misses       8195     8200       +5     
Flag Coverage Δ
servertests 29.05% <33.33%> (+0.12%) ⬆️
unittests 93.19% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@milas
Copy link
Copy Markdown
Contributor Author

milas commented May 22, 2026

Ah, apologies for vagueness. It was a discovered via downstream expectation when using motoserver!

Specifically, I was using moto for an integration test w/ Vector's S3 source (which uses SQS bucket notifications).

The Rust type's deserializer is strict and refuses unless it's got the Z at the end etc, so it refuses to handle the event notification messages currently. With these changes, it works as expected 🎉

// https://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html
#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(rename_all = "PascalCase")]
pub struct S3Event {
    pub records: Vec<S3EventRecord>,
}

#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct S3EventRecord {
    pub event_version: S3EventVersion,
    pub event_source: String,
    pub aws_region: String,
    pub event_name: S3EventName,
    pub event_time: DateTime<Utc>,

    pub s3: S3Message,
}

https://github.com/vectordotdev/vector/blob/2084d534f269bb1fa2d508bbc28a634d57d4dc32/src/sources/aws_s3/sqs.rs#L1004C1-L1021C2

@bpandola bpandola added this to the 5.2.2 milestone May 23, 2026
@bpandola bpandola merged commit 9614809 into getmoto:master May 23, 2026
129 of 130 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants