Skip to content

PBM-1472 sanitize double slashes in storage config parts#1268

Open
jcechace wants to merge 6 commits intopercona:devfrom
jcechace:PBM-1472-stg-path-double-slash
Open

PBM-1472 sanitize double slashes in storage config parts#1268
jcechace wants to merge 6 commits intopercona:devfrom
jcechace:PBM-1472-stg-path-double-slash

Conversation

@jcechace
Copy link
Collaborator

Add storage.TrimSlashes() helper to strip leading/trailing '/' from
bucket, prefix, and container config values. Extra slashes in these
values cause S3 signature errors and backup discovery failures because
none of the storage SDKs sanitize these inputs.
Trim leading/trailing slashes from Bucket and Prefix in s3.Config.Cast().
Add TestCastSlashTrimming tests.
Trim leading/trailing slashes from Bucket and Prefix in mio.Config.Cast().
Add TestCastSlashTrimming tests.
Trim leading/trailing slashes from Bucket and Prefix in gcs.Config.Cast().
Add TestCastSlashTrimming tests.
Trim leading/trailing slashes from Container and Prefix in
azure.Config.Cast(). Add TestCastSlashTrimming tests.
Trim leading/trailing slashes from Bucket and Prefix in oss.Config.Cast().
Add TestCastSlashTrimming tests.
@jcechace
Copy link
Collaborator Author

jcechace commented Feb 17, 2026

@boris-ilijic sanitizing in Cast() works well but...

  • pbm config --file will properly sanitize when creating config
  • pbm config --set storage.s3.bucket=bcp/ will not

PBM will still work fine (as Cast will normalize it before storage is used). However there are few weaknesses

pbm backup
Starting backup "2026-02-17T12:14:35Z"....Backup "2026-02-17T12:14:35Z" saved to remote store (path: "http://minio:9000/bcp//pbme2etest")

pbm status

Main storage:
  Type:       S3
  Region:     us-east-1
  Path:       http://minio:9000/bcp//pbme2etest

Here Path() still prints the path based on users actual configuration. Not neccessarily bad -- at least users can see their typo

A fix for pbm config --set storage.s3.bucket=bcp/ would be slightly akward as it owuld have to check the possible suffixes for which to trim the slashes. So in config.SetConfigVar we would need something like

if strings.HasSuffix(key, ".bucket") || strings.HasSuffix(key, ".container") || strings.HasSuffix(key, ".prefix") {
    s := strings.Trim(v.(string), "/")
}

which seems a bit fragile -- but that is the case for the entire function

@jcechace jcechace marked this pull request as ready for review February 17, 2026 20:32
@boris-ilijic
Copy link
Member

It's better to avoid having invalid config setting (eg. t1/ in example) and then mutate it all the time before the usage. That will be potential source of bugs, so that needs to be fixed.

$ pbm config --set storage.s3.prefix=t1/
[storage.s3.prefix=t1/]
$ pbm config
storage:
  type: s3
  s3:
    bucket: abc
    prefix: t1/

That check doesn't need to be fragile, it can be just case like existing one:

case "storage.filesystem.path":
if v.(string) == "" {
return errors.New("storage.filesystem.path can't be empty")
}

	case "storage.s3.prefix", "storage.s3.bucket":

Optionally:
I'd prefer to fail if param is invalid within the file, more than adjust, so from the ticket maybe better fix should be:

PBM validates bucket/prefix options and returns error about incorrect format

So something like this:
https://github.com/percona/percona-backup-mongodb/blob/01eb9981f58a6875f634eba60dc15b51b2d24037/pbm/storage/mio/config.go#L102C2-L104C3

@jcechace
Copy link
Collaborator Author

@boris-ilijic

That check doesn't need to be fragile, it can be just case like existing one:
There is 6x2=12 cases which need to be either validated or sanitized -- I'm for sanitization since it's basically the same step, just less bothersome for the user.

But overall I agree that this shouldn't likely be in Cast(). Will change that part

@boris-ilijic
Copy link
Member

But overall I agree that this shouldn't likely be in Cast(). Will change that part

While you are changing, it should be clearer and easier to report the error than to sanitize.

@boris-ilijic
Copy link
Member

@jcechace

I'm for sanitization since it's basically the same step, just less bothersome for the user.

Sorry I missed your comment, it's hidden by formatting in gray, see it above.
User doesn't care imo, he/she will be informed, and then correction will be made. I would do it as it is easier for maintenance, but it's up to you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments