Tracking issues of user metadata support

[Xuanwo]

This issue is used to track the implementation of user metadata support.

Context

There are two types of metadata across various storage services.

• System Defined: metadata like last-modified, content-type and so on. Some of them are editable, like content-type, but some are non-editable like last-modified.
• User Defined: extra metadata that provided by user, can be any valid string.

OpenDAL now only supports System Defined metadata, we should add support for User Defined metadata so that users can write most useful information along with the file data. For example, write the location for this file.

Tasks

• Implement user_metadata for metadata
  • Add user_metadata: HashMap<String, String> in Metadata
  • Add user_metadata in metakey
  • Add write_with_user_metadata in capability
  • Add user_metadata integration tests
• Impelment user_metadata for services
  • s3: feat(core/services-s3): support user defined metadata #5030
  • azblob: https://learn.microsoft.com/en-us/rest/api/storageservices/set-blob-metadata
  • oss: feat(core): support user defined metadata for oss #4881
  • gcs: feat(core/services-gcs): support user defined metadata #5276
  • cos: feat(services/cos): Added user metadata support for cos service #5510
  • obs: feat(services/obs): support user defined metadata #5405
  • swift: feat(services/swift): Added user metadata support for swift service #5601
  • azdls
  • azfile
  • b2
  • fs
  • upyun
  • webdav

[meteorgan]

Hi, I'm interested in this issue.
but I have one problem to confirm：how do clients intend to use this feature, like

let mut w = op
      .writer_with("path/to/file")
      .content_type("application/octet-stream")
      .user_metadata(map)
      .await?;

?

[Xuanwo]

but I have one problem to confirm：what client wants to use this feature, like

Hi, there are many use cases of this feature. For example, a db can write their own metadata like tenant, catalog, database, table in this map. So they can query those metadata without extra meta services.

[meteorgan]

emmm, I should make it clear: I've reviewed the use cases, and would like to confirm the API for users.
should I add an function to FutureWrite like system defined metadata ?

[Xuanwo]

should I add an function to FutureWrite like system defined metadata ?

Yes, we need a new API called user_metadata for both write and writer

[meteorgan]

should I add an function to FutureWrite like system defined metadata ?

Yes, we need a new API called user_metadata for both write and writer

Ok, i'll begin addressing this issue now