Skip to content

Support the binary store API (alpha) from Dapr 1.19 #1296

Closed

Description

@nelson-parente

Summary

Dapr 1.19 adds the binary store building block as an alpha API. An app uses it to store, read and delete files as byte streams through a binary store component. The Python SDK has no client API for it yet. The .NET SDK already supports it, so its implementation is a reference for this work.

Runtime API

gRPC, in the Dapr service (binarystore.proto):

  • SetBinaryFileAlpha1 is a client stream. The first message must contain initial_request with component_name, file_name and overwrite, and it must arrive within 5 seconds. Each later message contains one payload chunk (common.v1.StreamPayload). Its seq starts at 0 and goes up by 1 for each chunk. If overwrite is false (the default), the call fails when the file exists. If overwrite is true, the call replaces the file.
  • GetBinaryFileAlpha1 returns a server stream of payload chunks of up to 32 KiB, with seq from 0.
  • DeleteBinaryFileAlpha1 is a unary call.

HTTP, on /v1.0-alpha1/binarystore/{name}/{fileName}:

  • PUT stores the request body and replaces an existing file.
  • POST stores the request body, and returns 409 if the file exists.
  • GET returns the file as application/octet-stream.
  • DELETE deletes the file.

Components in 1.19: binarystore.aws.s3, binarystore.azure.blobstorage, binarystore.azure.datalake, binarystore.gcp.bucket, binarystore.oci.objectstorage and binarystore.in-memory.

Reference: the .NET SDK

Related links: the proposal (dapr/proposals#88), the runtime implementation (dapr/dapr#10367) and the components (dapr/components-contrib#4559 and dapr/components-contrib#4637). dapr/dotnet-sdk#1934 stores workflow data references in the binary store. That is a separate feature (dapr/dapr#8706), and it is not part of this issue.

Scope

  • Generate the protos from Dapr 1.19 with tools/regen_grpcclient.sh (for example DAPR_BRANCH=release-1.19). The script takes every runtime/v1 proto, so it picks up binarystore.proto. main and release-1.19 do not have it yet.
  • Add set, get and delete *_alpha1 methods to the sync client (dapr/clients/grpc/client.py) and the async client (dapr/aio/clients/grpc/client.py). Set takes bytes or a file-like object and an overwrite flag. Get returns bytes or an iterator of chunks.
  • Send the upload chunks in order after the initial request, with seq from 0.
  • Add tests, and an example in examples/ that uses the binarystore.in-memory component.
  • Add docs to the Python SDK section of dapr/docs.

The API is alpha, so its names and behavior can change in a later release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions