feat(core): expose metadata from YAML extension files#691
Conversation
Implements support for reading and exposing the optional metadata field from YAML extension files at runtime. This allows users to extend Substrait with custom metadata at three levels: - Extension-level: via ExtensionCollection.getExtensionMetadata(urn) - Type-level: via Type.metadata() - Function-level: via ScalarFunctionVariant/AggregateFunctionVariant/ WindowFunctionVariant.metadata() Also updates substrait submodule to v0.80.0 which includes the metadata schema support. Closes #690
benbellick
force-pushed
the
benbellick/expose-yaml-metadata
branch
from
a0b8dbc to
712b0ba
Compare
|
|
||
| public abstract Map<String, Option> options(); | ||
|
|
||
| public abstract Optional<Map<String, Object>> metadata(); |
There was a problem hiding this comment.
I'm wondering whether it would be useful to have an interface instead of Object for e.g. creating custom YAML deserializers / serializers and whether we should allow to add them to the ObjectMapper in SimpleExtension.load().
Otherwise I'm fine with the changes.
There was a problem hiding this comment.
That is a useful idea, but I wonder if we can leave that for future work? I think this simple approach works for now, and it satisfies the goal of being able to expose arbitrary data to users from YAML files.
There was a problem hiding this comment.
Well, if we switch to an interface instead of Object at a later point this would be a breaking change for consumers which we probably want to avoid.
There was a problem hiding this comment.
I can see that using Object is simpler for default types like String | Integer | Boolean
2 participants
Closes #690