RCL as Templating Language?

[JanBeelte]

Hello,

I just discovered this project and find the goals and ideas super interesting, first of all thanks for the great work you are doing!
I am currently looking for a solution for the automation of generating yaml files from (user controlled) configuration data.

I envision something like:

1. json data (+ optionally schema data for validation) - the values to be filled
2. an RCL file that takes the json input and generates the final output

Now the questions:

1. Does RCL support input parameters via command-line or json that can be processed during generation?
2. Can RCL be used as a library from the rust side? Ideally I am envisioning a solution comparable to a template library like e.g. https://docs.rs/liquid/latest/liquid/ where I can configure the input data from rust-side directly.

[ruuda]

1. In your RCL document, you can import "parameters.json" and use them where needed.
2. Yes, see Using rcl in another project #79.

[JanBeelte]

Fantastic, thanks for the quick response!

Have you ever thought about advertising this input feature more explicitly and maybe even exposing it not only via a file (which probably has to live in the same directory?) but also via a dedicated input (similar to other templating libraries)?

I think such a feature could make RCL a serious competitor to other templating languages :)

Best,
Jan

[ruuda]

Have you ever thought about advertising this input feature more explicitly

I’m curious to know, where did you look for it?

It is used in the example at https://rcl-lang.org/, it’s mentioned under the “Remove repetition” heading, and it has a chapter in the manual, though it’s not part of the tutorial, so maybe it’s not clear enough?

and maybe even exposing it not only via a file but also via a dedicated input (similar to other templating libraries)?

For the rcl command-line program, I have thought about the ability to pass variables, and I can see the appeal of it, but it would open up a can of worms. RCL documents on purpose take no input. It means they can always be evaluated, it means that there is only one code path that matters, and it means that the configuration is self-contained. When some of the input needs to be passed in separately, that’s configuration that now lives in an ad-hoc place, rather than in the document.

If you use the Rust crate though, then you can set whatever variables you like, and this is how rcl query works.

which probably has to live in the same directory?

It does not have to live in the same directory, it can live anywhere on the file system. By default RCL will not load files outside of the working directory, but you can disable that restriction with --sandbox=unrestricted.

[polarathene]

When some of the input needs to be passed in separately, that’s configuration that now lives in an ad-hoc place, rather than in the document.

I'm not sure how that differs from the use of import? You could have some way to restrict how input is received (projects using HCL for example would declare variable "SOME_ENV" {}, another example is a env() function to source from env).

---

I haven't touched KCL in a while (I very roughly documented a use-case with comparison to yq and generating config based on files I was interested in at runtime). KCL has option() for indicating inputs, these are defined top-level and are configured as CLI options to the kcl command.

There's two other related features to option() in KCL:

• A YAML file of inputs can be used:

  # /config/kcl.yaml
  kcl_cli_configs:
    files:
      - /config/dkim/keys/example.com/mail-ec/rspamd.k
      - /config/dkim/keys/example.com/mail-rsa/rspamd.k
      - /config/dkim/keys/example.org/mail-rsa/rspamd.k

  # Now works without requiring `find` + `xargs`
  $ kcl /config/main.k --setting /config/kcl.yaml
  
  domain:
    example.com:
      selectors:
      - path: /config/dkim/keys/example.com/mail-ec/private.pem
        selector: mail-ec
      - path: /config/dkim/keys/example.com/mail-rsa/private.pem
        selector: mail-rsa
    example.org:
      selectors:
      - path: /config/dkim/keys/example.org/mail-rsa/private.pem
        selector: mail-rsa

• Giving kcl a list of KCL config files, which is what I demonstrate using here for building a unified config file from smaller .k files (effectively dynamic path inputs for data not known ahead of time to be declared in main.k):

  $ docker run --rm -v "/tmp/config:/config:ro" kcllang/kcl:main bash -c \
    'find /config/dkim/keys -type f -name "rspamd.k" | xargs kcl /config/main.k'
  
  domain:
    example.com:
      selectors:
      - path: /config/dkim/keys/example.com/mail-ec/private.pem
        selector: mail-ec
      - path: /config/dkim/keys/example.com/mail-rsa/private.pem
        selector: mail-rsa
    example.org:
      selectors:
      - path: /config/dkim/keys/example.org/mail-rsa/private.pem
        selector: mail-rsa

So KCL is more flexible there and might be a good example for reference (and any user raised issues about it, I believe my interactions there at the time ran into some UX issues). FWIW one of the responses noted that previously the above wasn't supported well due to a security concern.

I shared on that linked discussion how I had been working around the issue initially when diving into KCL by using file redirection with cat:

kcl <(cat main.k $(find /config/dkim/keys -type f -name "rspamd.k"))

---

I've not looked into RCL much myself, just drive-by landed on this issue 😅 In my case I'm just looking for something nice/flexible for ephemeral generation of configs (which can just be simple objects with mappings to multi-line strings permutations to feed into Github Action inputs).

I noticed an example generating a GHA workflow but presently no marketplace action for using RCL within a workflow (I guess without support for inputs to the RCL command, there isn't much convenience from a dedicated action vs grabbing the executable from GH releases?).

[ruuda]

If you want to use RCL from GitHub Actions, the fastest way right now is indeed to download a binary. If you want to use RCL to generate repetitive GitHub Actions workflows, for that RCL itself does not need to be available on CI, you can just run it locally and commit the resulting workflow files. If you want to verify on CI that the generated files are up to date, for example with rcl build --check, then of course you do need to have rcl there.