Existing CI/CD process can be improved in terms of transparency, simplicity and speed.
Transparency:
- The public link to view the tests log and status is a bit hidden (http://prow.feast.ai). Clicking the status badge e.g. green badge for successful tests, should ideally link to the the CI/CD log so developers can easily fix the issues.
To encourage more contribution, I think we should make it more transparent what are being tested and allow tests log to be more accesible.
Simplicity:
- Existing integration tests include provisioning and destroying infrastructure using Terraform which takes up to 15 minutes in addition to running the actual tests.
- Existing CI pipeline https://github.com/gojek/feast/blob/v0.1.0/testing/integration/tests/redis-bq-dataflow/workflow.yaml can be simplified if we focus on testing Feast deployment via Helm charts only (w/o the required infrastructure provisioning, I think it's ok to assume Kubernetes Cluster and storage infrastructure e.g. BigQuery have been set up before installing Feast)
Separating infrastructure provisioning with the actual Feast deployment will make tests simpler.
Speed:
- Current CI pipeline does not support concurrent testing or it may cause the tests result to be unreliable if tests are run concurrently. This slows down development lifecyle because we have to wait for running test to complete before making any small commit changes (1 whole test cycle can take 20 minutes)
- Using Argo workflow allows CI pipeline to be defined in a flexible manner. However, it is useful to relate the CI process to various Git events such as push to master branch, commit tag (For instance, we may want to run a specific step in CI only when it is a commi tag). Doing this via Argo requires a lot of extra parameters and if-elses, which may make the workflow.yaml less easily understood.
- Also current tests are not automatically triggered on pull requests or push to the main repository. We need to automate this to discover bugs faster.
Update CI pipelines to allow concurrent run for faster development lifecyle and consider using other tools to define the CI steps more simply. So updates can be made faster
Existing CI/CD process can be improved in terms of transparency, simplicity and speed.
Transparency:
To encourage more contribution, I think we should make it more transparent what are being tested and allow tests log to be more accesible.
Simplicity:
Separating infrastructure provisioning with the actual Feast deployment will make tests simpler.
Speed:
Update CI pipelines to allow concurrent run for faster development lifecyle and consider using other tools to define the CI steps more simply. So updates can be made faster