Status of testing of Apache Airflow Helm Chart 1.20.0rc2

[jscheffl]

We are kindly requesting that contributors to Apache Airflow Helm Chart 1.20.0rc1 help test the RC.

Please let us know by commenting if the issue is addressed in the latest RC.

• Add missing support for: securityContexts and containerLifecycleHooks (#60677): @henry3260
• Remove automatic KUBERNETES_ENVIRONMENT_VARIABLES and KUBERNETES_SECRETS prefixes from chart helpers (#60750): @uplsh580 @jscheffl
• Drop support for all Airflow versions below 2.11 in Helm Chart (#61018): @Miretpl @jscheffl
  Linked issues:
  • Apache Airflow <= 2.10 support removal from Helm Chart (#60141)
• Add workers.celery.containerLifecycleHooks & workers.kubernetes.containerLifecycleHooks (#61369): @Miretpl
• Add workers.celery.podDisruptionBudget (#61414): @Miretpl
• Add LOG_MAX_SIZE environment variables to log groomer (#61559): @pcorliss
• Add workers.celery.keda section (#61820): @Miretpl
• feat(docker): add minute-level log retention to clean-logs script (#61855): @n-badtke-cg @pugarte7
  Linked issues:
  • Container Log Groomer should be configurable with a log retention time <1d - OTEL Collector FileLogReceiver (#61814)
• Add workers.celery.kerberosSidecar & workers.kubernetes.kerberosSidecar sections (#61881): @Miretpl
• Add workers.celery.resources & workers.kubernetes.resources (#61890): @Miretpl
• Add workers.celery.terminationGracePeriodSeconds & workers.kubernetes.terminationGracePeriodSeconds (#61892): @Miretpl
• Add workers.celery.nodeSelector & workers.kubernetes.nodeSelector (#61957): @Miretpl
• Allow to use short SPDX licence identifier for selected files (#62073): @potiuk
• Helm chart 1.19.0 released (#62091): @jedcunningham
• Fix webserver.defaultUser.enabled=false not honored (#62143): @jedcunningham @jscheffl
  Linked issues:
  • Remove breaking change for createUser in Chart (#61337)
• Refactor Git-Sync livenessProbe & deprecate readinessProbe & add startupProbe (#62334): @Miretpl @potiuk @jeff-cook
  Linked issues:
  • Can't expose port for git-sync metrics (#62592)
  • Support Native Kubernetes sidecars in our Chart (#35154)
• Add ingress deprecation warnings for apiServer, statsd, and pgbouncer (#62490): @jedcunningham
• Cleanup Helm Chart documentation (#62544): @Miretpl @jscheffl
  Linked issues:
  • Drop support for all Airflow versions below 2.11 in Helm Chart (#61018)
• Fix tpl rendering for TLS hosts in ingress templates #62358 (#62548): @haseebmalik18
• Fix all build-system/requires including transitive dependencies (#62570): @potiuk
• [main] Upgrade important CI environment (#62610): @potiuk
• Fix chart NOTES.txt showing deprecation warnings only without secret key (#62722): @jedcunningham
• Chart: warn on deprecated per-component securityContext values (#62729): @jedcunningham
• [main] Upgrade important CI environment (#62792): @potiuk
• Chart: Support Helm template expressions in podAnnotations (#63019): @devgonvarun @XD-DENG
  Linked issues:
  • helm checksums support in Helm Chart (#62698)
• chart: api-server Deployment always includes spec.replicas when HPA is enabled (#63187): @thejoeejoee
  Linked issues:
  • chart: api-server Deployment always includes spec.replicas when HPA is enabled (#63186)
• Remove JWT secrets from triggerer, worker and dag-processor (#63204): @deepujain
• Chart: Default airflow version to 3.1.8 (#63392): @vatsrahul1001
• More restrictive chart rendering logic (#63464): @Miretpl @jscheffl
  Linked issues:
  • Drop support for all Airflow versions below 2.11 in Helm Chart (#61018)
• Add *.iml to .gitignore in all distributions (#63636): @potiuk
• Add missing deprecation warnings for workers section (#63659): @Miretpl @csp33
  Linked issues:
  • Can't configure Kubernetes and Celery workers in Helm Chart (#28880)
  • Add workers.celery.nodeSelector & workers.kubernetes.nodeSelector (#61957)
• Update Helm Chart release notes for 1.20.0 (#63759): @jscheffl

Thanks to all who contributed to the release (probably not a complete list!):
@devgonvarun @potiuk @henry3260 @haseebmalik18 @jscheffl @vatsrahul1001 @pcorliss @deepujain @Miretpl @thejoeejoee @jedcunningham @XD-DENG @pugarte7 @uplsh580 @csp33 @n-badtke-cg @jeff-cook

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[jscheffl]

Checked Chart in our environment and looks good.

[n-badtke-cg]

For #61814 and #61559 being effective in the Helm Chart, there needs to be a new Airflow container image build. As of now, these features are just ignored.

[haseebmalik18]

Confirmed #62548 is working correctly, templated TLS hosts render properly in all ingress templates.

[henry3260]

#60677 works fine!

[Miretpl]

All my changes:

1. Add workers.celery.containerLifecycleHooks & workers.kubernetes.containerLifecycleHooks #61369
2. Add workers.celery.podDisruptionBudget #61414
3. Add workers.celery.keda section #61820
4. Add workers.celery.kerberosSidecar & workers.kubernetes.kerberosSidecar sections #61881
5. Add workers.celery.resources & workers.kubernetes.resources #61890
6. Add workers.celery.terminationGracePeriodSeconds & workers.kubernetes.terminationGracePeriodSeconds #61892
7. Add workers.celery.nodeSelector & workers.kubernetes.nodeSelector #61957
8. Refactor Git-Sync livenessProbe & deprecate readinessProbe & add startupProbe #62334
9. Cleanup Helm Chart documentation #62544
10. More restrictive chart rendering logic #63464
11. Add missing deprecation warnings for workers section #63659

looks good

[jscheffl]

Note, keeping the same issue for RC2, no additional fixes just the creation of the package is fixed.

[potiuk]

All my confirmed

[jscheffl]

Thank you everyone. New Helm-Chart is released.

I invite everyone to help improve providers for the next release, a list of open issues can be found here.

[n-badtke-cg]

For #61814 and #61559 being effective in the Helm Chart, there needs to be a new Airflow container image build. As of now, these features are just ignored.

#63859 (comment)

Sorry, small correction: #61559 is already effective with Airflow 3.1.8 . Only #61814 is missing with 3.1.8