Allow to specify arbitrary fields of the workspace Pod and containers using a Devfile

[l0rd]

Description

There are a few examples of workspaces Pods annotations (openshift.io/scc or runtimeClassName) or Pods spec fields (runtimeClassName, schedulerName) that users want to be able to specify.

The approach until now has been to use spec.template.attributes. But that requires code/spec changes for every new field/annotation that we want to support. To avoid that, and allow users to set arbitrary Pods fields, we could add a DevWorspace.spec.pods field for that:

kind: DevWorkspace
apiVersion: workspace.devfile.io/v1alpha2
metadata:
  name: myspacework
spec:
  started: true
+  pods:
+    metadata:
+      annotations:
+        io.openshift.userns: "true"
+        io.kubernetes.cri-o.userns-mode: "auto:size=65536;map-to-root=true"  # <-- user namespace
+        openshift.io/scc: container-build
+    spec:
+      runtimeClassName: kata
+      schedulerName: stork      
  template:
  (...)

Additional context

• Specify workspace pod schedulerName as an URL parameter eclipse-che/che#21420
• Support running a workspace in kata containers eclipse-che/che#21105
• Users should be able to configure their workspace pods securityContext capabilities eclipse-che/che#20459

Release Notes Text

Using a Devfile in the git repository, a developer can override any workspace Pod and containers fields. This includes runtimeClassName, schedulerName and kubernetes extended resources such as nvidia.com/gpu.

[amisevsk]

Opened Devfile API issue: devfile/api#860