GKE auto-detection logic can get it wrong

[caseydavenport]

Namely, there are some APIs in the networking.gke.io API group that can be installed even on non-GKE cluster.

See: https://github.com/nais/fqdn-policy

[bilalahmadpolitiet]

This issue was caused by the following CRD resource:

$ kubectl api-resources | grep networking.gke.io
fqdnnetworkpolicies                                                                                                        networking.gke.io/v1alpha3                  true         FQDNNetworkPolicy

During our upgrade of k8s version non of the pods managed to startup and was stuck in ContainerCreating with the following error message:

Warning  FailedCreatePodSandBox  82s                 kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "aa9df04beb254d1587aec038e68e51dfba9b887d6cfa0933a0a9c186b2c9be70": plugin type="calico" failed (add): failed to find plugin "calico" in path [/opt/cni/bin]

Because binDir was set to /home/kubernetes/bin instead of /opt/cni/bin.