Recently we had a requirement deploy Fortanix KMS for etcd secret encryption at rest.
https://github.com/fortanix/k8s-sdkms-plugin/blob/master/README.md
Now we need to have the KMS plugin run manually on the master nodes. This can be done via docker or a static pod.
We wanted to have that run as static pod as we get the plugin highly available and have Kubelet manage its state. Running through docker is another option, but we worry about docker getting deprecated with future versions.
Running as static pod looks promising if we think about future.
This is just 1 example of static pod requirement, but there can be many others.
We would like to have a provision of running static pods on the PMK clusters.