It is also possible to run spark on kubernetes (in combination with distributed storage like S3 or minio):
https://spark.apache.org/docs/latest/running-on-kubernetes.html

It will require some time to get your head around this, but note that you can do this with or without the spark operator installed on your kubernetes cluster.

Marc