mirror of
https://github.com/hanzo-ml/spark-operator.git
synced 2026-10-30 20:39:07 +00:00
Venkateswarlu KomirisettiandGitHub
5006eb7a3d
* feat: add REST-based spark submitter with mTLS support - Add RestSparkSubmitter with mTLS, retry with exponential backoff, and HTTP+TLS startup verification - Add typed SubmitErrorCode enum with structured error responses - Implement operator-side idempotent retry (DRIVER_POD_ALREADY_EXISTS + matching submission_id = success) - Switch retry logic from HTTP-status-based to error-code-based - Add RestSubmitter feature gate to toggle between default and REST submitter - Move feature gate check to start.go with newSparkSubmitter() helper - Document contract, request/response payloads, and error codes Signed-off-by: Venkateswarlu Komirisetti <[email protected]> * feat: integrate server-side idempotency (duplicate_submission) The submitter now handles duplicate submissions server-side, returning 200 OK with duplicate_submission=true instead of an error. Update the operator client to treat both 201 and 200 as success, and update DRIVER_POD_ALREADY_EXISTS to 409 (genuine conflict only). Signed-off-by: Venkateswarlu Komirisetti <[email protected]> * fix: check resp.Body.Close error to satisfy errcheck linter Signed-off-by: Venkateswarlu Komirisetti <[email protected]> * refactor: address PR review feedback for REST submitter - Pass context from SetupSignalHandler to newSparkSubmitter for proper signal propagation and return error instead of calling os.Exit - Move submitter-max-retries validation to PreRunE for early failure - Extract buildDriverPodTemplate/buildExecutorPodTemplate as pure functions to ensure pod templates include owner refs for both spark-submit and REST submission paths - Remove cert reload mention from comment (not yet implemented) Signed-off-by: Venkateswarlu Komirisetti <[email protected]> * refactor: simplify TLS flags and add submitter URL validation Replace --submitter-tls-cert-dir with --submitter-tls-enabled flag and direct path flags (--submitter-tls-cert-file, --submitter-tls-key-file, --submitter-tls-ca-file). Group all REST submitter validations under RestSubmitter feature gate check in PreRunE, including requiring --submitter-service-url when the gate is enabled. Signed-off-by: Venkateswarlu Komirisetti <[email protected]> * style: fix variable alignment to pass go-fmt Signed-off-by: Venkateswarlu Komirisetti <[email protected]> --------- Signed-off-by: Venkateswarlu Komirisetti <[email protected]>