4879adacdb
* new local website docs Signed-off-by: vara-bonthu <[email protected]> * homepage UX: ribbon logo, architecture diagram, CNCF footer, code-block fixes Signed-off-by: vara-bonthu <[email protected]> * docs: favicon, social cards, image optimization, consistent paragraph sizing Signed-off-by: vara-bonthu <[email protected]> * Potential fix for pull request finding Signed-off-by: Vara Bonthu <[email protected]> * docs: address PR review feedback (brand link, storage guard, cron, typos) Signed-off-by: vara-bonthu <[email protected]> * docs: address PR review (spec.mode, jar typo, CSS @import order, external-link host check, brand-link fallback) + strict docs build gate Signed-off-by: vara-bonthu <[email protected]> * docs: embed Kubeflow Spark Operator intro video on homepage Signed-off-by: vara-bonthu <[email protected]> * docs: clarify homepage video is the CNCF KubeCon NA 2024 best-practices talk Signed-off-by: vara-bonthu <[email protected]> * docs: fix header logo opening homepage in a new tab (exclude brand link from external-link handling) Signed-off-by: vara-bonthu <[email protected]> * docs: fix header logo navigation and point Kubeflow.org links to kubeflow.org Signed-off-by: vara-bonthu <[email protected]> * ci: add documentation build and link-check workflow Signed-off-by: vara-bonthu <[email protected]> * ci(docs): make linkcheck non-blocking; fix broken links and ignore JS-rendered anchors Signed-off-by: vara-bonthu <[email protected]> * docs: fix benchmarking links (Data on EKS benchmark kit and contributor handle) Signed-off-by: vara-bonthu <[email protected]> * docs: refine homepage copy and add muted color accents to landing cards Signed-off-by: vara-bonthu <[email protected]> * docs: move website under docs/website and manage Python deps with uv - Move the Sphinx site from docs/ to docs/website/ - Replace requirements.txt with pyproject.toml (uv 'docs' group) and uv.lock - Build with uv in Read the Docs, the Makefile targets, and CI - Leave pre-existing docs/{api-docs,kustomize-installation,release}.md in place Signed-off-by: vara-bonthu <[email protected]> * docs: improve sidebar navigation readability and hierarchy Signed-off-by: vara-bonthu <[email protected]> --------- Signed-off-by: vara-bonthu <[email protected]> Signed-off-by: Vara Bonthu <[email protected]> Co-authored-by: Hanzo Dev <[email protected]>
2.8 KiB
User Guide
In-depth guides for writing, configuring, scheduling, monitoring, and operating Spark applications with the Kubeflow Spark Operator.
Working with SparkApplications
:::::{grid} 1 1 2 2 :gutter: 3
::::{grid-item-card} Using SparkApplications 🔗 using-sparkapplication :link-type: doc
Create, list, check the status of, and delete SparkApplication objects
::::
::::{grid-item-card} Writing a SparkApplication 🔗 writing-sparkapplication :link-type: doc
The full anatomy of a SparkApplication spec — types, deps, pods, volumes, and more
::::
::::{grid-item-card} Working with SparkApplications 🔗 working-with-sparkapplication :link-type: doc
Restart policies, failure handling, and managing running applications ::::
::::{grid-item-card} Running on a Schedule 🔗 running-sparkapplication-on-schedule :link-type: doc
Use ScheduledSparkApplication to run Spark jobs on a cron schedule
::::
:::::
Operating the Operator
:::::{grid} 1 1 2 2 :gutter: 3
::::{grid-item-card} Customizing Spark Operator 🔗 customizing-spark-operator :link-type: doc
Tune operator behavior, flags, and Helm chart values ::::
::::{grid-item-card} Enabling Leader Election 🔗 leader-election :link-type: doc
Run the operator in high-availability mode with leader election ::::
::::{grid-item-card} Running Multiple Instances 🔗 running-multiple-instances-of-the-operator :link-type: doc
Deploy several operator instances scoped to different namespaces ::::
::::{grid-item-card} Resource Quota Enforcement 🔗 resource-quota-enforcement :link-type: doc
Enforce Kubernetes resource quotas on Spark workloads ::::
:::::
Monitoring & Scheduling
:::::{grid} 1 1 2 2 :gutter: 3
::::{grid-item-card} Monitoring with Prometheus & JMX 🔗 monitoring-with-jmx-and-prometheus :link-type: doc
Export Spark metrics to Prometheus using the JMX exporter ::::
::::{grid-item-card} Volcano Integration 🔗 volcano-integration :link-type: doc
Batch scheduling and gang scheduling with Volcano ::::
::::{grid-item-card} YuniKorn Integration 🔗 yunikorn-integration :link-type: doc
Resource-aware batch scheduling with Apache YuniKorn ::::
:::::
Integrations
:::::{grid} 1 1 2 2 :gutter: 3
::::{grid-item-card} Google Cloud Storage & BigQuery 🔗 gcp :link-type: doc
Read and write data with GCS and BigQuery on GKE ::::
::::{grid-item-card} Kubeflow Notebooks 🔗 notebooks-spark-operator :link-type: doc
Run PySpark jobs from Kubeflow Notebooks ::::
:::::
:hidden:
:maxdepth: 1
using-sparkapplication
writing-sparkapplication
working-with-sparkapplication
running-sparkapplication-on-schedule
customizing-spark-operator
leader-election
running-multiple-instances-of-the-operator
resource-quota-enforcement
monitoring-with-jmx-and-prometheus
volcano-integration
yunikorn-integration
gcp
notebooks-spark-operator