From db98d6a4f45fa51d51e152c8ca62ec0058e3dbfb Mon Sep 17 00:00:00 2001 From: wbsong111 Date: Mon, 9 Dec 2024 14:48:23 +0900 Subject: [PATCH] Add custom-values.yaml, CUSTOM-README.md --- charts/spark-operator/CUSTOM-README.md | 49 +++++++++++++++++ charts/spark-operator/custom-values.yaml | 69 ++++++++++++++++++++++++ 2 files changed, 118 insertions(+) create mode 100644 charts/spark-operator/CUSTOM-README.md create mode 100644 charts/spark-operator/custom-values.yaml diff --git a/charts/spark-operator/CUSTOM-README.md b/charts/spark-operator/CUSTOM-README.md new file mode 100644 index 0000000..bdf9b4b --- /dev/null +++ b/charts/spark-operator/CUSTOM-README.md @@ -0,0 +1,49 @@ + +# Spark Operator 배포 배포 + +## 1. 배포 방법 + +- 명령어 + + ``` sh + $ helm upgrade --instal spark-operator ../spark-operator \ + --version=2.0.2 \ + --namespace spark-operator \ + --create-namespace \ + -f custom-values.yaml + ``` + + + +## 2. custom-values.yaml 설명 + +- custom-values.yaml에 정의된 값에 대한 설명이다. + +### 1) 이미지 설정 +| Name | 설명 | 기본값 | +| ---------------------- | ------------------------------------------------------------ | ------ | +| `image.registry` | 오프라인 설치 시에 설정.
paasup 설치시에는 "paasup.io"으로 설정 | `""` | + +### 2) Controller 설정 + +| Name | 설명 | 기본값 | +| ------------------------ | ---------------------------------- | --------- | +| `contoller.worker` | spark contoller의 thread 설정
SparkApplication 객체를 동시에 처리할 수 있는 수 설정 | `10` | +| `contoller.logLevel` | controller의 log level 설정 | `info` | +| `controller.uiService.enable` | Spark Web UI의 Service 생성 여부 | `true` | +| `controller.uiIngress.enable` | Spark Web UI의 Ingress 생성 여부 | `true` | +| `controller.uiIngress.urlFormat` | 생성할 Ingress의 접속 URL 포맷 설정 | `"spark.example.org/{{$appNamespace}}/{{$appName}}"` | +| `controller.resources` | controller pod의 resource 설정. | `custom-values.yaml 참조` | + +### 3) webhook 설정 + +| Name | 설명 | 기본값 | +| ------------------------ | ---------------------------------- | --------- | +| `webhook.resources` | webhook pod의 resource 설정. | `custom-values.yaml 참조` | + + +### 4) spark 설정 + +| Name | 설명 | 기본값 | +| ------------------------ | ---------------------------------- | --------- | +| `spark.jobNamespace` | spark application이 배포될 수 있는 namespace를 설정. | `[]` | diff --git a/charts/spark-operator/custom-values.yaml b/charts/spark-operator/custom-values.yaml new file mode 100644 index 0000000..52325ee --- /dev/null +++ b/charts/spark-operator/custom-values.yaml @@ -0,0 +1,69 @@ +image: + registry: docker.io + repository: kubeflow/spark-operator + # version 목록 + ## spark : 3.5.2 + ## java : 11 + ## python : 3.8.10 + tag: "2.0.2" + + +controller: + # spark contoller의 thread 설정 + # SparkApplication 객체를 동시에 처리할 수 있는 수 + workers: 10 + + # controller log level 설정 + logLevel: info + + # spark application 실행 시 접근 가능한 UI Ingress 활성화 + ## spark 동작 중에만 접근 가능한 UI + uiService: + enable: true + uiIngress: + enable: true + urlFormat: "spark.example.org/{{$appNamespace}}/{{$appName}}" + + # controller resource 설정 + resources: + limits: + cpu: 100m + memory: 300Mi + requests: + cpu: 100m + memory: 300Mi + + + # Workqueue rate limiter configuration forwarded to the controller-runtime Reconciler. + workqueueRateLimiter: + # -- Specifies the average rate of items process by the workqueue rate limiter. + bucketQPS: 50 + # -- Specifies the maximum number of items that can be in the workqueue at any given time. + bucketSize: 500 + maxDelay: + # -- Specifies whether to enable max delay for the workqueue rate limiter. + # This is useful to avoid losing events when the workqueue is full. + enable: true + # -- Specifies the maximum delay duration for the workqueue rate limiter. + duration: 6h + +webhook: + resources: + limits: + cpu: 100m + memory: 300Mi + requests: + cpu: 100m + memory: 300Mi + +spark: + jobNamespaces: [] + +# prometheus: +# metrics: +# enable: false +# podMonitor: +# create: false +# labels: {} + +