update kubeflow dip-catalog

This commit is contained in:
ChanghoWoo
2025-01-13 02:31:27 +00:00
parent 1dc1181a03
commit 5451f16d72
1959 changed files with 602337 additions and 0 deletions
+5
View File
@@ -0,0 +1,5 @@
reviewers:
- annajung
- juliusvonkohout
approvers:
- juliusvonkohout
+4
View File
@@ -0,0 +1,4 @@
The contrib directory contains third-party applications maintained externally.
All contrib components must meet the [requirements](https://github.com/kubeflow/manifests/blob/master/proposals/20220926-contrib-component-guidelines.md#component-requirements)
to be considered active and failure to meet them will result in deprecation and removal from the Kubeflow manifest repository.
+23
View File
@@ -0,0 +1,23 @@
BENTOML_YATAI_IMAGE_BUILDER_VERSION ?= 1.2.28
BENTOML_YATAI_DEPLOYMENT_VERSION ?= 1.1.21
BENTOML_HELM_CHART_REPO ?= https://bentoml.github.io/helm-charts
.PHONY: bentoml-yatai-stack/bases
bentoml-yatai-stack/bases: clean-kustomize
mkdir -p bentoml-yatai-stack/bases/yatai-image-builder
cd bentoml-yatai-stack/bases/yatai-image-builder && helm template --skip-tests yatai-image-builder-crds yatai-image-builder-crds --repo ${BENTOML_HELM_CHART_REPO} --namespace kubeflow --version ${BENTOML_YATAI_IMAGE_BUILDER_VERSION} > resources.yaml
cd bentoml-yatai-stack/bases/yatai-image-builder && helm template --skip-tests -f ../../../sources/yatai-image-builder-values.yaml yatai-image-builder yatai-image-builder --repo ${BENTOML_HELM_CHART_REPO} --namespace kubeflow --version ${BENTOML_YATAI_IMAGE_BUILDER_VERSION} >> resources.yaml
cp sources/kustomization-template.yaml bentoml-yatai-stack/bases/yatai-image-builder/kustomization.yaml
mkdir -p bentoml-yatai-stack/bases/yatai-deployment
cd bentoml-yatai-stack/bases/yatai-deployment && helm template --skip-tests yatai-deployment-crds yatai-deployment-crds --repo ${BENTOML_HELM_CHART_REPO} --namespace kubeflow --version ${BENTOML_YATAI_DEPLOYMENT_VERSION} > resources.yaml
cd bentoml-yatai-stack/bases/yatai-deployment && helm template --skip-tests -f ../../../sources/yatai-deployment-values.yaml yatai-deployment yatai-deployment --repo ${BENTOML_HELM_CHART_REPO} --namespace kubeflow --version ${BENTOML_YATAI_DEPLOYMENT_VERSION} >> resources.yaml
cp sources/kustomization-template.yaml bentoml-yatai-stack/bases/yatai-deployment/kustomization.yaml
.PHONY: clean-kustomize
clean-kustomize:
rm -rf bentoml-yatai-stack/bases
.PHONY: test
test:
./test.sh
+3
View File
@@ -0,0 +1,3 @@
approvers:
- yubozhao
- juliusvonkohout
+86
View File
@@ -0,0 +1,86 @@
# BentoML on Kubeflow
Starting with the release of Kubeflow 1.7, BentoML provides a native integration with Kubeflow through [Yatai](https://github.com/bentoml/yatai-deployment). This integration allows you to package models trained in Kubeflow Notebooks or Pipelines as [Bentos](https://docs.bentoml.org/en/latest/concepts/bento.html), and deploy them as microservices in a Kubernetes cluster through BentoML's cloud native components and custom resource definitions (CRDs). This documentation provides a comprehensive guide on how to use BentoML and Kubeflow together to streamline the process of deploying models at scale.
## Requirements
* Kubernetes 1.20 - 1.25
## Installation
Run the following command to install BentoML Yatai. Note that the YAML assumes you will install in kubeflow namespace.
```bash
kustomize build bentoml-yatai-stack/default | kubectl apply -n kubeflow --server-side -f -
```
## Customizations
You can customize the container repository configurations and credentials for the `yatai-image-builder` operator to push Bento images to a container registry of your choice.
WARNING: The `yatai-image-builder` operator requires root privileges because it needs to access the Docker daemon, which requires elevated permissions. Granting root privileges can potentially be dangerous, as it can give a user unrestricted access to the underlying operating system.
```
dockerRegistry:
bentoRepositoryName: yatai-bentos
inClusterServer: docker-registry.kubeflow.svc.cluster.local:5000
password: ""
secure: false
server: 127.0.0.1:5000
username: ""
```
You can also supply AWS credentials for the `bento-image-builder` operator to download the Bento specified in the BentoRequest resource from S3.
```
aws:
accessKeyID: ''
secretAccessKey: ''
secretAccessKeyExistingSecretName: ''
secretAccessKeyExistingSecretKey: ''
```
Update the resources with the following command.
```bash
make bentoml-yatai-stack/bases
```
Re-install and apply resources.
```bash
kustomize build bentoml-yatai-stack/default | kubectl apply -n kubeflow --server-side -f -
```
## Upgrading
See [UPGRADE.md](UPGRADE.md)
## Why BentoML
[BentoML](https://github.com/bentoml/BentoML) is an open-source platform for building, shipping, and scaling AI applications.
- Building
- Unifies ML frameworks to run inference with any pre-trained models or bring your own
- Multi-model Inference graph support for complex AI solutions
- Python first framework that integrates with any ecosystem tooling
- Shipping
- Any environment, batch inference, streaming, or real-time serving
- Any public cloud for on-prem deployment
- Kubenetes native deployment
- Scaling
- Efficient resource utilization with autoscaling
- Adaptive batching for higher efficiency and throughput
- Distributed microservice architecture to run services on the most optimal hardware
## Workflow on Kubeflow Notebook
In this example, we will train three fraud detection models using the Kubeflow notebook and the [Kaggle IEEE-CIS Fraud Detection dataset](https://www.kaggle.com/c/ieee-fraud-detection). We will then create a BentoML service that can simultaneously invoke all three models and return a decision on whether a transaction is fraudulent and build it into a Bento. We will showcase two deployment workflows using BentoML's Kubernetes operators: deploying directly from the Bento, and deploying from an OCI image built from the Bento.
![image](https://raw.githubusercontent.com/bentoml/BentoML/main/docs/source/_static/img/kubeflow-fraud-detection.png)
See the [Fraud Detection Example](https://github.com/bentoml/BentoML/tree/main/examples/kubeflow) for a detailed workflow from model training to end-to-end deployment on Kubernetes.
## Workflow on Kubeflow Pipeline
This option will be available in Kubeflow release 1.8.
@@ -0,0 +1,35 @@
# Upgrading Documentation
## Updating manifests
In order to update manifests make sure you are running the commands in linux.
If you are running in another OS, please make sure to update the Makefile commands.
You can refresh the configuration by running:
```
make bentoml-yatai-stack/base
```
## Updating to specific version
Upgrading the version can be done by setting the `BENTOML_YATAI_STACK_VERSION` environment variable, such as:
```
# Set the desired version
export BENTOML_YATAI_IMAGE_BUILDER_VERSION=1.1.0
export BENTOML_YATAI_DEPLOYMENT_VERSION=1.1.0
# Rebuild the kustomize bases
make bentoml-yatai-stack/bases
# Run new manifests against cluster
kustomize build bentoml-yatai-stack/default | kubectl apply -f -
```
## Instructions for breaking changes
The [Yatai upgrading docs](https://docs.bentoml.org/projects/yatai) provide step by step overview of breaking changes across minor and patch versions.
@@ -0,0 +1,2 @@
resources:
- resources.yaml
@@ -0,0 +1,2 @@
resources:
- resources.yaml
@@ -0,0 +1,6 @@
# Adds namespace to all resources.
namespace: kubeflow
bases:
- ../bases/yatai-image-builder
- ../bases/yatai-deployment
@@ -0,0 +1,97 @@
apiVersion: resources.yatai.ai/v1alpha1
kind: Bento
metadata:
name: fraud-detection
namespace: kubeflow
spec:
image: docker.io/bentoml/fraud_detection:o5smnagbncigycvj
runners:
- name: ieee-fraud-detection-0
runnableType: XGBoost
- name: ieee-fraud-detection-1
runnableType: XGBoost
- name: ieee-fraud-detection-2
runnableType: XGBoost
tag: fraud_detection:o5smnagbncigycvj
---
apiVersion: serving.yatai.ai/v2alpha1
kind: BentoDeployment
metadata:
name: fraud-detection
namespace: kubeflow
spec:
autoscaling:
maxReplicas: 2
metrics:
- resource:
name: cpu
target:
averageUtilization: 80
type: Utilization
type: Resource
minReplicas: 1
bento: fraud-detection
ingress:
enabled: false
resources:
limits:
cpu: 1000m
memory: 1024Mi
requests:
cpu: 100m
memory: 200Mi
runners:
- autoscaling:
maxReplicas: 2
metrics:
- resource:
name: cpu
target:
averageUtilization: 80
type: Utilization
type: Resource
minReplicas: 1
name: ieee-fraud-detection-0
resources:
limits:
cpu: 1000m
memory: 1024Mi
requests:
cpu: 100m
memory: 200Mi
- autoscaling:
maxReplicas: 2
metrics:
- resource:
name: cpu
target:
averageUtilization: 80
type: Utilization
type: Resource
minReplicas: 1
name: ieee-fraud-detection-1
resources:
limits:
cpu: 1000m
memory: 1024Mi
requests:
cpu: 100m
memory: 200Mi
- autoscaling:
maxReplicas: 2
metrics:
- resource:
name: cpu
target:
averageUtilization: 80
type: Utilization
type: Resource
minReplicas: 1
name: ieee-fraud-detection-2
resources:
limits:
cpu: 1000m
memory: 1024Mi
requests:
cpu: 100m
memory: 200Mi
@@ -0,0 +1,94 @@
apiVersion: resources.yatai.ai/v1alpha1
kind: BentoRequest
metadata:
name: fraud-detection
namespace: kubeflow
spec:
bentoTag: fraud_detection:o5smnagbncigycvj
downloadUrl: s3://bentoml.com/kubeflow/fraud_detection.bento
runners:
- name: ieee-fraud-detection-0
- name: ieee-fraud-detection-1
- name: ieee-fraud-detection-2
---
apiVersion: serving.yatai.ai/v2alpha1
kind: BentoDeployment
metadata:
name: fraud-detection
namespace: kubeflow
spec:
autoscaling:
maxReplicas: 2
metrics:
- resource:
name: cpu
target:
averageUtilization: 80
type: Utilization
type: Resource
minReplicas: 1
bento: fraud-detection
ingress:
enabled: false
resources:
limits:
cpu: 1000m
memory: 1024Mi
requests:
cpu: 100m
memory: 200Mi
runners:
- autoscaling:
maxReplicas: 2
metrics:
- resource:
name: cpu
target:
averageUtilization: 80
type: Utilization
type: Resource
minReplicas: 1
name: ieee-fraud-detection-0
resources:
limits:
cpu: 1000m
memory: 1024Mi
requests:
cpu: 100m
memory: 200Mi
- autoscaling:
maxReplicas: 2
metrics:
- resource:
name: cpu
target:
averageUtilization: 80
type: Utilization
type: Resource
minReplicas: 1
name: ieee-fraud-detection-1
resources:
limits:
cpu: 1000m
memory: 1024Mi
requests:
cpu: 100m
memory: 200Mi
- autoscaling:
maxReplicas: 2
metrics:
- resource:
name: cpu
target:
averageUtilization: 80
type: Utilization
type: Resource
minReplicas: 1
name: ieee-fraud-detection-2
resources:
limits:
cpu: 1000m
memory: 1024Mi
requests:
cpu: 100m
memory: 200Mi
@@ -0,0 +1,2 @@
resources:
- resources.yaml
@@ -0,0 +1,20 @@
yataiSystem:
namespace: "kubeflow"
yatai:
endpoint: ''
layers:
network:
ingressClass: nginx
ingressAnnotations: {}
ingressPath: /
ingressPathType: ImplementationSpecific
domainSuffix: ''
automaticDomainSuffixGeneration: false
bentoDeploymentNamespaces: ["kubeflow"]
@@ -0,0 +1,13 @@
yataiSystem:
namespace: "kubeflow"
yatai:
endpoint: ''
dockerRegistry:
bentoRepositoryName: yatai-bentos
inClusterServer: docker-registry.kubeflow.svc.cluster.local:5000
password: ""
secure: false
server: 127.0.0.1:5000
username: ""
+440
View File
@@ -0,0 +1,440 @@
#!/bin/bash
set -xe
kubectl create ns kubeflow || echo "namespace kubeflow already exists"
kustomize build bentoml-yatai-stack/default | kubectl apply --server-side -f -
kubectl -n kubeflow wait --for=condition=available --timeout=600s deploy/yatai-image-builder
kubectl -n kubeflow wait --for=condition=available --timeout=600s deploy/yatai-deployment
sleep 5
kubectl apply -n kubeflow -f deployment_from_bento.yaml
sleep 5
kubectl -n kubeflow logs deploy/yatai-deployment
sleep 5
kubectl -n kubeflow wait --for=condition=available --timeout=600s deploy/fraud-detection
kubectl -n kubeflow wait --for=condition=available --timeout=600s deploy/fraud-detection-runner-0
kubectl -n kubeflow wait --for=condition=available --timeout=600s deploy/fraud-detection-runner-1
kubectl -n kubeflow wait --for=condition=available --timeout=600s deploy/fraud-detection-runner-2
kubectl -n kubeflow port-forward svc/fraud-detection 3333:3000 &
PID=$!
function trap_handler {
kill $PID
kubectl -n kubeflow logs -l yatai.ai/bento-deployment=fraud-detection --tail=100
kubectl -n kubeflow delete -f deployment_from_bento.yaml
kustomize build bentoml-yatai-stack/default | kubectl delete -f -
}
trap trap_handler EXIT
sleep 5
# FIXME: getting AttributeError: 'ColumnTransformer' object has no attribute '_name_to_fitted_passthrough'
# output=$(curl --fail -X 'POST' \
# 'http://localhost:3333/is_fraud' \
# -H 'accept: application/json' \
# -H 'Content-Type: application/json' \
# -d '[
# {
# "TransactionID": 2987000,
# "TransactionDT": 86400,
# "TransactionAmt": 68.5,
# "ProductCD": "W",
# "card1": 13926,
# "card2": null,
# "card3": 150,
# "card4": "discover",
# "card5": 142,
# "card6": "credit",
# "addr1": 315,
# "addr2": 87,
# "dist1": 19,
# "dist2": null,
# "P_emaildomain": null,
# "R_emaildomain": null,
# "C1": 1,
# "C2": 1,
# "C3": 0,
# "C4": 0,
# "C5": 0,
# "C6": 1,
# "C7": 0,
# "C8": 0,
# "C9": 1,
# "C10": 0,
# "C11": 2,
# "C12": 0,
# "C13": 1,
# "C14": 1,
# "D1": 14,
# "D2": null,
# "D3": 13,
# "D4": null,
# "D5": null,
# "D6": null,
# "D7": null,
# "D8": null,
# "D9": null,
# "D10": 13,
# "D11": 13,
# "D12": null,
# "D13": null,
# "D14": null,
# "D15": 0,
# "M1": "T",
# "M2": "T",
# "M3": "T",
# "M4": "M2",
# "M5": "F",
# "M6": "T",
# "M7": null,
# "M8": null,
# "M9": null,
# "V1": 1,
# "V2": 1,
# "V3": 1,
# "V4": 1,
# "V5": 1,
# "V6": 1,
# "V7": 1,
# "V8": 1,
# "V9": 1,
# "V10": 0,
# "V11": 0,
# "V12": 1,
# "V13": 1,
# "V14": 1,
# "V15": 0,
# "V16": 0,
# "V17": 0,
# "V18": 0,
# "V19": 1,
# "V20": 1,
# "V21": 0,
# "V22": 0,
# "V23": 1,
# "V24": 1,
# "V25": 1,
# "V26": 1,
# "V27": 0,
# "V28": 0,
# "V29": 0,
# "V30": 0,
# "V31": 0,
# "V32": 0,
# "V33": 0,
# "V34": 0,
# "V35": null,
# "V36": null,
# "V37": null,
# "V38": null,
# "V39": null,
# "V40": null,
# "V41": null,
# "V42": null,
# "V43": null,
# "V44": null,
# "V45": null,
# "V46": null,
# "V47": null,
# "V48": null,
# "V49": null,
# "V50": null,
# "V51": null,
# "V52": null,
# "V53": 1,
# "V54": 1,
# "V55": 1,
# "V56": 1,
# "V57": 0,
# "V58": 0,
# "V59": 0,
# "V60": 0,
# "V61": 1,
# "V62": 1,
# "V63": 0,
# "V64": 0,
# "V65": 1,
# "V66": 1,
# "V67": 1,
# "V68": 0,
# "V69": 0,
# "V70": 0,
# "V71": 0,
# "V72": 0,
# "V73": 0,
# "V74": 0,
# "V75": 1,
# "V76": 1,
# "V77": 1,
# "V78": 1,
# "V79": 0,
# "V80": 0,
# "V81": 0,
# "V82": 0,
# "V83": 0,
# "V84": 0,
# "V85": 0,
# "V86": 1,
# "V87": 1,
# "V88": 1,
# "V89": 0,
# "V90": 0,
# "V91": 0,
# "V92": 0,
# "V93": 0,
# "V94": 0,
# "V95": 0,
# "V96": 1,
# "V97": 0,
# "V98": 0,
# "V99": 0,
# "V100": 0,
# "V101": 0,
# "V102": 1,
# "V103": 0,
# "V104": 0,
# "V105": 0,
# "V106": 0,
# "V107": 1,
# "V108": 1,
# "V109": 1,
# "V110": 1,
# "V111": 1,
# "V112": 1,
# "V113": 1,
# "V114": 1,
# "V115": 1,
# "V116": 1,
# "V117": 1,
# "V118": 1,
# "V119": 1,
# "V120": 1,
# "V121": 1,
# "V122": 1,
# "V123": 1,
# "V124": 1,
# "V125": 1,
# "V126": 0,
# "V127": 117,
# "V128": 0,
# "V129": 0,
# "V130": 0,
# "V131": 0,
# "V132": 0,
# "V133": 117,
# "V134": 0,
# "V135": 0,
# "V136": 0,
# "V137": 0,
# "V138": null,
# "V139": null,
# "V140": null,
# "V141": null,
# "V142": null,
# "V143": null,
# "V144": null,
# "V145": null,
# "V146": null,
# "V147": null,
# "V148": null,
# "V149": null,
# "V150": null,
# "V151": null,
# "V152": null,
# "V153": null,
# "V154": null,
# "V155": null,
# "V156": null,
# "V157": null,
# "V158": null,
# "V159": null,
# "V160": null,
# "V161": null,
# "V162": null,
# "V163": null,
# "V164": null,
# "V165": null,
# "V166": null,
# "V167": null,
# "V168": null,
# "V169": null,
# "V170": null,
# "V171": null,
# "V172": null,
# "V173": null,
# "V174": null,
# "V175": null,
# "V176": null,
# "V177": null,
# "V178": null,
# "V179": null,
# "V180": null,
# "V181": null,
# "V182": null,
# "V183": null,
# "V184": null,
# "V185": null,
# "V186": null,
# "V187": null,
# "V188": null,
# "V189": null,
# "V190": null,
# "V191": null,
# "V192": null,
# "V193": null,
# "V194": null,
# "V195": null,
# "V196": null,
# "V197": null,
# "V198": null,
# "V199": null,
# "V200": null,
# "V201": null,
# "V202": null,
# "V203": null,
# "V204": null,
# "V205": null,
# "V206": null,
# "V207": null,
# "V208": null,
# "V209": null,
# "V210": null,
# "V211": null,
# "V212": null,
# "V213": null,
# "V214": null,
# "V215": null,
# "V216": null,
# "V217": null,
# "V218": null,
# "V219": null,
# "V220": null,
# "V221": null,
# "V222": null,
# "V223": null,
# "V224": null,
# "V225": null,
# "V226": null,
# "V227": null,
# "V228": null,
# "V229": null,
# "V230": null,
# "V231": null,
# "V232": null,
# "V233": null,
# "V234": null,
# "V235": null,
# "V236": null,
# "V237": null,
# "V238": null,
# "V239": null,
# "V240": null,
# "V241": null,
# "V242": null,
# "V243": null,
# "V244": null,
# "V245": null,
# "V246": null,
# "V247": null,
# "V248": null,
# "V249": null,
# "V250": null,
# "V251": null,
# "V252": null,
# "V253": null,
# "V254": null,
# "V255": null,
# "V256": null,
# "V257": null,
# "V258": null,
# "V259": null,
# "V260": null,
# "V261": null,
# "V262": null,
# "V263": null,
# "V264": null,
# "V265": null,
# "V266": null,
# "V267": null,
# "V268": null,
# "V269": null,
# "V270": null,
# "V271": null,
# "V272": null,
# "V273": null,
# "V274": null,
# "V275": null,
# "V276": null,
# "V277": null,
# "V278": null,
# "V279": 0,
# "V280": 0,
# "V281": 0,
# "V282": 1,
# "V283": 1,
# "V284": 0,
# "V285": 0,
# "V286": 0,
# "V287": 0,
# "V288": 0,
# "V289": 0,
# "V290": 1,
# "V291": 1,
# "V292": 1,
# "V293": 0,
# "V294": 1,
# "V295": 0,
# "V296": 0,
# "V297": 0,
# "V298": 0,
# "V299": 0,
# "V300": 0,
# "V301": 0,
# "V302": 0,
# "V303": 0,
# "V304": 0,
# "V305": 1,
# "V306": 0,
# "V307": 117,
# "V308": 0,
# "V309": 0,
# "V310": 0,
# "V311": 0,
# "V312": 0,
# "V313": 0,
# "V314": 0,
# "V315": 0,
# "V316": 0,
# "V317": 117,
# "V318": 0,
# "V319": 0,
# "V320": 0,
# "V321": 0,
# "V322": null,
# "V323": null,
# "V324": null,
# "V325": null,
# "V326": null,
# "V327": null,
# "V328": null,
# "V329": null,
# "V330": null,
# "V331": null,
# "V332": null,
# "V333": null,
# "V334": null,
# "V335": null,
# "V336": null,
# "V337": null,
# "V338": null,
# "V339": null
# }
# ]')
# echo "output: '${output}'"
# if [[ $output != *'false'* ]]; then
# echo "Test failed"
# exit 1
# fi
+56
View File
@@ -0,0 +1,56 @@
KSERVE_VERSION ?= 0.12.1
MODELS_WEBAPP_VERSION ?= 0.8.1
MODELS_WEBAPP_RELEASE_VERSION := $(shell echo ${MODELS_WEBAPP_VERSION} | cut -d "." -f1-2)
.PHONY: upgrade-kserve-manifests
upgrade-kserve-manifests: clean-kserve-manifests
curl -sSL 'https://github.com/kserve/kserve/releases/download/v$(KSERVE_VERSION)/kserve_kubeflow.yaml' -o 'kserve/kserve_kubeflow.yaml'
curl -sSL 'https://github.com/kserve/kserve/releases/download/v$(KSERVE_VERSION)/kserve-cluster-resources.yaml' -o 'kserve/kserve-cluster-resources.yaml'
curl -sSL 'https://github.com/kserve/kserve/releases/download/v$(KSERVE_VERSION)/kserve.yaml' -o 'kserve/kserve.yaml'
.PHONY: clean-kserve-manifests
clean-kserve-manifests:
cd kserve && rm -f kserve.yaml kserve-cluster-resources.yaml kserve_kubeflow.yaml
.PHONY: install-kserve
install-kserve:
# Create kubeflow namespace if not exists
kubectl create namespace kubeflow || echo "kubeflow namespace exists"
kubectl apply -k kserve
.PHONY: uninstall-kserve
uninstall-kserve:
kubectl delete -k kserve
.PHONY: test-kserve
test-kserve:
cd tests && pytest .
.PHONY: clean-models-webapp-manifests
clean-models-webapp-manifests:
rm -rf models-web-app
.PHONY: upgrade-models-webapp-manifests
upgrade-models-webapp-manifests: clean-models-webapp-manifests
git clone https://github.com/kserve/models-web-app.git kserve-models-web-app-src
cd kserve-models-web-app-src && git checkout release-${MODELS_WEBAPP_RELEASE_VERSION}
cp -r kserve-models-web-app-src/config ./models-web-app
# cleanup
rm -rf kserve-models-web-app-src
.PHONY: install-models-web-app
install-models-webapp:
# Create kubeflow namespace if not exists
kubectl create namespace kubeflow || echo "kubeflow namespace exists"
kubectl apply -k models-web-app/overlays/kubeflow
.PHONY: uninstall-models-web-app
uninstall-models-webapp:
kubectl delete -k models-web-app/overlays/kubeflow
.PHONY: test-models-webapp
test-models-webapp: install-models-webapp
kubectl wait --for=condition=Available --timeout=300s -n kubeflow deployment/kserve-models-web-app
@echo "Test Passed"
@echo "cleaning up ..."
kubectl delete -k models-web-app/overlays/kubeflow
+2
View File
@@ -0,0 +1,2 @@
approvers:
- yuzisun
+120
View File
@@ -0,0 +1,120 @@
# KServe
[![Go Report Card](https://goreportcard.com/badge/github.com/kserve/kserve)](https://goreportcard.com/report/github.com/kserve/kserve)
[![OpenSSF Best Practices](https://bestpractices.coreinfrastructure.org/projects/6643/badge)](https://bestpractices.coreinfrastructure.org/projects/6643)
[![Releases](https://img.shields.io/github/release-pre/kserve/kserve.svg?sort=semver)](https://github.com/kserve/kserve/releases)
[![LICENSE](https://img.shields.io/github/license/kserve/kserve.svg)](https://github.com/kserve/kserve/blob/master/LICENSE)
KServe provides a Kubernetes [Custom Resource Definition](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) for serving predictive and generative machine learning (ML) models. It aims to solve production model serving use cases by providing high abstraction interfaces for Tensorflow, XGBoost, ScikitLearn, PyTorch, Huggingface Transformer/LLM models using standardized data plane protocols.
It encapsulates the complexity of autoscaling, networking, health checking, and server configuration to bring cutting edge serving features like GPU Autoscaling, Scale to Zero, and Canary Rollouts to your ML deployments. It enables a simple, pluggable, and complete story for Production ML Serving including prediction, pre-processing, post-processing and explainability. KServe is being [used across various organizations.](https://kserve.github.io/website/master/community/adopters/)
For more details, visit the [KServe website](https://kserve.github.io/website/).
![KServe](assets/kserve_new.png)
*[KFServing has been rebranded to KServe since v0.7](https://blog.kubeflow.org/release/official/2021/09/27/kfserving-transition.html).*
## Why KServe?
- KServe is a standard, cloud agnostic **Model Inference Platform** for serving predictive and generative AI models on Kubernetes, built for highly scalable use cases.
- Provides performant, **standardized inference protocol** across ML frameworks including OpenAI specification for generative models.
- Support modern **serverless inference workload** with **request based autoscaling including scale-to-zero** on **CPU and GPU**.
- Provides **high scalability, density packing and intelligent routing** using **ModelMesh**.
- **Simple and pluggable production serving** for **inference**, **pre/post processing**, **monitoring** and **explainability**.
- Advanced deployments for **canary rollout**, **pipeline**, **ensembles** with **InferenceGraph**.
## :hammer_and_wrench: Installation
### Standalone Installation
- **[Serverless Installation](https://kserve.github.io/website/master/admin/serverless/serverless/)**: KServe by default installs Knative for **serverless deployment** for InferenceService.
- **[Raw Deployment Installation](https://kserve.github.io/website/master/admin/kubernetes_deployment)**: Compared to Serverless Installation, this is a more **lightweight** installation. However, this option does not support canary deployment and request based autoscaling with scale-to-zero.
- **[ModelMesh Installation](https://kserve.github.io/website/master/admin/modelmesh/)**: You can optionally install ModelMesh to enable **high-scale**, **high-density** and **frequently-changing model serving** use cases.
- **[Quick Installation](https://kserve.github.io/website/master/get_started/)**: Install KServe on your local machine.
### Kubeflow Installation
KServe is an important addon component of Kubeflow, please learn more from the [Kubeflow KServe documentation](https://www.kubeflow.org/docs/external-add-ons/kserve/kserve) and follow [KServe with Kubeflow on AWS](https://awslabs.github.io/kubeflow-manifests/main/docs/component-guides/kserve) to learn how to use KServe on AWS.
## :hammer_and_pick: Models Web App
The Models web app is responsible for allowing the user to manipulate the Model Servers in their Kubeflow cluster. To achieve this it provides a user friendly way to handle the lifecycle of InferenceService CRs.
Please follow the [Kserve Models UI documentation](https://www.kubeflow.org/docs/external-add-ons/kserve/webapp/) for more information.
## :rocket: Upgrading
For upgrading see [UPGRADE.md](UPGRADE.md)
## :microscope: Testing
### Testing Kserve
#### Prerequisite
1. Install Python >= 3.8
2. Install requirements
```sh
pip install -r tests/requirements.txt
```
3. Create kubeflow namespace
```sh
kubectl apply -k ../../common/kubeflow-namespace/base
```
4. Install cert manager
```sh
kubectl apply -k ../../common/cert-manager/cert-manager/base
kubectl apply -k ../../common/cert-manager/kubeflow-issuer/base
```
5. Install Istio
```sh
kubectl apply -k ../../common/istio-1-22/istio-crds/base
kubectl apply -k ../../common/istio-1-22/istio-namespace/base
kubectl apply -k ../../common/istio-1-22/istio-install/base
```
6. Install knative
```sh
kubectl apply -k ../../common/knative/knative-serving/overlays/gateways
kubectl apply -k ../../common/istio-1-22/cluster-local-gateway/base
kubectl apply -k ../../common/istio-1-22/kubeflow-istio-resources/base
```
7. Install kserve
```sh
make install-kserve
```
> **_NOTE:_** If resource/crd installation fails please re-run the commands.
#### Steps
1. Create test namespace
```sh
kubectl create ns kserve-test
```
2. Configure domain name
```sh
kubectl patch cm config-domain --patch '{"data":{"example.com":""}}' -n knative-serving
```
3. Port forward
```sh
# start a new terminal and run
INGRESS_GATEWAY_SERVICE=$(kubectl get svc --namespace istio-system --selector="app=istio-ingressgateway" --output jsonpath='{.items[0].metadata.name}')
kubectl port-forward --namespace istio-system svc/${INGRESS_GATEWAY_SERVICE} 8080:80
```
4. Run test
```sh
export KSERVE_INGRESS_HOST_PORT='localhost:8080'
make test-kserve
### Testing Models WebApp
#### Prerequisite
1. Running kubernetes cluster
2. `kubectl` configured to talk to the desired cluster.
#### Steps
1. Run the test
```sh
make test-models-webapp
```
## :flight_departure: [Create your first InferenceService](https://kserve.github.io/website/master/get_started/first_isvc)
## :blue_book: [InferenceService API Reference](https://kserve.github.io/website/master/reference/api/)
## :handshake: [Adopters](https://kserve.github.io/website/master/community/adopters/)
## :books: Learn More
To learn more about KServe, how to use various supported features, and how to participate in the KServe community,
please follow the [KServe website documentation](https://kserve.github.io/website).
Additionally, we have compiled a list of [presentations and demos](https://kserve.github.io/website/master/community/presentations/) to dive through various details.
+68
View File
@@ -0,0 +1,68 @@
# Upgrade Documentation
## Upgrade Kserve
### Prerequisites
- Running Kubernetes cluster with kubeflow installed.
- `kubectl` configured to talk to the desired cluster.
- `curl`
> **_NOTE:_** This documentation assumes that you are running the commands in linux.
If you are using another OS, please make sure to update the Makefile commands.
### To update the kserve manifests to specific version follow the below instructions.
1. Set the desired version to upgrade.
```sh
export KSERVE_VERSION=0.12.1
```
2. Rebuild the manifests.
```sh
make upgrade-kserve-manifests
```
3. Install the updated manifests.
```sh
make install-kserve
```
> **_NOTE:_** If resource/crd installation fails please re-run the commands.
### Testing
For testing refer [kserve readme](README.md#testing-kserve).
## Upgrade Models Webapp
### Prerequisites
- Running Kubernetes cluster with kubeflow installed.
- `kubectl` configured to talk to the desired cluster.
- `git`
> **_NOTE:_** This documentation assumes that you are running the commands in linux.
If you are using another OS, please make sure to update the Makefile commands.
### To update the kserve manifests to specific version follow the below instructions.
1. Set the desired version to upgrade.
```sh
export MODELS_WEBAPP_VERSION=0.8.1
```
2. Rebuild the manifests.
```sh
make upgrade-models-webapp-manifests
```
3. Install the updated manifests.
```sh
make install-models-webapp
```
> **_NOTE:_** If resource/crd installation fails please re-run the commands.
### Testing
For testing refer [kserve readme](README.md#testing-models-webapp).
Binary file not shown.

After

Width:  |  Height:  |  Size: 462 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 99 KiB

@@ -0,0 +1,89 @@
aggregationRule:
clusterRoleSelectors:
- matchLabels:
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-kserve-admin: "true"
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app: kserve
app.kubernetes.io/name: kserve
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-admin: "true"
name: kubeflow-kserve-admin
rules: []
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app: kserve
app.kubernetes.io/name: kserve
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-edit: "true"
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-kserve-admin: "true"
name: kubeflow-kserve-edit
rules:
- apiGroups:
- serving.kserve.io
resources:
- inferenceservices
verbs:
- get
- list
- watch
- create
- delete
- deletecollection
- patch
- update
- apiGroups:
- serving.knative.dev
resources:
- services
- services/status
- routes
- routes/status
- configurations
- configurations/status
- revisions
- revisions/status
verbs:
- get
- list
- watch
- create
- delete
- deletecollection
- patch
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app: kserve
app.kubernetes.io/name: kserve
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-view: "true"
name: kubeflow-kserve-view
rules:
- apiGroups:
- serving.kserve.io
resources:
- inferenceservices
verbs:
- get
- list
- watch
- apiGroups:
- serving.knative.dev
resources:
- services
- services/status
- routes
- routes/status
- configurations
- configurations/status
- revisions
- revisions/status
verbs:
- get
- list
@@ -0,0 +1,391 @@
apiVersion: serving.kserve.io/v1alpha1
kind: ClusterServingRuntime
metadata:
name: kserve-huggingfaceserver
spec:
annotations:
prometheus.kserve.io/path: /metrics
prometheus.kserve.io/port: "8080"
containers:
- args:
- --model_name={{.Name}}
image: kserve/huggingfaceserver:v0.13.0
name: kserve-container
resources:
limits:
cpu: "1"
memory: 2Gi
requests:
cpu: "1"
memory: 2Gi
protocolVersions:
- v2
- v1
supportedModelFormats:
- autoSelect: true
name: huggingface
priority: 1
version: "1"
---
apiVersion: serving.kserve.io/v1alpha1
kind: ClusterServingRuntime
metadata:
name: kserve-lgbserver
spec:
annotations:
prometheus.kserve.io/path: /metrics
prometheus.kserve.io/port: "8080"
containers:
- args:
- --model_name={{.Name}}
- --model_dir=/mnt/models
- --http_port=8080
- --nthread=1
image: kserve/lgbserver:v0.13.0
name: kserve-container
resources:
limits:
cpu: "1"
memory: 2Gi
requests:
cpu: "1"
memory: 2Gi
protocolVersions:
- v1
- v2
supportedModelFormats:
- autoSelect: true
name: lightgbm
priority: 1
version: "3"
---
apiVersion: serving.kserve.io/v1alpha1
kind: ClusterServingRuntime
metadata:
name: kserve-mlserver
spec:
annotations:
prometheus.kserve.io/path: /metrics
prometheus.kserve.io/port: "8080"
containers:
- env:
- name: MLSERVER_MODEL_IMPLEMENTATION
value: '{{.Labels.modelClass}}'
- name: MLSERVER_HTTP_PORT
value: "8080"
- name: MLSERVER_GRPC_PORT
value: "9000"
- name: MODELS_DIR
value: /mnt/models
image: docker.io/seldonio/mlserver:1.3.2
name: kserve-container
resources:
limits:
cpu: "1"
memory: 2Gi
requests:
cpu: "1"
memory: 2Gi
protocolVersions:
- v2
supportedModelFormats:
- autoSelect: true
name: sklearn
priority: 2
version: "0"
- autoSelect: true
name: sklearn
priority: 2
version: "1"
- autoSelect: true
name: xgboost
priority: 2
version: "1"
- autoSelect: true
name: lightgbm
priority: 2
version: "3"
- autoSelect: true
name: mlflow
priority: 1
version: "1"
---
apiVersion: serving.kserve.io/v1alpha1
kind: ClusterServingRuntime
metadata:
name: kserve-paddleserver
spec:
annotations:
prometheus.kserve.io/path: /metrics
prometheus.kserve.io/port: "8080"
containers:
- args:
- --model_name={{.Name}}
- --model_dir=/mnt/models
- --http_port=8080
image: kserve/paddleserver:v0.13.0
name: kserve-container
resources:
limits:
cpu: "1"
memory: 2Gi
requests:
cpu: "1"
memory: 2Gi
protocolVersions:
- v1
- v2
supportedModelFormats:
- autoSelect: true
name: paddle
priority: 1
version: "2"
---
apiVersion: serving.kserve.io/v1alpha1
kind: ClusterServingRuntime
metadata:
name: kserve-pmmlserver
spec:
annotations:
prometheus.kserve.io/path: /metrics
prometheus.kserve.io/port: "8080"
containers:
- args:
- --model_name={{.Name}}
- --model_dir=/mnt/models
- --http_port=8080
image: kserve/pmmlserver:v0.13.0
name: kserve-container
resources:
limits:
cpu: "1"
memory: 2Gi
requests:
cpu: "1"
memory: 2Gi
protocolVersions:
- v1
- v2
supportedModelFormats:
- autoSelect: true
name: pmml
priority: 1
version: "3"
- autoSelect: true
name: pmml
priority: 1
version: "4"
---
apiVersion: serving.kserve.io/v1alpha1
kind: ClusterServingRuntime
metadata:
name: kserve-sklearnserver
spec:
annotations:
prometheus.kserve.io/path: /metrics
prometheus.kserve.io/port: "8080"
containers:
- args:
- --model_name={{.Name}}
- --model_dir=/mnt/models
- --http_port=8080
image: kserve/sklearnserver:v0.13.0
name: kserve-container
resources:
limits:
cpu: "1"
memory: 2Gi
requests:
cpu: "1"
memory: 2Gi
protocolVersions:
- v1
- v2
supportedModelFormats:
- autoSelect: true
name: sklearn
priority: 1
version: "1"
---
apiVersion: serving.kserve.io/v1alpha1
kind: ClusterServingRuntime
metadata:
name: kserve-tensorflow-serving
spec:
annotations:
prometheus.kserve.io/path: /metrics
prometheus.kserve.io/port: "8080"
containers:
- args:
- --model_name={{.Name}}
- --port=9000
- --rest_api_port=8080
- --model_base_path=/mnt/models
- --rest_api_timeout_in_ms=60000
command:
- /usr/bin/tensorflow_model_server
image: tensorflow/serving:2.6.2
name: kserve-container
resources:
limits:
cpu: "1"
memory: 2Gi
requests:
cpu: "1"
memory: 2Gi
protocolVersions:
- v1
- grpc-v1
supportedModelFormats:
- autoSelect: true
name: tensorflow
priority: 2
version: "1"
- autoSelect: true
name: tensorflow
priority: 2
version: "2"
---
apiVersion: serving.kserve.io/v1alpha1
kind: ClusterServingRuntime
metadata:
name: kserve-torchserve
spec:
annotations:
prometheus.kserve.io/path: /metrics
prometheus.kserve.io/port: "8082"
containers:
- args:
- torchserve
- --start
- --model-store=/mnt/models/model-store
- --ts-config=/mnt/models/config/config.properties
env:
- name: TS_SERVICE_ENVELOPE
value: '{{.Labels.serviceEnvelope}}'
image: pytorch/torchserve-kfs:0.9.0
name: kserve-container
resources:
limits:
cpu: "1"
memory: 2Gi
requests:
cpu: "1"
memory: 2Gi
protocolVersions:
- v1
- v2
- grpc-v2
supportedModelFormats:
- autoSelect: true
name: pytorch
priority: 2
version: "1"
---
apiVersion: serving.kserve.io/v1alpha1
kind: ClusterServingRuntime
metadata:
name: kserve-tritonserver
spec:
annotations:
prometheus.kserve.io/path: /metrics
prometheus.kserve.io/port: "8002"
containers:
- args:
- tritonserver
- --model-store=/mnt/models
- --grpc-port=9000
- --http-port=8080
- --allow-grpc=true
- --allow-http=true
image: nvcr.io/nvidia/tritonserver:23.05-py3
name: kserve-container
resources:
limits:
cpu: "1"
memory: 2Gi
requests:
cpu: "1"
memory: 2Gi
protocolVersions:
- v2
- grpc-v2
supportedModelFormats:
- autoSelect: true
name: tensorrt
priority: 1
version: "8"
- autoSelect: true
name: tensorflow
priority: 1
version: "1"
- autoSelect: true
name: tensorflow
priority: 1
version: "2"
- autoSelect: true
name: onnx
priority: 1
version: "1"
- name: pytorch
version: "1"
- autoSelect: true
name: triton
priority: 1
version: "2"
---
apiVersion: serving.kserve.io/v1alpha1
kind: ClusterServingRuntime
metadata:
name: kserve-xgbserver
spec:
annotations:
prometheus.kserve.io/path: /metrics
prometheus.kserve.io/port: "8080"
containers:
- args:
- --model_name={{.Name}}
- --model_dir=/mnt/models
- --http_port=8080
- --nthread=1
image: kserve/xgbserver:v0.13.0
name: kserve-container
resources:
limits:
cpu: "1"
memory: 2Gi
requests:
cpu: "1"
memory: 2Gi
protocolVersions:
- v1
- v2
supportedModelFormats:
- autoSelect: true
name: xgboost
priority: 1
version: "1"
---
apiVersion: serving.kserve.io/v1alpha1
kind: ClusterStorageContainer
metadata:
name: default
spec:
container:
image: kserve/storage-initializer:v0.13.0
name: storage-initializer
resources:
limits:
cpu: "1"
memory: 1Gi
requests:
cpu: 100m
memory: 100Mi
supportedUriFormats:
- prefix: gs://
- prefix: s3://
- prefix: hdfs://
- prefix: webhdfs://
- regex: https://(.+?).blob.core.windows.net/(.+)
- regex: https://(.+?).file.core.windows.net/(.+)
- regex: https?://(.+)/(.+)
File diff suppressed because it is too large Load Diff
File diff suppressed because one or more lines are too long
@@ -0,0 +1,6 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
# Install Kserve in kubeflow namespace
- kserve_kubeflow.yaml
- kserve-cluster-resources.yaml
@@ -0,0 +1 @@
ingressGateway=kubeflow/kubeflow-gateway
@@ -0,0 +1,44 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: kserve-models-web-app
namespace: kserve
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/component: kserve-models-web-app
template:
metadata:
labels:
app.kubernetes.io/component: kserve-models-web-app
spec:
containers:
- image: kserve/models-web-app:latest
imagePullPolicy: Always
name: kserve-models-web-app
envFrom:
- configMapRef:
name: kserve-models-web-app-config
ports:
- containerPort: 5000
name: http
livenessProbe:
httpGet:
path: /healthz/liveness
port: http
initialDelaySeconds: 0
periodSeconds: 10
timeoutSeconds: 1
failureThreshold: 3
successThreshold: 1
readinessProbe:
httpGet:
path: /healthz/readiness
port: http
initialDelaySeconds: 0
periodSeconds: 10
timeoutSeconds: 1
failureThreshold: 3
successThreshold: 1
serviceAccountName: kserve-models-web-app
@@ -0,0 +1,21 @@
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: kserve-models-web-app
namespace: kserve
spec:
gateways:
- knative-serving/knative-ingress-gateway
hosts:
- '*'
http:
- match:
- uri:
prefix: /kserve-endpoints/
rewrite:
uri: /
route:
- destination:
host: kserve-models-web-app.kserve.svc.cluster.local
port:
number: 80
@@ -0,0 +1,20 @@
resources:
- rbac.yaml
- service.yaml
- deployment.yaml
- istio.yaml
namespace: kserve
commonLabels:
kustomize.component: kserve-models-web-app
app.kubernetes.io/component: kserve-models-web-app
images:
- name: kserve/models-web-app
newName: kserve/models-web-app
newTag: v0.13.0-rc.0
configMapGenerator:
- name: kserve-models-web-app-config
literals:
- APP_DISABLE_AUTH="True"
@@ -0,0 +1,68 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: kserve-models-web-app
namespace: kserve
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kserve-models-web-app-cluster-role
rules:
- apiGroups:
- authorization.k8s.io
resources:
- subjectaccessreviews
verbs:
- create
- apiGroups:
- ""
resources:
- namespaces
- pods
- pods/log
- events
verbs:
- get
- list
- apiGroups:
- serving.kserve.io
resources:
- inferenceservices
- inferenceservices/status
verbs:
- get
- list
- watch
- create
- delete
- deletecollection
- patch
- update
- apiGroups:
- serving.knative.dev
resources:
- services
- services/status
- routes
- routes/status
- configurations
- configurations/status
- revisions
- revisions/status
verbs:
- get
- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: kserve-models-web-app-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kserve-models-web-app-cluster-role
subjects:
- kind: ServiceAccount
name: kserve-models-web-app
namespace: kserve
@@ -0,0 +1,14 @@
apiVersion: v1
kind: Service
metadata:
name: kserve-models-web-app
namespace: kserve
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: 5000
selector:
app.kubernetes.io/component: kserve-models-web-app
type: ClusterIP
@@ -0,0 +1,42 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
# Adds namespace to all resources.
namespace: kubeflow
# Labels to add to all resources and selectors.
commonLabels:
app: kserve
app.kubernetes.io/name: kserve
bases:
- ../../base
- web-app-authorization-policy.yaml
patchesStrategicMerge:
- patches/web-app-sidecar.yaml
patchesJson6902:
- target:
group: networking.istio.io
version: v1beta1
kind: VirtualService
name: kserve-models-web-app
namespace: kserve
path: patches/web-app-vsvc.yaml
generatorOptions:
disableNameSuffixHash: true
# To make namespace for standalone installation kustomizable,
# variabalize ingress gateway, webhook service name and
# kserve namespace in webhook configurations
configMapGenerator:
- name: kserve-models-web-app-config
behavior: replace
literals:
- USERID_HEADER=kubeflow-userid
- APP_PREFIX=/kserve-endpoints
configurations:
- params.yaml
@@ -0,0 +1,4 @@
commonLabels:
- kind: AuthorizationPolicy
path: spec/selector/matchLabels
create: true
@@ -0,0 +1,10 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: kserve-models-web-app
namespace: kserve
spec:
template:
metadata:
annotations:
sidecar.istio.io/inject: "true"
@@ -0,0 +1,14 @@
- op: replace
path: /spec/gateways
value:
- kubeflow/kubeflow-gateway
- op: replace
path: /spec/http/0/route/0/destination
value:
host: kserve-models-web-app.kubeflow.svc.cluster.local
port:
number: 80
- op: replace
path: /spec/gateways
value:
- kubeflow/kubeflow-gateway
@@ -0,0 +1,18 @@
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
labels:
kustomize.component: kserve-models-web-app
app.kubernetes.io/component: kserve-models-web-app
name: kserve-models-web-app
spec:
action: ALLOW
selector:
matchLabels:
kustomize.component: kserve-models-web-app
app.kubernetes.io/component: kserve-models-web-app
rules:
- from:
- source:
principals:
- cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account
@@ -0,0 +1,6 @@
{
"instances": [
[6.8, 2.8, 4.8, 1.4],
[6.0, 3.4, 4.5, 1.6]
]
}
@@ -0,0 +1,4 @@
pytest>=7.0.0
kserve>=0.12.1
kubernetes>=18.20.0
requests>=2.18.4
@@ -0,0 +1,60 @@
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
from kubernetes import client
from kubernetes.client import V1ResourceRequirements
from kserve import (
constants,
KServeClient,
V1beta1InferenceService,
V1beta1InferenceServiceSpec,
V1beta1PredictorSpec,
V1beta1SKLearnSpec,
)
from utils import KSERVE_TEST_NAMESPACE
from utils import predict
def test_sklearn_kserve():
service_name = "isvc-sklearn"
predictor = V1beta1PredictorSpec(
min_replicas=1,
sklearn=V1beta1SKLearnSpec(
storage_uri="gs://kfserving-examples/models/sklearn/1.0/model",
resources=V1ResourceRequirements(
requests={"cpu": "50m", "memory": "128Mi"},
limits={"cpu": "100m", "memory": "256Mi"},
),
),
)
isvc = V1beta1InferenceService(
api_version=constants.KSERVE_V1BETA1,
kind=constants.KSERVE_KIND,
metadata=client.V1ObjectMeta(
name=service_name, namespace=KSERVE_TEST_NAMESPACE
),
spec=V1beta1InferenceServiceSpec(predictor=predictor),
)
kserve_client = KServeClient(
config_file=os.environ.get("KUBECONFIG", "~/.kube/config")
)
kserve_client.create(isvc)
kserve_client.wait_isvc_ready(service_name, namespace=KSERVE_TEST_NAMESPACE)
res = predict(service_name, "./data/iris_input.json")
assert res["predictions"] == [1, 1]
kserve_client.delete(service_name, KSERVE_TEST_NAMESPACE)
@@ -0,0 +1,125 @@
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import json
import logging
import os
import time
from urllib.parse import urlparse
import requests
from kubernetes import client
from kserve import KServeClient
from kserve import constants
logging.basicConfig(level=logging.INFO)
KSERVE_NAMESPACE = "kserve"
KSERVE_TEST_NAMESPACE = "kserve-test"
MODEL_CLASS_NAME = "modelClass"
class M2mTokenNotAvailable(Exception):
pass
def get_cluster_ip(name="istio-ingressgateway", namespace="istio-system"):
api_instance = client.CoreV1Api(client.ApiClient())
service = api_instance.read_namespaced_service(name, namespace)
if service.status.load_balancer.ingress is None:
cluster_ip = service.spec.cluster_ip
else:
if service.status.load_balancer.ingress[0].hostname:
cluster_ip = service.status.load_balancer.ingress[0].hostname
else:
cluster_ip = service.status.load_balancer.ingress[0].ip
return os.environ.get("KSERVE_INGRESS_HOST_PORT", cluster_ip)
def get_m2m_auth_token(env_name="KSERVE_M2M_TOKEN"):
try:
return os.environ[env_name]
except KeyError:
raise M2mTokenNotAvailable(env_name)
def predict(
service_name,
input_json,
protocol_version="v1",
version=constants.KSERVE_V1BETA1_VERSION,
model_name=None,
):
with open(input_json) as json_file:
data = json.load(json_file)
return predict_str(
service_name=service_name,
input_json=json.dumps(data),
protocol_version=protocol_version,
version=version,
model_name=model_name,
)
def predict_str(
service_name,
input_json,
protocol_version="v1",
version=constants.KSERVE_V1BETA1_VERSION,
model_name=None,
):
kfs_client = KServeClient(
config_file=os.environ.get("KUBECONFIG", "~/.kube/config")
)
isvc = kfs_client.get(
service_name,
namespace=KSERVE_TEST_NAMESPACE,
version=version,
)
# temporary sleep until this is fixed https://github.com/kserve/kserve/issues/604
time.sleep(10)
cluster_ip = get_cluster_ip()
host = urlparse(isvc["status"]["url"]).netloc
headers = {
"Host": host,
"Content-Type": "application/json",
}
try:
token = get_m2m_auth_token()
headers.update({"Authorization": f"Bearer {token}"})
logging.info("M2M Token Found.")
except M2mTokenNotAvailable:
logging.warn("M2M Token Not found, client authentication disabled.")
if model_name is None:
model_name = service_name
url = f"http://{cluster_ip}/v1/models/{model_name}:predict"
if protocol_version == "v2":
url = f"http://{cluster_ip}/v2/models/{model_name}/infer"
logging.info("Sending Header = %s", headers)
logging.info("Sending url = %s", url)
logging.info("Sending request data: %s", input_json)
response = requests.post(url, input_json, headers=headers)
logging.info(
"Got response code %s, content %s", response.status_code, response.content
)
if response.status_code == 200:
preds = json.loads(response.content.decode("utf-8"))
return preds
else:
response.raise_for_status()
@@ -0,0 +1,18 @@
apiVersion: kpt.dev/v1
kind: Kptfile
metadata:
name: metacontroller
upstream:
type: git
git:
repo: https://github.com/kubeflow/pipelines
directory: /manifests/kustomize/third-party/metacontroller
ref: 2.0.0-alpha.3
updateStrategy: resource-merge
upstreamLock:
type: git
git:
repo: https://github.com/kubeflow/pipelines
directory: /manifests/kustomize/third-party/metacontroller
ref: 2.0.0-alpha.3
commit: fe333d49fc08f66ec867d3dd7dbd756927bf390c
@@ -0,0 +1,63 @@
BUILD_DIR=./build
.PHONY: hydrate
hydrate:
rm -r -f "${BUILD_DIR}"
mkdir -p "${BUILD_DIR}"
kustomize build -o "${BUILD_DIR}" ./base
echo "Succesfully built into ${BUILD_DIR}."
.PHONY: clean-build
clean-build:
rm -r -f "${BUILD_DIR}"
echo "Succesfully cleaned ${BUILD_DIR}."
.PHONY: apply
apply: hydrate
kubectl apply -f "${BUILD_DIR}"
echo "Succesfully applied from ${BUILD_DIR}."
.PHONY: delete
delete: hydrate
kubectl delete -f "${BUILD_DIR}"
echo "Succesfully deleted metacontroller."
.PHONY: test
test: test-deploy test-catset-controller test-deploy-cleanup
.PHONY: test-deploy
test-deploy:
kubectl create namespace kubeflow || echo "Found 'kubeflow' namespace"
kustomize build ./base | kubectl apply -f -
sleep 30
# Check CRDs
kubectl -n kubeflow wait --for=condition=established --timeout=60s crd/compositecontrollers.metacontroller.k8s.io
kubectl -n kubeflow wait --for=condition=established --timeout=60s crd/controllerrevisions.metacontroller.k8s.io
kubectl -n kubeflow wait --for=condition=established --timeout=60s crd/decoratorcontrollers.metacontroller.k8s.io
# Check ServiceAccount
kubectl get -n kubeflow serviceaccount/meta-controller-service
kubectl get -n kubeflow clusterrolebinding/meta-controller-cluster-role-binding
# Check StatefulSet
kubectl -n kubeflow rollout status --watch --timeout=60s StatefulSet/metacontroller
echo "[Test] contrib/metacontroller deployment PASSED"
.PHONY: test-deploy-cleanup
test-deploy-cleanup:
# Cleanup
kustomize build ./base | kubectl delete -f -
.PHONY: test-catset-controller
test-catset-controller:
# Create a sample CRD
kubectl apply -f test/catset-controller.yaml
sleep 30
# Check a custom CRD
kubectl -n kubeflow get compositecontrollers/catset-controller
# Cleanup
kubectl delete compositecontrollers/catset-controller
echo "[Test] contrib/metacontroller catset-controller PASSED"
.PHONY: pull
pull:
# Check ./UPGRADE.md for instructions on how to pull updated manifests
kpt pkg update .@${KFP_VERSION}
@@ -0,0 +1,6 @@
approvers:
- zijianjoy
- gkcalat
reviewers:
- zijianjoy
- gkcalat
@@ -0,0 +1,40 @@
# Metacontroller
- [Official documentation](https://metacontroller.github.io/metacontroller/)
- [Official repository](https://github.com/metacontroller/metacontroller)
Metacontroller is an add-on for Kubernetes that makes it easy to write and deploy custom controllers.
## Prerequisites
- Kubernetes v1.16+ (because of maintainability, e2e test suite might not cover all releases).
- You should have `kubectl` available and configured to talk to the desired cluster.
- `kustomize`.
## Compile manifests
```bash
make hydrate
```
## Install Metacontroller
```bash
make apply
```
## Verify deployment
```bash
make test
```
## Uninstall Metacontroller
```bash
make delete
```
## Upgrade Metacontroller
To upgrade to the lates version used in Kubeflow, follow the steps in [UPGRADE.md](./UPDGRADE.md).
@@ -0,0 +1,9 @@
# Upgrade Metacontroller
Metacontroller is pulled from [Kubeflow Pipelines's (KFP) third-party folder](https://github.com/kubeflow/pipelines/tree/master/manifests/kustomize/third-party/metacontroller). To update this component specify the desired Kubeflow Pipelines version as `KFP_VERSION` environment variable and run `make pull` in console:
```bash
KFP_VERSION=2.0.0-alpha.3 make pull
```
Alternatively, you can copy the content from [Kubeflow Pipelines third-party folder](https://github.com/kubeflow/pipelines/tree/master/manifests/kustomize/third-party/metacontroller) by choosing the appropriate `TAG` in that repository.
@@ -0,0 +1,11 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata: # kpt-merge: /meta-controller-cluster-role-binding
name: meta-controller-cluster-role-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: meta-controller-service
@@ -0,0 +1,530 @@
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata: # kpt-merge: /compositecontrollers.metacontroller.k8s.io
annotations:
"api-approved.kubernetes.io": "unapproved, request not yet submitted"
name: compositecontrollers.metacontroller.k8s.io
spec:
group: metacontroller.k8s.io
names:
kind: CompositeController
listKind: CompositeControllerList
plural: compositecontrollers
shortNames:
- cc
- cctl
singular: compositecontroller
scope: Cluster
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
properties:
childResources:
items:
properties:
apiVersion:
type: string
resource:
type: string
updateStrategy:
properties:
method:
type: string
statusChecks:
properties:
conditions:
items:
properties:
reason:
type: string
status:
type: string
type:
type: string
required:
- type
type: object
type: array
type: object
type: object
required:
- apiVersion
- resource
type: object
type: array
generateSelector:
type: boolean
hooks:
properties:
customize:
properties:
webhook:
properties:
path:
type: string
service:
properties:
name:
type: string
namespace:
type: string
port:
format: int32
type: integer
protocol:
type: string
required:
- name
- namespace
type: object
timeout:
type: string
url:
type: string
type: object
type: object
finalize:
properties:
webhook:
properties:
path:
type: string
service:
properties:
name:
type: string
namespace:
type: string
port:
format: int32
type: integer
protocol:
type: string
required:
- name
- namespace
type: object
timeout:
type: string
url:
type: string
type: object
type: object
postUpdateChild:
properties:
webhook:
properties:
path:
type: string
service:
properties:
name:
type: string
namespace:
type: string
port:
format: int32
type: integer
protocol:
type: string
required:
- name
- namespace
type: object
timeout:
type: string
url:
type: string
type: object
type: object
preUpdateChild:
properties:
webhook:
properties:
path:
type: string
service:
properties:
name:
type: string
namespace:
type: string
port:
format: int32
type: integer
protocol:
type: string
required:
- name
- namespace
type: object
timeout:
type: string
url:
type: string
type: object
type: object
sync:
properties:
webhook:
properties:
path:
type: string
service:
properties:
name:
type: string
namespace:
type: string
port:
format: int32
type: integer
protocol:
type: string
required:
- name
- namespace
type: object
timeout:
type: string
url:
type: string
type: object
type: object
type: object
parentResource:
properties:
apiVersion:
type: string
resource:
type: string
revisionHistory:
properties:
fieldPaths:
items:
type: string
type: array
type: object
required:
- apiVersion
- resource
type: object
resyncPeriodSeconds:
format: int32
type: integer
required:
- parentResource
type: object
status:
type: object
required:
- metadata
- spec
type: object
served: true
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata: # kpt-merge: /controllerrevisions.metacontroller.k8s.io
annotations:
"api-approved.kubernetes.io": "unapproved, request not yet submitted"
name: controllerrevisions.metacontroller.k8s.io
spec:
group: metacontroller.k8s.io
names:
kind: ControllerRevision
listKind: ControllerRevisionList
plural: controllerrevisions
singular: controllerrevision
scope: Namespaced
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
children:
items:
properties:
apiGroup:
type: string
kind:
type: string
names:
items:
type: string
type: array
required:
- apiGroup
- kind
- names
type: object
type: array
kind:
description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
parentPatch:
type: object
required:
- metadata
- parentPatch
type: object
served: true
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata: # kpt-merge: /decoratorcontrollers.metacontroller.k8s.io
annotations:
"api-approved.kubernetes.io": "unapproved, request not yet submitted"
name: decoratorcontrollers.metacontroller.k8s.io
spec:
group: metacontroller.k8s.io
names:
kind: DecoratorController
listKind: DecoratorControllerList
plural: decoratorcontrollers
shortNames:
- dec
- decorators
singular: decoratorcontroller
scope: Cluster
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
properties:
attachments:
items:
properties:
apiVersion:
type: string
resource:
type: string
updateStrategy:
properties:
method:
type: string
type: object
required:
- apiVersion
- resource
type: object
type: array
hooks:
properties:
customize:
properties:
webhook:
properties:
path:
type: string
service:
properties:
name:
type: string
namespace:
type: string
port:
format: int32
type: integer
protocol:
type: string
required:
- name
- namespace
type: object
timeout:
type: string
url:
type: string
type: object
type: object
finalize:
properties:
webhook:
properties:
path:
type: string
service:
properties:
name:
type: string
namespace:
type: string
port:
format: int32
type: integer
protocol:
type: string
required:
- name
- namespace
type: object
timeout:
type: string
url:
type: string
type: object
type: object
sync:
properties:
webhook:
properties:
path:
type: string
service:
properties:
name:
type: string
namespace:
type: string
port:
format: int32
type: integer
protocol:
type: string
required:
- name
- namespace
type: object
timeout:
type: string
url:
type: string
type: object
type: object
type: object
resources:
items:
properties:
annotationSelector:
properties:
matchAnnotations:
additionalProperties:
type: string
type: object
matchExpressions:
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
type: object
apiVersion:
type: string
labelSelector:
description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
resource:
type: string
required:
- apiVersion
- resource
type: object
type: array
resyncPeriodSeconds:
format: int32
type: integer
required:
- resources
type: object
status:
type: object
required:
- metadata
- spec
type: object
served: true
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
@@ -0,0 +1,14 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: kubeflow
resources:
- cluster-role-binding.yaml
- crd.yaml
- service-account.yaml
- stateful-set.yaml
commonLabels:
kustomize.component: metacontroller
# Update metacontroller CRD:
# Copy the upstream file to crd.yaml in this folder.
# Upstream file: https://github.com/metacontroller/metacontroller/blob/master/manifests/production/metacontroller-crds-v1.yaml
@@ -0,0 +1,4 @@
apiVersion: v1
kind: ServiceAccount
metadata: # kpt-merge: /meta-controller-service
name: meta-controller-service
@@ -0,0 +1,45 @@
apiVersion: apps/v1
kind: StatefulSet
metadata: # kpt-merge: /metacontroller
labels:
app: metacontroller
name: metacontroller
spec:
replicas: 1
selector:
matchLabels:
app: metacontroller
serviceName: ""
template:
metadata:
labels:
app: metacontroller
annotations:
sidecar.istio.io/inject: "false"
spec:
containers:
- resources:
limits:
cpu: '1'
memory: 1Gi
requests:
cpu: 100m
memory: 256Mi
command:
- /usr/bin/metacontroller
- --zap-log-level=4
- '--discovery-interval=3600s' # less insane than 10 seconds
securityContext:
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 1000
privileged: false
allowPrivilegeEscalation: false
name: metacontroller
image: 'docker.io/metacontrollerio/metacontroller:v2.0.4'
serviceAccountName: meta-controller-service
# Workaround for https://github.com/kubernetes-sigs/kustomize/issues/677
volumeClaimTemplates: []
@@ -0,0 +1,27 @@
apiVersion: metacontroller.k8s.io/v1alpha1
kind: CompositeController
metadata: # kpt-merge: /catset-controller
name: catset-controller
spec:
parentResource:
apiVersion: ctl.enisoc.com/v1
resource: catsets
revisionHistory:
fieldPaths:
- spec.template
childResources:
- apiVersion: v1
resource: pods
updateStrategy:
method: RollingRecreate
statusChecks:
conditions:
- type: Ready
status: "True"
- apiVersion: v1
resource: persistentvolumeclaims
hooks:
sync:
webhook:
url: http://catset-controller.metacontroller/sync
timeout: 10s
+12
View File
@@ -0,0 +1,12 @@
KUBERAY_RELEASE_VERSION ?= 1.1.1
KUBERAY_HELM_CHART_REPO ?= https://ray-project.github.io/kuberay-helm/
.PHONY: kuberay-operator/base
kuberay-operator/base:
mkdir -p kuberay-operator/base
cd kuberay-operator/base && helm template --include-crds kuberay-operator kuberay-operator --version ${KUBERAY_RELEASE_VERSION} --repo ${KUBERAY_HELM_CHART_REPO} > resources.yaml
.PHONY: test
test:
./test.sh
+5
View File
@@ -0,0 +1,5 @@
approvers:
- juliusvonkohout
reviewers:
- juliusvonkohout
- kimwnasptd
+133
View File
@@ -0,0 +1,133 @@
TODO
- The ray dashboard, worker and head must only be available from inside your kubeflow user namespace
- Reenable the istio sidecar for the ray head and worker in the user namespace and provide the corresponding Istio Authorizationpolicies. We can keep the istio sidecar for the deployment kuberay-operator in the namespace kubeflow, since it does NOT use a webhook, but something else to reconcile rayclusters. This means we also do not need a networkpolicy for the ray operator.
> Credit: This manifest refers a lot to the engineering blog ["Building a Machine Learning Platform with Kubeflow and Ray on Google Kubernetes Engine"](https://cloud.google.com/blog/products/ai-machine-learning/build-a-ml-platform-with-kubeflow-and-ray-on-gke) from Google Cloud.
# Ray
[Ray](https://github.com/ray-project/ray) is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a toolkit of libraries (Ray AIR) for simplifying ML compute.
<figure>
<img
src="assets/map-of-ray.png"
alt="Ray">
<figcaption>Stack of Ray libraries - unified toolkit for ML workloads. (ref: https://docs.ray.io/en/latest/ray-overview/index.html)</figcaption>
</figure>
# KubeRay
[KubeRay](https://github.com/ray-project/kuberay) is an open-source Kubernetes operator for Ray. It provides several CRDs to simplify managing Ray clusters on Kubernetes. We will integrate Kubeflow and KubeRay in this document.
# Requirements
* Dependencies
* `kustomize`: v5.2.1+ (Kubeflow manifest is sensitive to `kustomize` version.)
* `Kubernetes`: v1.29+
* Computing resources:
* 16GB RAM
* 8 CPUs
# Example
<figure>
<img
src="assets/architecture.svg"
alt="ray/kubeflow integration">
<figcaption>Note: (1) Kubeflow Central Dashboard will be renamed to workbench in the future. (2) Kubeflow Pipeline (KFP) is an important component of Kubeflow, but it is not included in this example.</figcaption>
</figure>
## Step 1: Install Kubeflow v1.7-branch
* This example installs Kubeflow with the [v1.9-branch](https://github.com/kubeflow/manifests/tree/v1.9-branch).
* Install all Kubeflow official components and all common services using [one command](https://github.com/kubeflow/manifests/tree/v1.7-branch#install-with-a-single-command).
* If you do not want to install all components, you can comment out **KNative**, **Katib**, **Tensorboards Controller**, **Tensorboard Web App**, **Training Operator**, and **KServe** from [example/kustomization.yaml](https://github.com/kubeflow/manifests/blob/v1.7-branch/example/kustomization.yaml).
## Step 2: Install KubeRay operator
We never ever break Kubernetes standards and do not use the "default" namespace, but a proper one, in our case "kubeflow" for the ray operator.
```sh
# Install a KubeRay operator and custom resource definitions.
kustomize build kuberay-operator/overlays/kubeflow | kubectl apply --server-side -f -
# Check KubeRay operator
kubectl get pod -l app.kubernetes.io/component=kuberay-operator -n kubeflow
# NAME READY STATUS RESTARTS AGE
# kuberay-operator-5b8cd69758-rkpvh 1/1 Running 0 6m23s
```
## Step 3: Install RayCluster
```sh
# Create a RayCluster CR, and the KubeRay operator will reconcile a Ray cluster
# with 1 head Pod and 1 worker Pod.
# $MY_KUBEFLOW_USER_NAMESPACE is a proper Kubeflow user namespace with istio sidecar injection and never ever the wrong "default"
export MY_KUBEFLOW_USER_NAMESPACE=development
kubectl apply -f raycluster_example.yaml -n $MY_KUBEFLOW_USER_NAMESPACE
# Check RayCluster
kubectl get pod -l ray.io/cluster=kubeflow-raycluster -n $MY_KUBEFLOW_USER_NAMESPACE
# NAME READY STATUS RESTARTS AGE
# kubeflow-raycluster-head-p6dpk 1/1 Running 0 70s
# kubeflow-raycluster-worker-small-group-l7j6c 1/1 Running 0 70s
```
* `raycluster_example.yaml` uses `rayproject/ray:2.23.0-py311-cpu` as its OCI image. Ray is very sensitive to the Python versions and Ray versions between the server (RayCluster) and client (JupyterLab) sides. This image uses:
* Python 3.11
* Ray 2.23.0
## Step 4: Forward the port of Istio's Ingress-Gateway
* Follow the [instructions](https://github.com/kubeflow/manifests/tree/v1.7-branch#port-forward) to forward the port of Istio's Ingress-Gateway and log in to Kubeflow Central Dashboard.
## Step 5: Create a JupyterLab via Kubeflow Central Dashboard
* Click "Notebooks" icon in the left panel.
* Click "New Notebook"
* Select `kubeflownotebookswg/jupyter-scipy:v1.9.0` as OCI image (or any other with the same python version)
* Click "Launch"
* Click "CONNECT" to connect into the JupyterLab instance.
## Step 6: Use Ray client in the JupyterLab to connect to the RayCluster
* As I mentioned in Step 3, Ray is very sensitive to the Python versions and Ray versions between the server (RayCluster) and client (JupyterLab) sides.
```sh
# Check Python version. The version's MAJOR and MINOR should match with RayCluster (i.e. Python 3.11.9)
python --version
# Python 3.11.9
pip install -U ray[default]==2.23.0
```
* Connect to RayCluster via Ray client.
```python
# Open a new .ipynb page.
import ray
# For other namespaces use ray://${RAYCLUSTER_HEAD_SVC}.${NAMESPACE}.svc.cluster.local:${RAY_CLIENT_PORT}
# But we use of course our per namespace ray cluster to have multi-tenancy and
# We never ever use "default" as namespace since this would violate Kubernetes standards
ray.init(address="ray://kubeflow-raycluster-head-svc:10001")
print(ray.cluster_resources())
# {'node:10.244.0.41': 1.0, 'memory': 3000000000.0, 'node:10.244.0.40': 1.0, 'object_store_memory': 805386239.0, 'CPU': 2.0}
# Try Ray task
@ray.remote
def f(x):
return x * x
futures = [f.remote(i) for i in range(4)]
print(ray.get(futures)) # [0, 1, 4, 9]
# Try Ray actor
@ray.remote
class Counter(object):
def __init__(self):
self.n = 0
def increment(self):
self.n += 1
def read(self):
return self.n
counters = [Counter.remote() for i in range(4)]
[c.increment.remote() for c in counters]
futures = [c.read.remote() for c in counters]
print(ray.get(futures)) # [1, 1, 1, 1]
```
# Upgrading
See [UPGRADE.md](UPGRADE.md) for more details.
+6
View File
@@ -0,0 +1,6 @@
# Upgrading
```sh
# Step 1: Update KUBERAY_RELEASE_VERSION in Makefile
# Step 2: Create new KubeRay operator manifest
make kuberay-operator/base
```
File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 217 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 70 KiB

@@ -0,0 +1,52 @@
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kubeflow-kuberay-admin
labels:
app: kuberay-operator
app.kubernetes.io/name: kuberay-operator
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-admin: "true"
rules: []
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kubeflow-kuberay-edit
labels:
app: kuberay-operator
app.kubernetes.io/name: kuberay-operator
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-edit: "true"
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-admin: "true"
rules:
- apiGroups:
- ray.io
resources:
- "*"
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kubeflow-kuberay-view
labels:
app: kuberay-operator
app.kubernetes.io/name: kuberay-operator
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-view: "true"
rules:
- apiGroups:
- ray.io
resources:
- "*"
verbs:
- get
- list
- watch
---
@@ -0,0 +1,20 @@
patches:
# Add securityContext to KubeRay operator Pod.
- target:
kind: Deployment
labelSelector: "app.kubernetes.io/name=kuberay-operator"
patch: |-
- op: add
path: /spec/template/spec/containers/0/securityContext
value:
runAsUser: 1000
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
namespace: kubeflow
resources:
- resources.yaml
- aggregated-roles.yaml
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,3 @@
namespace: kubeflow
resources:
- ../../base
@@ -0,0 +1,4 @@
namespace: kubeflow
resources:
- ../../base
- namespace.yaml
@@ -0,0 +1,4 @@
apiVersion: v1
kind: Namespace
metadata:
name: kubeflow
@@ -0,0 +1,128 @@
apiVersion: ray.io/v1alpha1
kind: RayCluster
metadata:
name: kubeflow-raycluster
spec:
rayVersion: '2.23.0'
# Ray head pod configuration
headGroupSpec:
# Kubernetes Service Type.
serviceType: ClusterIP
# The following params are used to complete the ray start: ray start --head --block --dashboard-host: '0.0.0.0' ...
rayStartParams:
dashboard-host: '0.0.0.0'
block: 'true'
# pod template
template:
metadata:
# Custom labels. NOTE: To avoid conflicts with KubeRay operator, do not define custom labels start with `raycluster`.
# Refer to https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
# The ray head must not have an Istio sidecar
# TODO add an authorizationpolicy in the future for the ray head
labels:
sidecar.istio.io/inject: "false"
spec:
containers:
- name: ray-head
image: rayproject/ray:2.23.0-py311-cpu
ports:
- containerPort: 6379
name: gcs
- containerPort: 8265
name: dashboard
- containerPort: 10001
name: client
lifecycle:
preStop:
exec:
command: ["/bin/sh","-c","ray stop"]
volumeMounts:
- mountPath: /tmp/ray
name: ray-logs
# The resource requests and limits in this config are too small for production!
# It is better to use a few large Ray pod than many small ones.
# For production, it is ideal to size each Ray pod to take up the
# entire Kubernetes node on which it is scheduled.
resources:
limits:
cpu: "1"
memory: "2G"
requests:
cpu: "100m"
memory: "2G"
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
volumes:
- name: ray-logs
emptyDir: {}
workerGroupSpecs:
# the pod replicas in this group typed worker
- replicas: 1
minReplicas: 1
maxReplicas: 10
# logical group name, for this called small-group, also can be functional
groupName: small-group
rayStartParams:
block: 'true'
#pod template
template:
metadata:
labels:
# Disable the sidecars for the ray wokers
# TODO add an authorizationpolicy in the future for the ray worker
sidecar.istio.io/inject: "false"
spec:
containers:
- name: ray-worker
image: rayproject/ray:2.23.0-py311-cpu
lifecycle:
preStop:
exec:
command: ["/bin/sh","-c","ray stop"]
# use volumeMounts.Optional.
# Refer to https://kubernetes.io/docs/concepts/storage/volumes/
volumeMounts:
- mountPath: /tmp/ray
name: ray-logs
# The resource requests and limits in this config are too small for production!
# It is better to use a few large Ray pod than many small ones.
# For production, it is ideal to size each Ray pod to take up the
# entire Kubernetes node on which it is scheduled.
resources:
limits:
cpu: "1"
memory: "1G"
requests:
cpu: "300m"
memory: "1G"
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
initContainers:
# the env var $RAY_IP is set by the operator if missing, with the value of the head service name
- name: init
image: busybox:1.36
# Change the cluster postfix if you don't have a default setting
command: ['sh', '-c', "until nslookup $RAY_IP.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for K8s Service $RAY_IP; sleep 2; done"]
securityContext:
runAsUser: 1000
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
# use volumes
# Refer to https://kubernetes.io/docs/concepts/storage/volumes/
volumes:
- name: ray-logs
emptyDir: {}
+69
View File
@@ -0,0 +1,69 @@
#!/bin/bash
set -euxo
NAMESPACE=kubeflow
TIMEOUT=120 # timeout in seconds
SLEEP_INTERVAL=30 # interval between checks in seconds
RAY_VERSION=2.23.0
function trap_handler {
kill $PID
# Delete RayCluster
kubectl -n $NAMESPACE delete -f raycluster_example.yaml
# Wait for all Ray Pods to be deleted.
start_time=$(date +%s)
while true; do
pods=$(kubectl -n $NAMESPACE get pods -o json | jq '.items | length')
if [ "$pods" -eq 1 ]; then
break
fi
current_time=$(date +%s)
elapsed_time=$((current_time - start_time))
if [ "$elapsed_time" -ge "$TIMEOUT" ]; then
echo "Timeout exceeded. Exiting loop."
exit 1
fi
sleep $SLEEP_INTERVAL
done
# Delete KubeRay operator
kustomize build kuberay-operator/base | kubectl -n $NAMESPACE delete -f -
}
trap trap_handler EXIT
# Install KubeRay operator
kustomize build kuberay-operator/overlays/standalone | kubectl -n $NAMESPACE apply --server-side -f -
# Wait for the operator to be ready.
kubectl -n $NAMESPACE wait --for=condition=available --timeout=600s deploy/kuberay-operator
kubectl -n $NAMESPACE get pod -l app.kubernetes.io/component=kuberay-operator
# Create a RayCluster custom resource.
kubectl -n $NAMESPACE apply -f raycluster_example.yaml
# Wait for the RayCluster to be ready.
sleep 5
kubectl -n $NAMESPACE wait --for=condition=ready pod -l ray.io/cluster=kubeflow-raycluster --timeout=900s
kubectl -n $NAMESPACE logs -l ray.io/cluster=kubeflow-raycluster,ray.io/node-type=head
# Forward the port of Dashboard
sleep 5
kubectl -n $NAMESPACE port-forward --address 0.0.0.0 svc/kubeflow-raycluster-head-svc 8265:8265 &
PID=$!
echo "Forward the port 8265 of Ray head in the background process: $PID"
# Send a curl command to test basic Ray functionality.
sleep 5
output=$(curl -H "Content-Type: application/json" localhost:8265/api/version)
echo "output: ${output}"
# output format: {"version": ..., "ray_version": RAY_VERSION, "ray_commit": ...}
if echo "${output}" | grep -q $RAY_VERSION; then
echo "Test succeeded!"
else
echo "Test failed!"
exit 1
fi
@@ -0,0 +1,9 @@
apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component
patches:
- path: patches/kubeflow-labels.yaml
- path: patches/istio-labels.yaml
- path: patches/cert-manager-labels.yaml
- path: patches/dex-labels.yaml
- path: patches/oauth2-proxy-labels.yaml
@@ -0,0 +1,6 @@
apiVersion: v1
kind: Namespace
metadata:
name: cert-manager
labels:
pod-security.kubernetes.io/enforce: baseline
@@ -0,0 +1,6 @@
apiVersion: v1
kind: Namespace
metadata:
name: auth
labels:
pod-security.kubernetes.io/enforce: baseline
@@ -0,0 +1,6 @@
apiVersion: v1
kind: Namespace
metadata:
name: istio-system
labels:
pod-security.kubernetes.io/enforce: baseline
@@ -0,0 +1,6 @@
apiVersion: v1
kind: Namespace
metadata:
name: kubeflow
labels:
pod-security.kubernetes.io/enforce: baseline
@@ -0,0 +1,6 @@
apiVersion: v1
kind: Namespace
metadata:
name: oauth2-proxy
labels:
pod-security.kubernetes.io/enforce: baseline
@@ -0,0 +1,9 @@
apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component
patches:
- path: patches/kubeflow-labels.yaml
- path: patches/istio-labels.yaml
- path: patches/cert-manager-labels.yaml
- path: patches/dex-labels.yaml
- path: patches/oauth2-proxy-labels.yaml
@@ -0,0 +1,6 @@
apiVersion: v1
kind: Namespace
metadata:
name: cert-manager
labels:
pod-security.kubernetes.io/enforce: restricted
@@ -0,0 +1,6 @@
apiVersion: v1
kind: Namespace
metadata:
name: auth
labels:
pod-security.kubernetes.io/enforce: restricted
@@ -0,0 +1,6 @@
apiVersion: v1
kind: Namespace
metadata:
name: istio-system
labels:
pod-security.kubernetes.io/enforce: restricted
@@ -0,0 +1,6 @@
apiVersion: v1
kind: Namespace
metadata:
name: kubeflow
labels:
pod-security.kubernetes.io/enforce: restricted
@@ -0,0 +1,6 @@
apiVersion: v1
kind: Namespace
metadata:
name: oauth2-proxy
labels:
pod-security.kubernetes.io/enforce: restricted
+46
View File
@@ -0,0 +1,46 @@
SELDON_VERSION ?= 1.18.1
# This could be replaced by a local path if required
SELDON_OPERATOR_CHART ?= seldon-core-operator --repo https://storage.googleapis.com/seldon-charts
seldon-core-operator/base: clean-kustomize
mkdir -p seldon-core-operator/base
cd seldon-core-operator/base && helm template -f ../../values.yaml seldon-core ${SELDON_OPERATOR_CHART} --namespace kubeflow --version ${SELDON_VERSION} > resources.yaml
sed -i 's#cert-manager.io/inject-ca-from:.*#cert-manager.io/inject-ca-from: kubeflow/seldon-serving-cert#g' seldon-core-operator/base/resources.yaml
sed -i "s#'seldon-webhook-service.kubeflow.svc.cluster.local'#"'seldon-webhook-service.$$(SERVICE_NAMESPACE).svc.cluster.local#' seldon-core-operator/base/resources.yaml
sed -i "s#commonName: 'seldon-webhook-service.kubeflow.svc'#"'commonName: seldon-webhook-service.$$(CERTIFICATE_NAMESPACE).svc#' seldon-core-operator/base/resources.yaml
sed -i "s#'seldon-webhook-service.kubeflow.svc'#"'seldon-webhook-service.$$(SERVICE_NAMESPACE).svc#' seldon-core-operator/base/resources.yaml
sed -i 's#kubeflow/kubeflow-gateway#$$(SERVICE_NAMESPACE)/kubeflow-gateway#' seldon-core-operator/base/resources.yaml
cp kustomization.tpl seldon-core-operator/base/kustomization.yaml
cp kustomizeconfig.tpl seldon-core-operator/base/kustomizeconfig.yaml
.PHONY:clean-kustomize
clean-kustomize:
rm -rf seldon-core-operator/base
test:
# Create seldon namespace
kubectl create namespace seldon || echo "seldon namespace exists"
# Create kubeflow namespace if not exists
kubectl create namespace kubeflow || echo "kubeflow namespace exists"
# Creating resources
kustomize build seldon-core-operator/base | kubectl apply -n kubeflow -f -
# Waiting for resources
kubectl wait --for condition=Available -n kubeflow deployment/seldon-controller-manager
# Wait for manager resources to be created
sleep 5
# Create example deployment
kubectl apply -n seldon -f example.yaml
# Wait for manager resources to be created
sleep 5
kubectl get deploy -A
sleep 5
kubectl logs -n kubeflow deployment/seldon-controller-manager
sleep 5
kubectl get deploy -A
# Wait for deployment
kubectl wait --for condition=Available -n seldon deployment/echo-default-0-classifier --timeout=360s
# Remove deployment
kubectl delete -n seldon -f example.yaml
# Cleaning resources
kustomize build seldon-core-operator/base | kubectl delete -n kubeflow -f -
+4
View File
@@ -0,0 +1,4 @@
approvers:
- adriangonz
- ahousley
+92
View File
@@ -0,0 +1,92 @@
# Seldon Core
[Seldon Core](https://github.com/SeldonIO/seldon-core/) is a framework to deploy your machine learning models on Kubernetes at scale.
# Requirements
* Kubernetes 1.18 - 1.24
Support for Kubernetes 1.25 is currently part of [SeldonIO/seldon-core#4172](https://github.com/SeldonIO/seldon-core/pull/4172)
## Install Seldon Core Operator
* The yaml assumes you will install in kubeflow namespace
* You need to have installed istio first
```
kustomize build seldon-core-operator/base | kubectl apply -n kubeflow -f -
```
## Updating
See [UPGRADE.md](UPGRADE.md)
## Testing
```
make test
```
# Overview
We can create a test model once the "Install Seldon Operator" is configured
```
# Create namespace for model
kubectl create namespace seldon
```
We can create an echo model with the following command:
```
kubectl apply -f - << ENDapiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
name: echo
namespace: seldon
spec:
predictors:
- name: default
replicas: 1
graph:
name: classifier
type: MODEL
componentSpecs:
- spec:
containers:
- image: seldonio/echo-model:1.17.1
name: classifier
END
```
We can verify that model is running:
```
kubectl get pods -n seldon
NAME READY STATUS RESTARTS AGE
echo-default-0-classifier-679cb5fb68-qd4nm 2/2 Running 0 25m
```
Also we can verify that the correct virtualservice was created:
```
kubectl get virtualservice -n seldon
NAME GATEWAYS HOSTS AGE
echo ["kubeflow/kubeflow-gateway"] ["*"] 42m
```
Finally we can send a request (you will need to fetch the Dex Auth Token / Cookie):
```
export CLUSTER_IP=# Your cluster IP
export SESSION=# Your dex session
curl -H "Content-Type: application/json" -H "Cookie: authservice_session=${SESSION}" \
-d '{"data": {"ndarray":[[1.0, 2.0, 5.0]]}}' \
http://{CLUSTER_IP}/seldon/seldon/echo/api/v1.0/predictions
{"data":{"names":["t:0","t:1","t:2"],"ndarray":[[1.0,2.0,5.0]]},"meta":{"metrics":[{"key":"mycounter","type":"COUNTER","value":1},{"key":"mygauge","type":"GAUGE","value":100},{"key":"mytimer","type":"TIMER","value":20.2}],"requestPath":{"classifier":"seldonio/echo-model:1.17.1"}}}
```
+33
View File
@@ -0,0 +1,33 @@
# Upgrading Documentation
## Updating manifests
In order to update manifests make sure you are running the commands in linux.
If you are running in another OS, please make sure to update the Makefile commands.
You can refresh the configuration by running:
```
make seldon-core-operator/base
```
## Updating to specific version
Upgrading the version can be done by setting the `SELDON_VERSION` environment variable, such as:
```
# Set the desired version
export SELDON_VERSION=1.14.0
# Rebuild the kustomize base
make seldon-core-operator/base
# Run new manifests against cluster
kustomize build seldon-core-operator/base | kubectl apply -f -
```
## Instructions for breaking changes
The [core upgrading docs](https://docs.seldon.io/projects/seldon-core/en/latest/reference/upgrading.html.) provide step by step overview of breaking changes across minor and patch versions.
@@ -0,0 +1,15 @@
apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
name: echo
spec:
predictors:
- name: default
graph:
name: classifier
type: MODEL
componentSpecs:
- spec:
containers:
- image: seldonio/echo-model:1.17.1
name: classifier
@@ -0,0 +1,31 @@
# List of resource files that kustomize reads, modifies
# and emits as a YAML string
resources:
- resources.yaml
configurations:
- kustomizeconfig.yaml
vars:
- name: CERTIFICATE_NAMESPACE # namespace of the certificate CR
objref:
kind: Certificate
group: cert-manager.io
version: v1
name: seldon-serving-cert # this name should match the one in certificate.yaml
fieldref:
fieldpath: metadata.namespace
- name: CERTIFICATE_NAME
objref:
kind: Certificate
group: cert-manager.io
version: v1
name: seldon-serving-cert # this name should match the one in certificate.yaml
- name: SERVICE_NAMESPACE # namespace of the service
objref:
kind: Deployment
version: v1
group: apps
name: seldon-controller-manager
fieldref:
fieldpath: metadata.namespace
@@ -0,0 +1,9 @@
varReference:
- path: metadata/annotations
- kind: Certificate
group: cert-manager.io
path: spec/commonName
- kind: Certificate
group: cert-manager.io
path: spec/dnsNames
@@ -0,0 +1,31 @@
# List of resource files that kustomize reads, modifies
# and emits as a YAML string
resources:
- resources.yaml
configurations:
- kustomizeconfig.yaml
vars:
- name: CERTIFICATE_NAMESPACE # namespace of the certificate CR
objref:
kind: Certificate
group: cert-manager.io
version: v1
name: seldon-serving-cert # this name should match the one in certificate.yaml
fieldref:
fieldpath: metadata.namespace
- name: CERTIFICATE_NAME
objref:
kind: Certificate
group: cert-manager.io
version: v1
name: seldon-serving-cert # this name should match the one in certificate.yaml
- name: SERVICE_NAMESPACE # namespace of the service
objref:
kind: Deployment
version: v1
group: apps
name: seldon-controller-manager
fieldref:
fieldpath: metadata.namespace
@@ -0,0 +1,9 @@
varReference:
- path: metadata/annotations
- kind: Certificate
group: cert-manager.io
path: spec/commonName
- kind: Certificate
group: cert-manager.io
path: spec/dnsNames
@@ -0,0 +1,709 @@
---
# Source: seldon-core-operator/templates/serviceaccount_seldon-manager.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app: seldon
app.kubernetes.io/instance: 'seldon-core'
app.kubernetes.io/name: 'seldon-core-operator'
app.kubernetes.io/version: '1.18.1'
name: 'seldon-manager'
namespace: 'kubeflow'
---
# Source: seldon-core-operator/templates/clusterrole_seldon-manager-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
labels:
app: seldon
app.kubernetes.io/instance: 'seldon-core'
app.kubernetes.io/name: 'seldon-core-operator'
app.kubernetes.io/version: '1.18.1'
name: seldon-manager-role-kubeflow
rules:
- apiGroups:
- ''
resources:
- events
verbs:
- create
- patch
- apiGroups:
- ''
resources:
- namespaces
verbs:
- get
- list
- watch
- apiGroups:
- ''
resources:
- services
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- apps
resources:
- deployments
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- apps
resources:
- deployments/status
verbs:
- get
- patch
- update
- apiGroups:
- autoscaling
resources:
- horizontalpodautoscalers
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- autoscaling
resources:
- horizontalpodautoscalers/status
verbs:
- get
- patch
- update
- apiGroups:
- getambassador.io
resources:
- mappings
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- getambassador.io
resources:
- mappings/status
verbs:
- get
- patch
- update
- apiGroups:
- getambassador.io
resources:
- tlscontexts
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- getambassador.io
resources:
- tlscontexts/status
verbs:
- get
- patch
- update
- apiGroups:
- keda.sh
resources:
- scaledobjects
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- keda.sh
resources:
- scaledobjects/finalizers
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- keda.sh
resources:
- scaledobjects/status
verbs:
- get
- patch
- update
- apiGroups:
- machinelearning.seldon.io
resources:
- seldondeployments
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- machinelearning.seldon.io
resources:
- seldondeployments/finalizers
verbs:
- get
- patch
- update
- apiGroups:
- machinelearning.seldon.io
resources:
- seldondeployments/status
verbs:
- get
- patch
- update
- apiGroups:
- networking.istio.io
resources:
- destinationrules
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- networking.istio.io
resources:
- destinationrules/status
verbs:
- get
- patch
- update
- apiGroups:
- networking.istio.io
resources:
- virtualservices
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- networking.istio.io
resources:
- virtualservices/status
verbs:
- get
- patch
- update
- apiGroups:
- policy
resources:
- poddisruptionbudgets
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- policy
resources:
- poddisruptionbudgets/status
verbs:
- get
- patch
- update
- apiGroups:
- v1
resources:
- namespaces
verbs:
- get
- list
- watch
- apiGroups:
- v1
resources:
- services
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- v1
resources:
- services/status
verbs:
- get
- patch
- update
---
# Source: seldon-core-operator/templates/clusterrole_seldon-manager-sas-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
labels:
app: seldon
app.kubernetes.io/instance: 'seldon-core'
app.kubernetes.io/name: 'seldon-core-operator'
app.kubernetes.io/version: '1.18.1'
name: seldon-manager-sas-role-kubeflow
rules:
- apiGroups:
- ''
resources:
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- ''
resources:
- configmaps
verbs:
- get
- list
- watch
- apiGroups:
- ''
resources:
- serviceaccounts
verbs:
- get
- list
- watch
---
# Source: seldon-core-operator/templates/clusterrole_seldon-webhook-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
labels:
app: seldon
app.kubernetes.io/instance: 'seldon-core'
app.kubernetes.io/name: 'seldon-core-operator'
app.kubernetes.io/version: '1.18.1'
name: seldon-webhook-role-kubeflow
rules:
- apiGroups:
- admissionregistration.k8s.io
resources:
- mutatingwebhookconfigurations
- validatingwebhookconfigurations
verbs:
- get
- list
- create
- delete
- update
- apiGroups:
- apps
resources:
- deployments/finalizers
verbs:
- get
- patch
- update
- apiGroups:
- apiextensions.k8s.io
resources:
- customresourcedefinitions
verbs:
- create
- get
- list
- apiGroups:
- apiextensions.k8s.io
resources:
- customresourcedefinitions/finalizers
verbs:
- get
- patch
- update
---
# Source: seldon-core-operator/templates/clusterrolebinding_seldon-manager-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app: seldon
app.kubernetes.io/instance: 'seldon-core'
app.kubernetes.io/name: 'seldon-core-operator'
app.kubernetes.io/version: '1.18.1'
name: seldon-manager-rolebinding-kubeflow
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: seldon-manager-role-kubeflow
subjects:
- kind: ServiceAccount
name: 'seldon-manager'
namespace: 'kubeflow'
---
# Source: seldon-core-operator/templates/clusterrolebinding_seldon-manager-sas-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app: seldon
app.kubernetes.io/instance: 'seldon-core'
app.kubernetes.io/name: 'seldon-core-operator'
app.kubernetes.io/version: '1.18.1'
name: seldon-manager-sas-rolebinding-kubeflow
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: seldon-manager-sas-role-kubeflow
subjects:
- kind: ServiceAccount
name: seldon-manager
namespace: 'kubeflow'
---
# Source: seldon-core-operator/templates/clusterrolebinding_seldon-webhook-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app: seldon
app.kubernetes.io/instance: 'seldon-core'
app.kubernetes.io/name: 'seldon-core-operator'
app.kubernetes.io/version: '1.18.1'
name: seldon-webhook-rolebinding-kubeflow
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: seldon-webhook-role-kubeflow
subjects:
- kind: ServiceAccount
name: seldon-manager
namespace: 'kubeflow'
---
# Source: seldon-core-operator/templates/role_seldon-leader-election-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app: seldon
app.kubernetes.io/instance: 'seldon-core'
app.kubernetes.io/name: 'seldon-core-operator'
app.kubernetes.io/version: '1.18.1'
name: seldon-leader-election-role
namespace: 'kubeflow'
rules:
- apiGroups:
- ''
resources:
- configmaps/status
verbs:
- get
- update
- patch
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
- apiGroups:
- ''
resources:
- configmaps
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
- apiGroups:
- ''
resources:
- events
verbs:
- create
- patch
---
# Source: seldon-core-operator/templates/rolebinding_seldon-leader-election-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app: seldon
app.kubernetes.io/instance: 'seldon-core'
app.kubernetes.io/name: 'seldon-core-operator'
app.kubernetes.io/version: '1.18.1'
name: seldon-leader-election-rolebinding
namespace: 'kubeflow'
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: seldon-leader-election-role
subjects:
- kind: ServiceAccount
name: 'seldon-manager'
namespace: 'kubeflow'
---
# Source: seldon-core-operator/templates/deployment_seldon-controller-manager.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: seldon
app.kubernetes.io/instance: 'seldon-core'
app.kubernetes.io/name: 'seldon-core-operator'
app.kubernetes.io/version: '1.18.1'
control-plane: seldon-controller-manager
name: seldon-controller-manager
namespace: 'kubeflow'
spec:
replicas: 1
selector:
matchLabels:
app: seldon
app.kubernetes.io/instance: seldon1
app.kubernetes.io/name: seldon
app.kubernetes.io/version: v0.5
control-plane: seldon-controller-manager
template:
metadata:
annotations:
prometheus.io/port: '8080'
prometheus.io/scrape: 'true'
sidecar.istio.io/inject: 'false'
labels:
app: seldon
app.kubernetes.io/instance: seldon1
app.kubernetes.io/name: seldon
app.kubernetes.io/version: v0.5
control-plane: seldon-controller-manager
spec:
containers:
- args:
- --enable-leader-election
- --webhook-port=4443
- --metrics-addr=:8080
- --create-resources=$(MANAGER_CREATE_RESOURCES)
- --log-level=$(MANAGER_LOG_LEVEL)
- --leader-election-id=$(MANAGER_LEADER_ELECTION_ID)
- --leader-election-resource-lock=$(MANAGER_LEADER_ELECTION_RESOURCE_LOCK)
- --leader-election-lease-duration-secs=$(MANAGER_LEADER_ELECTION_LEASE_DURATION_SECS)
- --leader-election-renew-deadline-secs=$(MANAGER_LEADER_ELECTION_RENEW_DEADLINE_SECS)
- --leader-election-retry-period-secs=$(MANAGER_LEADER_ELECTION_RETRY_PERIOD_SECS)
- ''
command:
- /manager
env:
- name: MANAGER_LEADER_ELECTION_ID
value: 'a33bd623.machinelearning.seldon.io'
- name: MANAGER_LEADER_ELECTION_RESOURCE_LOCK
value: ''
- name: MANAGER_LEADER_ELECTION_LEASE_DURATION_SECS
value: '15'
- name: MANAGER_LEADER_ELECTION_RENEW_DEADLINE_SECS
value: '10'
- name: MANAGER_LEADER_ELECTION_RETRY_PERIOD_SECS
value: '2'
- name: MANAGER_LOG_LEVEL
value: 'INFO'
- name: WATCH_NAMESPACE
value: ''
- name: RELATED_IMAGE_EXECUTOR
value: ''
- name: RELATED_IMAGE_STORAGE_INITIALIZER
value: ''
- name: RELATED_IMAGE_SKLEARNSERVER
value: ''
- name: RELATED_IMAGE_SKLEARNSERVER_V2
value: ''
- name: RELATED_IMAGE_XGBOOSTSERVER
value: ''
- name: RELATED_IMAGE_XGBOOSTSERVER_V2
value: ''
- name: RELATED_IMAGE_MLFLOWSERVER
value: ''
- name: RELATED_IMAGE_MLFLOWSERVER_V2
value: ''
- name: RELATED_IMAGE_TFPROXY
value: ''
- name: RELATED_IMAGE_TENSORFLOW
value: ''
- name: RELATED_IMAGE_EXPLAINER
value: ''
- name: RELATED_IMAGE_EXPLAINER_V2
value: ''
- name: RELATED_IMAGE_MOCK_CLASSIFIER
value: ''
- name: MANAGER_CREATE_RESOURCES
value: 'true'
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: CONTROLLER_ID
value: ''
- name: AMBASSADOR_ENABLED
value: 'false'
- name: AMBASSADOR_VERSION
value: 'v2'
- name: AMBASSADOR_SINGLE_NAMESPACE
value: 'false'
- name: PREDICTIVE_UNIT_HTTP_SERVICE_PORT
value: '9000'
- name: PREDICTIVE_UNIT_GRPC_SERVICE_PORT
value: '9500'
- name: PREDICTIVE_UNIT_DEFAULT_ENV_SECRET_REF_NAME
value: ''
- name: PREDICTIVE_UNIT_METRICS_PORT_NAME
value: 'metrics'
- name: ISTIO_ENABLED
value: 'true'
- name: KEDA_ENABLED
value: 'false'
- name: ISTIO_GATEWAY
value: '$(SERVICE_NAMESPACE)/kubeflow-gateway'
- name: ISTIO_TLS_MODE
value: ''
- name: USE_EXECUTOR
value: 'true'
- name: EXECUTOR_CONTAINER_IMAGE_AND_VERSION
value: 'docker.io/seldonio/seldon-core-executor:1.18.1'
- name: EXECUTOR_CONTAINER_IMAGE_PULL_POLICY
value: 'IfNotPresent'
- name: EXECUTOR_PROMETHEUS_PATH
value: '/prometheus'
- name: EXECUTOR_SERVER_PORT
value: '8000'
- name: EXECUTOR_CONTAINER_USER
value: '8888'
- name: EXECUTOR_CONTAINER_SERVICE_ACCOUNT_NAME
value: 'default'
- name: EXECUTOR_SERVER_METRICS_PORT_NAME
value: 'metrics'
- name: EXECUTOR_REQUEST_LOGGER_DEFAULT_ENDPOINT
value: 'http://default-broker'
- name: EXECUTOR_REQUEST_LOGGER_WORK_QUEUE_SIZE
value: '10000'
- name: EXECUTOR_REQUEST_LOGGER_WRITE_TIMEOUT_MS
value: '2000'
- name: DEFAULT_USER_ID
value: '8888'
- name: EXECUTOR_DEFAULT_CPU_REQUEST
value: '500m'
- name: EXECUTOR_DEFAULT_MEMORY_REQUEST
value: '512Mi'
- name: EXECUTOR_DEFAULT_CPU_LIMIT
value: '500m'
- name: EXECUTOR_DEFAULT_MEMORY_LIMIT
value: '512Mi'
- name: DEPLOYMENT_NAME_AS_PREFIX
value: 'false'
- name: EXECUTOR_FULL_HEALTH_CHECKS
value: 'false'
image: 'docker.io/seldonio/seldon-core-operator:1.18.1'
imagePullPolicy: 'IfNotPresent'
name: manager
ports:
- containerPort: 4443
name: webhook-server
protocol: TCP
- containerPort: 8080
name: metrics
protocol: TCP
resources:
limits:
cpu: '500m'
memory: '300Mi'
requests:
cpu: '100m'
memory: '200Mi'
hostNetwork: false
priorityClassName: ''
securityContext:
runAsUser: 8888
serviceAccountName: 'seldon-manager'
terminationGracePeriodSeconds: 10
---
# Source: seldon-core-operator/templates/certificate_seldon-serving-cert.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
labels:
app: seldon
app.kubernetes.io/instance: 'seldon-core'
app.kubernetes.io/name: 'seldon-core-operator'
app.kubernetes.io/version: '1.18.1'
name: seldon-serving-cert
namespace: 'kubeflow'
spec:
commonName: seldon-webhook-service.$(CERTIFICATE_NAMESPACE).svc
dnsNames:
- seldon-webhook-service.$(SERVICE_NAMESPACE).svc.cluster.local
- seldon-webhook-service.$(SERVICE_NAMESPACE).svc
issuerRef:
kind: Issuer
name: seldon-selfsigned-issuer
secretName: seldon-webhook-server-cert
---
# Source: seldon-core-operator/templates/issuer_seldon-selfsigned-issuer.yaml
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
labels:
app: seldon
app.kubernetes.io/instance: 'seldon-core'
app.kubernetes.io/name: 'seldon-core-operator'
app.kubernetes.io/version: '1.18.1'
name: seldon-selfsigned-issuer
namespace: 'kubeflow'
spec:
selfSigned: {}
@@ -0,0 +1,41 @@
apiVersion: app.k8s.io/v1beta1
kind: Application
metadata:
name: seldon-core-operator
spec:
selector:
matchLabels:
app.kubernetes.io/component: seldon
app.kubernetes.io/instance: seldon-1.2.1
app.kubernetes.io/managed-by: kfctl
app.kubernetes.io/name: seldon
app.kubernetes.io/part-of: kubeflow
app.kubernetes.io/version: '1.2.1'
componentKinds:
- group: apps/v1
kind: StatefulSet
- group: v1
kind: Service
- group: apps/v1
kind: Deployment
- group: v1
kind: Secret
- group: v1
kind: ConfigMap
descriptor:
type: seldon-core-operator
version: v1
description: "Seldon allows users to create ML Inference Graphs to deploy their models and serve predictions"
maintainers:
- email: dev@seldon.io
name: Seldon
owners:
- email: dev@seldon.io
name: Seldon
keywords:
- seldon
- inference
links:
- description: Docs
url: https://docs.seldon.io/projects/seldon-core/en/v1.1.0/
addOwnerRef: true
@@ -0,0 +1,10 @@
apiVersion: kustomize.config.k8s.io/v1beta1
bases:
- ../../base
commonLabels:
app.kubernetes.io/component: seldon
app.kubernetes.io/name: seldon-core-operator
kind: Kustomization
resources:
- application.yaml
@@ -0,0 +1,22 @@
# # Seldon Core Operator
# Below are the overrides for kubeflow defaults
# Istio gateway must be set to the kubeflow default
istio:
enabled: true
gateway: kubeflow/kubeflow-gateway
tlsMode: ""
# Certmanager using existing cert
certManager:
enabled: true
# Creation of resources automated as CRD too large for apply
managerCreateResources: true
# Disable CRD in helm charts as manager creates resources
crd:
create: false
kubeflow: true