update kubeflow dip-catalog

This commit is contained in:
ChanghoWoo
2025-01-13 02:31:27 +00:00
parent 1dc1181a03
commit 5451f16d72
1959 changed files with 602337 additions and 0 deletions
+201
View File
@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+11
View File
@@ -0,0 +1,11 @@
approvers:
- juliusvonkohout
- kimwnasptd
reviewers:
- juliusvonkohout
- kimwnasptd
emeritus_approvers:
- elikatsis
- PatrickXYS
- StefanoFioravanzo
- yanniszark
+553
View File
@@ -0,0 +1,553 @@
# Kubeflow Manifests
## Table of Contents
<!-- toc -->
- [Overview of the Kubeflow Platform](#overview)
- [Kubeflow components versions](#kubeflow-components-versions)
- [Installation](#installation)
* [Prerequisites](#prerequisites)
* [Install with a single command](#install-with-a-single-command)
* [Install individual components](#install-individual-components)
* [Connect to your Kubeflow Cluster](#connect-to-your-kubeflow-cluster)
* [Change default user password](#change-default-user-password)
- [Upgrading and extending](#upgrading-and-extending)
- [Release process](#release-process)
- [CVE Scanning](#cve-scanning)
- [Frequently Asked Questions](#frequently-asked-questions)
<!-- tocstop -->
## Overview of the Kubeflow Platform
This repository is owned by the [Manifests Working Group](https://github.com/kubeflow/community/blob/master/wg-manifests/charter.md).
If you are a contributor authoring or editing the packages please see [Best Practices](./documents/KustomizeBestPractices.md).
You can join the CNCF Slack and access our meetings at the [Kubeflow Community](https://www.kubeflow.org/docs/about/community/) website. Our channel on the CNCF Slack is here [**#kubeflow-platform**](https://app.slack.com/client/T08PSQ7BQ/C073W572LA2). You can also find there our [biweekly meetings](https://bit.ly/kf-wg-manifests-meet), including the commentable [Agenda](https://bit.ly/kf-wg-manifests-notes).
The Kubeflow Manifests repository is organized under three main directories, which include manifests for installing:
| Directory | Purpose |
| - | - |
| `apps` | Kubeflow's official components, as maintained by the respective Kubeflow WGs |
| `common` | Common services, as maintained by the Manifests WG |
| `contrib` | 3rd party contributed applications (e.g. Ray, Kserve), which are maintained externally and are not part of a Kubeflow WG |
All components are deployable with `kustomize`. You can choose to deploy the whole Kubeflow platform or individual components.
## Kubeflow components versions
### Kubeflow Version: master
This repo periodically syncs all official Kubeflow components from their respective upstream repos. The following matrix shows the git version that we include for each component:
| Component | Local Manifests Path | Upstream Revision |
| - | - | - |
| Training Operator | apps/training-operator/upstream | [v1.8.0](https://github.com/kubeflow/training-operator/tree/v1.8.0/manifests) |
| Notebook Controller | apps/jupyter/notebook-controller/upstream | [v1.9.0](https://github.com/kubeflow/kubeflow/tree/v1.9.0/components/notebook-controller/config) |
| PVC Viewer Controller | apps/pvcviewer-roller/upstream | [v1.9.0](https://github.com/kubeflow/kubeflow/tree/v1.9.0/components/pvcviewer-controller/config) |
| Tensorboard Controller | apps/tensorboard/tensorboard-controller/upstream | [v1.9.0](https://github.com/kubeflow/kubeflow/tree/v1.9.0/components/tensorboard-controller/config) |
| Central Dashboard | apps/centraldashboard/upstream | [v1.9.0](https://github.com/kubeflow/kubeflow/tree/v1.9.0/components/centraldashboard/manifests) |
| Profiles + KFAM | apps/profiles/upstream | [v1.9.0](https://github.com/kubeflow/kubeflow/tree/v1.9.0/components/profile-controller/config) |
| PodDefaults Webhook | apps/admission-webhook/upstream | [v1.9.0](https://github.com/kubeflow/kubeflow/tree/v1.9.0/components/admission-webhook/manifests) |
| Jupyter Web App | apps/jupyter/jupyter-web-app/upstream | [v1.9.0](https://github.com/kubeflow/kubeflow/tree/v1.9.0/components/crud-web-apps/jupyter/manifests) |
| Tensorboards Web App | apps/tensorboard/tensorboards-web-app/upstream | [v1.9.0](https://github.com/kubeflow/kubeflow/tree/v1.9.0/components/crud-web-apps/tensorboards/manifests) |
| Volumes Web App | apps/volumes-web-app/upstream | [v1.9.0](https://github.com/kubeflow/kubeflow/tree/v1.9.0/components/crud-web-apps/volumes/manifests) |
| Katib | apps/katib/upstream | [v0.17.0](https://github.com/kubeflow/katib/tree/v0.17.0/manifests/v1beta1) |
| KServe | contrib/kserve/kserve | [0.13.0](https://github.com/kserve/kserve/releases/tag/v0.13.0) |
| KServe Models Web App | contrib/kserve/models-web-app | [0.13.0](https://github.com/kserve/models-web-app/tree/0.13.0/config) |
| Kubeflow Pipelines | apps/pipeline/upstream | [2.2.0](https://github.com/kubeflow/pipelines/tree/2.2.0/manifests/kustomize) |
| Kubeflow Tekton Pipelines | apps/kfp-tekton/upstream | [2.0.5](https://github.com/kubeflow/kfp-tekton/tree/2.0.5/manifests/kustomize) |
| Kubeflow Model Registry | apps/model-registry/upstream | [v0.2.1-alpha](https://github.com/kubeflow/model-registry/tree/v0.2.1-alpha/manifests/kustomize) |
The following is also a matrix with versions from common components that are
used from the different projects of Kubeflow:
| Component | Local Manifests Path | Upstream Revision |
| - | - | - |
| Istio | common/istio-1-22 | [1.22.1](https://github.com/istio/istio/releases/tag/1.22.1) |
| Knative | common/knative/knative-serving <br /> common/knative/knative-eventing | [v1.12.4](https://github.com/knative/serving/releases/tag/knative-v1.12.4) <br /> [v1.12.6](https://github.com/knative/eventing/releases/tag/knative-v1.12.6) |
| Cert Manager | common/cert-manager | [1.14.5](https://github.com/cert-manager/cert-manager/releases/tag/v1.12.2) |
## Installation
This is for the installation from scratch. For the in-place upgrade guide please jump to the upgrading and extending section.
The Manifests WG provides two options for installing Kubeflow official components and common services with kustomize. The aim is to help end users install easily and to help distribution owners build their opinionated distributions from a tested starting point:
1. Single-command installation of all components under `apps` and `common`
2. Multi-command, individual components installation for `apps` and `common`
Option 1 targets ease of deployment for end users. \
Option 2 targets customization and ability to pick and choose individual components.
The `example` directory contains an example kustomization for the single command to be able to run.
:warning: In both options, we use a default email (`user@example.com`) and password (`12341234`). For any production Kubeflow deployment, you should change the default password by following [the relevant section](#change-default-user-password).
### Prerequisites
- This is the master branch which targets Kubernetes 1.29+
- For the specific Kubernetes version per release consult the [release notes](https://github.com/kubeflow/manifests/releases)
- Either our local Kind (installed below) or your own Kubernetes cluster with a default [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/)
- Kustomize [5.2.1+](https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv5.2.1)
- Kubectl in a version that is [compatible with your Kubernetes cluster](https://kubernetes.io/releases/version-skew-policy/#kubectl)
---
**NOTE**
`kubectl apply` commands may fail on the first try. This is inherent in how Kubernetes and `kubectl` work (e.g., CR must be created after CRD becomes ready). The solution is to simply re-run the command until it succeeds. For the single-line command, we have included a bash one-liner to retry the command.
---
### Install with a single command
#### Prerequisites
- 32 GB of RAM recommended
- 16 CPU cores recommended
- `kind`
- `docker`
- Linux kernel subsystem changes to support many pods
- `sudo sysctl fs.inotify.max_user_instances=2280`
- `sudo sysctl fs.inotify.max_user_watches=1255360`
#### Create kind cluster
```sh
cat <<EOF | kind create cluster --name=kubeflow --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
image: kindest/node:v1.29.4
kubeadmConfigPatches:
- |
kind: ClusterConfiguration
apiServer:
extraArgs:
"service-account-issuer": "kubernetes.default.svc"
"service-account-signing-key-file": "/etc/kubernetes/pki/sa.key"
EOF
```
#### Save kubeconfig
```sh
kind get kubeconfig --name kubeflow > /tmp/kubeflow-config
export KUBECONFIG=/tmp/kubeflow-config
```
#### Create a Secret based on existing credentials in order to pull the images
```sh
docker login
kubectl create secret generic regcred \
--from-file=.dockerconfigjson=/home/to/.docker/config.json \
--type=kubernetes.io/dockerconfigjson
```
You can install all Kubeflow official components (residing under `apps`) and all common services (residing under `common`) using the following command:
```sh
while ! kustomize build example | kubectl apply -f -; do echo "Retrying to apply resources"; sleep 20; done
```
Once, everything is installed successfully, you can access the Kubeflow Central Dashboard [by logging in to your cluster](#connect-to-your-kubeflow-cluster).
Congratulations! You can now start experimenting and running your end-to-end ML workflows with Kubeflow.
### Install individual components
In this section, we will install each Kubeflow official component (under `apps`) and each common service (under `common`) separately, using just `kubectl` and `kustomize`.
If all the following commands are executed, the result is the same as in the above section of the single command installation. The purpose of this section is to:
- Provide a description of each component and insight on how it gets installed.
- Enable the user or distribution owner to pick and choose only the components they need.
---
**Troubleshooting note**
We've seen errors like the following when applying the kustomizations of different components:
```
error: resource mapping not found for name: "<RESOURCE_NAME>" namespace: "<SOME_NAMESPACE>" from "STDIN": no matches for kind "<CRD_NAME>" in version "<CRD_FULL_NAME>"
ensure CRDs are installed first
```
This is because a kustomization applies both a CRD and a CR very quickly, and the CRD
hasn't become [`Established`](https://github.com/kubernetes/apiextensions-apiserver/blob/a7ee7f91a2d0805f729998b85680a20cfba208d2/pkg/apis/apiextensions/types.go#L276-L279) yet. You can learn more about this in https://github.com/kubernetes/kubectl/issues/1117 and https://github.com/helm/helm/issues/4925.
If you bump into this error we advise to re-apply the kustomization of the component.
---
#### cert-manager
Cert-manager is used by many Kubeflow components to provide certificates for
admission webhooks.
Install cert-manager:
```sh
kustomize build common/cert-manager/cert-manager/base | kubectl apply -f -
echo "Waiting for cert-manager to be ready ..."
kubectl wait --for=condition=ready pod -l 'app in (cert-manager,webhook)' --timeout=180s -n cert-manager
kubectl wait --for=jsonpath='{.subsets[0].addresses[0].targetRef.kind}'=Pod endpoints -l 'app in (cert-manager,webhook)' --timeout=180s -n cert-manager
```
In case you get this error:
```
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s": dial tcp 10.96.202.64:443: connect: connection refused
```
This is because the webhook is not yet ready to receive request. Wait a couple seconds and retry applying the manfiests.
For more troubleshooting info also check out https://cert-manager.io/docs/troubleshooting/webhook/
#### Istio
Istio is used by most Kubeflow components to secure their traffic, enforce
network authorization and implement routing policies.
Install Istio:
```sh
echo "Installing Istio configured with external authorization..."
cd common/istio-1-22
kustomize build common/istio-1-22/istio-crds/base | kubectl apply -f -
kustomize build common/istio-1-22/istio-namespace/base | kubectl apply -f -
kustomize build common/istio-1-22/istio-install/overlays/oauth2-proxy | kubectl apply -f -
echo "Waiting for all Istio Pods to become ready..."
kubectl wait --for=condition=Ready pods --all -n istio-system --timeout 300s
```
#### Oauth2-proxy
The oauth2-proxy extends your Istio Ingress-Gateway capabilities, to be able to function as an OIDC client:
```sh
echo "Installing oauth2-proxy..."
kustomize build common/oidc-client/oauth2-proxy/overlays/m2m-self-signed/ | kubectl apply -f -
kubectl wait --for=condition=ready pod -l 'app.kubernetes.io/name=oauth2-proxy' --timeout=180s -n oauth2-proxy
```
It supports user sessions as well as proper token-based machine to machine atuhhentication.
#### Dex
Dex is an OpenID Connect Identity (OIDC) with multiple authentication backends. In this default installation, it includes a static user with email `user@example.com`. By default, the user's password is `12341234`. For any production Kubeflow deployment, you should change the default password by following [the relevant section](#change-default-user-password).
Install Dex:
```sh
kustomize build common/dex/overlays/oauth2-proxy | kubectl apply -f -
```
#### Knative
Knative is used by the KServe official Kubeflow component.
Install Knative Serving:
```sh
kustomize build common/knative/knative-serving/overlays/gateways | kubectl apply -f -
kustomize build common/istio-1-22/cluster-local-gateway/base | kubectl apply -f -
```
Optionally, you can install Knative Eventing which can be used for inference request logging:
```sh
kustomize build common/knative/knative-eventing/base | kubectl apply -f -
```
#### Kubeflow Namespace
Create the namespace where the Kubeflow components will live in. This namespace
is named `kubeflow`.
Install kubeflow namespace:
```sh
kustomize build common/kubeflow-namespace/base | kubectl apply -f -
```
#### Kubeflow Roles
Create the Kubeflow ClusterRoles, `kubeflow-view`, `kubeflow-edit` and
`kubeflow-admin`. Kubeflow components aggregate permissions to these
ClusterRoles.
Install kubeflow roles:
```sh
kustomize build common/kubeflow-roles/base | kubectl apply -f -
```
#### Kubeflow Pipelines
Install the [Multi-User Kubeflow Pipelines](https://www.kubeflow.org/docs/components/pipelines/multi-user/) official Kubeflow component:
```sh
kustomize build apps/pipeline/upstream/env/cert-manager/platform-agnostic-multi-user | kubectl apply -f -
```
This installs argo with the runasnonroot emissary executor. Please note that you are still responsible to analyze the security issues that arise when containers are run with root access and to decide if the kubeflow pipeline main containers are run as runasnonroot. It is in general strongly recommended that all user-accessible OCI containers run with Pod Security Standards [restricted]
(https://kubernetes.io/docs/concepts/security/pod-security-standards/#restricted)
**Multi-User Kubeflow Pipelines dependencies**
* Istio
* Kubeflow Roles
* OIDC Auth Service (or cloud provider specific auth service)
* Profiles + KFAM
**Alternative: Kubeflow Pipelines Standalone**
You can install [Kubeflow Pipelines Standalone](https://www.kubeflow.org/docs/components/pipelines/installation/standalone-deployment/) which
* does not support multi user separation
* has no dependencies on the other services mentioned here
You can learn more about their differences in [Installation Options for Kubeflow Pipelines
](https://www.kubeflow.org/docs/components/pipelines/installation/overview/).
Besides installation instructions in Kubeflow Pipelines Standalone documentation, you need to apply two virtual services to expose [Kubeflow Pipelines UI](https://github.com/kubeflow/pipelines/blob/1.7.0/manifests/kustomize/base/installs/multi-user/virtual-service.yaml) and [Metadata API](https://github.com/kubeflow/pipelines/blob/1.7.0/manifests/kustomize/base/metadata/options/istio/virtual-service.yaml) in kubeflow-gateway.
#### KServe
KFServing was rebranded to KServe.
Install the KServe component:
```sh
kustomize build contrib/kserve/kserve | kubectl apply -f -
```
Install the Models web application:
```sh
kustomize build contrib/kserve/models-web-app/overlays/kubeflow | kubectl apply -f -
```
#### Katib
Install the Katib official Kubeflow component:
```sh
kustomize build apps/katib/upstream/installs/katib-with-kubeflow | kubectl apply -f -
```
#### Central Dashboard
Install the Central Dashboard official Kubeflow component:
```sh
kustomize build apps/centraldashboard/upstream/overlays/kserve | kubectl apply -f -
```
#### Admission Webhook
Install the Admission Webhook for PodDefaults:
```sh
kustomize build apps/admission-webhook/upstream/overlays/cert-manager | kubectl apply -f -
```
#### Notebooks 1.0
Install the Notebook Controller official Kubeflow component:
```sh
kustomize build apps/jupyter/notebook-controller/upstream/overlays/kubeflow | kubectl apply -f -
```
Install the Jupyter Web App official Kubeflow component:
```sh
kustomize build apps/jupyter/jupyter-web-app/upstream/overlays/istio | kubectl apply -f -
```
#### Workspaces (Notebooks 2.0)
It is still in development.
#### PVC Viewer Controller
Install the PVC Viewer Controller official Kubeflow component:
```sh
kustomize build apps/pvcviewer-controller/upstream/default | kubectl apply -f -
```
#### Profiles + KFAM
Install the Profile Controller and the Kubeflow Access-Management (KFAM) official Kubeflow
components:
```sh
kustomize build apps/profiles/upstream/overlays/kubeflow | kubectl apply -f -
```
#### Volumes Web Application
Install the Volumes Web App official Kubeflow component:
```sh
kustomize build apps/volumes-web-app/upstream/overlays/istio | kubectl apply -f -
```
#### Tensorboard
Install the Tensorboards Web App official Kubeflow component:
```sh
kustomize build apps/tensorboard/tensorboards-web-app/upstream/overlays/istio | kubectl apply -f -
```
Install the Tensorboard Controller official Kubeflow component:
```sh
kustomize build apps/tensorboard/tensorboard-controller/upstream/overlays/kubeflow | kubectl apply -f -
```
#### Training Operator
Install the Training Operator official Kubeflow component:
```sh
kustomize build apps/training-operator/upstream/overlays/kubeflow | kubectl apply -f -
```
#### User Namespaces
Finally, create a new namespace for the default user (named `kubeflow-user-example-com`).
```sh
kustomize build common/user-namespace/base | kubectl apply -f -
```
### Connect to your Kubeflow Cluster
After installation, it will take some time for all Pods to become ready. Make sure all Pods are ready before trying to connect, otherwise you might get unexpected errors. To check that all Kubeflow-related Pods are ready, use the following commands:
```sh
kubectl get pods -n cert-manager
kubectl get pods -n istio-system
kubectl get pods -n auth
kubectl get pods -n knative-eventing
kubectl get pods -n knative-serving
kubectl get pods -n kubeflow
kubectl get pods -n kubeflow-user-example-com
```
#### Port-Forward
The default way of accessing Kubeflow is via port-forward. This enables you to get started quickly without imposing any requirements on your environment. Run the following to port-forward Istio's Ingress-Gateway to local port `8080`:
```sh
kubectl port-forward svc/istio-ingressgateway -n istio-system 8080:80
```
After running the command, you can access the Kubeflow Central Dashboard by doing the following:
1. Open your browser and visit `http://localhost:8080`. You should get the Dex login screen.
2. Login with the default user's credentials. The default email address is `user@example.com` and the default password is `12341234`.
#### NodePort / LoadBalancer / Ingress
In order to connect to Kubeflow using NodePort / LoadBalancer / Ingress, you need to setup HTTPS. The reason is that many of our web applications (e.g., Tensorboard Web Application, Jupyter Web Application, Katib UI) use [Secure Cookies](https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies#restrict_access_to_cookies), so accessing Kubeflow with HTTP over a non-localhost domain does not work.
Exposing your Kubeflow cluster with proper HTTPS is a simple proces, but dependent on your environment.
There are also third-party commercial [distributions](https://www.kubeflow.org/docs/started/installing-kubeflow/#install-a-packaged-kubeflow-distribution) available.
---
**NOTE**
If you absolutely need to expose Kubeflow over HTTP, you can disable the `Secure Cookies` feature by setting the `APP_SECURE_COOKIES` environment variable to `false` in every relevant web app. This is not recommended, as it poses security risks.
---
### Change default user password
For security reasons, we don't want to use the default password for the default Kubeflow user when installing in security-sensitive environments. Instead, you should define your own password and apply it either **before creating the cluster** or **after creating the cluster**.
Pick a password for the default user, with email `user@example.com`, and hash it using `bcrypt`:
```sh
python3 -c 'from passlib.hash import bcrypt; import getpass; print(bcrypt.using(rounds=12, ident="2y").hash(getpass.getpass()))'
```
For example, running the above command locally with required packages like _passlib_ would look as follows:
```sh
python3 -c 'from passlib.hash import bcrypt; import getpass; print(bcrypt.using(rounds=12, ident="2y").hash(getpass.getpass()))'
Password: <--- Enter the password here
$2y$12$vIm8CANhuWui0J1p3jYeGeuM28Qcn76IFMaFWvZCG5ZkKZ4MjTF4u <--- GENERATED_HASH_FOR_ENTERED_PASSWORD
```
#### Before creating the cluster:
1. Edit `common/dex/base/dex-passwords.yaml` and fill the relevant field with the hash of the password you chose:
```yaml
...
stringData:
DEX_USER_PASSWORD: <REPLACE_WITH_HASH>
```
#### After creating the cluster:
1. Delete the existing secret _dex-passwords_ in auth namespace using the following command:
```sh
kubectl delete secret dex-passwords -n auth
```
2. Create secret dex-passwords with new hash using the following command:
```sh
kubectl create secret generic dex-passwords --from-literal=DEX_USER_PASSWORD='REPLACE_WITH_HASH' -n auth
```
3. Recreate the _dex_ pod in auth namespace using the following command:
```sh
kubectl delete pods --all -n auth
```
4. Try to login using the new dex password.
## Upgrading and extending
For modifications and in place upgrades of the Kubeflow platform we provide a rough description for advanced users:
- Never ever edit the manifests directly, use Kustomize overlays and [components](https://github.com/kubernetes-sigs/kustomize/blob/master/examples/components.md) on top of the [example.yaml](https://github.com/kubeflow/manifests/blob/master/example/kustomization.yaml).
- This allows you to upgrade by just referencing the new manifests, building with kustomize and running `kubectl apply` again.
- You might have to adjust your over the top overlays and components if needed.
- You might have to prune old resources. For that you would add [labels](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/labels/) to all your resources from the start.
- With labels you can use `kubectl apply` with `--prune` and `--dry-run` to list prunable resources.
- Sometimes there are major changes, e.g. in the 1.9 release we switch to oauth2-proxy, which need additional attention.
- Nevertheless with a bit of Kubernetes knowledge one should be able to upgrade.
## Release process
The Manifest Working Group releases Kubeflow based on the [release timeline](https://github.com/kubeflow/community/blob/master/releases/handbook.md#timeline).
The community and the release team work closely with the Manifest Working Group to define the specific dates at the start of the [release cycle](https://github.com/kubeflow/community/blob/master/releases/handbook.md#releasing)
and follow the [release versioning policy](https://github.com/kubeflow/community/blob/master/releases/handbook.md#versioning-policy),
as defined in the [Kubeflow release handbook](https://github.com/kubeflow/community/blob/master/releases/handbook.md).
## CVE Scanning
To view all past security scans, head to the [Image Extracting and Security Scanning GitHub Action workflow](https://github.com/kubeflow/manifests/actions/workflows/trivy.yaml). In the logs of the workflow you can expand the `Run image extracting and security scanning script` step to view the CVE logs. You will find a per-image CVE scan and a JSON dump of per-WorkingGroup aggregated metrics.
You can run the Python script from the workflow file locally on your machine to obtain the detailed JSON files for any git commit.
The Kubeflow security working group follows a responsible disclosure policy for CVE results:
- **Internal Review**: All CVE findings are initially reviewed internally by the security working group.
- **Severity Assessment**: Each CVE is assessed for severity and potential impact on the Kubeflow project.
- **Disclosure**: For high and critical severity CVEs, the security working group will:
- Notify the maintainers and contributors
- Try to provide a fix or mitigation strategy
- Publicly disclose the CVE details
## Frequently Asked Questions
- **Q:** What versions of Istio, Knative, Cert-Manager, Argo, ... are compatible with Kubeflow? \
**A:** Please refer to each individual component's documentation for a dependency compatibility range. For Istio, Knative, Dex, Cert-Manager and OIDC-AuthService, the versions in `common` are the ones we have validated.
- **Q:** Can I use earlier version of Kustomize with Kubeflow manifests?
**A:** No, it is not supported anymore, although it might be possible with manual effort.
@@ -0,0 +1,11 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cluster-role-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-role
subjects:
- kind: ServiceAccount
name: service-account
@@ -0,0 +1,65 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cluster-role
rules:
- apiGroups:
- kubeflow.org
resources:
- poddefaults
verbs:
- get
- watch
- list
- update
- create
- patch
- delete
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kubeflow-poddefaults-admin
labels:
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-admin: "true"
aggregationRule:
clusterRoleSelectors:
- matchLabels:
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-poddefaults-admin: "true"
rules: []
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kubeflow-poddefaults-edit
labels:
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-edit: "true"
aggregationRule:
clusterRoleSelectors:
- matchLabels:
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-poddefaults-edit: "true"
rules: []
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kubeflow-poddefaults-view
labels:
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-poddefaults-admin: "true"
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-poddefaults-edit: "true"
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-view: "true"
rules:
- apiGroups:
- kubeflow.org
resources:
- poddefaults
verbs:
- get
- list
- watch
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,25 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment
spec:
template:
metadata:
annotations:
sidecar.istio.io/inject: "false"
spec:
containers:
- image: docker.io/kubeflownotebookswg/poddefaults-webhook
name: admission-webhook
volumeMounts:
- mountPath: /etc/webhook/certs
name: webhook-cert
readOnly: true
ports:
- name: https-webhook
containerPort: 4443
volumes:
- name: webhook-cert
secret:
secretName: webhook-certs
serviceAccountName: service-account
@@ -0,0 +1,52 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- cluster-role-binding.yaml
- cluster-role.yaml
- deployment.yaml
- mutating-webhook-configuration.yaml
- service-account.yaml
- service.yaml
- crd.yaml
commonLabels:
app: poddefaults
kustomize.component: poddefaults
app.kubernetes.io/component: poddefaults
app.kubernetes.io/name: poddefaults
images:
- name: docker.io/kubeflownotebookswg/poddefaults-webhook
newName: docker.io/kubeflownotebookswg/poddefaults-webhook
newTag: v1.9.0
namespace: kubeflow
generatorOptions:
disableNameSuffixHash: true
vars:
# These vars are used to substitute in the namespace, service name and
# deployment name into the mutating WebHookConfiguration.
# Since its a CR kustomize isn't aware of those fields and won't
# transform them.
# We need the var names to be relatively unique so that when we
# compose with other applications they won't conflict.
- fieldref:
fieldPath: metadata.namespace
name: podDefaultsNamespace
objref:
apiVersion: v1
kind: Service
name: service
- fieldref:
fieldPath: metadata.name
name: podDefaultsServiceName
objref:
apiVersion: v1
kind: Service
name: service
- fieldref:
fieldPath: metadata.name
name: podDefaultsDeploymentName
objref:
apiVersion: apps/v1
kind: Deployment
name: deployment
configurations:
- params.yaml
@@ -0,0 +1,28 @@
apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
name: mutating-webhook-configuration
webhooks:
- admissionReviewVersions:
- v1beta1
- v1
clientConfig:
caBundle: ""
service:
name: service
path: /apply-poddefault
sideEffects: None
failurePolicy: Fail
name: $(podDefaultsDeploymentName).kubeflow.org
namespaceSelector:
matchLabels:
app.kubernetes.io/part-of: kubeflow-profile
rules:
- apiGroups:
- ""
apiVersions:
- v1
operations:
- CREATE
resources:
- pods
@@ -0,0 +1,19 @@
varReference:
- path: webhooks/clientConfig/service/namespace
kind: MutatingWebhookConfiguration
- path: webhooks/clientConfig/service/name
kind: MutatingWebhookConfiguration
- path: webhooks/name
kind: MutatingWebhookConfiguration
nameReference:
- kind: Service
version: v1
fieldSpecs:
- kind: MutatingWebhookConfiguration
group: admissionregistration.k8s.io
path: webhooks/clientConfig/service/name
namespace:
- kind: MutatingWebhookConfiguration
group: admissionregistration.k8s.io
path: webhooks/clientConfig/service/namespace
create: true
@@ -0,0 +1,4 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: service-account
@@ -0,0 +1,9 @@
apiVersion: v1
kind: Service
metadata:
name: service
spec:
ports:
- name: https-webhook
port: 443
targetPort: https-webhook
@@ -0,0 +1,23 @@
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: cert
spec:
isCA: true
commonName: $(podDefaultsServiceName).$(podDefaultsNamespace).svc
dnsNames:
- $(podDefaultsServiceName).$(podDefaultsNamespace).svc
- $(podDefaultsServiceName).$(podDefaultsNamespace).svc.cluster.local
issuerRef:
kind: Issuer
name: selfsigned-issuer
secretName: webhook-certs
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: selfsigned-issuer
spec:
selfSigned: {}
@@ -0,0 +1,12 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment
spec:
template:
spec:
containers:
- name: admission-webhook
args:
- --tlsCertFile=/etc/webhook/certs/tls.crt
- --tlsKeyFile=/etc/webhook/certs/tls.key
@@ -0,0 +1,47 @@
# This overlay uses CertManager to provision a certificate for the
# PodDefaults admission controller. This is preferred over the old
# way of using "bootstrap" which was running a shell script to create
# the certificate.
# TODO(jlewi): We should eventually refactor the manifests to delete
# bootstrap and use certmanager by default.
bases:
- ../../base
resources:
- certificate.yaml
namespace: kubeflow
namePrefix: admission-webhook-
commonLabels:
app: poddefaults
kustomize.component: poddefaults
app.kubernetes.io/component: poddefaults
app.kubernetes.io/name: poddefaults
patchesStrategicMerge:
- mutating-webhook-configuration.yaml
- deployment.yaml
generatorOptions:
disableNameSuffixHash: true
vars:
# These vars are used to substitute in the namespace, service name and
# deployment name into the mutating WebHookConfiguration.
# Since its a CR kustomize isn't aware of those fields and won't
# transform them.
# We need the var names to be relatively unique so that when we
# compose with other applications they won't conflict.
- name: podDefaultsCertName
objref:
kind: Certificate
group: cert-manager.io
version: v1
name: cert
fieldref:
fieldpath: metadata.name
configurations:
- params.yaml
@@ -0,0 +1,7 @@
apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
name: mutating-webhook-configuration
annotations:
cert-manager.io/inject-ca-from: $(podDefaultsNamespace)/$(podDefaultsCertName)
@@ -0,0 +1,16 @@
varReference:
- path: spec/commonName
kind: Certificate
- path: spec/dnsNames
kind: Certificate
- path: spec/issuerRef/name
kind: Certificate
- path: metadata/annotations
kind: MutatingWebhookConfiguration
nameReference:
- kind: Issuer
group: cert-manager.io
fieldSpecs:
- kind: Certificate
group: cert-manager.io
path: spec/issuerRef/name
@@ -0,0 +1,14 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app: centraldashboard
name: centraldashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: centraldashboard
subjects:
- kind: ServiceAccount
name: centraldashboard
namespace: kubeflow
@@ -0,0 +1,17 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app: centraldashboard
name: centraldashboard
rules:
- apiGroups:
- ""
resources:
- events
- namespaces
- nodes
verbs:
- get
- list
- watch
@@ -0,0 +1,120 @@
apiVersion: v1
data:
settings: |-
{
"DASHBOARD_FORCE_IFRAME": true
}
links: |-
{
"menuLinks": [
{
"icon": "book",
"link": "/jupyter/",
"text": "Notebooks",
"type": "item"
},
{
"icon": "assessment",
"link": "/tensorboards/",
"text": "TensorBoards",
"type": "item"
},
{
"icon": "device:storage",
"link": "/volumes/",
"text": "Volumes",
"type": "item"
},
{
"icon": "kubeflow:katib",
"link": "/katib/",
"text": "Katib Experiments",
"type": "item"
},
{
"icon": "kubeflow:pipeline-centered",
"items": [
{
"link": "/pipeline/#/pipelines",
"text": "Pipelines",
"type": "item"
},
{
"link": "/pipeline/#/experiments",
"text": "Experiments",
"type": "item"
},
{
"link": "/pipeline/#/runs",
"text": "Runs",
"type": "item"
},
{
"link": "/pipeline/#/recurringruns",
"text": "Recurring Runs",
"type": "item"
},
{
"link": "/pipeline/#/artifacts",
"text": "Artifacts",
"type": "item"
},
{
"link": "/pipeline/#/executions",
"text": "Executions",
"type": "item"
}
],
"text": "Pipelines",
"type": "section"
}
],
"externalLinks": [],
"documentationItems": [
{
"desc": "The Kubeflow website",
"link": "https://www.kubeflow.org/",
"text": "Kubeflow Website"
},
{
"desc": "Documentation for Kubeflow Pipelines",
"link": "https://www.kubeflow.org/docs/components/pipelines/",
"text": "Kubeflow Pipelines Documentation"
},
{
"desc": "Documentation for Kubeflow Notebooks",
"link": "https://www.kubeflow.org/docs/components/notebooks/",
"text": "Kubeflow Notebooks Documentation"
},
{
"desc": "Documentation for Kubeflow Training Operator",
"link": "https://www.kubeflow.org/docs/components/training/",
"text": "Kubeflow Training Operator Documentation"
},
{
"desc": "Documentation for Katib",
"link": "https://www.kubeflow.org/docs/components/katib/",
"text": "Katib Documentation"
}
],
"quickLinks": [
{
"desc": "Kubeflow Notebooks",
"link": "/jupyter/new",
"text": "Create a new Notebook"
},
{
"desc": "Kubeflow Pipelines",
"link": "/pipeline/#/pipelines",
"text": "Upload a Pipeline"
},
{
"desc": "Pipelines",
"link": "/pipeline/#/runs",
"text": "View Pipeline Runs"
}
]
}
kind: ConfigMap
metadata:
name: centraldashboard-config
@@ -0,0 +1,49 @@
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: centraldashboard
name: centraldashboard
spec:
replicas: 1
selector:
matchLabels:
app: centraldashboard
template:
metadata:
labels:
app: centraldashboard
annotations:
sidecar.istio.io/inject: "true"
spec:
containers:
- name: centraldashboard
image: docker.io/kubeflownotebookswg/centraldashboard
imagePullPolicy: IfNotPresent
livenessProbe:
httpGet:
path: /healthz
port: 8082
initialDelaySeconds: 30
periodSeconds: 30
ports:
- containerPort: 8082
protocol: TCP
env:
- name: USERID_HEADER
value: CD_USERID_HEADER_PLACEHOLDER
- name: USERID_PREFIX
value: CD_USERID_PREFIX_PLACEHOLDER
- name: PROFILES_KFAM_SERVICE_HOST
value: profiles-kfam.kubeflow
- name: REGISTRATION_FLOW
value: CD_REGISTRATION_FLOW_PLACEHOLDER
- name: DASHBOARD_CONFIGMAP
value: CD_CONFIGMAP_NAME_PLACEHOLDER
- name: LOGOUT_URL
value: '/authservice/logout'
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
serviceAccountName: centraldashboard
@@ -0,0 +1,83 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: kubeflow
resources:
- clusterrole-binding.yaml
- clusterrole.yaml
- deployment.yaml
- role-binding.yaml
- role.yaml
- service-account.yaml
- service.yaml
- configmap.yaml
images:
- name: docker.io/kubeflownotebookswg/centraldashboard
newName: docker.io/kubeflownotebookswg/centraldashboard
newTag: v1.9.0
configMapGenerator:
- envs:
- params.env
name: centraldashboard-parameters
generatorOptions:
disableNameSuffixHash: true
labels:
- includeSelectors: true
pairs:
app: centraldashboard
app.kubernetes.io/component: centraldashboard
app.kubernetes.io/name: centraldashboard
kustomize.component: centraldashboard
replacements:
- source:
fieldPath: data.CD_USERID_HEADER
kind: ConfigMap
name: centraldashboard-parameters
version: v1
targets:
- fieldPaths:
- spec.template.spec.containers.0.env.0.value
select:
group: apps
kind: Deployment
name: centraldashboard
version: v1
- source:
fieldPath: data.CD_USERID_PREFIX
kind: ConfigMap
name: centraldashboard-parameters
version: v1
targets:
- fieldPaths:
- spec.template.spec.containers.0.env.1.value
select:
group: apps
kind: Deployment
name: centraldashboard
version: v1
- source:
fieldPath: data.CD_REGISTRATION_FLOW
kind: ConfigMap
name: centraldashboard-parameters
version: v1
targets:
- fieldPaths:
- spec.template.spec.containers.0.env.3.value
select:
group: apps
kind: Deployment
name: centraldashboard
version: v1
- source:
fieldPath: metadata.name
kind: ConfigMap
name: centraldashboard-config
version: v1
targets:
- fieldPaths:
- spec.template.spec.containers.0.env.4.value
select:
group: apps
kind: Deployment
name: centraldashboard
version: v1
@@ -0,0 +1,4 @@
CD_CLUSTER_DOMAIN=cluster.local
CD_USERID_HEADER=kubeflow-userid
CD_USERID_PREFIX=
CD_REGISTRATION_FLOW=false
@@ -0,0 +1,14 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app: centraldashboard
name: centraldashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: centraldashboard
subjects:
- kind: ServiceAccount
name: centraldashboard
namespace: kubeflow
@@ -0,0 +1,26 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app: centraldashboard
name: centraldashboard
rules:
- apiGroups:
- ""
- "app.k8s.io"
resources:
- applications
- pods
- pods/exec
- pods/log
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- secrets
- configmaps
verbs:
- get
@@ -0,0 +1,4 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: centraldashboard
@@ -0,0 +1,15 @@
apiVersion: v1
kind: Service
metadata:
labels:
app: centraldashboard
name: centraldashboard
spec:
ports:
- port: 80
protocol: TCP
targetPort: 8082
selector:
app: centraldashboard
sessionAffinity: None
type: ClusterIP
@@ -0,0 +1,14 @@
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: central-dashboard
spec:
action: ALLOW
rules:
- from:
- source:
principals:
- cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account
selector:
matchLabels:
app: centraldashboard
@@ -0,0 +1,49 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
- virtual-service.yaml
- authorizationpolicy.yaml
namespace: kubeflow
replacements:
- source:
fieldPath: metadata.namespace
kind: Service
name: centraldashboard
version: v1
targets:
- fieldPaths:
- spec.http.0.route.0.destination.host
options:
delimiter: .
index: 1
select:
group: networking.istio.io
kind: VirtualService
name: centraldashboard
version: v1alpha3
- source:
fieldPath: data.CD_CLUSTER_DOMAIN
kind: ConfigMap
name: centraldashboard-parameters
version: v1
targets:
- fieldPaths:
- spec.http.0.route.0.destination.host
options:
delimiter: .
index: 3
select:
group: networking.istio.io
kind: VirtualService
name: centraldashboard
version: v1alpha3
configurations:
- params.yaml
labels:
- includeSelectors: true
pairs:
app: centraldashboard
app.kubernetes.io/component: centraldashboard
app.kubernetes.io/name: centraldashboard
kustomize.component: centraldashboard
@@ -0,0 +1,3 @@
varReference:
- path: spec/http/route/destination/host
kind: VirtualService
@@ -0,0 +1,20 @@
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: centraldashboard
spec:
gateways:
- kubeflow-gateway
hosts:
- '*'
http:
- match:
- uri:
prefix: /
rewrite:
uri: /
route:
- destination:
host: centraldashboard.CD_NAMESPACE_PLACEHOLDER.svc.CD_CLUSTER_DOMAIN_PLACEHOLDER
port:
number: 80
@@ -0,0 +1,13 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../istio
labels:
- includeSelectors: true
pairs:
app: centraldashboard
app.kubernetes.io/component: centraldashboard
app.kubernetes.io/name: centraldashboard
kustomize.component: centraldashboard
patches:
- path: patches/configmap.yaml
@@ -0,0 +1,126 @@
apiVersion: v1
data:
settings: |-
{
"DASHBOARD_FORCE_IFRAME": true
}
links: |-
{
"menuLinks": [
{
"icon": "book",
"link": "/jupyter/",
"text": "Notebooks",
"type": "item"
},
{
"icon": "assessment",
"link": "/tensorboards/",
"text": "TensorBoards",
"type": "item"
},
{
"icon": "device:storage",
"link": "/volumes/",
"text": "Volumes",
"type": "item"
},
{
"icon": "kubeflow:katib",
"link": "/katib/",
"text": "Katib Experiments",
"type": "item"
},
{
"type": "item",
"link": "/kserve-endpoints/",
"text": "KServe Endpoints",
"icon": "kubeflow:models"
},
{
"icon": "kubeflow:pipeline-centered",
"items": [
{
"link": "/pipeline/#/pipelines",
"text": "Pipelines",
"type": "item"
},
{
"link": "/pipeline/#/experiments",
"text": "Experiments",
"type": "item"
},
{
"link": "/pipeline/#/runs",
"text": "Runs",
"type": "item"
},
{
"link": "/pipeline/#/recurringruns",
"text": "Recurring Runs",
"type": "item"
},
{
"link": "/pipeline/#/artifacts",
"text": "Artifacts",
"type": "item"
},
{
"link": "/pipeline/#/executions",
"text": "Executions",
"type": "item"
}
],
"text": "Pipelines",
"type": "section"
}
],
"externalLinks": [],
"documentationItems": [
{
"desc": "The Kubeflow website",
"link": "https://www.kubeflow.org/",
"text": "Kubeflow Website"
},
{
"desc": "Documentation for Kubeflow Pipelines",
"link": "https://www.kubeflow.org/docs/components/pipelines/",
"text": "Kubeflow Pipelines Documentation"
},
{
"desc": "Documentation for Kubeflow Notebooks",
"link": "https://www.kubeflow.org/docs/components/notebooks/",
"text": "Kubeflow Notebooks Documentation"
},
{
"desc": "Documentation for Kubeflow Training Operator",
"link": "https://www.kubeflow.org/docs/components/training/",
"text": "Kubeflow Training Operator Documentation"
},
{
"desc": "Documentation for Katib",
"link": "https://www.kubeflow.org/docs/components/katib/",
"text": "Katib Documentation"
}
],
"quickLinks": [
{
"desc": "Kubeflow Notebooks",
"link": "/jupyter/new",
"text": "Create a new Notebook"
},
{
"desc": "Kubeflow Pipelines",
"link": "/pipeline/#/pipelines",
"text": "Upload a Pipeline"
},
{
"desc": "Pipelines",
"link": "/pipeline/#/runs",
"text": "View Pipeline Runs"
}
]
}
kind: ConfigMap
metadata:
name: centraldashboard-config
@@ -0,0 +1,9 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
# Using kserve overlay because it's also used in example installation.
- ../kserve
components:
- ../../../../../common/oidc-client/oauth2-proxy/components/central-dashboard
@@ -0,0 +1,11 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cluster-role-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-role
subjects:
- kind: ServiceAccount
name: service-account
@@ -0,0 +1,114 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cluster-role
rules:
- apiGroups:
- authorization.k8s.io
resources:
- subjectaccessreviews
verbs:
- create
- apiGroups:
- kubeflow.org
resources:
- notebooks
- notebooks/finalizers
- poddefaults
verbs:
- get
- list
- create
- delete
- patch
- update
- apiGroups:
- ""
resources:
- persistentvolumeclaims
verbs:
- create
- delete
- get
- list
- apiGroups:
- ""
resources:
- events
- nodes
verbs:
- list
- apiGroups:
- storage.k8s.io
resources:
- storageclasses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- pods
- pods/log
verbs:
- list
- get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kubeflow-notebook-ui-admin
labels:
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-admin: "true"
rules: []
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kubeflow-notebook-ui-edit
labels:
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-edit: "true"
rules:
- apiGroups:
- kubeflow.org
resources:
- notebooks
- notebooks/finalizers
- poddefaults
verbs:
- get
- list
- create
- delete
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kubeflow-notebook-ui-view
labels:
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-view: "true"
rules:
- apiGroups:
- kubeflow.org
resources:
- notebooks
- notebooks/finalizers
- poddefaults
verbs:
- get
- list
- apiGroups:
- storage.k8s.io
resources:
- storageclasses
verbs:
- get
- list
- watch
@@ -0,0 +1,191 @@
apiVersion: v1
data:
jupyter-icon.svg: |
<svg width="44" height="51" viewBox="0 0 44 51" version="2.0" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:figma="http://www.figma.com/figma/ns">
<desc>Created using Figma 0.90</desc>
<g id="Canvas" transform="translate(-1640 -2453)" figma:type="canvas">
<g id="Group" style="mix-blend-mode:normal;" figma:type="group">
<g id="Group" style="mix-blend-mode:normal;" figma:type="group">
<g id="Group" style="mix-blend-mode:normal;" figma:type="group">
<g id="g" style="mix-blend-mode:normal;" figma:type="group">
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
<g id="path9 fill" style="mix-blend-mode:normal;" figma:type="vector">
<use xlink:href="#path0_fill" transform="translate(1640.54 2474.36)" fill="#4E4E4E" style="mix-blend-mode:normal;"/>
</g>
</g>
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
<g id="path10 fill" style="mix-blend-mode:normal;" figma:type="vector">
<use xlink:href="#path1_fill" transform="translate(1645.68 2474.37)" fill="#4E4E4E" style="mix-blend-mode:normal;"/>
</g>
</g>
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
<g id="path11 fill" style="mix-blend-mode:normal;" figma:type="vector">
<use xlink:href="#path2_fill" transform="translate(1653.39 2474.26)" fill="#4E4E4E" style="mix-blend-mode:normal;"/>
</g>
</g>
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
<g id="path12 fill" style="mix-blend-mode:normal;" figma:type="vector">
<use xlink:href="#path3_fill" transform="translate(1660.43 2474.39)" fill="#4E4E4E" style="mix-blend-mode:normal;"/>
</g>
</g>
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
<g id="path13 fill" style="mix-blend-mode:normal;" figma:type="vector">
<use xlink:href="#path4_fill" transform="translate(1667.55 2472.54)" fill="#4E4E4E" style="mix-blend-mode:normal;"/>
</g>
</g>
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
<g id="path14 fill" style="mix-blend-mode:normal;" figma:type="vector">
<use xlink:href="#path5_fill" transform="translate(1672.47 2474.29)" fill="#4E4E4E" style="mix-blend-mode:normal;"/>
</g>
</g>
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
<g id="path15 fill" style="mix-blend-mode:normal;" figma:type="vector">
<use xlink:href="#path6_fill" transform="translate(1679.98 2474.24)" fill="#4E4E4E" style="mix-blend-mode:normal;"/>
</g>
</g>
</g>
</g>
<g id="g" style="mix-blend-mode:normal;" figma:type="group">
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
<g id="path16 fill" style="mix-blend-mode:normal;" figma:type="vector">
<use xlink:href="#path7_fill" transform="translate(1673.48 2453.69)" fill="#767677" style="mix-blend-mode:normal;"/>
</g>
</g>
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
<g id="path17 fill" style="mix-blend-mode:normal;" figma:type="vector">
<use xlink:href="#path8_fill" transform="translate(1643.21 2484.27)" fill="#F37726" style="mix-blend-mode:normal;"/>
</g>
</g>
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
<g id="path18 fill" style="mix-blend-mode:normal;" figma:type="vector">
<use xlink:href="#path9_fill" transform="translate(1643.21 2457.88)" fill="#F37726" style="mix-blend-mode:normal;"/>
</g>
</g>
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
<g id="path19 fill" style="mix-blend-mode:normal;" figma:type="vector">
<use xlink:href="#path10_fill" transform="translate(1643.28 2496.09)" fill="#9E9E9E" style="mix-blend-mode:normal;"/>
</g>
</g>
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
<g id="path20 fill" style="mix-blend-mode:normal;" figma:type="vector">
<use xlink:href="#path11_fill" transform="translate(1641.87 2458.43)" fill="#616262" style="mix-blend-mode:normal;"/>
</g>
</g>
</g>
</g>
</g>
</g>
<defs>
<path id="path0_fill" d="M 1.74498 5.47533C 1.74498 7.03335 1.62034 7.54082 1.29983 7.91474C 0.943119 8.23595 0.480024 8.41358 0 8.41331L 0.124642 9.3036C 0.86884 9.31366 1.59095 9.05078 2.15452 8.56466C 2.45775 8.19487 2.6834 7.76781 2.818 7.30893C 2.95261 6.85005 2.99341 6.36876 2.93798 5.89377L 2.93798 0L 1.74498 0L 1.74498 5.43972L 1.74498 5.47533Z"/>
<path id="path1_fill" d="M 5.50204 4.76309C 5.50204 5.43081 5.50204 6.02731 5.55545 6.54368L 4.496 6.54368L 4.42478 5.48423C 4.20318 5.85909 3.88627 6.16858 3.50628 6.38125C 3.12628 6.59392 2.69675 6.70219 2.26135 6.69503C 1.22861 6.69503 0 6.13415 0 3.84608L 0 0.0445149L 1.193 0.0445149L 1.193 3.6057C 1.193 4.84322 1.57583 5.67119 2.65309 5.67119C 2.87472 5.67358 3.09459 5.63168 3.29982 5.54796C 3.50505 5.46424 3.69149 5.34039 3.84822 5.18366C 4.00494 5.02694 4.1288 4.84049 4.21252 4.63527C 4.29623 4.43004 4.33813 4.21016 4.33575 3.98853L 4.33575 0L 5.52874 0L 5.52874 4.72748L 5.50204 4.76309Z"/>
<path id="path2_fill" d="M 0.0534178 2.27264C 0.0534178 1.44466 0.0534178 0.768036 0 0.153731L 1.06836 0.153731L 1.12177 1.2666C 1.3598 0.864535 1.70247 0.534594 2.11325 0.311954C 2.52404 0.0893145 2.98754 -0.0176786 3.45435 0.00238095C 5.03908 0.00238095 6.23208 1.32892 6.23208 3.30538C 6.23208 5.63796 4.7987 6.79535 3.24958 6.79535C 2.85309 6.81304 2.45874 6.7281 2.10469 6.54874C 1.75064 6.36937 1.44888 6.10166 1.22861 5.77151L 1.22861 5.77151L 1.22861 9.33269L 0.0534178 9.33269L 0.0534178 2.29935L 0.0534178 2.27264ZM 1.22861 4.00872C 1.23184 4.17026 1.24972 4.33117 1.28203 4.48948C 1.38304 4.88479 1.61299 5.23513 1.93548 5.48506C 2.25798 5.735 2.65461 5.87026 3.06262 5.86944C 4.31794 5.86944 5.05689 4.8456 5.05689 3.3588C 5.05689 2.05897 4.36246 0.946096 3.10714 0.946096C 2.61036 0.986777 2.14548 1.20726 1.79965 1.5662C 1.45382 1.92514 1.25079 2.3979 1.22861 2.89585L 1.22861 4.00872Z"/>
<path id="path3_fill" d="M 1.31764 0.0178059L 2.75102 3.85499C 2.90237 4.28233 3.06262 4.7987 3.16946 5.18153C 3.2941 4.7898 3.42764 4.29123 3.5879 3.82828L 4.88773 0.0178059L 6.14305 0.0178059L 4.36246 4.64735C 3.47216 6.87309 2.92908 8.02158 2.11 8.71601C 1.69745 9.09283 1.19448 9.35658 0.649917 9.48166L 0.356119 8.48453C 0.736886 8.35942 1.09038 8.16304 1.39777 7.90584C 1.8321 7.55188 2.17678 7.10044 2.4038 6.5882C 2.45239 6.49949 2.48551 6.40314 2.50173 6.3033C 2.49161 6.19586 2.46457 6.0907 2.42161 5.9917L 0 0L 1.29983 0L 1.31764 0.0178059Z"/>
<path id="path4_fill" d="M 2.19013 0L 2.19013 1.86962L 3.8995 1.86962L 3.8995 2.75992L 2.19013 2.75992L 2.19013 6.26769C 2.19013 7.06896 2.42161 7.53191 3.08043 7.53191C 3.31442 7.53574 3.54789 7.5088 3.77486 7.45179L 3.82828 8.34208C 3.48794 8.45999 3.12881 8.51431 2.76882 8.50234C 2.53042 8.51726 2.29161 8.48043 2.06878 8.39437C 1.84595 8.30831 1.64438 8.17506 1.47789 8.00377C 1.11525 7.51873 0.949826 6.91431 1.01494 6.31221L 1.01494 2.75102L 0 2.75102L 0 1.86072L 1.03274 1.86072L 1.03274 0.275992L 2.19013 0Z"/>
<path id="path5_fill" d="M 1.17716 3.57899C 1.153 3.88093 1.19468 4.18451 1.29933 4.46876C 1.40398 4.75301 1.5691 5.01114 1.78329 5.22532C 1.99747 5.43951 2.2556 5.60463 2.53985 5.70928C 2.8241 5.81393 3.12768 5.85561 3.42962 5.83145C 4.04033 5.84511 4.64706 5.72983 5.21021 5.49313L 5.41498 6.38343C 4.72393 6.66809 3.98085 6.80458 3.23375 6.78406C 2.79821 6.81388 2.36138 6.74914 1.95322 6.59427C 1.54505 6.43941 1.17522 6.19809 0.869071 5.88688C 0.562928 5.57566 0.327723 5.2019 0.179591 4.79125C 0.0314584 4.38059 -0.0260962 3.94276 0.0108748 3.50777C 0.0108748 1.54912 1.17716 0 3.0824 0C 5.21911 0 5.75329 1.86962 5.75329 3.06262C 5.76471 3.24644 5.76471 3.43079 5.75329 3.61461L 1.15046 3.61461L 1.17716 3.57899ZM 4.66713 2.6887C 4.70149 2.45067 4.68443 2.20805 4.61709 1.97718C 4.54976 1.74631 4.43372 1.53255 4.2768 1.35031C 4.11987 1.16808 3.92571 1.0216 3.70739 0.920744C 3.48907 0.81989 3.25166 0.767006 3.01118 0.765656C 2.52201 0.801064 2.06371 1.01788 1.72609 1.37362C 1.38847 1.72935 1.19588 2.19835 1.18607 2.6887L 4.66713 2.6887Z"/>
<path id="path6_fill" d="M 0.0534178 2.19228C 0.0534178 1.42663 0.0534178 0.767806 0 0.162404L 1.06836 0.162404L 1.06836 1.43553L 1.12177 1.43553C 1.23391 1.04259 1.4656 0.694314 1.78468 0.439049C 2.10376 0.183783 2.4944 0.034196 2.90237 0.0110538C 3.01466 -0.00368459 3.12839 -0.00368459 3.24068 0.0110538L 3.24068 1.12393C 3.10462 1.10817 2.9672 1.10817 2.83114 1.12393C 2.427 1.13958 2.04237 1.30182 1.7491 1.58035C 1.45583 1.85887 1.27398 2.23462 1.23751 2.63743C 1.20422 2.8196 1.18635 3.00425 1.1841 3.18941L 1.1841 6.65267L 0.00890297 6.65267L 0.00890297 2.20118L 0.0534178 2.19228Z"/>
<path id="path7_fill" d="M 6.03059 2.83565C 6.06715 3.43376 5.92485 4.02921 5.6218 4.54615C 5.31875 5.0631 4.86869 5.47813 4.32893 5.73839C 3.78917 5.99864 3.18416 6.09233 2.59097 6.00753C 1.99778 5.92272 1.44326 5.66326 0.998048 5.26219C 0.552837 4.86113 0.23709 4.33661 0.0910307 3.75546C -0.0550287 3.17431 -0.0247891 2.56283 0.177897 1.99893C 0.380583 1.43503 0.746541 0.944221 1.22915 0.589037C 1.71176 0.233853 2.28918 0.0303686 2.88784 0.00450543C 3.28035 -0.0170932 3.67326 0.0391144 4.04396 0.169896C 4.41467 0.300677 4.75587 0.503453 5.04794 0.766561C 5.34 1.02967 5.57718 1.34792 5.74582 1.70301C 5.91446 2.0581 6.01124 2.44303 6.03059 2.83565L 6.03059 2.83565Z"/>
<path id="path8_fill" d="M 18.6962 7.12238C 10.6836 7.12238 3.64131 4.24672 0 0C 1.41284 3.82041 3.96215 7.1163 7.30479 9.44404C 10.6474 11.7718 14.623 13.0196 18.6962 13.0196C 22.7695 13.0196 26.745 11.7718 30.0877 9.44404C 33.4303 7.1163 35.9796 3.82041 37.3925 4.0486e-13C 33.7601 4.24672 26.7445 7.12238 18.6962 7.12238Z"/>
<path id="path9_fill" d="M 18.6962 5.89725C 26.7089 5.89725 33.7512 8.77291 37.3925 13.0196C 35.9796 9.19922 33.4303 5.90333 30.0877 3.57559C 26.745 1.24785 22.7695 4.0486e-13 18.6962 0C 14.623 4.0486e-13 10.6474 1.24785 7.30479 3.57559C 3.96215 5.90333 1.41284 9.19922 0 13.0196C 3.64131 8.76401 10.648 5.89725 18.6962 5.89725Z"/>
<path id="path10_fill" d="M 7.59576 3.56656C 7.64276 4.31992 7.46442 5.07022 7.08347 5.72186C 6.70251 6.3735 6.13619 6.89698 5.45666 7.22561C 4.77713 7.55424 4.01515 7.67314 3.26781 7.56716C 2.52046 7.46117 1.82158 7.13511 1.26021 6.63051C 0.698839 6.12591 0.300394 5.46561 0.115637 4.73375C -0.0691191 4.00188 -0.0318219 3.23159 0.222777 2.52099C 0.477376 1.8104 0.93775 1.19169 1.54524 0.743685C 2.15274 0.295678 2.87985 0.0386595 3.63394 0.00537589C 4.12793 -0.0210471 4.62229 0.0501173 5.08878 0.214803C 5.55526 0.37949 5.98473 0.63447 6.35264 0.965179C 6.72055 1.29589 7.01971 1.69584 7.233 2.1422C 7.4463 2.58855 7.56957 3.07256 7.59576 3.56656L 7.59576 3.56656Z"/>
<path id="path11_fill" d="M 2.25061 4.37943C 1.81886 4.39135 1.39322 4.27535 1.02722 4.04602C 0.661224 3.81668 0.371206 3.48424 0.193641 3.09052C 0.0160762 2.69679 -0.0411078 2.25935 0.0292804 1.83321C 0.0996686 1.40707 0.294486 1.01125 0.589233 0.695542C 0.883981 0.37983 1.2655 0.158316 1.68581 0.0588577C 2.10611 -0.0406005 2.54644 -0.0135622 2.95143 0.136572C 3.35641 0.286707 3.70796 0.553234 3.96186 0.902636C 4.21577 1.25204 4.3607 1.66872 4.37842 2.10027C 4.39529 2.6838 4.18131 3.25044 3.78293 3.67715C 3.38455 4.10387 2.83392 4.35623 2.25061 4.37943Z"/>
</defs>
</svg>
jupyterlab-logo.svg: |
<svg xmlns="http://www.w3.org/2000/svg" width="200" viewBox="0 0 1860.8 475">
<g class="jp-icon2" fill="#4E4E4E" transform="translate(480.136401, 64.271493)">
<g transform="translate(0.000000, 58.875566)">
<g transform="translate(0.087603, 0.140294)">
<path d="M-426.9,169.8c0,48.7-3.7,64.7-13.6,76.4c-10.8,10-25,15.5-39.7,15.5l3.7,29 c22.8,0.3,44.8-7.9,61.9-23.1c17.8-18.5,24-44.1,24-83.3V0H-427v170.1L-426.9,169.8L-426.9,169.8z"/>
</g>
</g>
<g transform="translate(155.045296, 56.837104)">
<g transform="translate(1.562453, 1.799842)">
<path d="M-312,148c0,21,0,39.5,1.7,55.4h-31.8l-2.1-33.3h-0.8c-6.7,11.6-16.4,21.3-28,27.9 c-11.6,6.6-24.8,10-38.2,9.8c-31.4,0-69-17.7-69-89V0h36.4v112.7c0,38.7,11.6,64.7,44.6,64.7c10.3-0.2,20.4-3.5,28.9-9.4 c8.5-5.9,15.1-14.3,18.9-23.9c2.2-6.1,3.3-12.5,3.3-18.9V0.2h36.4V148H-312L-312,148z"/>
</g>
</g>
<g transform="translate(390.013322, 53.479638)">
<g transform="translate(1.706458, 0.231425)">
<path d="M-478.6,71.4c0-26-0.8-47-1.7-66.7h32.7l1.7,34.8h0.8c7.1-12.5,17.5-22.8,30.1-29.7 c12.5-7,26.7-10.3,41-9.8c48.3,0,84.7,41.7,84.7,103.3c0,73.1-43.7,109.2-91,109.2c-12.1,0.5-24.2-2.2-35-7.8 c-10.8-5.6-19.9-13.9-26.6-24.2h-0.8V291h-36v-220L-478.6,71.4L-478.6,71.4z M-442.6,125.6c0.1,5.1,0.6,10.1,1.7,15.1 c3,12.3,9.9,23.3,19.8,31.1c9.9,7.8,22.1,12.1,34.7,12.1c38.5,0,60.7-31.9,60.7-78.5c0-40.7-21.1-75.6-59.5-75.6 c-12.9,0.4-25.3,5.1-35.3,13.4c-9.9,8.3-16.9,19.7-19.6,32.4c-1.5,4.9-2.3,10-2.5,15.1V125.6L-442.6,125.6L-442.6,125.6z"/>
</g>
</g>
<g transform="translate(606.740726, 56.837104)">
<g transform="translate(0.751226, 1.989299)">
<path d="M-440.8,0l43.7,120.1c4.5,13.4,9.5,29.4,12.8,41.7h0.8c3.7-12.2,7.9-27.7,12.8-42.4 l39.7-119.2h38.5L-346.9,145c-26,69.7-43.7,105.4-68.6,127.2c-12.5,11.7-27.9,20-44.6,23.9l-9.1-31.1 c11.7-3.9,22.5-10.1,31.8-18.1c13.2-11.1,23.7-25.2,30.6-41.2c1.5-2.8,2.5-5.7,2.9-8.8c-0.3-3.3-1.2-6.6-2.5-9.7L-480.2,0.1 h39.7L-440.8,0L-440.8,0z"/>
</g>
</g>
<g transform="translate(822.748104, 0.000000)">
<g transform="translate(1.464050, 0.378914)">
<path d="M-413.7,0v58.3h52v28.2h-52V196c0,25,7,39.5,27.3,39.5c7.1,0.1,14.2-0.7,21.1-2.5 l1.7,27.7c-10.3,3.7-21.3,5.4-32.2,5c-7.3,0.4-14.6-0.7-21.3-3.4c-6.8-2.7-12.9-6.8-17.9-12.1c-10.3-10.9-14.1-29-14.1-52.9 V86.5h-31V58.3h31V9.6L-413.7,0L-413.7,0z"/>
</g>
</g>
<g transform="translate(974.433286, 53.479638)">
<g transform="translate(0.990034, 0.610339)">
<path d="M-445.8,113c0.8,50,32.2,70.6,68.6,70.6c19,0.6,37.9-3,55.3-10.5l6.2,26.4 c-20.9,8.9-43.5,13.1-66.2,12.6c-61.5,0-98.3-41.2-98.3-102.5C-480.2,48.2-444.7,0-386.5,0c65.2,0,82.7,58.3,82.7,95.7 c-0.1,5.8-0.5,11.5-1.2,17.2h-140.6H-445.8L-445.8,113z M-339.2,86.6c0.4-23.5-9.5-60.1-50.4-60.1 c-36.8,0-52.8,34.4-55.7,60.1H-339.2L-339.2,86.6L-339.2,86.6z"/>
</g>
</g>
<g transform="translate(1201.961058, 53.479638)">
<g transform="translate(1.179640, 0.705068)">
<path d="M-478.6,68c0-23.9-0.4-44.5-1.7-63.4h31.8l1.2,39.9h1.7c9.1-27.3,31-44.5,55.3-44.5 c3.5-0.1,7,0.4,10.3,1.2v34.8c-4.1-0.9-8.2-1.3-12.4-1.2c-25.6,0-43.7,19.7-48.7,47.4c-1,5.7-1.6,11.5-1.7,17.2v108.3h-36V68 L-478.6,68z"/>
</g>
</g>
</g>
<g class="jp-icon-warn0" fill="#F37726">
<path d="M1352.3,326.2h37V28h-37V326.2z M1604.8,326.2c-2.5-13.9-3.4-31.1-3.4-48.7v-76 c0-40.7-15.1-83.1-77.3-83.1c-25.6,0-50,7.1-66.8,18.1l8.4,24.4c14.3-9.2,34-15.1,53-15.1c41.6,0,46.2,30.2,46.2,47v4.2 c-78.6-0.4-122.3,26.5-122.3,75.6c0,29.4,21,58.4,62.2,58.4c29,0,50.9-14.3,62.2-30.2h1.3l2.9,25.6H1604.8z M1565.7,257.7 c0,3.8-0.8,8-2.1,11.8c-5.9,17.2-22.7,34-49.2,34c-18.9,0-34.9-11.3-34.9-35.3c0-39.5,45.8-46.6,86.2-45.8V257.7z M1698.5,326.2 l1.7-33.6h1.3c15.1,26.9,38.7,38.2,68.1,38.2c45.4,0,91.2-36.1,91.2-108.8c0.4-61.7-35.3-103.7-85.7-103.7 c-32.8,0-56.3,14.7-69.3,37.4h-0.8V28h-36.6v245.7c0,18.1-0.8,38.6-1.7,52.5H1698.5z M1704.8,208.2c0-5.9,1.3-10.9,2.1-15.1 c7.6-28.1,31.1-45.4,56.3-45.4c39.5,0,60.5,34.9,60.5,75.6c0,46.6-23.1,78.1-61.8,78.1c-26.9,0-48.3-17.6-55.5-43.3 c-0.8-4.2-1.7-8.8-1.7-13.4V208.2z"/>
</g>
</svg>
group-two-icon.svg: |-
<?xml version="1.0" encoding="utf-8"?>
<!-- Generator: Adobe Illustrator 13.0.2, SVG Export Plug-In . SVG Version: 6.00 Build 14948) -->
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg version="1.1" id="Ebene_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
width="14.35px" height="42.88px" viewBox="0 0 14.35 42.88" enable-background="new 0 0 14.35 42.88" xml:space="preserve">
<g>
<path d="M12.583,28.057v2.027H1.228c-0.016-0.508,0.066-0.996,0.246-1.465c0.289-0.773,0.752-1.535,1.389-2.285
s1.557-1.617,2.76-2.602c1.867-1.531,3.129-2.744,3.785-3.639s0.984-1.74,0.984-2.537c0-0.836-0.299-1.541-0.896-2.115
s-1.377-0.861-2.338-0.861c-1.016,0-1.828,0.305-2.438,0.914s-0.918,1.453-0.926,2.531l-2.168-0.223
c0.148-1.617,0.707-2.85,1.676-3.697s2.27-1.271,3.902-1.271c1.648,0,2.953,0.457,3.914,1.371s1.441,2.047,1.441,3.398
c0,0.688-0.141,1.363-0.422,2.027s-0.748,1.363-1.4,2.098s-1.736,1.742-3.252,3.023c-1.266,1.063-2.078,1.783-2.438,2.162
s-0.656,0.76-0.891,1.143H12.583z"/>
</g>
</svg>
group-two-logo.svg: |-
<?xml version="1.0" encoding="utf-8"?>
<!-- Generator: Adobe Illustrator 13.0.2, SVG Export Plug-In . SVG Version: 6.00 Build 14948) -->
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg version="1.1" id="Ebene_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
width="14.35px" height="42.88px" viewBox="0 0 14.35 42.88" enable-background="new 0 0 14.35 42.88" xml:space="preserve">
<g>
<path d="M12.583,28.057v2.027H1.228c-0.016-0.508,0.066-0.996,0.246-1.465c0.289-0.773,0.752-1.535,1.389-2.285
s1.557-1.617,2.76-2.602c1.867-1.531,3.129-2.744,3.785-3.639s0.984-1.74,0.984-2.537c0-0.836-0.299-1.541-0.896-2.115
s-1.377-0.861-2.338-0.861c-1.016,0-1.828,0.305-2.438,0.914s-0.918,1.453-0.926,2.531l-2.168-0.223
c0.148-1.617,0.707-2.85,1.676-3.697s2.27-1.271,3.902-1.271c1.648,0,2.953,0.457,3.914,1.371s1.441,2.047,1.441,3.398
c0,0.688-0.141,1.363-0.422,2.027s-0.748,1.363-1.4,2.098s-1.736,1.742-3.252,3.023c-1.266,1.063-2.078,1.783-2.438,2.162
s-0.656,0.76-0.891,1.143H12.583z"/>
</g>
</svg>
group-one-icon.svg: |-
<?xml version="1.0" encoding="utf-8"?>
<!-- Generator: Adobe Illustrator 13.0.2, SVG Export Plug-In . SVG Version: 6.00 Build 14948) -->
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg version="1.1" id="Ebene_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
width="14.35px" height="42.88px" viewBox="0 0 14.35 42.88" enable-background="new 0 0 14.35 42.88" xml:space="preserve">
<g>
<path d="M9.442,30.084H7.333V16.643c-0.508,0.484-1.174,0.969-1.998,1.453s-1.564,0.848-2.221,1.09v-2.039
c1.18-0.555,2.211-1.227,3.094-2.016s1.508-1.555,1.875-2.297h1.359V30.084z"/>
</g>
</svg>
group-one-logo.svg: |-
<?xml version="1.0" encoding="utf-8"?>
<!-- Generator: Adobe Illustrator 13.0.2, SVG Export Plug-In . SVG Version: 6.00 Build 14948) -->
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg version="1.1" id="Ebene_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
width="14.35px" height="42.88px" viewBox="0 0 14.35 42.88" enable-background="new 0 0 14.35 42.88" xml:space="preserve">
<g>
<path d="M9.442,30.084H7.333V16.643c-0.508,0.484-1.174,0.969-1.998,1.453s-1.564,0.848-2.221,1.09v-2.039
c1.18-0.555,2.211-1.227,3.094-2.016s1.508-1.555,1.875-2.297h1.359V30.084z"/>
</g>
</svg>
kind: ConfigMap
metadata:
name: logos
@@ -0,0 +1,306 @@
# --------------------------------------------------------------
# Configuration file for the Kubeflow Notebooks UI.
#
# About the `readOnly` configs:
# - when `readOnly` is set to "true", the respective option
# will be disabled for users and only set by the admin
# - when 'readOnly' is missing, it defaults to 'false'
# --------------------------------------------------------------
spawnerFormDefaults:
################################################################
# Container Images
################################################################
# if users can input custom images, or only select from dropdowns
allowCustomImage: true
# if the registry of the container image is hidden from display
hideRegistry: true
# if the tag of the container image is hidden from display
hideTag: false
# configs for the ImagePullPolicy
imagePullPolicy:
readOnly: false
# the default ImagePullPolicy
# (possible values: "Always", "IfNotPresent", "Never")
value: IfNotPresent
################################################################
# Jupyter-like Container Images
#
# NOTES:
# - the `image` section is used for "Jupyter-like" apps whose
# HTTP path is configured by the "NB_PREFIX" environment variable
################################################################
image:
# the default container image
value: kubeflownotebookswg/jupyter-scipy:latest
# the list of available container images in the dropdown
options:
- kubeflownotebookswg/jupyter-scipy:latest
- kubeflownotebookswg/jupyter-pytorch-full:latest
- kubeflownotebookswg/jupyter-pytorch-cuda-full:latest
- kubeflownotebookswg/jupyter-tensorflow-full:latest
- kubeflownotebookswg/jupyter-tensorflow-cuda-full:latest
################################################################
# VSCode-like Container Images (Group 1)
#
# NOTES:
# - the `imageGroupOne` section is used for "VSCode-like" apps that
# expose themselves under the HTTP root path "/" and support path
# rewriting without breaking
# - the annotation `notebooks.kubeflow.org/http-rewrite-uri: "/"` is
# set on Notebooks spawned by this group, to make Istio rewrite
# the path of HTTP requests to the HTTP root
################################################################
imageGroupOne:
# the default container image
value: kubeflownotebookswg/codeserver-python:latest
# the list of available container images in the dropdown
options:
- kubeflownotebookswg/codeserver-python:latest
################################################################
# RStudio-like Container Images (Group 2)
#
# NOTES:
# - the `imageGroupTwo` section is used for "RStudio-like" apps whose
# HTTP path is configured by the "X-RStudio-Root-Path" header
# - the annotation `notebooks.kubeflow.org/http-rewrite-uri: "/"` is
# set on Notebooks spawned by this group, to make Istio rewrite
# the path of HTTP requests to the HTTP root
# - the annotation `notebooks.kubeflow.org/http-headers-request-set` is
# set on Notebooks spawned by this group, such that Istio injects the
# "X-RStudio-Root-Path" header to all request
################################################################
imageGroupTwo:
# the default container image
value: kubeflownotebookswg/rstudio-tidyverse:latest
# the list of available container images in the dropdown
options:
- kubeflownotebookswg/rstudio-tidyverse:latest
################################################################
# CPU Resources
################################################################
cpu:
readOnly: false
# the default cpu request for the container
value: "0.5"
# a factor by which to multiply the CPU request calculate the cpu limit
# (to disable cpu limits, set as "none")
limitFactor: "1.2"
################################################################
# Memory Resources
################################################################
memory:
readOnly: false
# the default memory request for the container
value: "1.0Gi"
# a factor by which to multiply the memory request calculate the memory limit
# (to disable memory limits, set as "none")
limitFactor: "1.2"
################################################################
# GPU/Device-Plugin Resources
################################################################
gpus:
readOnly: false
# configs for gpu/device-plugin limits of the container
# https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/#using-device-plugins
value:
# the `limitKey` of the default vendor
# (to have no default, set as "")
vendor: ""
# the list of available vendors in the dropdown
# `limitsKey` - what will be set as the actual limit
# `uiName` - what will be displayed in the dropdown UI
vendors:
- limitsKey: "nvidia.com/gpu"
uiName: "NVIDIA"
- limitsKey: "amd.com/gpu"
uiName: "AMD"
# the default value of the limit
# (possible values: "none", "1", "2", "4", "8")
num: "none"
################################################################
# Workspace Volumes
################################################################
workspaceVolume:
readOnly: false
# the default workspace volume to be created and mounted
# (to have no default, set `value: null`)
value:
mount: /home/jovyan
# pvc configs for creating new workspace volumes
# https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.24/#persistentvolumeclaim-v1-core
newPvc:
metadata:
# "{notebook-name}" is replaced with the Notebook name
name: "{notebook-name}-workspace"
spec:
#storageClassName: my-storage-class
resources:
requests:
storage: 5Gi
accessModes:
- ReadWriteOnce
################################################################
# Data Volumes
################################################################
dataVolumes:
readOnly: false
# a list of additional data volumes to be created and/or mounted
value: []
#value:
# - mount: /home/jovyan/datavol-1
# newPvc:
# metadata:
# name: "{notebook-name}-datavol-1"
# spec:
# resources:
# requests:
# storage: 5Gi
# accessModes:
# - ReadWriteOnce
#
# - mount: /home/jovyan/datavol-1
# existingSource:
# persistentVolumeClaim:
# claimName: "test-pvc"
################################################################
# Affinity
################################################################
affinityConfig:
readOnly: false
# the `configKey` of the default affinity config
# (to have no default, set as "")
# (if `readOnly`, the default `value` will be the only accessible option)
value: ""
# the list of available affinity configs in the dropdown
options: []
#options:
# - configKey: "dedicated_node_per_notebook"
# displayName: "Dedicated Node Per Notebook"
# affinity:
# # Require a Node with label `lifecycle=kubeflow-notebook`
# nodeAffinity:
# requiredDuringSchedulingIgnoredDuringExecution:
# nodeSelectorTerms:
# - matchExpressions:
# - key: "lifecycle"
# operator: "In"
# values:
# - "kubeflow-notebook"
#
# # Require a Node WITHOUT an existing Pod having `notebook-name` label
# podAntiAffinity:
# requiredDuringSchedulingIgnoredDuringExecution:
# - labelSelector:
# matchExpressions:
# - key: "notebook-name"
# operator: "Exists"
# topologyKey: "kubernetes.io/hostname"
# # WARNING: `namespaceSelector` is Beta in 1.22 and Stable in 1.24,
# # setting to {} is required for affinity to work across Namespaces
# namespaceSelector: {}
################################################################
# Tolerations
################################################################
tolerationGroup:
readOnly: false
# the `groupKey` of the default toleration group
# (to have no default, set as "")
# (if `readOnly`, the default `value` will be the only accessible option)
value: ""
# the list of available toleration groups in the dropdown
options: []
#options:
# - groupKey: "group_1"
# displayName: "4 CPU 8Gb Mem at ~$X.XXX USD per day"
# tolerations:
# - key: "dedicated"
# operator: "Equal"
# value: "kubeflow-c5.xlarge"
# effect: "NoSchedule"
#
# - groupKey: "group_2"
# displayName: "8 CPU 16Gb Mem at ~$X.XXX USD per day"
# tolerations:
# - key: "dedicated"
# operator: "Equal"
# value: "kubeflow-c5.2xlarge"
# effect: "NoSchedule"
#
# - groupKey: "group_3"
# displayName: "16 CPU 32Gb Mem at ~$X.XXX USD per day"
# tolerations:
# - key: "dedicated"
# operator: "Equal"
# value: "kubeflow-c5.4xlarge"
# effect: "NoSchedule"
#
# - groupKey: "group_4"
# displayName: "32 CPU 256Gb Mem at ~$X.XXX USD per day"
# tolerations:
# - key: "dedicated"
# operator: "Equal"
# value: "kubeflow-r5.8xlarge"
# effect: "NoSchedule"
################################################################
# Shared Memory
################################################################
shm:
readOnly: false
# the default state of the "Enable Shared Memory" toggle
value: true
################################################################
# PodDefaults
################################################################
configurations:
readOnly: false
# the list of PodDefault names that are selected by default
# (take care to ensure these PodDefaults exist in Profile Namespaces)
value: []
#value:
# - my-pod-default
################################################################
# Environment
#
# NOTE:
# - these configs are only used by the ROK "flavor" of the UI
################################################################
environment:
readOnly: false
value: {}
@@ -0,0 +1,37 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment
spec:
replicas: 1
template:
spec:
containers:
- name: jupyter-web-app
image: docker.io/kubeflownotebookswg/jupyter-web-app
ports:
- containerPort: 5000
volumeMounts:
- mountPath: /etc/config
name: config-volume
- mountPath: /src/apps/default/static/assets/logos
name: logos-volume
env:
- name: APP_PREFIX
value: $(JWA_PREFIX)
- name: UI
value: $(JWA_UI)
- name: USERID_HEADER
value: $(JWA_USERID_HEADER)
- name: USERID_PREFIX
value: $(JWA_USERID_PREFIX)
- name: APP_SECURE_COOKIES
value: $(JWA_APP_SECURE_COOKIES)
serviceAccountName: service-account
volumes:
- configMap:
name: config
name: config-volume
- configMap:
name: jupyter-web-app-logos
name: logos-volume
@@ -0,0 +1,85 @@
# TODO(https://github.com/kubeflow/manifests/issues/774):
# This is a legacy package. Hopefully we can get rid of it once
# 774 is complete.
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
# TODO(jlewi): We can't depend on base because of the deployment_patch.
# but maybe if we changed that to use ConfigMapRef then the patch would correctly
# override the patch applied in base_v3
resources:
- cluster-role-binding.yaml
- cluster-role.yaml
- deployment.yaml
- role-binding.yaml
- role.yaml
- service-account.yaml
- service.yaml
- configs/logos-configmap.yaml
namePrefix: jupyter-web-app-
namespace: kubeflow
commonLabels:
app: jupyter-web-app
kustomize.component: jupyter-web-app
images:
- name: docker.io/kubeflownotebookswg/jupyter-web-app
newName: docker.io/kubeflownotebookswg/jupyter-web-app
newTag: v1.9.0
# We need the name to be unique without the suffix because the original name is what
# gets used with patches
configMapGenerator:
- envs:
- params.env
name: parameters
- files:
- configs/spawner_ui_config.yaml
name: config
vars:
- fieldref:
fieldPath: data.JWA_CLUSTER_DOMAIN
name: JWA_CLUSTER_DOMAIN
objref:
apiVersion: v1
kind: ConfigMap
name: parameters
- fieldref:
fieldPath: metadata.namespace
name: JWA_NAMESPACE
objref:
apiVersion: v1
kind: Service
name: service
- fieldref:
fieldPath: data.JWA_USERID_HEADER
name: JWA_USERID_HEADER
objref:
apiVersion: v1
kind: ConfigMap
name: parameters
- fieldref:
fieldPath: data.JWA_USERID_PREFIX
name: JWA_USERID_PREFIX
objref:
apiVersion: v1
kind: ConfigMap
name: parameters
- fieldref:
fieldPath: data.JWA_UI
name: JWA_UI
objref:
apiVersion: v1
kind: ConfigMap
name: parameters
- fieldref:
fieldPath: data.JWA_PREFIX
name: JWA_PREFIX
objref:
apiVersion: v1
kind: ConfigMap
name: parameters
- name: JWA_APP_SECURE_COOKIES
fieldref:
fieldPath: data.JWA_APP_SECURE_COOKIES
objref:
apiVersion: v1
kind: ConfigMap
name: parameters
@@ -0,0 +1,6 @@
JWA_UI=default
JWA_PREFIX=/jupyter
JWA_CLUSTER_DOMAIN=cluster.local
JWA_USERID_HEADER=kubeflow-userid
JWA_USERID_PREFIX=
JWA_APP_SECURE_COOKIES=true
@@ -0,0 +1,11 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: jupyter-notebook-role-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: jupyter-notebook-role
subjects:
- kind: ServiceAccount
name: jupyter-notebook
@@ -0,0 +1,48 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: jupyter-notebook-role
rules:
- apiGroups:
- authorization.k8s.io
resources:
- subjectaccessreviews
verbs:
- create
- apiGroups:
- kubeflow.org
resources:
- notebooks
- notebooks/finalizers
- poddefaults
verbs:
- get
- list
- create
- delete
- patch
- update
- apiGroups:
- ""
resources:
- persistentvolumeclaims
verbs:
- create
- delete
- get
- list
- apiGroups:
- ""
resources:
- events
- nodes
verbs:
- list
- apiGroups:
- storage.k8s.io
resources:
- storageclasses
verbs:
- get
- list
- watch
@@ -0,0 +1,4 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: service-account
@@ -0,0 +1,13 @@
apiVersion: v1
kind: Service
metadata:
labels:
run: jupyter-web-app
name: service
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: 5000
type: ClusterIP
@@ -0,0 +1,14 @@
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: jupyter-web-app
spec:
action: ALLOW
rules:
- from:
- source:
principals:
- cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account
selector:
matchLabels:
app: jupyter-web-app
@@ -0,0 +1,9 @@
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: jupyter-web-app
spec:
host: jupyter-web-app-service.kubeflow.svc.cluster.local
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
@@ -0,0 +1,13 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
- virtual-service.yaml
- authorization-policy.yaml
- destination-rule.yaml
namespace: kubeflow
commonLabels:
app: jupyter-web-app
kustomize.component: jupyter-web-app
configurations:
- params.yaml
@@ -0,0 +1,3 @@
varReference:
- path: spec/http/route/destination/host
kind: VirtualService
@@ -0,0 +1,24 @@
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: jupyter-web-app-jupyter-web-app
spec:
gateways:
- kubeflow-gateway
hosts:
- '*'
http:
- headers:
request:
add:
x-forwarded-prefix: /jupyter
match:
- uri:
prefix: /jupyter/
rewrite:
uri: /
route:
- destination:
host: jupyter-web-app-service.$(JWA_NAMESPACE).svc.$(JWA_CLUSTER_DOMAIN)
port:
number: 80
@@ -0,0 +1,30 @@
### Manifests
This folder contains manifests for installing `notebook-controller`. The structure is the following:
```
.
├── crd
├── default
├── manager
├── rbac
├── samples
├── base
├── overlays
│ ├── kubeflow
│ └── standalone
```
The breakdown is the following:
- `crd`, `default`, `manager`, `rbac`, `samples`: Kubebuilder-generated structure. We keep this in order to be compatible with kubebuilder workflows. This is not meant for the consumer of the manifests.
- `base`, `overlays`: Kustomizations meant for consumption by the user:
- `overlays/kubeflow`: Installs `notebook-controller` as part of Kubeflow. The resulting manifests should be the same as the result of the [deprecated `base_v3` from kubeflow/manifests](https://github.com/kubeflow/manifests/tree/306d02979124bc29e48152272ddd60a59be9306c/profiles/base_v3). At a glance, it makes the following changes:
- Use namespace `kubeflow`.
- Remove namespace resource.
- Add KFAM container.
- Add KFAM Service and VirtualService.
- `overlays/standalone`: Install `notebook-controller` in its own namespace. Useful for testing or for users that prefer to install just the controller.
### CRD Issue
We patch the kubebuilder-generated CRD with an older version. That's because the validation was more relaxed in a previous version and now we ended up with some clients and resources in a state that fails more detailed validation, but works correctly. For more information, see: https://github.com/kubeflow/kubeflow/issues/5722
@@ -0,0 +1,8 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../default
images:
- name: docker.io/kubeflownotebookswg/notebook-controller
newName: docker.io/kubeflownotebookswg/notebook-controller
newTag: v1.9.0
@@ -0,0 +1,32 @@
# This kustomization.yaml is not intended to be run by itself,
# since it depends on service name and namespace that are out of this kustomize package.
# It should be run by config/default
resources:
- bases/kubeflow.org_notebooks.yaml
# +kubebuilder:scaffold:crdkustomizeresource
patchesStrategicMerge:
- patches/trivial_conversion_patch.yaml
# [WEBHOOK] To enable webhook, uncomment all the sections with [WEBHOOK] prefix.
# patches here are for enabling the conversion webhook for each CRD
#- patches/webhook_in_notebooks.yaml
# +kubebuilder:scaffold:crdkustomizewebhookpatch
# [CERTMANAGER] To enable webhook, uncomment all the sections with [CERTMANAGER] prefix.
# patches here are for enabling the CA injection for each CRD
#- patches/cainjection_in_notebooks.yaml
# +kubebuilder:scaffold:crdkustomizecainjectionpatch
# the following config is for teaching kustomize how to do kustomization for CRDs.
configurations:
- kustomizeconfig.yaml
patchesJson6902:
- target:
group: apiextensions.k8s.io
version: v1
kind: CustomResourceDefinition
name: notebooks.kubeflow.org
path: patches/validation_patches.yaml
@@ -0,0 +1,17 @@
# This file is for teaching kustomize how to substitute name and namespace reference in CRD
nameReference:
- kind: Service
version: v1
fieldSpecs:
- kind: CustomResourceDefinition
group: apiextensions.k8s.io
path: spec/conversion/webhookClientConfig/service/name
namespace:
- kind: CustomResourceDefinition
group: apiextensions.k8s.io
path: spec/conversion/webhookClientConfig/service/namespace
create: false
varReference:
- path: metadata/annotations
@@ -0,0 +1,8 @@
# The following patch adds a directive for certmanager to inject CA into the CRD
# CRD conversion requires k8s 1.13 or later.
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
certmanager.k8s.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME)
name: notebooks.kubeflow.org
@@ -0,0 +1,9 @@
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: notebooks.kubeflow.org
spec:
preserveUnknownFields: false # TODO: Remove in Kubeflow 1.7 release
conversion:
strategy: None
@@ -0,0 +1,29 @@
- op: replace
path: /spec/versions/0/schema/openAPIV3Schema/properties/spec/properties/template/properties/spec/properties/containers/items/required
value:
- name
- image
- op: replace
path: /spec/versions/1/schema/openAPIV3Schema/properties/spec/properties/template/properties/spec/properties/containers/items/required
value:
- name
- image
- op: replace
path: /spec/versions/2/schema/openAPIV3Schema/properties/spec/properties/template/properties/spec/properties/containers/items/required
value:
- name
- image
- op: add
path: /spec/versions/0/schema/openAPIV3Schema/properties/spec/properties/template/properties/spec/properties/containers/minItems
value: 1
- op: add
path: /spec/versions/1/schema/openAPIV3Schema/properties/spec/properties/template/properties/spec/properties/containers/minItems
value: 1
- op: add
path: /spec/versions/2/schema/openAPIV3Schema/properties/spec/properties/template/properties/spec/properties/containers/minItems
value: 1
@@ -0,0 +1,17 @@
# The following patch enables conversion webhook for CRD
# CRD conversion requires k8s 1.13 or later.
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: notebooks.kubeflow.org
spec:
conversion:
strategy: Webhook
webhookClientConfig:
# this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank,
# but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager)
caBundle: Cg==
service:
namespace: system
name: webhook-service
path: /convert
@@ -0,0 +1,75 @@
# Adds namespace to all resources.
namespace: notebook-controller-system
# Value of this field is prepended to the
# names of all resources, e.g. a deployment named
# "wordpress" becomes "alices-wordpress".
# Note that it should also match with the prefix (text before '-') of the namespace
# field above.
namePrefix: notebook-controller-
# Labels to add to all resources and selectors.
commonLabels:
app: notebook-controller
kustomize.component: notebook-controller
bases:
- ../rbac
- ../manager
- ../crd
# [WEBHOOK] To enable webhook, uncomment all the sections with [WEBHOOK] prefix including the one in crd/kustomization.yaml
#- ../webhook
# [CERTMANAGER] To enable cert-manager, uncomment all sections with 'CERTMANAGER'. 'WEBHOOK' components are required.
#- ../certmanager
#patchesStrategicMerge:
#- manager_image_patch.yaml
# Protect the /metrics endpoint by putting it behind auth.
# Only one of manager_auth_proxy_patch.yaml and
# manager_prometheus_metrics_patch.yaml should be enabled.
#- manager_auth_proxy_patch.yaml
# If you want your controller-manager to expose the /metrics
# endpoint w/o any authn/z, uncomment the following line and
# comment manager_auth_proxy_patch.yaml.
# Only one of manager_auth_proxy_patch.yaml and
# manager_prometheus_metrics_patch.yaml should be enabled.
#- manager_prometheus_metrics_patch.yaml
# [WEBHOOK] To enable webhook, uncomment all the sections with [WEBHOOK] prefix including the one in crd/kustomization.yaml
#- manager_webhook_patch.yaml
# [CERTMANAGER] To enable cert-manager, uncomment all sections with 'CERTMANAGER'.
# Uncomment 'CERTMANAGER' sections in crd/kustomization.yaml to enable the CA injection in the admission webhooks.
# 'CERTMANAGER' needs to be enabled to use ca injection
#- webhookcainjection_patch.yaml
# the following config is for teaching kustomize how to do var substitution
vars:
# [CERTMANAGER] To enable cert-manager, uncomment all sections with 'CERTMANAGER' prefix.
# - name: CERTIFICATE_NAMESPACE # namespace of the certificate CR
# objref:
# kind: Certificate
# group: certmanager.k8s.io
# version: v1alpha1
# name: serving-cert # this name should match the one in certificate.yaml
# fieldref:
# fieldpath: metadata.namespace
# - name: CERTIFICATE_NAME
# objref:
# kind: Certificate
# group: certmanager.k8s.io
# version: v1alpha1
# name: serving-cert # this name should match the one in certificate.yaml
# - name: SERVICE_NAMESPACE # namespace of the service
# objref:
# kind: Service
# version: v1
# name: webhook-service
# fieldref:
# fieldpath: metadata.namespace
# - name: SERVICE_NAME
# objref:
# kind: Service
# version: v1
# name: webhook-service
@@ -0,0 +1,25 @@
# This patch inject a sidecar container which is a HTTP proxy for the controller manager,
# it performs RBAC authorization against the Kubernetes API using SubjectAccessReviews.
apiVersion: apps/v1
kind: Deployment
metadata:
name: controller-manager
namespace: system
spec:
template:
spec:
containers:
- name: kube-rbac-proxy
image: gcr.io/kubebuilder/kube-rbac-proxy:v0.4.0
args:
- "--secure-listen-address=0.0.0.0:8443"
- "--upstream=http://127.0.0.1:8080/"
- "--logtostderr=true"
- "--v=10"
ports:
- containerPort: 8443
name: https
- name: manager
args:
- "--metrics-addr=127.0.0.1:8080"
- "--enable-leader-election"
@@ -0,0 +1,12 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: controller-manager
namespace: system
spec:
template:
spec:
containers:
# Change the value of image field below to your controller image URL
- image: IMAGE_URL
name: manager
@@ -0,0 +1,19 @@
# This patch enables Prometheus scraping for the manager pod.
apiVersion: apps/v1
kind: Deployment
metadata:
name: controller-manager
namespace: system
spec:
template:
metadata:
annotations:
prometheus.io/scrape: 'true'
spec:
containers:
# Expose the prometheus metrics on default port
- name: manager
ports:
- containerPort: 8080
name: metrics
protocol: TCP
@@ -0,0 +1,23 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: controller-manager
namespace: system
spec:
template:
spec:
containers:
- name: manager
ports:
- containerPort: 443
name: webhook-server
protocol: TCP
volumeMounts:
- mountPath: /tmp/k8s-webhook-server/serving-certs
name: cert
readOnly: true
volumes:
- name: cert
secret:
defaultMode: 420
secretName: webhook-server-cert
@@ -0,0 +1,15 @@
# This patch add annotation to admission webhook config and
# the variables $(CERTIFICATE_NAMESPACE) and $(CERTIFICATE_NAME) will be substituted by kustomize.
apiVersion: admissionregistration.k8s.io/v1beta1
kind: MutatingWebhookConfiguration
metadata:
name: mutating-webhook-configuration
annotations:
certmanager.k8s.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME)
---
apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingWebhookConfiguration
metadata:
name: validating-webhook-configuration
annotations:
certmanager.k8s.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME)
@@ -0,0 +1,8 @@
resources:
- manager.yaml
- service-account.yaml
- service.yaml
configMapGenerator:
- name: config
envs:
- params.env
@@ -0,0 +1,73 @@
apiVersion: v1
kind: Namespace
metadata:
labels:
control-plane: controller-manager
name: system
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment
spec:
template:
metadata:
labels:
app: notebook-controller
kustomize.component: notebook-controller
spec:
containers:
- name: manager
image: docker.io/kubeflownotebookswg/notebook-controller
command:
- /manager
env:
- name: USE_ISTIO
valueFrom:
configMapKeyRef:
name: config
key: USE_ISTIO
- name: ISTIO_GATEWAY
valueFrom:
configMapKeyRef:
name: config
key: ISTIO_GATEWAY
- name: ISTIO_HOST
valueFrom:
configMapKeyRef:
name: config
key: ISTIO_HOST
- name: CLUSTER_DOMAIN
valueFrom:
configMapKeyRef:
name: config
key: CLUSTER_DOMAIN
- name: ENABLE_CULLING
valueFrom:
configMapKeyRef:
name: config
key: ENABLE_CULLING
- name: CULL_IDLE_TIME
valueFrom:
configMapKeyRef:
name: config
key: CULL_IDLE_TIME
- name: IDLENESS_CHECK_PERIOD
valueFrom:
configMapKeyRef:
name: config
key: IDLENESS_CHECK_PERIOD
imagePullPolicy: IfNotPresent
livenessProbe:
httpGet:
path: /healthz
port: 8081
initialDelaySeconds: 5
periodSeconds: 10
readinessProbe:
httpGet:
path: /readyz
port: 8081
initialDelaySeconds: 5
periodSeconds: 10
serviceAccountName: service-account
@@ -0,0 +1,7 @@
USE_ISTIO=true
ISTIO_GATEWAY=kubeflow/kubeflow-gateway
ISTIO_HOST=*
CLUSTER_DOMAIN=cluster.local
ENABLE_CULLING=false
CULL_IDLE_TIME=1440
IDLENESS_CHECK_PERIOD=1
@@ -0,0 +1,4 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: service-account
@@ -0,0 +1,13 @@
apiVersion: v1
kind: Service
metadata:
labels:
app: notebook-controller
kustomize.component: notebook-controller
name: service
spec:
ports:
- port: 443
selector:
app: notebook-controller
kustomize.component: notebook-controller
@@ -0,0 +1,13 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
namespace: kubeflow
patchesStrategicMerge:
- patches/remove-namespace.yaml
configMapGenerator:
- name: config
behavior: merge
literals:
- USE_ISTIO=true
- ISTIO_GATEWAY=kubeflow/kubeflow-gateway
@@ -0,0 +1,5 @@
$patch: delete
apiVersion: v1
kind: Namespace
metadata:
name: notebook-controller-system
@@ -0,0 +1,10 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
namespace: notebook-controller-system
configMapGenerator:
- name: config
behavior: merge
literals:
- USE_ISTIO=false
@@ -0,0 +1,13 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: proxy-role
rules:
- apiGroups: ["authentication.k8s.io"]
resources:
- tokenreviews
verbs: ["create"]
- apiGroups: ["authorization.k8s.io"]
resources:
- subjectaccessreviews
verbs: ["create"]
@@ -0,0 +1,11 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: proxy-rolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: proxy-role
subjects:
- kind: ServiceAccount
name: service-account
@@ -0,0 +1,18 @@
apiVersion: v1
kind: Service
metadata:
annotations:
prometheus.io/port: "8443"
prometheus.io/scheme: https
prometheus.io/scrape: "true"
labels:
control-plane: controller-manager
name: controller-manager-metrics-service
namespace: system
spec:
ports:
- name: https
port: 8443
targetPort: https
selector:
control-plane: controller-manager
@@ -0,0 +1,12 @@
resources:
- role.yaml
- role_binding.yaml
- leader_election_role.yaml
- leader_election_role_binding.yaml
- user_cluster_roles.yaml
# Comment the following 3 lines if you want to disable
# the auth proxy (https://github.com/brancz/kube-rbac-proxy)
# which protects your /metrics endpoint.
# - auth_proxy_service.yaml
# - auth_proxy_role.yaml
# - auth_proxy_role_binding.yaml
@@ -0,0 +1,32 @@
# permissions to do leader election.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: leader-election-role
rules:
- apiGroups:
- ""
resources:
- configmaps
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
- apiGroups:
- ""
resources:
- configmaps/status
verbs:
- get
- update
- patch
- apiGroups:
- ""
resources:
- events
verbs:
- create
@@ -0,0 +1,11 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: leader-election-rolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: leader-election-role
subjects:
- kind: ServiceAccount
name: service-account
@@ -0,0 +1,51 @@
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
name: role
rules:
- apiGroups:
- apps
resources:
- statefulsets
verbs:
- '*'
- apiGroups:
- ""
resources:
- events
verbs:
- create
- get
- list
- patch
- watch
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- services
verbs:
- '*'
- apiGroups:
- kubeflow.org
resources:
- notebooks
- notebooks/finalizers
- notebooks/status
verbs:
- '*'
- apiGroups:
- networking.istio.io
resources:
- virtualservices
verbs:
- '*'
@@ -0,0 +1,11 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: role-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: role
subjects:
- kind: ServiceAccount
name: service-account
@@ -0,0 +1,55 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kubeflow-notebooks-admin
labels:
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-admin: "true"
aggregationRule:
clusterRoleSelectors:
- matchLabels:
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-notebooks-admin: "true"
rules: []
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kubeflow-notebooks-edit
labels:
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-edit: "true"
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-notebooks-admin: "true"
rules:
- apiGroups:
- kubeflow.org
resources:
- notebooks
- notebooks/status
verbs:
- get
- list
- watch
- create
- delete
- deletecollection
- patch
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kubeflow-notebooks-view
labels:
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-view: "true"
rules:
- apiGroups:
- kubeflow.org
resources:
- notebooks
- notebooks/status
verbs:
- get
- list
- watch
@@ -0,0 +1,11 @@
---
apiVersion: kubeflow.org/v1
kind: Notebook
metadata:
name: notebook-sample-v1
spec:
template:
spec:
containers:
- name: notebook-sample-v1
image: kubeflownotebookswg/jupyter:latest
@@ -0,0 +1,11 @@
---
apiVersion: kubeflow.org/v1alpha1
kind: Notebook
metadata:
name: notebook-sample-v1alpha1
spec:
template:
spec:
containers:
- name: notebook-sample-v1
image: kubeflownotebookswg/jupyter:latest
@@ -0,0 +1,11 @@
---
apiVersion: kubeflow.org/v1beta1
kind: Notebook
metadata:
name: notebook-sample-v1beta1
spec:
template:
spec:
containers:
- name: notebook-sample-v1
image: kubeflownotebookswg/jupyter:latest
@@ -0,0 +1,69 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: katib-controller
namespace: kubeflow
labels:
katib.kubeflow.org/component: controller
spec:
replicas: 1
selector:
matchLabels:
katib.kubeflow.org/component: controller
template:
metadata:
labels:
katib.kubeflow.org/component: controller
katib.kubeflow.org/metrics-collector-injection: disabled
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "8080"
sidecar.istio.io/inject: "false"
spec:
serviceAccountName: katib-controller
containers:
- name: katib-controller
image: docker.io/kubeflowkatib/katib-controller
command: ["./katib-controller"]
args:
- --katib-config=/katib-config.yaml
ports:
- containerPort: 8443
name: webhook
protocol: TCP
- containerPort: 8080
name: metrics
protocol: TCP
- containerPort: 18080
name: healthz
protocol: TCP
readinessProbe:
httpGet:
path: /readyz
port: healthz
livenessProbe:
httpGet:
path: /healthz
port: healthz
env:
- name: KATIB_CORE_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- mountPath: /tmp/cert
name: cert
readOnly: true
- mountPath: /katib-config.yaml
name: katib-config
subPath: katib-config.yaml
readOnly: true
volumes:
- name: cert
secret:
defaultMode: 420
secretName: katib-webhook-cert
- name: katib-config
configMap:
name: katib-config
@@ -0,0 +1,9 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- controller.yaml
- rbac.yaml
- service.yaml
- trial-templates.yaml
@@ -0,0 +1,148 @@
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: katib-controller
rules:
- apiGroups:
- ""
resources:
- services
verbs:
- "get"
- "list"
- "watch"
- "create"
- "delete"
- apiGroups:
- ""
resources:
- events
verbs:
- "create"
- "patch"
- "update"
- apiGroups:
- ""
resources:
- serviceaccounts
- persistentvolumes
- persistentvolumeclaims
verbs:
- "get"
- "list"
- "watch"
- "create"
- apiGroups:
- ""
resources:
- namespaces
- configmaps
verbs:
- "get"
- "list"
- "watch"
- apiGroups:
- ""
resources:
- pods
- pods/status
verbs:
- "get"
- apiGroups:
- ""
resources:
- secrets
verbs:
- "get"
- "list"
- "watch"
- "patch"
- apiGroups:
- apps
resources:
- deployments
verbs:
- "get"
- "list"
- "watch"
- "create"
- "delete"
- apiGroups:
- rbac.authorization.k8s.io
resources:
- roles
- rolebindings
verbs:
- "get"
- "create"
- "list"
- "watch"
- apiGroups:
- batch
resources:
- jobs
- cronjobs
verbs:
- "get"
- "list"
- "watch"
- "create"
- "delete"
- apiGroups:
- kubeflow.org
resources:
- tfjobs
- pytorchjobs
- mpijobs
- xgboostjobs
- mxjobs
verbs:
- "get"
- "list"
- "watch"
- "create"
- "delete"
- apiGroups:
- kubeflow.org
resources:
- experiments
- experiments/status
- experiments/finalizers
- trials
- trials/status
- trials/finalizers
- suggestions
- suggestions/status
- suggestions/finalizers
verbs:
- "*"
- apiGroups:
- admissionregistration.k8s.io
resources:
- validatingwebhookconfigurations
- mutatingwebhookconfigurations
verbs:
- "get"
- "watch"
- "list"
- "patch"
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: katib-controller
namespace: kubeflow
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: katib-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: katib-controller
subjects:
- kind: ServiceAccount
name: katib-controller
namespace: kubeflow
@@ -0,0 +1,26 @@
---
apiVersion: v1
kind: Service
metadata:
name: katib-controller
namespace: kubeflow
labels:
katib.kubeflow.org/component: controller
annotations:
prometheus.io/port: "8080"
prometheus.io/scheme: http
prometheus.io/scrape: "true"
spec:
ports:
- port: 443
protocol: TCP
targetPort: 8443
name: webhook
- name: metrics
port: 8080
targetPort: 8080
- name: healthz
port: 18080
targetPort: 18080
selector:
katib.kubeflow.org/component: controller
@@ -0,0 +1,77 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: trial-templates
namespace: kubeflow
labels:
katib.kubeflow.org/component: trial-templates
data:
defaultTrialTemplate.yaml: |-
apiVersion: batch/v1
kind: Job
spec:
template:
spec:
containers:
- name: training-container
image: docker.io/kubeflowkatib/pytorch-mnist-cpu:v0.17.0
command:
- "python3"
- "/opt/pytorch-mnist/mnist.py"
- "--epochs=1"
- "--batch-size=16"
- "--lr=${trialParameters.learningRate}"
- "--momentum=${trialParameters.momentum}"
restartPolicy: Never
# For ConfigMap templates double quotes must set in commands to correct parse JSON parameters in Trial Template (e.g nn_config, architecture)
enasCPUTemplate: |-
apiVersion: batch/v1
kind: Job
spec:
template:
spec:
containers:
- name: training-container
image: docker.io/kubeflowkatib/enas-cnn-cifar10-cpu:v0.17.0
command:
- python3
- -u
- RunTrial.py
- --num_epochs=1
- "--architecture=\"${trialParameters.neuralNetworkArchitecture}\""
- "--nn_config=\"${trialParameters.neuralNetworkConfig}\""
restartPolicy: Never
pytorchJobTemplate: |-
apiVersion: kubeflow.org/v1
kind: PyTorchJob
spec:
pytorchReplicaSpecs:
Master:
replicas: 1
restartPolicy: OnFailure
template:
spec:
containers:
- name: pytorch
image: docker.io/kubeflowkatib/pytorch-mnist-cpu:v0.17.0
command:
- "python3"
- "/opt/pytorch-mnist/mnist.py"
- "--epochs=1"
- "--lr=${trialParameters.learningRate}"
- "--momentum=${trialParameters.momentum}"
Worker:
replicas: 2
restartPolicy: OnFailure
template:
spec:
containers:
- name: pytorch
image: docker.io/kubeflowkatib/pytorch-mnist-cpu:v0.17.0
command:
- "python3"
- "/opt/pytorch-mnist/mnist.py"
- "--epochs=1"
- "--lr=${trialParameters.learningRate}"
- "--momentum=${trialParameters.momentum}"
@@ -0,0 +1,36 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: experiments.kubeflow.org
spec:
group: kubeflow.org
scope: Namespaced
versions:
- name: v1beta1
served: true
storage: true
additionalPrinterColumns:
- name: Type
type: string
jsonPath: .status.conditions[-1:].type
- name: Status
type: string
jsonPath: .status.conditions[-1:].status
- name: Age
type: date
jsonPath: .metadata.creationTimestamp
subresources:
status: {}
schema:
openAPIV3Schema:
type: object
x-kubernetes-preserve-unknown-fields: true
names:
kind: Experiment
singular: experiment
plural: experiments
categories:
- all
- kubeflow
- katib
@@ -0,0 +1,8 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- experiment.yaml
- suggestion.yaml
- trial.yaml
@@ -0,0 +1,42 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: suggestions.kubeflow.org
spec:
group: kubeflow.org
scope: Namespaced
versions:
- name: v1beta1
served: true
storage: true
additionalPrinterColumns:
- name: Type
type: string
jsonPath: .status.conditions[-1:].type
- name: Status
type: string
jsonPath: .status.conditions[-1:].status
- name: Requested
type: string
jsonPath: .spec.requests
- name: Assigned
type: string
jsonPath: .status.suggestionCount
- name: Age
type: date
jsonPath: .metadata.creationTimestamp
subresources:
status: {}
schema:
openAPIV3Schema:
type: object
x-kubernetes-preserve-unknown-fields: true
names:
kind: Suggestion
singular: suggestion
plural: suggestions
categories:
- all
- kubeflow
- katib
@@ -0,0 +1,36 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: trials.kubeflow.org
spec:
group: kubeflow.org
scope: Namespaced
versions:
- name: v1beta1
served: true
storage: true
additionalPrinterColumns:
- name: Type
type: string
jsonPath: .status.conditions[-1:].type
- name: Status
type: string
jsonPath: .status.conditions[-1:].status
- name: Age
type: date
jsonPath: .metadata.creationTimestamp
subresources:
status: {}
schema:
openAPIV3Schema:
type: object
x-kubernetes-preserve-unknown-fields: true
names:
kind: Trial
singular: trial
plural: trials
categories:
- all
- kubeflow
- katib
@@ -0,0 +1,42 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: katib-db-manager
namespace: kubeflow
labels:
katib.kubeflow.org/component: db-manager
spec:
replicas: 1
selector:
matchLabels:
katib.kubeflow.org/component: db-manager
template:
metadata:
labels:
katib.kubeflow.org/component: db-manager
annotations:
sidecar.istio.io/inject: "false"
spec:
containers:
- name: katib-db-manager
image: docker.io/kubeflowkatib/katib-db-manager
env:
- name: DB_NAME
value: "mysql"
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: katib-mysql-secrets
key: MYSQL_ROOT_PASSWORD
command:
- "./katib-db-manager"
ports:
- name: api
containerPort: 6789
livenessProbe:
grpc:
port: 6789
initialDelaySeconds: 10
periodSeconds: 60
failureThreshold: 5
@@ -0,0 +1,7 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- db-manager.yaml
- service.yaml
@@ -0,0 +1,16 @@
---
apiVersion: v1
kind: Service
metadata:
name: katib-db-manager
namespace: kubeflow
labels:
katib.kubeflow.org/component: db-manager
spec:
type: ClusterIP
ports:
- port: 6789
protocol: TCP
name: api
selector:
katib.kubeflow.org/component: db-manager
@@ -0,0 +1,9 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- mysql.yaml
- pvc.yaml
- secret.yaml
- service.yaml
@@ -0,0 +1,74 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: katib-mysql
namespace: kubeflow
labels:
katib.kubeflow.org/component: mysql
spec:
replicas: 1
selector:
matchLabels:
katib.kubeflow.org/component: mysql
strategy:
type: Recreate
template:
metadata:
labels:
katib.kubeflow.org/component: mysql
annotations:
sidecar.istio.io/inject: "false"
spec:
containers:
- name: katib-mysql
image: mysql:8.0.29
args:
- --datadir
- /var/lib/mysql/datadir
env:
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: katib-mysql-secrets
key: MYSQL_ROOT_PASSWORD
- name: MYSQL_ALLOW_EMPTY_PASSWORD
value: "true"
- name: MYSQL_DATABASE
value: "katib"
ports:
- name: dbapi
containerPort: 3306
readinessProbe:
exec:
command:
- "/bin/bash"
- "-c"
- "mysql -D ${MYSQL_DATABASE} -u root -p${MYSQL_ROOT_PASSWORD} -e 'SELECT 1'"
initialDelaySeconds: 10
periodSeconds: 5
failureThreshold: 10
livenessProbe:
exec:
command:
- "/bin/bash"
- "-c"
- "mysqladmin ping -u root -p${MYSQL_ROOT_PASSWORD}"
initialDelaySeconds: 10
periodSeconds: 5
failureThreshold: 10
startupProbe:
exec:
command:
- "/bin/bash"
- "-c"
- "mysqladmin ping -u root -p${MYSQL_ROOT_PASSWORD}"
periodSeconds: 15
failureThreshold: 60
volumeMounts:
- name: katib-mysql
mountPath: /var/lib/mysql
volumes:
- name: katib-mysql
persistentVolumeClaim:
claimName: katib-mysql
@@ -0,0 +1,12 @@
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: katib-mysql
namespace: kubeflow
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi

Some files were not shown because too many files have changed in this diff Show More