update kubeflow dip-catalog

2025-01-13 02:31:27 +00:00
parent 1dc1181a03
commit 5451f16d72
1959 changed files with 602337 additions and 0 deletions
@@ -0,0 +1,43 @@
+# Unauthenticated Issuer Discovery
+
+If you are using Kubernetes with tools like kind, vCluster, minikube, or similar solutions for local
+development, it's highly likely that the Kubernetes OIDC Issuer operates within the cluster and is
+secured with self-signed certificates.
+
+To facilitate the use of m2m tokens with `oauth2-proxy` and Istio, both tools must perform OIDC
+Connect Discovery on the Token Issuer. Kubernetes offers OIDC Discovery functionality at a specific
+URI:
+```
+https://kubernetes.default.svc.cluster.local/.well-known/openid-configuration
+```
+
+Access to this endpoint is blocked by default due to RBAC policies, but it can be enabled by
+creating a `ClusterRoleBinding`. This binding associates the predefined `ClusterRole`
+`system:service-account-issuer-discovery` with the `system:unauthenticated` group. The
+configuration for this is detailed in the resource file `clusterrolebinding.unauthenticated-oidc-viewer.yaml`.
+
+Once this step is completed, the endpoint can be accessed to reveal the Issuer URL:
+```bash
+$ curl -k https://kubernetes.default.svc.cluster.local/.well-known/openid-configuration
+
+# Example output in kind:
+{"issuer":"https://kubernetes.default.svc.cluster.local","jwks_uri":"https://172.18.0.5:6443/openid/v1/jwks","response_types_supported":["id_token"],"subject_types_supported":["public"],"id_token_signing_alg_values_supported":["RS256"]}
+
+# Example output in vCluster:
+{"issuer":"https://kubernetes.default.svc.cluster.local","jwks_uri":"https://1.2.3.4:6443/openid/v1/jwks","response_types_supported":["id_token"],"subject_types_supported":["public"],"id_token_signing_alg_values_supported":["RS256"]}
+
+# Example output in AWS EKS:
+{"issuer":"https://oidc.eks.region.amazonaws.com/id/123abc","jwks_uri":"https://ip-1-2-3-4.eu-central-1.compute.internal:443/openid/v1/jwks","response_types_supported":["id_token"],"subject_types_supported":["public"],"id_token_signing_alg_values_supported":["RS256"]}
+```
+
+If you're operating in a `vCluster`, access to the endpoint specified under `jwks_uri` (for example,
+`https://1.2.3.4:6443/openid/v1/jwks`) is managed separately, and merely creating the previously
+mentioned `ClusterRoleBinding` is insufficient. To circumvent this limitation, you can configure the
+`kube-apiserver` to allow anonymous authentication by setting `--anonymous-auth=true`. This is
+achieved by appending `--kube-apiserver-arg=anonymous-auth=true` to the list of arguments in the
+Helm Chart Values file:
+```yaml
+vcluster:
+    extraArgs:
+    - --kube-apiserver-arg=anonymous-auth=true
+```
@@ -0,0 +1,12 @@
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: unauthenticated-oidc-viewer
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: system:service-account-issuer-discovery
+subjects:
+- apiGroup: rbac.authorization.k8s.io
+  kind: Group
+  name: system:unauthenticated
@@ -0,0 +1,5 @@
+apiVersion: kustomize.config.k8s.io/v1alpha1
+kind: Component
+
+resources:
+- clusterrolebinding.unauthenticated-oidc-viewer.yaml
@@ -0,0 +1,5 @@
+apiVersion: kustomize.config.k8s.io/v1alpha1
+kind: Component
+
+patches:
+- path: patches/deployment.logout-url.yaml
@@ -0,0 +1,19 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: centraldashboard
+  namespace: kubeflow
+spec:
+  template:
+    spec:
+      containers:
+      - name: centraldashboard
+        env:
+        # This successfully logs out the user but the user is not redirected to
+        # the home page. This is because of custom integration with
+        # oidc-authservice which will provide response with 'afterLogoutURL'.
+        # https://github.com/arrikto/oidc-authservice/blob/0c4ea9a/server.go#L509
+        # https://github.com/kubeflow/kubeflow/blob/c6c4492/components/centraldashboard/public/components/logout-button.js#L50
+        # Please refresh the page after logging out.
+        - name: LOGOUT_URL
+          value: /oauth2/sign_out
@@ -0,0 +1,41 @@
+# Configure Istio with Self-Signed Kubernetes OIDC Issuer
+
+## Overview
+
+This kustomize component is designed for scenarios where the Kubernetes OIDC Issuer is behind
+self-signed certificates, causing trust issues with Istio in retrieving the JWKS. It creates a
+Kubernetes Job and necessary RBAC configurations to address this issue. The job facilitates the
+retrieval of JWKS from the OIDC Issuer and embeds it directly into Istio's configuration, thereby
+bypassing trust issues with self-signed certificates.
+
+## Configuration Persistence and JWKS Public Accessibility
+
+The configuration created by this job is stored exclusively in etcd and is not persisted elsewhere.
+This setup is compatible with ArgoCD, which by default does not delete properties absent in the
+desired manifest. However, if the `RequestAuthentication` resource is modified erroneously or
+deleted, the configuration would be lost, necessitating a rerun of the job.
+
+To circumvent the need for rerunning the job, it is advisable to store the JWKS in a repository.
+JWKS are typically publicly accessible and contain no sensitive information, making them safe for
+repository storage. By persisting the JWKS in a repository, consistent access is ensured regardless
+of changes in the cluster or accidental deletions.
+
+## Functionality
+
+- **Reading OIDC Issuer URL**: The Job reads the `RequestAuthentication` resource to extract the
+  OIDC Issuer URL.
+- **Fetching JWKS**: After identifying the Issuer URL, the Job retrieves the JWKS.
+- **Patching RequestAuthentication**: The JWKS is used to patch the `RequestAuthentication`
+  resource, embedding the JWKS directly.
+- **Static JWKS Configuration for Istio**: Ensures that Istio uses the JWKS provided by the Job
+  instead of requesting it independently.
+
+## Use Case
+
+This setup is particularly useful when:
+- Kubernetes serves as an OIDC provider.
+- The Kubernetes API is not served with publicly trusted certificates.
+
+This component ensures seamless M2M authentication by handling JWKS retrieval and configuration
+internally, thus circumventing certificate validation issues for Istio fetching JWKS from an OIDC
+provider with self-signed or private CA certificates.
@@ -0,0 +1,39 @@
+apiVersion: batch/v1
+kind: CronJob
+metadata:
+  name: kubeflow-m2m-oidc-configurator
+  namespace: istio-system
+spec:
+  schedule: '*/5 * * * *'
+  concurrencyPolicy: Forbid
+  jobTemplate:
+    spec:
+      backoffLimit: 3
+      ttlSecondsAfterFinished: 600
+      template:
+        metadata:
+          labels: {}
+        spec:
+          restartPolicy: OnFailure
+          serviceAccountName: kubeflow-m2m-oidc-configurator
+          containers:
+            - image: docker.io/curlimages/curl
+              name: kubeflow-m2m-oidc-configurator
+              command:
+                - /script.sh
+              envFrom:
+                - configMapRef:
+                    name: kubeflow-m2m-oidc-configurator-envs
+              volumeMounts:
+                - mountPath: /script.sh
+                  name: script
+                  subPath: script.sh
+              resources: {}
+          volumes:
+            - name: script
+              configMap:
+                name: kubeflow-m2m-oidc-configurator-script
+                defaultMode: 0777
+                items:
+                  - key: script.sh
+                    path: script.sh
@@ -0,0 +1,24 @@
+apiVersion: kustomize.config.k8s.io/v1alpha1
+kind: Component
+
+resources:
+- cronjob.kubeflow-m2m-oidc-configurator.yaml
+- rbac.yaml
+
+configMapGenerator:
+- name: kubeflow-m2m-oidc-configurator-script
+  namespace: istio-system
+  files:
+  - script.sh=script.sh
+
+- name: kubeflow-m2m-oidc-configurator-envs
+  namespace: istio-system
+  literals:
+  - ISTIO_ROOT_NAMESPACE=istio-system
+  - REQUEST_AUTHENTICATION_NAME=m2m-token-issuer
+  - KUBERNETES_API_SERVER_URL=https://kubernetes.default.svc
+
+- name: oauth2-proxy-parameters
+  behavior: merge
+  literals:
+  - ALLOW_SELF_SIGNED_ISSUER=true
@@ -0,0 +1,35 @@
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: kubeflow-m2m-oidc-configurator
+  namespace: istio-system
+
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: kubeflow-m2m-oidc-configurator
+  namespace: istio-system
+rules:
+  - apiGroups:
+      - security.istio.io
+    resources:
+      - requestauthentications
+    verbs:
+      - get
+      - patch
+
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: kubeflow-m2m-oidc-configurator
+  namespace: istio-system
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: kubeflow-m2m-oidc-configurator
+subjects:
+  - kind: ServiceAccount
+    name: kubeflow-m2m-oidc-configurator
+    namespace: istio-system
@@ -0,0 +1,143 @@
+#!/bin/sh
+set -e
+
+KUBERNETES_API_SERVER_URL="${KUBERNETES_API_SERVER_URL:-https://kubernetes.default.svc}"
+ISTIO_ROOT_NAMESPACE="${ISTIO_ROOT_NAMESPACE:-istio-system}"
+REQUEST_AUTHENTICATION_NAME="${REQUEST_AUTHENTICATION_NAME:-m2m-token-issuer}"
+
+RESOURCE_URL="\
+${KUBERNETES_API_SERVER_URL}\
+/apis/security.istio.io/v1/namespaces/\
+${ISTIO_ROOT_NAMESPACE}\
+/requestauthentications/\
+${REQUEST_AUTHENTICATION_NAME}"
+
+wait_for_resource_ready() {
+  while true; do
+    response="$(
+      curl -s -o /dev/null \
+        --url "${RESOURCE_URL}" \
+        -w "%{http_code}" \
+        --header "Authorization: Bearer $(cat /run/secrets/kubernetes.io/serviceaccount/token)" \
+        --insecure
+    )"
+    if [ "${response}" = "200" ]; then
+      break
+    fi
+    sleep 5
+  done
+}
+
+get_request_authentication_obj() {
+  curl -s --request GET \
+    --url "${RESOURCE_URL}" \
+    --header "Authorization: Bearer $(cat /run/secrets/kubernetes.io/serviceaccount/token)" \
+    --insecure
+}
+
+get_issuer_url_from_obj() {
+  obj="${1}"
+  echo "${obj}" | awk -F'"' '/"issuer":/ { print $4 }'
+}
+
+get_current_escaped_jwks_from_obj() {
+  obj="${1}"
+  echo "${obj}" | awk -F'"' '/"jwks":/' | sed -n 's/^.*"jwks": "\(.*\)".*$/\1/p'
+}
+
+get_jwks_uri() {
+  issuer_url="${1}"
+  curl -s --request GET \
+      --url "${issuer_url}/.well-known/openid-configuration" \
+      --header "Authorization: Bearer $(cat /run/secrets/kubernetes.io/serviceaccount/token)" \
+      --insecure |
+      grep -o '"jwks_uri":"https:\/\/[^"]\+"' |
+      sed 's/"jwks_uri":"\(.*\)"/\1/'
+  }
+
+get_jwks_from_uri() {
+  jwks_uri="${1}"
+  curl -s --request GET \
+      --url "${jwks_uri}" \
+      --header "Authorization: Bearer $(cat /run/secrets/kubernetes.io/serviceaccount/token)" \
+      --insecure
+}
+
+# Format JWKS in a way that can be accepted in resource patch.
+parse_escaped_jwks() {
+  jwks="${1}"
+  echo "${jwks}" | sed 's/"/\\"/g'
+}
+
+are_jwks_equal() {
+  jwks1="${1}"
+  jwks2="${2}"
+  test "$(echo "${jwks1}" | base64 -w0)" = "$(echo "${jwks2}" | base64 -w0)"
+}
+
+patch_request_authentication_with_escaped_jwks() {
+  jwks_escaped="${1}"
+  curl -s --request PATCH \
+    --url "${RESOURCE_URL}" \
+    --header "Content-Type: application/json-patch+json" \
+    --header "Authorization: Bearer $(cat /run/secrets/kubernetes.io/serviceaccount/token)" \
+    -d '[{ "op": "add", "path": "/spec/jwtRules/0/jwks", "value": "'"${jwks_escaped}"'" }]' \
+    --insecure
+    echo
+}
+
+patch_request_authentication_with_jwks_if_required() {
+  echo "Getting RequestAuthentication object."
+  REQUEST_AUTHENTICATION_OBJ="$(get_request_authentication_obj)"
+
+  ISSUER_URL="$(get_issuer_url_from_obj "${REQUEST_AUTHENTICATION_OBJ}")"
+  echo "Issuer Url in RequestAuthentication: ${ISSUER_URL}"
+
+  CURRENT_JWKS_ESCAPED="$(get_current_escaped_jwks_from_obj "${REQUEST_AUTHENTICATION_OBJ}")"
+  printf "Current Jwks (escaped):\n%s\n" "${CURRENT_JWKS_ESCAPED}"
+
+  JWKS_URI="$(get_jwks_uri "${ISSUER_URL}")"
+  echo "Jwks Uri from Well Known OpenID Configuration: ${JWKS_URI}"
+
+  JWKS="$(get_jwks_from_uri "${JWKS_URI}")"
+  JWKS_ESCAPED="$(parse_escaped_jwks "${JWKS}")"
+  printf "JWKS from Well Known OpenID Configuration (escaped): \n%s\n" "${JWKS_ESCAPED}"
+
+  if are_jwks_equal "${JWKS_ESCAPED}" "${CURRENT_JWKS_ESCAPED}"; then
+    echo "JWKS in RequestAuthentication ${REQUEST_AUTHENTICATION_NAME} is configured correctly."
+  else
+    echo "JWKS in RequestAuthentication ${REQUEST_AUTHENTICATION_NAME} needs to be configured."
+    patch_request_authentication_with_escaped_jwks "${JWKS_ESCAPED}"
+  fi
+}
+
+verify_jwks_in_request_authentication() {
+  REQUEST_AUTHENTICATION_OBJ="$(get_request_authentication_obj)"
+  ISSUER_URL="$(get_issuer_url_from_obj "${REQUEST_AUTHENTICATION_OBJ}")"
+  CURRENT_JWKS_ESCAPED="$(get_current_escaped_jwks_from_obj "${REQUEST_AUTHENTICATION_OBJ}")"
+  JWKS_URI="$(get_jwks_uri "${ISSUER_URL}")"
+  JWKS="$(get_jwks_from_uri "${JWKS_URI}")"
+  JWKS_ESCAPED="$(parse_escaped_jwks "${JWKS}")"
+  if ! are_jwks_equal "${JWKS_ESCAPED}" "${CURRENT_JWKS_ESCAPED}"; then
+    echo "JWKS not properly configured, exit with error code 1"
+    exit 1
+  fi
+}
+
+main() {
+  echo "Wait until resource RequestAuthentication ${REQUEST_AUTHENTICATION_NAME} in namespace ${ISTIO_ROOT_NAMESPACE} is ready."
+  wait_for_resource_ready
+  echo "Resource RequestAuthentication ${REQUEST_AUTHENTICATION_NAME} in namespace ${ISTIO_ROOT_NAMESPACE} is ready."
+
+  echo "Patch RequestAuthentication with JWKS if required."
+  patch_request_authentication_with_jwks_if_required
+
+  echo "Wait 5 seconds before verifying RequestAuthentication JWKS configuration."
+  sleep 5
+
+  echo "Verify if RequestAuthentication is properly configured with JWKS..."
+  verify_jwks_in_request_authentication
+  echo "RequestAuthentication is properly configured with JWKS."
+}
+
+main
@@ -0,0 +1,6 @@
+apiVersion: kustomize.config.k8s.io/v1alpha1
+kind: Component
+
+patches:
+- path: patches/cm.enable-oauth2-proxy.yaml
+- path: patches/deployment.jwt-refresh-interval.yaml
@@ -0,0 +1,36 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: istio
+  namespace: istio-system
+data:
+  mesh: |-
+    accessLogFile: /dev/stdout
+    defaultConfig:
+      discoveryAddress: istiod.istio-system.svc:15012
+      proxyMetadata: {}
+      tracing: {}
+    enablePrometheusMerge: true
+    rootNamespace: istio-system
+    tcpKeepalive:
+      interval: 5s
+      probes: 3
+      time: 10s
+    trustDomain: cluster.local
+    extensionProviders:
+    - envoyExtAuthzHttp:
+        headersToDownstreamOnDeny:
+        - content-type
+        - set-cookie
+        headersToUpstreamOnAllow:
+        - authorization
+        - path
+        - x-auth-request-email
+        - x-auth-request-groups
+        - x-auth-request-user
+        includeRequestHeadersInCheck:
+        - authorization
+        - cookie
+        service: oauth2-proxy.oauth2-proxy.svc.cluster.local
+        port: 80
+      name: oauth2-proxy
@@ -0,0 +1,18 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: istiod
+  namespace: istio-system
+spec:
+  template:
+    spec:
+      containers:
+        - name: discovery
+          env:
+            # Istio will refresh the JWT Public Keys provided with
+            # RequestAuthentication by default every 20 minutes. For deployment
+            # from scratch this is not ideal because there is a high chance that
+            # the istiod will be available before dex is available, triggering
+            # Istio mechanism to use a placeholder jwt until refreshed.
+            - name: PILOT_JWT_PUB_KEY_REFRESH_INTERVAL
+              value: "1m"
@@ -0,0 +1,23 @@
+apiVersion: security.istio.io/v1beta1
+kind: AuthorizationPolicy
+metadata:
+  name: istio-ingressgateway-oauth2-proxy
+  namespace: istio-system
+spec:
+  action: CUSTOM
+  provider:
+    name: oauth2-proxy
+  selector:
+    matchLabels:
+      app: istio-ingressgateway
+  rules:
+  - to:
+    - operation:
+        notPaths:
+        - /favicon*
+        - /webcomponentsjs*
+        - /vendor.bundle.js
+        - /app.bundle.js
+        - /dashboard_lib.bundle.js
+        - /assets*
+        - /app.css
@@ -0,0 +1,14 @@
+apiVersion: security.istio.io/v1beta1
+kind: AuthorizationPolicy
+metadata:
+  name: istio-ingressgateway-oauth2-proxy
+  namespace: istio-system
+spec:
+  action: CUSTOM
+  provider:
+    name: oauth2-proxy
+  selector:
+    matchLabels:
+      app: istio-ingressgateway
+  rules:
+  - {}
@@ -0,0 +1,10 @@
+apiVersion: kustomize.config.k8s.io/v1alpha1
+kind: Component
+
+resources:
+- authorizationpolicy.istio-ingressgateway-oauth2-proxy.yaml
+
+# If you're running Kubeflow behind CloudFlare, use 
+# authorizationpolicy.istio-ingressgateway-oauth2-proxy-cloudflare.yaml
+# instead of 
+# authorizationpolicy.istio-ingressgateway-oauth2-proxy.yaml
@@ -0,0 +1,19 @@
+apiVersion: security.istio.io/v1beta1
+kind: RequestAuthentication
+metadata:
+  name: dex-jwt
+  namespace: istio-system
+spec:
+  jwtRules:
+    # forwardOriginalToken must be set to true so the authorization header will
+    # be passed between Kubeflow Components and Istio can configure the
+    # Kubeflow Auth Headers based on this request authorization header.
+    - forwardOriginalToken: true
+      issuer: http://dex.auth.svc.cluster.local:5556/dex
+      # These 5 lines provides integration of istio/oauth2-proxy with
+      # Kubeflow custom auth headers.
+      outputClaimToHeaders:
+        - header: kubeflow-userid
+          claim: email
+        - header: kubeflow-groups
+          claim: groups
@@ -0,0 +1,19 @@
+# Kubernetes M2M Authentication with Istio and RequestAuthentication
+
+## Overview
+
+This kustomize component enables M2M (Machine-to-Machine) authentication in Kubernetes, using
+Istio and the `RequestAuthentication` object. It configures Istio to trust JWTs (JSON Web Tokens)
+in Authorization Bearer tokens when the JWT issuer matches the one in `RequestAuthentication`. The
+default setup uses Kubernetes' self-served OIDC issuer with self-signed certificates.
+
+In Kubernetes clusters managed by platform providers, the OIDC issuer is usually managed by the
+provider and served behind publicly trusted certificates. In these cases, it's advisable to use
+the platform-managed Kubernetes OIDC issuer in the `RequestAuthentication` for seamless integration
+and authentication compliance with the platform's security standards.
+
+For scenarios where the OIDC issuer is served behind self-signed certificates, the kustomize
+overlay using this component should include the `common/oidc-client/oauth2-proxy/components/configure-self-signed-kubernetes-oidc-issuer`
+component. This additional configuration is necessary to handle the self-signed nature of the
+certificates. This setup is the default in the Kustomize overlay defined in `common/oidc-client/oauth2-proxy/overlays/m2m-self-signed`,
+which is tailored for environments with self-signed OIDC issuers.
@@ -0,0 +1,29 @@
+apiVersion: kustomize.config.k8s.io/v1alpha1
+kind: Component
+
+resources:
+- requestauthentication.yaml
+
+configMapGenerator:
+- name: istio-m2m-params
+  envs:
+  - m2m.env
+
+replacements:
+- source:
+    kind: ConfigMap
+    version: v1
+    name: istio-m2m-params
+    fieldPath: data.M2M_TOKEN_ISSUER
+  targets:
+  - fieldPaths:
+    - spec.jwtRules.0.issuer
+    select:
+      group: security.istio.io
+      version: v1beta1
+      kind: RequestAuthentication
+      name: m2m-token-issuer
+      namespace: istio-system
+
+configurations:
+- params.yaml
@@ -0,0 +1 @@
+M2M_TOKEN_ISSUER=issuer
@@ -0,0 +1,21 @@
+apiVersion: v1
+data:
+  M2M_TOKEN_ISSUER: issuer
+kind: ConfigMap
+metadata:
+  name: istio-m2m-params-gfd5m8bd92
+---
+apiVersion: security.istio.io/v1beta1
+kind: RequestAuthentication
+metadata:
+  name: m2m-token-issuer
+  namespace: istio-system
+spec:
+  jwtRules:
+  - forwardOriginalToken: true
+    issuer: issuer
+    outputClaimToHeaders:
+    - claim: sub
+      header: x-auth-request-user
+    - claim: sub
+      header: kubeflow-userid
@@ -0,0 +1,3 @@
+varReference:
+- path: spec/jwtRules/issuer
+  kind: RequestAuthentication
@@ -0,0 +1,14 @@
+apiVersion: security.istio.io/v1beta1
+kind: RequestAuthentication
+metadata:
+  name: m2m-token-issuer
+  namespace: istio-system
+spec:
+  jwtRules:
+    - forwardOriginalToken: true
+      issuer: M2M_TOKEN_ISSUER_PLACEHOLDER
+      outputClaimToHeaders:
+        - claim: sub
+          header: x-auth-request-user
+        - claim: email
+          header: kubeflow-userid