update kubeflow dip-catalog

2025-01-13 02:31:27 +00:00
parent 1dc1181a03
commit 5451f16d72
1959 changed files with 602337 additions and 0 deletions
@@ -0,0 +1,95 @@
+# Istio
+
+## Upgrade Istio Manifests
+
+Istio ships with an installer called `istioctl`, which is a deployment /
+debugging / configuration management tool for Istio all in one package.
+In this section, we explain how to upgrade our istio kustomize packages
+by leveraging `istioctl`. Assuming the new version is `X.Y.Z` and the
+old version is `X1.Y1.Z1`:
+
+1.  Make a copy of the old istio manifests tree, which will become the
+    kustomization for the new Istio version:
+
+        $ export MANIFESTS_SRC=<path/to/manifests/repo>
+        $ export ISTIO_OLD=$MANIFESTS_SRC/common/istio-X1-Y1
+        $ export ISTIO_NEW=$MANIFESTS_SRC/common/istio-X-Y
+        $ cp -a $ISTIO_OLD $ISTIO_NEW
+
+2.  Download `istioctl` for version `X.Y.Z`:
+
+        $ ISTIO_VERSION="X.Y.Z"
+        $ wget "https://github.com/istio/istio/releases/download/${ISTIO_VERSION}/istio-${ISTIO_VERSION}-linux-amd64.tar.gz"
+        $ tar xvfz istio-${ISTIO_VERSION}-linux-amd64.tar.gz
+        # sudo mv istio-${ISTIO_VERSION}/bin/istioctl /usr/local/bin/istioctl
+
+3.  Use `istioctl` to generate an `IstioOperator` resource, the
+    CustomResource used to describe the Istio Control Plane:
+
+        $ cd $ISTIO_NEW
+        $ istioctl profile dump default > profile.yaml
+
+    ---
+    **NOTE**
+
+    `istioctl` comes with a bunch of [predefined profiles](https://istio.io/latest/docs/setup/additional-setup/config-profiles/)
+    (`default`, `demo`, `minimal`, etc.). The `default` profile is installed by default.
+
+    ---
+
+4.  Generate manifests and add them to their respective packages. We
+    will generate manifests using `istioctl`, the
+    `profile.yaml` file from upstream and the
+    `profile-overlay.yaml` file that contains our desired
+    changes:
+
+        $ export PATH="$MANIFESTS_SRC/scripts:$PATH"
+        $ cd $ISTIO_NEW
+        $ istioctl manifest generate --cluster-specific -f profile.yaml -f profile-overlay.yaml > dump.yaml
+        $ ./split-istio-packages -f dump.yaml
+        $ mv $ISTIO_NEW/crd.yaml $ISTIO_NEW/istio-crds/base
+        $ mv $ISTIO_NEW/install.yaml $ISTIO_NEW/istio-install/base
+        $ mv $ISTIO_NEW/cluster-local-gateway.yaml $ISTIO_NEW/cluster-local-gateway/base
+        $ rm dump.yaml
+
+    ---
+    **NOTE**
+
+    `split-istio-packages` is a python script in the same folder as this file.
+    The `ruamel.yaml` version used is 0.16.12.
+
+    `--cluster-specific` is a flag that determines if a current K8s cluster context will be used to dynamically 
+    detect default settings. Ensure you have a target cluster ready before running the above commands. 
+    We set this flag because `istioctl manifest generate` generates manifest files with resources that are no 
+    longer supported in Kubernetes 1.25 (`policy/v1beta1`). See: https://github.com/istio/istio/issues/41220
+    
+    ---
+
+## Changes to Istio's upstream manifests
+
+### Changes to the upstream IstioOperator profile
+
+Changes to Istio's upstream profile `default` are the following:
+
+-   Add a `cluster-local-gateway` component for Kserve. Knative-local-gateway is now obsolete https://github.com/kubeflow/manifests/pull/2355/commits/adc00b804404ea08685a044ae595be0bed9adb59.
+-   Disable the EgressGateway component. We do not use it and it adds unnecessary complexity.
+
+Those changes are captured in the [profile-overlay.yaml](profile-overlay.yaml)
+file.
+
+### Changes to the upstream manifests using kustomize
+
+The Istio kustomizations make the following changes:
+
+- Remove PodDisruptionBudget from `istio-install` and `cluster-local-gateway` kustomizations. See:
+    - https://github.com/istio/istio/issues/12602
+    - https://github.com/istio/istio/issues/24000
+- Add EnvoyFilter for adding an `X-Forwarded-For` header in requests passing through the Istio Ingressgateway, inside the `istio-install` kustomization.
+- Add Istio AuthorizationPolicy to allow all requests to the Istio Ingressgateway and the Istio cluster-local gateway.
+- Add Istio AuthorizationPolicy in Istio's root namespace, so that sidecars deny traffic by default (explicit deny-by-default authorization model).
+- Add Gateway CRs for the Istio Ingressgateway and the Istio cluster-local gateway, as `istioctl` stopped generating them in later versions.
+- Add the istio-system namespace object to `istio-namespace`, as `istioctl` stopped generating it in later versions.
+- Configure TCP KeepAlives.
+- Disable tracing as it causes DNS breakdown. See:
+  https://github.com/istio/istio/issues/29898
+- Set ENABLE_DEBUG_ON_HTTP=false according to https://istio.io/latest/docs/ops/best-practices/security/#control-plane
@@ -0,0 +1,344 @@
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  labels:
+    app: cluster-local-gateway
+    install.operator.istio.io/owning-resource: unknown
+    istio: cluster-local-gateway
+    istio.io/rev: default
+    operator.istio.io/component: IngressGateways
+    release: istio
+  name: cluster-local-gateway-service-account
+  namespace: istio-system
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  labels:
+    app: cluster-local-gateway
+    install.operator.istio.io/owning-resource: unknown
+    istio: cluster-local-gateway
+    istio.io/rev: default
+    operator.istio.io/component: IngressGateways
+    release: istio
+  name: cluster-local-gateway
+  namespace: istio-system
+spec:
+  selector:
+    matchLabels:
+      app: cluster-local-gateway
+      istio: cluster-local-gateway
+  strategy:
+    rollingUpdate:
+      maxSurge: 100%
+      maxUnavailable: 25%
+  template:
+    metadata:
+      annotations:
+        istio.io/rev: default
+        prometheus.io/path: /stats/prometheus
+        prometheus.io/port: '15020'
+        prometheus.io/scrape: 'true'
+        sidecar.istio.io/inject: 'false'
+      labels:
+        app: cluster-local-gateway
+        chart: gateways
+        heritage: Tiller
+        install.operator.istio.io/owning-resource: unknown
+        istio: cluster-local-gateway
+        istio.io/rev: default
+        operator.istio.io/component: IngressGateways
+        release: istio
+        service.istio.io/canonical-name: cluster-local-gateway
+        service.istio.io/canonical-revision: latest
+        sidecar.istio.io/inject: 'false'
+    spec:
+      affinity:
+        nodeAffinity:
+          preferredDuringSchedulingIgnoredDuringExecution:
+          requiredDuringSchedulingIgnoredDuringExecution:
+      containers:
+      - args:
+        - proxy
+        - router
+        - --domain
+        - $(POD_NAMESPACE).svc.cluster.local
+        - --proxyLogLevel=warning
+        - --proxyComponentLogLevel=misc:error
+        - --log_output_level=default:info
+        env:
+        - name: ISTIO_META_ROUTER_MODE
+          value: sni-dnat
+        - name: PILOT_CERT_PROVIDER
+          value: istiod
+        - name: CA_ADDR
+          value: istiod.istio-system.svc:15012
+        - name: NODE_NAME
+          valueFrom:
+            fieldRef:
+              apiVersion: v1
+              fieldPath: spec.nodeName
+        - name: POD_NAME
+          valueFrom:
+            fieldRef:
+              apiVersion: v1
+              fieldPath: metadata.name
+        - name: POD_NAMESPACE
+          valueFrom:
+            fieldRef:
+              apiVersion: v1
+              fieldPath: metadata.namespace
+        - name: INSTANCE_IP
+          valueFrom:
+            fieldRef:
+              apiVersion: v1
+              fieldPath: status.podIP
+        - name: HOST_IP
+          valueFrom:
+            fieldRef:
+              apiVersion: v1
+              fieldPath: status.hostIP
+        - name: ISTIO_CPU_LIMIT
+          valueFrom:
+            resourceFieldRef:
+              resource: limits.cpu
+        - name: SERVICE_ACCOUNT
+          valueFrom:
+            fieldRef:
+              fieldPath: spec.serviceAccountName
+        - name: ISTIO_META_WORKLOAD_NAME
+          value: cluster-local-gateway
+        - name: ISTIO_META_OWNER
+          value: 
+            kubernetes://apis/apps/v1/namespaces/istio-system/deployments/cluster-local-gateway
+        - name: ISTIO_META_MESH_ID
+          value: cluster.local
+        - name: TRUST_DOMAIN
+          value: cluster.local
+        - name: ISTIO_META_UNPRIVILEGED_POD
+          value: 'true'
+        - name: ISTIO_META_CLUSTER_ID
+          value: Kubernetes
+        - name: ISTIO_META_NODE_NAME
+          valueFrom:
+            fieldRef:
+              fieldPath: spec.nodeName
+        image: docker.io/istio/proxyv2:1.22.1
+        name: istio-proxy
+        ports:
+        - containerPort: 15020
+          protocol: TCP
+        - containerPort: 8080
+          protocol: TCP
+        - containerPort: 15090
+          name: http-envoy-prom
+          protocol: TCP
+        readinessProbe:
+          failureThreshold: 30
+          httpGet:
+            path: /healthz/ready
+            port: 15021
+            scheme: HTTP
+          initialDelaySeconds: 1
+          periodSeconds: 2
+          successThreshold: 1
+          timeoutSeconds: 1
+        resources:
+          limits:
+            cpu: 2000m
+            memory: 1024Mi
+          requests:
+            cpu: 100m
+            memory: 128Mi
+        securityContext:
+          allowPrivilegeEscalation: false
+          capabilities:
+            drop:
+            - ALL
+          privileged: false
+          readOnlyRootFilesystem: true
+        volumeMounts:
+        - mountPath: /var/run/secrets/workload-spiffe-uds
+          name: workload-socket
+        - mountPath: /var/run/secrets/credential-uds
+          name: credential-socket
+        - mountPath: /var/run/secrets/workload-spiffe-credentials
+          name: workload-certs
+        - mountPath: /etc/istio/proxy
+          name: istio-envoy
+        - mountPath: /etc/istio/config
+          name: config-volume
+        - mountPath: /var/run/secrets/istio
+          name: istiod-ca-cert
+        - mountPath: /var/run/secrets/tokens
+          name: istio-token
+          readOnly: true
+        - mountPath: /var/lib/istio/data
+          name: istio-data
+        - mountPath: /etc/istio/pod
+          name: podinfo
+        - mountPath: /etc/istio/ingressgateway-certs
+          name: ingressgateway-certs
+          readOnly: true
+        - mountPath: /etc/istio/ingressgateway-ca-certs
+          name: ingressgateway-ca-certs
+          readOnly: true
+      securityContext:
+        runAsGroup: 1337
+        runAsNonRoot: true
+        runAsUser: 1337
+      serviceAccountName: cluster-local-gateway-service-account
+      volumes:
+      - emptyDir: {}
+        name: workload-socket
+      - emptyDir: {}
+        name: credential-socket
+      - emptyDir: {}
+        name: workload-certs
+      - configMap:
+          name: istio-ca-root-cert
+        name: istiod-ca-cert
+      - downwardAPI:
+          items:
+          - fieldRef:
+              fieldPath: metadata.labels
+            path: labels
+          - fieldRef:
+              fieldPath: metadata.annotations
+            path: annotations
+        name: podinfo
+      - emptyDir: {}
+        name: istio-envoy
+      - emptyDir: {}
+        name: istio-data
+      - name: istio-token
+        projected:
+          sources:
+          - serviceAccountToken:
+              audience: istio-ca
+              expirationSeconds: 43200
+              path: istio-token
+      - configMap:
+          name: istio
+          optional: true
+        name: config-volume
+      - name: ingressgateway-certs
+        secret:
+          optional: true
+          secretName: istio-ingressgateway-certs
+      - name: ingressgateway-ca-certs
+        secret:
+          optional: true
+          secretName: istio-ingressgateway-ca-certs
+---
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+  labels:
+    app: cluster-local-gateway
+    install.operator.istio.io/owning-resource: unknown
+    istio: cluster-local-gateway
+    istio.io/rev: default
+    operator.istio.io/component: IngressGateways
+    release: istio
+  name: cluster-local-gateway
+  namespace: istio-system
+spec:
+  minAvailable: 1
+  selector:
+    matchLabels:
+      app: cluster-local-gateway
+      istio: cluster-local-gateway
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  labels:
+    install.operator.istio.io/owning-resource: unknown
+    istio.io/rev: default
+    operator.istio.io/component: IngressGateways
+    release: istio
+  name: cluster-local-gateway-sds
+  namespace: istio-system
+rules:
+- apiGroups:
+  - ''
+  resources:
+  - secrets
+  verbs:
+  - get
+  - watch
+  - list
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  labels:
+    install.operator.istio.io/owning-resource: unknown
+    istio.io/rev: default
+    operator.istio.io/component: IngressGateways
+    release: istio
+  name: cluster-local-gateway-sds
+  namespace: istio-system
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: cluster-local-gateway-sds
+subjects:
+- kind: ServiceAccount
+  name: cluster-local-gateway-service-account
+---
+apiVersion: autoscaling/v2
+kind: HorizontalPodAutoscaler
+metadata:
+  labels:
+    app: cluster-local-gateway
+    install.operator.istio.io/owning-resource: unknown
+    istio: cluster-local-gateway
+    istio.io/rev: default
+    operator.istio.io/component: IngressGateways
+    release: istio
+  name: cluster-local-gateway
+  namespace: istio-system
+spec:
+  maxReplicas: 5
+  metrics:
+  - resource:
+      name: cpu
+      target:
+        averageUtilization: 80
+        type: Utilization
+    type: Resource
+  minReplicas: 1
+  scaleTargetRef:
+    apiVersion: apps/v1
+    kind: Deployment
+    name: cluster-local-gateway
+---
+apiVersion: v1
+kind: Service
+metadata:
+  annotations:
+  labels:
+    app: cluster-local-gateway
+    install.operator.istio.io/owning-resource: unknown
+    istio: cluster-local-gateway
+    istio.io/rev: default
+    operator.istio.io/component: IngressGateways
+    release: istio
+  name: cluster-local-gateway
+  namespace: istio-system
+spec:
+  ports:
+  - name: status-port
+    port: 15020
+    protocol: TCP
+    targetPort: 15020
+  - name: http2
+    port: 80
+    protocol: TCP
+    targetPort: 8080
+  selector:
+    app: cluster-local-gateway
+    istio: cluster-local-gateway
+  type: ClusterIP
@@ -0,0 +1,14 @@
+# Allow all traffic to the cluster-local-gateway
+apiVersion: security.istio.io/v1beta1
+kind: AuthorizationPolicy
+metadata:
+  name: cluster-local-gateway
+spec:
+  action: ALLOW
+  selector:
+    # Same as the cluster-local-gateway Service selector
+    matchLabels:
+      app: cluster-local-gateway
+      istio: cluster-local-gateway
+  rules:
+  - {}
@@ -0,0 +1,17 @@
+apiVersion: networking.istio.io/v1alpha3
+kind: Gateway
+metadata:
+  name: cluster-local-gateway
+  labels:
+    release: istio
+spec:
+  selector:
+    app: cluster-local-gateway
+    istio: cluster-local-gateway
+  servers:
+  - port:
+      number: 80
+      name: http
+      protocol: HTTP
+    hosts:
+    - '*'
@@ -0,0 +1,12 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+namespace: istio-system
+
+resources:
+- cluster-local-gateway.yaml
+- gateway-authorizationpolicy.yaml
+- gateway.yaml
+
+patches:
+- path: patches/remove-pdb.yaml
@@ -0,0 +1,6 @@
+$patch: delete
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+  name: cluster-local-gateway
+  namespace: istio-system
@@ -0,0 +1,5 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+- crd.yaml
+namespace: istio-system
@@ -0,0 +1,9 @@
+# Enforce an explicit deny-by-default authorization model, similar to
+# the deprecated Istio RBAC
+apiVersion: security.istio.io/v1beta1
+kind: AuthorizationPolicy
+metadata:
+  name: global-deny-all
+  namespace: istio-system
+spec:
+  {}
@@ -0,0 +1,17 @@
+apiVersion: networking.istio.io/v1alpha3
+kind: Gateway
+metadata:
+  name: istio-ingressgateway
+  labels:
+    release: istio
+spec:
+  selector:
+    app: istio-ingressgateway
+    istio: ingressgateway
+  servers:
+  - port:
+      number: 80
+      name: http
+      protocol: HTTP
+    hosts:
+    - '*'
@@ -0,0 +1,15 @@
+# Allow all traffic to the istio-ingressgateway
+apiVersion: security.istio.io/v1beta1
+kind: AuthorizationPolicy
+metadata:
+  name: istio-ingressgateway
+  namespace: istio-system
+spec:
+  action: ALLOW
+  selector:
+    # Same as the istio-ingressgateway Service selector
+    matchLabels:
+      app: istio-ingressgateway
+      istio: ingressgateway
+  rules:
+  - {}
@@ -0,0 +1,18 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+namespace: istio-system
+
+resources:
+- install.yaml
+- gateway_authorizationpolicy.yaml
+- deny_all_authorizationpolicy.yaml
+- gateway.yaml
+- x-forwarded-host.yaml
+
+patches:
+- path: patches/service.yaml
+- path: patches/istio-configmap-disable-tracing.yaml
+- path: patches/disable-debugging.yaml
+- path: patches/istio-ingressgateway-remove-pdb.yaml
+- path: patches/istiod-remove-pdb.yaml
@@ -0,0 +1,18 @@
+# Penetration test enahncement: check port 15010 & 8080 in istiod: According to https://istio.io/latest/docs/ops/best-practices/security/#control-plane port 15010
+# is not that problematic (only resource discovery). Other parts of the documentation also say| 15010 | GRPC | XDS and CA services (Plaintext, only for secure networks) | 
+# We have a secure network layer and only XDS is served. 
+# Port 8080 is not listed in the service and even if it would be somehow reachable by IP it only "offers read access". 
+# Nevertheless we set ENABLE_DEBUG_ON_HTTP=false do disable it entirely.
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: istiod
+  namespace: istio-system
+spec:
+  template:
+    spec:
+      containers:
+      - name: discovery
+        env:
+        - name: ENABLE_DEBUG_ON_HTTP
+          value: 'false'
@@ -0,0 +1,20 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: istio
+  namespace: istio-system
+data:
+  # Configuration file for the mesh networks to be used by the Split Horizon EDS.
+  mesh: |-
+    accessLogFile: /dev/stdout
+    defaultConfig:
+      discoveryAddress: istiod.istio-system.svc:15012
+      proxyMetadata: {}
+      tracing: {}
+    enablePrometheusMerge: true
+    rootNamespace: istio-system
+    tcpKeepalive:
+      interval: 5s
+      probes: 3
+      time: 10s
+    trustDomain: cluster.local
@@ -0,0 +1,6 @@
+$patch: delete
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+  name: istio-ingressgateway
+  namespace: istio-system
@@ -0,0 +1,6 @@
+$patch: delete
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+  name: istiod
+  namespace: istio-system
@@ -0,0 +1,7 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: istio-ingressgateway
+  namespace: istio-system
+spec:
+  type: ClusterIP
@@ -0,0 +1,42 @@
+# EnvoyFilter for adding the X-Forwarded-Host header.
+# Needed for the Rok GW to work correctly.
+# Older manifests used an Istio rule, but that relies on Mixer which
+# is deprecated. This way is more performant and up-to-date.
+#
+# TODO: X-Forwarded-Host needs to be applied in two steps:
+# 1. Put old host in `X-Forwarded-Host`
+# 2. Update Host in request
+#
+# This filter only does (1). It can't do (2) because it doesn't know the new
+# host yet. See if we can add an EnvoyFilter for (2). We currently have to do
+# this per VirtualService, in each app that needs it.
+apiVersion: networking.istio.io/v1alpha3
+kind: EnvoyFilter
+metadata:
+  name: x-forwarded-host
+spec:
+  workloadSelector:
+    labels:
+      istio: ingressgateway
+  configPatches:
+    # The first patch adds the lua filter to the listener/http connection manager
+  - applyTo: HTTP_FILTER
+    match:
+      context: GATEWAY
+      listener:
+        filterChain:
+          filter:
+            name: "envoy.http_connection_manager"
+            subFilter:
+              name: "envoy.router"
+    patch:
+      operation: INSERT_BEFORE
+      value:
+       name: envoy.filters.http.lua
+       typed_config:
+          "@type": "type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua"
+          inlineCode: |
+            function envoy_on_request(request_handle)
+              local host = request_handle:headers():get(":authority")
+              request_handle:headers():add("x-forwarded-host", host)
+            end
@@ -0,0 +1,8 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+- ../../base
+
+components:
+- ../../../../oidc-client/oauth2-proxy/components/istio-external-auth-patches
@@ -0,0 +1,6 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+- namespace.yaml
+namespace: istio-system
@@ -0,0 +1,7 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: istio-system
+  labels:
+    istio-operator-managed: Reconcile
+    istio-injection: disabled
@@ -0,0 +1,55 @@
+---
+
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: kubeflow-istio-admin
+  labels:
+    rbac.authorization.kubeflow.org/aggregate-to-kubeflow-admin: "true"
+aggregationRule:
+  clusterRoleSelectors:
+  - matchLabels:
+      rbac.authorization.kubeflow.org/aggregate-to-kubeflow-istio-admin: "true"
+rules: []
+
+---
+
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: kubeflow-istio-edit
+  labels:
+    rbac.authorization.kubeflow.org/aggregate-to-kubeflow-edit: "true"
+    rbac.authorization.kubeflow.org/aggregate-to-kubeflow-istio-admin: "true"
+rules:
+- apiGroups: 
+  - istio.io
+  - networking.istio.io
+  resources: ["*"]
+  verbs:
+  - get
+  - list
+  - watch
+  - create
+  - delete
+  - deletecollection
+  - patch
+  - update
+
+---
+
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: kubeflow-istio-view
+  labels:
+    rbac.authorization.kubeflow.org/aggregate-to-kubeflow-view: "true"
+rules:
+- apiGroups:
+  - istio.io
+  - networking.istio.io
+  resources: ["*"]
+  verbs:
+  - get
+  - list
+  - watch
@@ -0,0 +1,14 @@
+apiVersion: networking.istio.io/v1alpha3
+kind: Gateway
+metadata:
+  name: kubeflow-gateway
+spec:
+  selector:
+    istio: ingressgateway
+  servers:
+  - port:
+      number: 80
+      name: http
+      protocol: HTTP
+    hosts:
+    - "*"
@@ -0,0 +1,6 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+- kf-istio-resources.yaml
+- cluster-roles.yaml
+namespace: kubeflow
@@ -0,0 +1,56 @@
+apiVersion: install.istio.io/v1alpha1
+kind: IstioOperator
+spec:
+  meshConfig:
+    tcpKeepalive:
+      time: 10s
+      interval: 5s
+      probes: 3
+  components:
+    ingressGateways:
+      # Cluster-local gateway for KFServing
+      - enabled: true
+        name: cluster-local-gateway
+        # https://github.com/istio/istio/issues/19263#issuecomment-615833092
+        label:
+          app: cluster-local-gateway
+          istio: cluster-local-gateway
+        k8s:
+          env:
+          - name: ISTIO_META_ROUTER_MODE
+            value: sni-dnat
+          hpaSpec:
+            maxReplicas: 5
+            metrics:
+            - resource:
+                name: cpu
+                target:
+                  type: Utilization
+                  averageUtilization: 80
+              type: Resource
+            minReplicas: 1
+            scaleTargetRef:
+              apiVersion: apps/v1
+              kind: Deployment
+              name: cluster-local-gateway
+          resources:
+            limits:
+              cpu: 2000m
+              memory: 1024Mi
+            requests:
+              cpu: 100m
+              memory: 128Mi
+          service:
+            type: ClusterIP
+            ports:
+            - name: status-port
+              port: 15020
+              targetPort: 15020
+            - name: http2
+              port: 80
+              targetPort: 8080
+
+    # Disable EgressGateway
+    egressGateways:
+    - enabled: false
+      name: istio-egressgateway
@@ -0,0 +1,26 @@
+apiVersion: install.istio.io/v1alpha1
+kind: IstioOperator
+spec:
+  components:
+    base:
+      enabled: true
+    egressGateways:
+    - enabled: false
+      name: istio-egressgateway
+    ingressGateways:
+    - enabled: true
+      name: istio-ingressgateway
+    pilot:
+      enabled: true
+  hub: docker.io/istio
+  profile: default
+  tag: 1.22.1
+  values:
+    defaultRevision: ""
+    gateways:
+      istio-egressgateway: {}
+      istio-ingressgateway: {}
+    global:
+      configValidation: true
+      istioNamespace: istio-system
+
@@ -0,0 +1,72 @@
+#!/usr/bin/env python3
+
+import sys
+import argparse
+import ruamel.yaml
+
+
+DESCRIPTION = """ Separate Istio YAML into separate components.
+
+Separate Istio YAML definitions into four separate components: crds, install
+and cluster-local-gateway.
+"""
+
+
+class YAMLEmitterNoVersionDirective(ruamel.yaml.emitter.Emitter):
+    """YAML Emitter that doesn't emit the YAML version directive."""
+
+    def write_version_directive(self, version_text):
+        """Disable emitting version directive, i.e., %YAML 1.1."""
+        pass
+
+
+class YAML(ruamel.yaml.YAML):
+    """Wrapper of the ruamel.yaml.YAML class with our custom settings."""
+
+    def __init__(self, *args, **kwargs):
+        super(YAML, self).__init__(*args, **kwargs)
+        # XXX: Explicitly set version for producing K8s compatible manifests.
+        # https://yaml.readthedocs.io/en/latest/detail.html#document-version-support
+        self.version = (1, 1)
+        # XXX: Do not emit version directive since tools might fail to
+        # parse manifests.
+        self.Emitter = YAMLEmitterNoVersionDirective
+
+
+yaml = YAML()
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description=DESCRIPTION,
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+    parser.add_argument("-f", "--manifest-file", type=str, required=True,
+                        dest="manifest_file",
+                        help="Istio YAML, generated by istioctl.")
+    return parser.parse_args()
+
+
+def main():
+    args = parse_args()
+    with open(args.manifest_file, "r") as f:
+        objects = [obj for obj in list(yaml.load_all(f)) if obj]
+    crds, install, cluster_local = [], [], []
+    for obj in objects:
+        if obj.get("kind") == "CustomResourceDefinition":
+            crds.append(obj)
+        elif (obj.get("metadata", {}).get("name", "").
+              startswith("cluster-local-gateway")):
+            cluster_local.append(obj)
+        else:
+            install.append(obj)
+
+    with open("crd.yaml", "w") as f:
+        yaml.dump_all(crds, f)
+    with open("install.yaml", "w") as f:
+        yaml.dump_all(install, f)
+    with open("cluster-local-gateway.yaml", "w") as f:
+        yaml.dump_all(cluster_local, f)
+
+
+if __name__ == "__main__":
+    sys.exit(main())