update kubeflow dip-catalog
This commit is contained in:
@@ -0,0 +1,135 @@
|
||||
# Kubeflow Authentication using OIDC
|
||||
|
||||
## Istio Envoy Filter
|
||||
|
||||
> EnvoyFilter provides a mechanism to customize the Envoy configuration generated by Istio Pilot. Use EnvoyFilter to modify values for certain fields, add specific filters, or even add entirely new listeners, clusters, etc.[^1]
|
||||
|
||||
Kubeflow will use an Envoy Filter for every incoming request when is used
|
||||
with `oidc-authservice`.
|
||||
|
||||
Usage of EnvoyFilter is currently not recommended. The preferred method for configuring External
|
||||
Authentication in Istio is the `envoyExtAuthzHttp` extension provider[^2].
|
||||
|
||||
Envoy Filter is set up with [oidc-authservice](https://github.com/arrikto/oidc-authservice).
|
||||
|
||||
## Istio envoyExtAuthzHttp
|
||||
|
||||
This is Istio's recommended approach for External Authorization[^2]. It is not limited to the use
|
||||
of `oauth2-proxy`[^3] alone. This method is an industry standard, meeting all of Kubeflow's
|
||||
current and foreseeable authentication needs.
|
||||
|
||||
## Kubeflow Pipelines User and M2M Authentication and Authorization
|
||||
|
||||
Kubeflow Pipelines component relies on the built-in kubernetes functionalities to authenticate and authorize
|
||||
user requests, specifically the TokenReviews[^4] and SubjectAccessReview[^5].
|
||||
|
||||
The best way to describe how it works is to explain with an example. Lets analyze the flow
|
||||
when client calls API to list the KF Pipeline Runs:
|
||||
|
||||
1. api-server starts endpoints in:
|
||||
|
||||
https://github.com/kubeflow/pipelines/blob/2.0.5/backend/src/apiserver/main.go#L95
|
||||
|
||||
Focusing on the pipelines run service:
|
||||
|
||||
1. Register Run Service:
|
||||
* https://github.com/kubeflow/pipelines/blob/2.0.5/backend/src/apiserver/main.go#L116
|
||||
2. proto RPC definition of ListRunsV1
|
||||
* https://github.com/kubeflow/pipelines/blob/2.0.5/backend/api/v1beta1/run.proto#L80
|
||||
3. code definition of ListRunsV1
|
||||
* https://github.com/kubeflow/pipelines/blob/2.0.5/backend/src/apiserver/server/run_server.go#L226
|
||||
4. ListRunsV1 calls internal method `listRuns`
|
||||
* https://github.com/kubeflow/pipelines/blob/2.0.5/backend/src/apiserver/server/run_server.go#L188
|
||||
5. `listRuns` calls internal method `canAccessRun` which itself calls `s.resourceManager.IsAuthorized`
|
||||
* https://github.com/kubeflow/pipelines/blob/2.0.5/backend/src/apiserver/server/run_server.go#L637
|
||||
6. `ResourceManager.IsAuthorized` first tries to authenticate over every available authenticator, which are the `TokenReviewAuthenticator` and `HTTPHeaderAuthenticator`
|
||||
* here the user identity is either the user email provided directly in the `kubeflow-userid` header or the user identity obtained from provided token
|
||||
* https://github.com/kubeflow/pipelines/blob/master/backend/src/apiserver/resource/resource_manager.go#L1667
|
||||
7. `TokenReviewAuthenticator.GetUserIdentity` gets the token from `Authorization` header and calls the K8s Auth `authv1.TokenReview` with given token which in return provides `userInfo := review.Status.User`. `GetUserIdentity` return `userInfo.Username` which at this point is the `system:serviceaccount:default:default`.
|
||||
* https://github.com/kubeflow/pipelines/blob/2.0.5/backend/src/apiserver/auth/authenticator_token_review.go#L53
|
||||
8. Next in `ResourceManager.IsAuthorized` a SubjectAccessReview is created with `r.subjectAccessReviewClient.Create` with arguments specifying RBAC verbs provided in code definition of `RunServer.listRuns`. If the user (sa) is not authorized, an error is thrown
|
||||
* https://github.com/kubeflow/pipelines/blob/master/backend/src/apiserver/resource/resource_manager.go#L1703
|
||||
* if the identity was obtained from token (service account), the `rolebinding.rbac.authorization.k8s.io/default-editor` provides the RBAC permission
|
||||
* if the identity was obtained from header (user), the `rolebinding.rbac.authorization.k8s.io/user-example-com` or similar provides the RBAC permission
|
||||
2. User calls api to list pipeline runs as unauthorized service account.
|
||||
|
||||
* This can be done by running Pod with curl in `default` namespace:
|
||||
```bash
|
||||
$ kubectl -n default run -ti --rm curl --image curlimages/curl --command -- sh
|
||||
# v1beta1
|
||||
~ $ curl "istio-ingressgateway.istio-system/pipeline/apis/v1beta1/runs?resource_reference_key.type=NAMESPACE&resource_reference_key.id=kubeflow-user-example-com" -H "Authorization: Bearer $(cat /run/secrets/kubernetes.io/serviceaccount/token)"
|
||||
{"error":"Failed to list v1beta1 runs: Failed to list runs due to authorization error. Check if you have permission to access namespace kubeflow-user-example-com: Failed to access run . Check if you have access to namespace kubeflow-user-example-com: PermissionDenied: User 'system:serviceaccount:default:default' is not authorized with reason: (request: \u0026ResourceAttributes{Namespace:kubeflow-user-example-com,Verb:list,Group:pipelines.kubeflow.org,Version:v1beta1,Resource:runs,Subresource:,Name:,}): Unauthorized access","code":7,"message":"Failed to list v1beta1 runs: Failed to list runs due to authorization error. Check if you have permission to access namespace kubeflow-user-example-com: Failed to access run . Check if you have access to namespace kubeflow-user-example-com: PermissionDenied: User 'system:serviceaccount:default:default' is not authorized with reason: (request: \u0026ResourceAttributes{Namespace:kubeflow-user-example-com,Verb:list,Group:pipelines.kubeflow.org,Version:v1beta1,Resource:runs,Subresource:,Name:,}): Unauthorized access","details":[{"@type":"type.googleapis.com/google.rpc.Status","code":7,"message":"User 'system:serviceaccount:default:default' is not authorized with reason: (request: \u0026ResourceAttributes{Namespace:kubeflow-user-example-com,Verb:list,Group:pipelines.kubeflow.org,Version:v1beta1,Resource:runs,Subresource:,Name:,})"}]}
|
||||
# v2beta1
|
||||
~ $ curl istio-ingressgateway.istio-system/pipeline/apis/v2beta1/runs?namespace=kubeflow-user-example-com -H "Authorization: Bearer $(cat /run/secrets/kubernetes.io/serviceaccount/token)"
|
||||
{"error":"Failed to list runs: Failed to list runs due to authorization error. Check if you have permission to access namespace kubeflow-user-example-com: Failed to access run . Check if you have access to namespace kubeflow-user-example-com: PermissionDenied: User 'system:serviceaccount:default:default' is not authorized with reason: (request: \u0026ResourceAttributes{Namespace:kubeflow-user-example-com,Verb:list,Group:pipelines.kubeflow.org,Version:v1beta1,Resource:runs,Subresource:,Name:,}): Unauthorized access","code":7,"message":"Failed to list runs: Failed to list runs due to authorization error. Check if you have permission to access namespace kubeflow-user-example-com: Failed to access run . Check if you have access to namespace kubeflow-user-example-com: PermissionDenied: User 'system:serviceaccount:default:default' is not authorized with reason: (request: \u0026ResourceAttributes{Namespace:kubeflow-user-example-com,Verb:list,Group:pipelines.kubeflow.org,Version:v1beta1,Resource:runs,Subresource:,Name:,}): Unauthorized access","details":[{"@type":"type.googleapis.com/google.rpc.Status","code":7,"message":"User 'system:serviceaccount:default:default' is not authorized with reason: (request: \u0026ResourceAttributes{Namespace:kubeflow-user-example-com,Verb:list,Group:pipelines.kubeflow.org,Version:v1beta1,Resource:runs,Subresource:,Name:,})"}]}
|
||||
```
|
||||
3. User calls api to list pipeline runs as authorized service account.
|
||||
|
||||
* This can be done by running Pod with curl in `kubeflow-user-example-com` namespace specifying correct service account:
|
||||
```bash
|
||||
$ kubectl -n kubeflow-user-example-com run -ti --rm curl --image curlimages/curl --command --overrides='{"spec": {"serviceAccountName": "default-editor"}}' -- sh
|
||||
# v1beta1
|
||||
~ $ curl "istio-ingressgateway.istio-system/pipeline/apis/v1beta1/runs?resource_reference_key.type=NAMESPACE&resource_reference_key.id=kubeflow-user-example-com" -H "Authorization: Bearer $(cat /run/secrets/kubernetes.io/serviceaccount/token)"
|
||||
{} # empty response which is fine because no pipeline runs exist
|
||||
# v2beta1
|
||||
~ $ curl istio-ingressgateway.istio-system/pipeline/apis/v2beta1/runs?namespace=kubeflow-user-example-com -H "Authorization: Bearer $(cat /run/secrets/kubernetes.io/serviceaccount/token)"
|
||||
{} # empty response which is fine because no pipeline runs exist
|
||||
```
|
||||
|
||||
### Auth analysis diagram for Kubeflow Pipelines
|
||||
|
||||

|
||||
|
||||
## Kubeflow Notebooks User and M2M Authentication and Authorization
|
||||
|
||||
The underlying mechanism is the same as in Kubeflow Pipelines.
|
||||
|
||||
Similarly, to explain how it works, let's analyze the code step by step, starting from the api route definition
|
||||
for listing notebooks:
|
||||
|
||||
* list notebooks api route definition
|
||||
* https://github.com/kubeflow/kubeflow/blob/v1.8.0/components/crud-web-apps/jupyter/backend/apps/common/routes/get.py#L53
|
||||
* this calls `crud_backend/api/notebook.py::list_notebooks`
|
||||
* `crud_backend/api/notebook.py::list_notebooks` calls `authz.ensure_authorized`
|
||||
* https://github.com/kubeflow/kubeflow/blob/v1.8.0/components/crud-web-apps/common/backend/kubeflow/kubeflow/crud_backend/api/notebook.py#L24
|
||||
* `crud_backend/authz.py::ensure_authorized` calls `crud_backend/authn.py::get_username`
|
||||
* https://github.com/kubeflow/kubeflow/blob/v1.8.0/components/crud-web-apps/common/backend/kubeflow/kubeflow/crud_backend/authz.py#L101
|
||||
* https://github.com/kubeflow/kubeflow/blob/v1.8.0/components/crud-web-apps/common/backend/kubeflow/kubeflow/crud_backend/authn.py#L12
|
||||
* `crud_backend/authn.py::get_username` gets the user id from userid header (email or sa in format `system:serviceaccount:kubeflowusernamespace:default-editor`)
|
||||
* `crud_backend/authz.py::ensure_authorized` calls `crud_backend/authz.py::is_authorized`
|
||||
* https://github.com/kubeflow/kubeflow/blob/v1.8.0/components/crud-web-apps/common/backend/kubeflow/kubeflow/crud_backend/authz.py#L46
|
||||
* this calls `create_subject_access_review` which uses the same mechanism as pipelines with `r.subjectAccessReviewClient.Create`
|
||||
|
||||
## KServe Authentication
|
||||
|
||||
The analysis of KServe auth capabilities suggests that while it's possible to limit access to only authenticated agents,
|
||||
there might be some improvements required to enable access only to authorized agents.
|
||||
|
||||
This is based on the following:
|
||||
|
||||
1. KServe Controller Manager patch integrating kube-rbac-proxy[^6].
|
||||
|
||||
This suggests the kserve **might** use the same mechanism based on
|
||||
`SubjectAccessReviews`. Having a look at the kubeflow/manifests I see it's
|
||||
not enabled.
|
||||
2. Search through the docs and code:
|
||||
|
||||
* https://github.com/kserve/kserve/tree/v0.12.0/docs/samples/istio-dex
|
||||
* https://github.com/kserve/kserve/tree/v0.12.0/docs/samples/gcp-iap
|
||||
|
||||
The docs above mention that while it's possible to enable authentication,
|
||||
authorization is more complicated and probably we need to add
|
||||
`AuthorizationPolicy`...
|
||||
|
||||
> create an [Istio AuthorizationPolicy](https://istio.io/latest/docs/reference/config/security/authorization-policy/) to grant access to the pods or disable it
|
||||
|
||||
Most probably some work is needed to enable authorized access to kserve models.
|
||||
|
||||
## Links
|
||||
|
||||
[^1]: [Envoy Filter](https://istio.io/latest/docs/reference/config/networking/envoy-filter/)
|
||||
[^2]: [External Authorization](https://istio.io/latest/docs/tasks/security/authorization/authz-custom/)
|
||||
[^3]: [oauth2-proxy](https://github.com/oauth2-proxy/oauth2-proxy)
|
||||
[^4]: [Kubernetes TokenReview](https://kubernetes.io/docs/reference/kubernetes-api/authentication-resources/token-review-v1/)
|
||||
[^5]: [Kubernetes SubjectAccessReview](https://kubernetes.io/docs/reference/kubernetes-api/authorization-resources/subject-access-review-v3/)
|
||||
[^6]: [Kube RBAC Proxy](https://github.com/brancz/kube-rbac-proxy)
|
||||
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 636 KiB |
@@ -0,0 +1,3 @@
|
||||
approvers:
|
||||
- axel7083
|
||||
- kromanow94
|
||||
@@ -0,0 +1,231 @@
|
||||
# Kubeflow with oauth2-proxy and envoyExtAuthzHttp
|
||||
|
||||
For a quick install, see [Example installation](#example-installation).
|
||||
|
||||
|
||||
## Description
|
||||
|
||||
Kubeflow authorization operates using custom authentication headers:
|
||||
* `kubeflow-userid`: Contains the user's email address.
|
||||
* `kubeflow-groups`: Holds a comma-separated list of user groups.
|
||||
* Note: The functionality for `kubeflow-groups` is not fully operational at this time.
|
||||
|
||||
This feature was implemented using a custom, minimalistic authorization tool from Arrikto
|
||||
named `oidc-authservice`. This tool was integrated into Istio using `EnvoyFilter`.
|
||||
|
||||
The adoption of `envoyExtAuthzHttp` for authentication offers several advantages:
|
||||
* **Simplified Authorization Process**: `envoyExtAuthzHttp` extension streamlines adding
|
||||
external authorization to the Envoy proxy within an Istio service mesh. It allows for
|
||||
declarative policy configuration, reducing the complexity associated with direct
|
||||
`EnvoyFilter` modifications.
|
||||
* **Standardization**: Istio recommends `envoyExtAuthzHttp` for its standardized approach.
|
||||
This promotes consistency across Istio deployments and simplifies understanding and
|
||||
maintenance of the authorization logic.
|
||||
* **Separate Policy Management**: Authorization policies are defined in distinct
|
||||
configuration resources, not directly in `EnvoyFilter`. This separation eases policy
|
||||
management and facilitates updates or replacements of authorization logic without
|
||||
altering `EnvoyFilter` configurations.
|
||||
* **Seamless Integration with Istio**: `envoyExtAuthzHttp` harmonizes with Istio features
|
||||
like `AuthorizationPolicy` and `VirtualService`, enabling smoother integration within
|
||||
Istio's ecosystem and taking advantage of its architecture and capabilities.
|
||||
* **Future-Proofing**: There is a high likelihood that Kubeflow's internal authentication
|
||||
decisions will transition to relying directly on JWTs instead of custom auth headers.
|
||||
|
||||
Additional information on the benefits of moving away from `EnvoyFilter` can be found in these
|
||||
resources:
|
||||
* [Istio GitHub Issue #27790](https://github.com/istio/istio/issues/27790)
|
||||
* [Istio Documentation on Authorization with Custom Authentication](https://istio.io/latest/docs/tasks/security/authorization/authz-custom/)
|
||||
|
||||
While `envoyExtAuthzHttp` could potentially integrate with `oidc-authservice`, `oauth2-proxy`
|
||||
emerges as a more advanced authentication proxy. It boasts broader community support and is
|
||||
widely used in the industry, including in the official Istio documentation on [External
|
||||
Authorization](https://istio.io/latest/docs/tasks/security/authorization/authz-custom).
|
||||
|
||||
For more details on the `oauth2-proxy`, refer to the [official documentation](https://oauth2-proxy.github.io/oauth2-proxy/docs/behaviour).
|
||||
|
||||
## Available Components
|
||||
|
||||
Below is a list of the available Kustomize Components with brief descriptions. Click on each for more details.
|
||||
|
||||
* **[allow-unauthenticated-issuer-discovery](./allow-unauthenticated-issuer-discovery.md)** -
|
||||
Creates a ClusterRoleBinding for anonymous access to Kubernetes OIDC
|
||||
discovery.
|
||||
|
||||
* **[central-dashboard](./central-dashboard.md)** - Configures the central
|
||||
dashboard to use oauth2-proxy logout URL.
|
||||
|
||||
* **[configure-self-signed-kubernetes-oidc-issuer](./configure-self-signed-kubernetes-oidc-issuer.md)** -
|
||||
Creates a Kubernetes Job with RBAC to discover and configure the in-cluster,
|
||||
self-signed default Kubernetes OIDC issuer, essential for using Kubernetes
|
||||
OIDC for M2M Tokens.
|
||||
|
||||
* **[istio-external-auth](./istio-external-auth.md)** - Modifies Istio
|
||||
configuration to define oauth2-proxy as external authentication middleware via
|
||||
envoyExtAuthzHttp extension provider. Adds RequestAuthentication to trust Dex
|
||||
as IdP and AuthorizationPolicies to delegate authentication to oauth2-proxy.
|
||||
|
||||
* **[istio-m2m](./istio-m2m.md)** - Creates RequestAuthentication for Istio to
|
||||
trust the OIDC Issuer specified in parameters. This allows the generation of
|
||||
JWTs for authenticating requests, typically as Bearer Tokens in the
|
||||
Authorization header. By default, the OIDC Issuer is the in-cluster Kubernetes
|
||||
OIDC.
|
||||
|
||||
|
||||
## OpenShift
|
||||
|
||||
This deployment of `oauth2-proxy` does not support OpenShift out of the box. To facilitate
|
||||
integration with OpenShift:
|
||||
* Utilize the OpenShift-specific distribution of `oauth2-proxy`, accessible at:
|
||||
* [OpenShift OAuth Proxy on GitHub](https://github.com/openshift/oauth-proxy)
|
||||
* Enable RBAC for token reviews by incorporating the `rbac.tokenreviews.yaml` file into
|
||||
your kustomize. For more details, refer to the relevant comment in `kustomization.yaml`.
|
||||
|
||||
## CloudFlare
|
||||
|
||||
CloudFlare requires that certain static, standard web browser assets are accessible without
|
||||
user authentication. This is crucial because CloudFlare aims to cache these assets for
|
||||
enhanced performance. If these assets necessitate user authentication, CloudFlare robots
|
||||
will be redirected to the authentication page, potentially causing access issues with the
|
||||
Kubeflow instance behind CloudFlare.
|
||||
|
||||
This issue can be resolved by defining a set of assets in the Istio `AuthorizationPolicy`
|
||||
that do not require authentication. An example `AuthorizationPolicy` for this purpose is
|
||||
provided in the file `authorizationpolicy.istio-ingressgateway-oauth2-proxy-cloudflare.yaml`.
|
||||
|
||||
## Explaining the Auth Routine
|
||||
|
||||
1. Istio is configured with the `envoyExtAuthzHttp` extension provider pointing to the
|
||||
`oauth2-proxy` service. This configuration enables the use of this extension in
|
||||
`AuthorizationPolicy` for adding external authorization to the service mesh.
|
||||
2. The Istio service mesh has an `AuthorizationPolicy` named `istio-ingressgateway-oauth2-proxy`
|
||||
in the `istio-system` (Istio root) namespace. This policy is set with `action: CUSTOM` and
|
||||
specifies the `oauth2-proxy` provider. Consequently, every request to the
|
||||
`istio-ingressgateway` must pass through the external authorization service, `oauth2-proxy`.
|
||||
3. `oauth2-proxy` decides based on the cookie named `oauth2_proxy_kubeflow`. If this cookie
|
||||
is absent, expired, or invalid, `oauth2-proxy` redirects to the configured OIDC provider,
|
||||
typically `dex`. The authentication redirect includes:
|
||||
* A `redirect_uri` for redirecting the user post-authentication,
|
||||
* A code challenge to guard against interception and replay attacks,
|
||||
* A state parameter to validate the authorization response's authenticity and
|
||||
ensure it originates from the initial request.
|
||||
|
||||
Post-authentication, a cookie is set in the user's browser, used by Istio and `oauth2-proxy`
|
||||
for authorizing requests.
|
||||
|
||||
`oauth2-proxy` also checks for an `Authorization` header with a JWT bearer token. It trusts
|
||||
JWTs issued by the configured OIDC provider (default `dex`) and can be configured to trust
|
||||
additional JWTs. If a valid JWT is present, `oauth2-proxy` forwards the request to Istio
|
||||
along with the authorization header.
|
||||
|
||||
The key objective here is to supply an `Authorization` header with a JWT Bearer Token,
|
||||
later used in `RequestAuthentication`. This step configures Istio to trust the JWT, parse
|
||||
its claims to include user email and groups in custom Kubeflow authorization headers, and
|
||||
make routing and authorization decisions based on these JWT claims.
|
||||
4. With this setup, Istio always receives the JWT in requests, enabling authorization
|
||||
decisions based on JWT claims.
|
||||
|
||||
## Using HTTPS
|
||||
|
||||
`oauth2-proxy` is initially set up with an `http` endpoint, secured by the Istio Service Mesh.
|
||||
As a result, `oauth2-proxy` may default to assuming that the authentication redirect URI should
|
||||
also use `http`. To enforce the use of `https`, modify the variable `FORCE_HTTPS` in
|
||||
`kustomization.yaml` to `true`. This adjustment leverages the `oauth2-proxy` configuration
|
||||
option `--cookie-secure`, ensuring redirection occurs with `https`.
|
||||
|
||||
## Istio JWT Public Key Refresh Interval
|
||||
|
||||
In the initial setup of Kubeflow, it's common for `istiod` to become available before `dex`.
|
||||
Istio's `RequestAuthentication` is configured to retrieve the JWT Public Key from the Issuer
|
||||
URL. If the Issuer is not yet operational, placeholder keys are set, which can render the
|
||||
setup nonfunctional until Istio can access the correct JWT Public Key. To address this,
|
||||
`istiod` is configured with the environment variable `PILOT_JWT_PUB_KEY_REFRESH_INTERVAL="1m"`.
|
||||
This setting ensures the JWT Public Key is refreshed every minute, rather than the default
|
||||
20 minutes.
|
||||
|
||||
Without this configuration, users may encounter the following Istio error:
|
||||
```
|
||||
Jwks doesn't have key to match kid or alg from Jwt
|
||||
```
|
||||
|
||||
## Issues with This Setup
|
||||
|
||||
While not an inherent issue with Istio or `oauth2-proxy`, the current Kubeflow configuration
|
||||
automatically redirects to a URL specified in the logout response body's `afterLogoutURL` key.
|
||||
This behavior stems from custom integration with the `oidc-authservice` component. While
|
||||
`oauth2-proxy` is capable of redirecting to the base Kubeflow page, this custom setup results
|
||||
in users being logged out (with the authentication cookie removed) but not redirected back to
|
||||
the Kubeflow Home Page.
|
||||
|
||||
Details of this custom integration are available at:
|
||||
* [oidc-authservice server.go](https://github.com/arrikto/oidc-authservice/blob/0c4ea9a/server.go#L509)
|
||||
* [Kubeflow logout-button.js](https://github.com/kubeflow/kubeflow/blob/c6c4492/components/centraldashboard/public/components/logout-button.js#L50)
|
||||
|
||||
To log in again, users must manually refresh the page.
|
||||
|
||||
## Example Installation
|
||||
|
||||
To install Kubeflow configured to use `oauth2-proxy` with Istio's `envoyExtAuthzHttp` extension,
|
||||
make the following changes to the `example/kustomization.yaml` file:
|
||||
* use `oauth2-proxy` overlay for istio-install
|
||||
```
|
||||
# from
|
||||
- ../common/istio-1-22/istio-install/base
|
||||
# to
|
||||
- ../common/istio-1-22/istio-install/overlays/oauth2-proxy
|
||||
```
|
||||
* change `OIDC Authservice` to `oauth2-proxy for OIDC` and use overlay for m2m
|
||||
bearer tokens with self-signed in-cluster issuer
|
||||
```
|
||||
# from
|
||||
- ../common/oidc-client/oidc-authservice/base
|
||||
# to
|
||||
- ../common/oidc-client/oauth2-proxy/overlays/m2m-self-signed
|
||||
```
|
||||
* change Dex overlay
|
||||
```
|
||||
# from
|
||||
- ../common/dex/overlays/istio
|
||||
# to
|
||||
- ../common/dex/overlays/oauth2-proxy
|
||||
* change Central Dashboard overlay to use oauth2-proxy for logout
|
||||
```
|
||||
# from
|
||||
- ../apps/centraldashboard/upstream/overlays/kserve
|
||||
# to
|
||||
- ../apps/centraldashboard/upstream/overlays/oauth2-proxy
|
||||
```
|
||||
|
||||
All those changes combined can be done with this single command:
|
||||
```diff
|
||||
$ git apply <<EOF
|
||||
diff --git a/example/kustomization.yaml b/example/kustomization.yaml
|
||||
index c1a85789..4a50440c 100644
|
||||
--- a/example/kustomization.yaml
|
||||
+++ b/example/kustomization.yaml
|
||||
@@ -38,11 +38,11 @@ resources:
|
||||
# Istio
|
||||
- ../common/istio-1-22/istio-crds/base
|
||||
- ../common/istio-1-22/istio-namespace/base
|
||||
-- ../common/istio-1-22/istio-install/base
|
||||
-# OIDC Authservice
|
||||
-- ../common/oidc-client/oidc-authservice/base
|
||||
+- ../common/istio-1-22/istio-install/overlays/oauth2-proxy
|
||||
+# oauth2-proxy for OIDC
|
||||
+- ../common/oidc-client/oauth2-proxy/overlays/m2m-self-signed
|
||||
# Dex
|
||||
-- ../common/dex/overlays/istio
|
||||
+- ../common/dex/overlays/oauth2-proxy
|
||||
# KNative
|
||||
- ../common/knative/knative-serving/overlays/gateways
|
||||
- ../common/knative/knative-eventing/base
|
||||
@@ -60,7 +60,7 @@ resources:
|
||||
# Katib
|
||||
- ../apps/katib/upstream/installs/katib-with-kubeflow
|
||||
# Central Dashboard
|
||||
-- ../apps/centraldashboard/upstream/overlays/kserve
|
||||
+- ../apps/centraldashboard/upstream/overlays/oauth2-proxy
|
||||
# Admission Webhook
|
||||
- ../apps/admission-webhook/upstream/overlays/cert-manager
|
||||
# Jupyter Web App
|
||||
EOF
|
||||
```
|
||||
@@ -0,0 +1,24 @@
|
||||
# oauth2-proxy
|
||||
|
||||
## `oauth2-proxy` Deployment
|
||||
|
||||
This deployment of `oauth2-proxy` has been configured to align closely with the official
|
||||
`oauth2-proxy` Helm installation. This approach facilitates easier integration with any
|
||||
existing `oauth2-proxy` deployments that may already be present on the cluster.
|
||||
|
||||
### Upgrading `oauth2-proxy`
|
||||
|
||||
The `oauth2-proxy` component is designed for easy upgrading, thanks to its foundation on the
|
||||
official `oauth2-proxy` Helm chart. The use of the standard Helm chart simplifies the upgrade
|
||||
process, closely following the upgrades of the official `oauth2-proxy` releases.
|
||||
|
||||
### Stateless Nature of `oauth2-proxy`
|
||||
|
||||
`oauth2-proxy` operates as a stateless application. This statelessness simplifies many
|
||||
aspects of its operation, particularly upgrades, as there are no concerns about complex state
|
||||
management or data migration. Additionally, while `oauth2-proxy` is integrated into the
|
||||
Kubernetes environment, this integration is limited to running the application, thereby
|
||||
minimizing the impact on Kubernetes infrastructure during upgrades.
|
||||
|
||||
These characteristics make the upgrade process for `oauth2-proxy` more predictable and
|
||||
manageable in Kubernetes environments.
|
||||
@@ -0,0 +1,85 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: oauth2-proxy
|
||||
labels:
|
||||
app: oauth2-proxy
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: oauth2-proxy
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: oauth2-proxy
|
||||
spec:
|
||||
volumes:
|
||||
- name: configmain
|
||||
configMap:
|
||||
name: oauth2-proxy
|
||||
defaultMode: 420
|
||||
containers:
|
||||
- name: oauth2-proxy
|
||||
image: quay.io/oauth2-proxy/oauth2-proxy:latest
|
||||
args:
|
||||
- --http-address=0.0.0.0:4180
|
||||
- --config=/etc/oauth2_proxy/oauth2_proxy.cfg
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 4180
|
||||
protocol: TCP
|
||||
- name: metrics
|
||||
containerPort: 44180
|
||||
protocol: TCP
|
||||
env:
|
||||
- name: OAUTH2_PROXY_CLIENT_ID
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: oauth2-proxy
|
||||
key: client-id
|
||||
- name: OAUTH2_PROXY_CLIENT_SECRET
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: oauth2-proxy
|
||||
key: client-secret
|
||||
- name: OAUTH2_PROXY_COOKIE_SECRET
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: oauth2-proxy
|
||||
key: cookie-secret
|
||||
- name: OAUTH2_PROXY_COOKIE_SECURE
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: oauth2-proxy-parameters
|
||||
key: FORCE_HTTPS
|
||||
- name: OAUTH2_PROXY_SSL_INSECURE_SKIP_VERIFY
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: oauth2-proxy-parameters
|
||||
key: ALLOW_SELF_SIGNED_ISSUER
|
||||
- name: OAUTH2_PROXY_SKIP_JWT_BEARER_TOKENS
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: oauth2-proxy-parameters
|
||||
key: ENABLE_M2M_TOKENS
|
||||
- name: OAUTH2_PROXY_EXTRA_JWT_ISSUERS
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: oauth2-proxy-parameters
|
||||
key: EXTRA_JWT_ISSUERS
|
||||
volumeMounts:
|
||||
- name: configmain
|
||||
mountPath: /etc/oauth2_proxy/oauth2_proxy.cfg
|
||||
subPath: oauth2_proxy.cfg
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /ping
|
||||
port: http
|
||||
scheme: HTTP
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /ping
|
||||
port: http
|
||||
scheme: HTTP
|
||||
resources: {}
|
||||
@@ -0,0 +1,64 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
namespace: oauth2-proxy
|
||||
|
||||
resources:
|
||||
- namespace.yaml
|
||||
- deployment.yaml
|
||||
- serviceaccount.yaml
|
||||
- service.yaml
|
||||
- oauth2-proxy-config.yaml
|
||||
# To use oauth2-proxy in OpenShift, use openshift distribution of oauth2-proxy
|
||||
# available here: https://github.com/openshift/oauth-proxy
|
||||
# and add RBAC to enable creation of tokenreviews
|
||||
# Token Reviews for OpenShift
|
||||
# - rbac.tokenreviews.yaml
|
||||
|
||||
secretGenerator:
|
||||
- name: oauth2-proxy
|
||||
type: Opaque
|
||||
envs:
|
||||
- secrets.env
|
||||
|
||||
configMapGenerator:
|
||||
- name: oauth2-proxy-parameters
|
||||
literals:
|
||||
# This will configure oauth2-proxy option --cookie-secure which can force
|
||||
# auth redirect with redirect_uri parameter using https.
|
||||
- FORCE_HTTPS=false
|
||||
|
||||
# If Kubernetes is managed by kind, vCluster, minikube or similar tool,
|
||||
# most probably the Kubernetes OIDC Issuer will be server in-cluster
|
||||
# behind self-signed certs. This option will configure
|
||||
# --ssl-insecure-skip-verify which will accept self-signed-certificates.
|
||||
- ALLOW_SELF_SIGNED_ISSUER=true
|
||||
|
||||
# This will configure oauth2-proxy option --skip-jwt-bearer-tokens which
|
||||
# will pass the requests with 'Authorization' header with Bearer Token
|
||||
# matching --extra-jwt-issuers.
|
||||
- ENABLE_M2M_TOKENS=true
|
||||
|
||||
# Bearer tokens issued by these OIDC Issuers will be verified against the
|
||||
# Issuer and accepted on success.
|
||||
# The format is exactly like in --extra-jwt-issuers.
|
||||
# If extra jwt issuers are meant to be used with m2m bearer tokens,
|
||||
# each issuer has to be added with RequestAuthentication so Istio can
|
||||
# verify, trust and use the JWT. See
|
||||
# 'common/oidc-client/oauth2-proxy/components/istio-m2m' for details.
|
||||
# Examples:
|
||||
# - EXTRA_JWT_ISSUERS=https://kubernetes.default.svc.cluster.local=https://kubernetes.default.svc.cluster.local
|
||||
# - EXTRA_JWT_ISSUERS=https://oidc.eks.region.amazonaws.com/id/1234abcd=https://kubernetes.default.svc
|
||||
- EXTRA_JWT_ISSUERS=
|
||||
|
||||
patches:
|
||||
- target:
|
||||
kind: ConfigMap
|
||||
name: oauth2-proxy
|
||||
path: patch-configmap.yaml
|
||||
|
||||
|
||||
images:
|
||||
- name: quay.io/oauth2-proxy/oauth2-proxy
|
||||
newName: quay.io/oauth2-proxy/oauth2-proxy
|
||||
newTag: v7.6.0
|
||||
@@ -0,0 +1,4 @@
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: oauth2-proxy
|
||||
@@ -0,0 +1,57 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: oauth2-proxy
|
||||
labels:
|
||||
app: oauth2-proxy
|
||||
data:
|
||||
oauth2_proxy.cfg: |
|
||||
provider = "oidc"
|
||||
oidc_issuer_url = "http://dex.auth.svc.cluster.local:5556/dex"
|
||||
scope = "profile email groups openid"
|
||||
upstreams = "static://200"
|
||||
email_domains = [ "*" ]
|
||||
skip_auth_regex=["/dex/.*"]
|
||||
# ---
|
||||
# OIDC Discovery has to be skipped and login url has to be provided directly
|
||||
# in order to enable relative auth redirect.
|
||||
# Turning On OIDC Discovery would set the auth redirect location as the dex
|
||||
# Issuer URL which is http://dex.auth.svc.cluster.local:5556 in the default,
|
||||
# example installation. This address is usuallynot available through the Web
|
||||
# Browser. If you have a setup where dex has it's url as other than the
|
||||
# in-cluster service, this is optional.
|
||||
skip_oidc_discovery = true
|
||||
login_url = "/dex/auth"
|
||||
redeem_url = "http://dex.auth.svc.cluster.local:5556/dex/token"
|
||||
oidc_jwks_url = "http://dex.auth.svc.cluster.local:5556/dex/keys"
|
||||
# ---
|
||||
# Go to dex login page directly instead of showing the oauth2-proxy login
|
||||
# page.
|
||||
skip_provider_button = true
|
||||
# ---
|
||||
# Set Authorization Bearer response header. This is needed in order to
|
||||
# forward the Authorization Bearer token to Istio and enable authorization
|
||||
# based on JWT.
|
||||
set_authorization_header = true
|
||||
# ---
|
||||
# set X-Auth-Request-User, X-Auth-Request-Groups, X-Auth-Request-Email and
|
||||
# X-Auth-Request-Preferred-Username. This is optional for Kubeflow but you
|
||||
# may have other services that use standard auth headers.
|
||||
set_xauthrequest = true
|
||||
# ---
|
||||
cookie_name = "oauth2_proxy_kubeflow"
|
||||
# ---
|
||||
# Dex default cookie expiration is 24h. If set to 168h (default oauth2-proxy),
|
||||
# Istio will not be able to use the JWT after 24h but oauth2-proxy will still
|
||||
# consider the cookie valid.
|
||||
# It's possible to configure the JWT Refresh Token to enable longer login
|
||||
# session.
|
||||
cookie_expire = "24h"
|
||||
cookie_refresh = 0
|
||||
# ---
|
||||
code_challenge_method = "S256"
|
||||
# ---
|
||||
redirect_url = "/oauth2/callback"
|
||||
relative_redirect_url = true
|
||||
|
||||
binaryData: {}
|
||||
@@ -0,0 +1,5 @@
|
||||
varReference:
|
||||
- path: spec/configPatches/patch/value/typed_config/http_service/server_uri/uri
|
||||
kind: EnvoyFilter
|
||||
- path: spec/configPatches/patch/value/typed_config/http_service/server_uri/cluster
|
||||
kind: EnvoyFilter
|
||||
@@ -0,0 +1,53 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: oauth2-proxy
|
||||
labels:
|
||||
app: oauth2-proxy
|
||||
data:
|
||||
oauth2_proxy.cfg: |
|
||||
provider = "keycloak-oidc"
|
||||
oidc_issuer_url = ""
|
||||
scope = "profile email roles openid"
|
||||
upstreams = "static://200"
|
||||
email_domains = [ "*" ]
|
||||
skip_auth_regex=["/dex/.*"]
|
||||
# ---
|
||||
# OIDC Discovery has to be skipped and login url has to be provided directly
|
||||
# in order to enable relative auth redirect.
|
||||
# Turning On OIDC Discovery would set the auth redirect location as the dex
|
||||
# Issuer URL which is http://dex.auth.svc.cluster.local:5556 in the default,
|
||||
# example installation. This address is usuallynot available through the Web
|
||||
# Browser. If you have a setup where dex has it's url as other than the
|
||||
# in-cluster service, this is optional.
|
||||
# ---
|
||||
# Go to dex login page directly instead of showing the oauth2-proxy login
|
||||
# page.
|
||||
skip_provider_button = true
|
||||
# ---
|
||||
# Set Authorization Bearer response header. This is needed in order to
|
||||
# forward the Authorization Bearer token to Istio and enable authorization
|
||||
# based on JWT.
|
||||
set_authorization_header = true
|
||||
# ---
|
||||
# set X-Auth-Request-User, X-Auth-Request-Groups, X-Auth-Request-Email and
|
||||
# X-Auth-Request-Preferred-Username. This is optional for Kubeflow but you
|
||||
# may have other services that use standard auth headers.
|
||||
set_xauthrequest = true
|
||||
# ---
|
||||
cookie_name = "oauth2_proxy_kubeflow"
|
||||
# ---
|
||||
# Dex default cookie expiration is 24h. If set to 168h (default oauth2-proxy),
|
||||
# Istio will not be able to use the JWT after 24h but oauth2-proxy will still
|
||||
# consider the cookie valid.
|
||||
# It's possible to configure the JWT Refresh Token to enable longer login
|
||||
# session.
|
||||
cookie_expire = "24h"
|
||||
cookie_refresh = "5m"
|
||||
# ---
|
||||
code_challenge_method = "S256"
|
||||
# ---
|
||||
redirect_url = "/oauth2/callback"
|
||||
relative_redirect_url = true
|
||||
|
||||
binaryData: {}
|
||||
@@ -0,0 +1,24 @@
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: authn-delegator
|
||||
rules:
|
||||
- apiGroups:
|
||||
- authentication.k8s.io
|
||||
resources:
|
||||
- tokenreviews
|
||||
verbs:
|
||||
- create
|
||||
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: authn-delegators
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: authn-delegator
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: oauth2-proxy
|
||||
@@ -0,0 +1,3 @@
|
||||
client-id=kubeflow-oidc-authservice
|
||||
client-secret=5kD2WwXGUYQ3lEUpy6dkTIVXHpVeemuA
|
||||
cookie-secret=094f9651100c4ee4a3a7337e405d8650
|
||||
@@ -0,0 +1,13 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: oauth2-proxy
|
||||
spec:
|
||||
type: ClusterIP
|
||||
selector:
|
||||
app.kubernetes.io/name: oauth2-proxy
|
||||
ports:
|
||||
- port: 80
|
||||
name: http
|
||||
targetPort: http
|
||||
publishNotReadyAddresses: true
|
||||
@@ -0,0 +1,4 @@
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: oauth2-proxy
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
# Unauthenticated Issuer Discovery
|
||||
|
||||
If you are using Kubernetes with tools like kind, vCluster, minikube, or similar solutions for local
|
||||
development, it's highly likely that the Kubernetes OIDC Issuer operates within the cluster and is
|
||||
secured with self-signed certificates.
|
||||
|
||||
To facilitate the use of m2m tokens with `oauth2-proxy` and Istio, both tools must perform OIDC
|
||||
Connect Discovery on the Token Issuer. Kubernetes offers OIDC Discovery functionality at a specific
|
||||
URI:
|
||||
```
|
||||
https://kubernetes.default.svc.cluster.local/.well-known/openid-configuration
|
||||
```
|
||||
|
||||
Access to this endpoint is blocked by default due to RBAC policies, but it can be enabled by
|
||||
creating a `ClusterRoleBinding`. This binding associates the predefined `ClusterRole`
|
||||
`system:service-account-issuer-discovery` with the `system:unauthenticated` group. The
|
||||
configuration for this is detailed in the resource file `clusterrolebinding.unauthenticated-oidc-viewer.yaml`.
|
||||
|
||||
Once this step is completed, the endpoint can be accessed to reveal the Issuer URL:
|
||||
```bash
|
||||
$ curl -k https://kubernetes.default.svc.cluster.local/.well-known/openid-configuration
|
||||
|
||||
# Example output in kind:
|
||||
{"issuer":"https://kubernetes.default.svc.cluster.local","jwks_uri":"https://172.18.0.5:6443/openid/v1/jwks","response_types_supported":["id_token"],"subject_types_supported":["public"],"id_token_signing_alg_values_supported":["RS256"]}
|
||||
|
||||
# Example output in vCluster:
|
||||
{"issuer":"https://kubernetes.default.svc.cluster.local","jwks_uri":"https://1.2.3.4:6443/openid/v1/jwks","response_types_supported":["id_token"],"subject_types_supported":["public"],"id_token_signing_alg_values_supported":["RS256"]}
|
||||
|
||||
# Example output in AWS EKS:
|
||||
{"issuer":"https://oidc.eks.region.amazonaws.com/id/123abc","jwks_uri":"https://ip-1-2-3-4.eu-central-1.compute.internal:443/openid/v1/jwks","response_types_supported":["id_token"],"subject_types_supported":["public"],"id_token_signing_alg_values_supported":["RS256"]}
|
||||
```
|
||||
|
||||
If you're operating in a `vCluster`, access to the endpoint specified under `jwks_uri` (for example,
|
||||
`https://1.2.3.4:6443/openid/v1/jwks`) is managed separately, and merely creating the previously
|
||||
mentioned `ClusterRoleBinding` is insufficient. To circumvent this limitation, you can configure the
|
||||
`kube-apiserver` to allow anonymous authentication by setting `--anonymous-auth=true`. This is
|
||||
achieved by appending `--kube-apiserver-arg=anonymous-auth=true` to the list of arguments in the
|
||||
Helm Chart Values file:
|
||||
```yaml
|
||||
vcluster:
|
||||
extraArgs:
|
||||
- --kube-apiserver-arg=anonymous-auth=true
|
||||
```
|
||||
+12
@@ -0,0 +1,12 @@
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: unauthenticated-oidc-viewer
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: system:service-account-issuer-discovery
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:unauthenticated
|
||||
+5
@@ -0,0 +1,5 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
|
||||
resources:
|
||||
- clusterrolebinding.unauthenticated-oidc-viewer.yaml
|
||||
+5
@@ -0,0 +1,5 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
|
||||
patches:
|
||||
- path: patches/deployment.logout-url.yaml
|
||||
+19
@@ -0,0 +1,19 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: centraldashboard
|
||||
namespace: kubeflow
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: centraldashboard
|
||||
env:
|
||||
# This successfully logs out the user but the user is not redirected to
|
||||
# the home page. This is because of custom integration with
|
||||
# oidc-authservice which will provide response with 'afterLogoutURL'.
|
||||
# https://github.com/arrikto/oidc-authservice/blob/0c4ea9a/server.go#L509
|
||||
# https://github.com/kubeflow/kubeflow/blob/c6c4492/components/centraldashboard/public/components/logout-button.js#L50
|
||||
# Please refresh the page after logging out.
|
||||
- name: LOGOUT_URL
|
||||
value: /oauth2/sign_out
|
||||
+41
@@ -0,0 +1,41 @@
|
||||
# Configure Istio with Self-Signed Kubernetes OIDC Issuer
|
||||
|
||||
## Overview
|
||||
|
||||
This kustomize component is designed for scenarios where the Kubernetes OIDC Issuer is behind
|
||||
self-signed certificates, causing trust issues with Istio in retrieving the JWKS. It creates a
|
||||
Kubernetes Job and necessary RBAC configurations to address this issue. The job facilitates the
|
||||
retrieval of JWKS from the OIDC Issuer and embeds it directly into Istio's configuration, thereby
|
||||
bypassing trust issues with self-signed certificates.
|
||||
|
||||
## Configuration Persistence and JWKS Public Accessibility
|
||||
|
||||
The configuration created by this job is stored exclusively in etcd and is not persisted elsewhere.
|
||||
This setup is compatible with ArgoCD, which by default does not delete properties absent in the
|
||||
desired manifest. However, if the `RequestAuthentication` resource is modified erroneously or
|
||||
deleted, the configuration would be lost, necessitating a rerun of the job.
|
||||
|
||||
To circumvent the need for rerunning the job, it is advisable to store the JWKS in a repository.
|
||||
JWKS are typically publicly accessible and contain no sensitive information, making them safe for
|
||||
repository storage. By persisting the JWKS in a repository, consistent access is ensured regardless
|
||||
of changes in the cluster or accidental deletions.
|
||||
|
||||
## Functionality
|
||||
|
||||
- **Reading OIDC Issuer URL**: The Job reads the `RequestAuthentication` resource to extract the
|
||||
OIDC Issuer URL.
|
||||
- **Fetching JWKS**: After identifying the Issuer URL, the Job retrieves the JWKS.
|
||||
- **Patching RequestAuthentication**: The JWKS is used to patch the `RequestAuthentication`
|
||||
resource, embedding the JWKS directly.
|
||||
- **Static JWKS Configuration for Istio**: Ensures that Istio uses the JWKS provided by the Job
|
||||
instead of requesting it independently.
|
||||
|
||||
## Use Case
|
||||
|
||||
This setup is particularly useful when:
|
||||
- Kubernetes serves as an OIDC provider.
|
||||
- The Kubernetes API is not served with publicly trusted certificates.
|
||||
|
||||
This component ensures seamless M2M authentication by handling JWKS retrieval and configuration
|
||||
internally, thus circumventing certificate validation issues for Istio fetching JWKS from an OIDC
|
||||
provider with self-signed or private CA certificates.
|
||||
+39
@@ -0,0 +1,39 @@
|
||||
apiVersion: batch/v1
|
||||
kind: CronJob
|
||||
metadata:
|
||||
name: kubeflow-m2m-oidc-configurator
|
||||
namespace: istio-system
|
||||
spec:
|
||||
schedule: '*/5 * * * *'
|
||||
concurrencyPolicy: Forbid
|
||||
jobTemplate:
|
||||
spec:
|
||||
backoffLimit: 3
|
||||
ttlSecondsAfterFinished: 600
|
||||
template:
|
||||
metadata:
|
||||
labels: {}
|
||||
spec:
|
||||
restartPolicy: OnFailure
|
||||
serviceAccountName: kubeflow-m2m-oidc-configurator
|
||||
containers:
|
||||
- image: docker.io/curlimages/curl
|
||||
name: kubeflow-m2m-oidc-configurator
|
||||
command:
|
||||
- /script.sh
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: kubeflow-m2m-oidc-configurator-envs
|
||||
volumeMounts:
|
||||
- mountPath: /script.sh
|
||||
name: script
|
||||
subPath: script.sh
|
||||
resources: {}
|
||||
volumes:
|
||||
- name: script
|
||||
configMap:
|
||||
name: kubeflow-m2m-oidc-configurator-script
|
||||
defaultMode: 0777
|
||||
items:
|
||||
- key: script.sh
|
||||
path: script.sh
|
||||
+24
@@ -0,0 +1,24 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
|
||||
resources:
|
||||
- cronjob.kubeflow-m2m-oidc-configurator.yaml
|
||||
- rbac.yaml
|
||||
|
||||
configMapGenerator:
|
||||
- name: kubeflow-m2m-oidc-configurator-script
|
||||
namespace: istio-system
|
||||
files:
|
||||
- script.sh=script.sh
|
||||
|
||||
- name: kubeflow-m2m-oidc-configurator-envs
|
||||
namespace: istio-system
|
||||
literals:
|
||||
- ISTIO_ROOT_NAMESPACE=istio-system
|
||||
- REQUEST_AUTHENTICATION_NAME=m2m-token-issuer
|
||||
- KUBERNETES_API_SERVER_URL=https://kubernetes.default.svc
|
||||
|
||||
- name: oauth2-proxy-parameters
|
||||
behavior: merge
|
||||
literals:
|
||||
- ALLOW_SELF_SIGNED_ISSUER=true
|
||||
+35
@@ -0,0 +1,35 @@
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: kubeflow-m2m-oidc-configurator
|
||||
namespace: istio-system
|
||||
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: kubeflow-m2m-oidc-configurator
|
||||
namespace: istio-system
|
||||
rules:
|
||||
- apiGroups:
|
||||
- security.istio.io
|
||||
resources:
|
||||
- requestauthentications
|
||||
verbs:
|
||||
- get
|
||||
- patch
|
||||
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: kubeflow-m2m-oidc-configurator
|
||||
namespace: istio-system
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: kubeflow-m2m-oidc-configurator
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: kubeflow-m2m-oidc-configurator
|
||||
namespace: istio-system
|
||||
+143
@@ -0,0 +1,143 @@
|
||||
#!/bin/sh
|
||||
set -e
|
||||
|
||||
KUBERNETES_API_SERVER_URL="${KUBERNETES_API_SERVER_URL:-https://kubernetes.default.svc}"
|
||||
ISTIO_ROOT_NAMESPACE="${ISTIO_ROOT_NAMESPACE:-istio-system}"
|
||||
REQUEST_AUTHENTICATION_NAME="${REQUEST_AUTHENTICATION_NAME:-m2m-token-issuer}"
|
||||
|
||||
RESOURCE_URL="\
|
||||
${KUBERNETES_API_SERVER_URL}\
|
||||
/apis/security.istio.io/v1/namespaces/\
|
||||
${ISTIO_ROOT_NAMESPACE}\
|
||||
/requestauthentications/\
|
||||
${REQUEST_AUTHENTICATION_NAME}"
|
||||
|
||||
wait_for_resource_ready() {
|
||||
while true; do
|
||||
response="$(
|
||||
curl -s -o /dev/null \
|
||||
--url "${RESOURCE_URL}" \
|
||||
-w "%{http_code}" \
|
||||
--header "Authorization: Bearer $(cat /run/secrets/kubernetes.io/serviceaccount/token)" \
|
||||
--insecure
|
||||
)"
|
||||
if [ "${response}" = "200" ]; then
|
||||
break
|
||||
fi
|
||||
sleep 5
|
||||
done
|
||||
}
|
||||
|
||||
get_request_authentication_obj() {
|
||||
curl -s --request GET \
|
||||
--url "${RESOURCE_URL}" \
|
||||
--header "Authorization: Bearer $(cat /run/secrets/kubernetes.io/serviceaccount/token)" \
|
||||
--insecure
|
||||
}
|
||||
|
||||
get_issuer_url_from_obj() {
|
||||
obj="${1}"
|
||||
echo "${obj}" | awk -F'"' '/"issuer":/ { print $4 }'
|
||||
}
|
||||
|
||||
get_current_escaped_jwks_from_obj() {
|
||||
obj="${1}"
|
||||
echo "${obj}" | awk -F'"' '/"jwks":/' | sed -n 's/^.*"jwks": "\(.*\)".*$/\1/p'
|
||||
}
|
||||
|
||||
get_jwks_uri() {
|
||||
issuer_url="${1}"
|
||||
curl -s --request GET \
|
||||
--url "${issuer_url}/.well-known/openid-configuration" \
|
||||
--header "Authorization: Bearer $(cat /run/secrets/kubernetes.io/serviceaccount/token)" \
|
||||
--insecure |
|
||||
grep -o '"jwks_uri":"https:\/\/[^"]\+"' |
|
||||
sed 's/"jwks_uri":"\(.*\)"/\1/'
|
||||
}
|
||||
|
||||
get_jwks_from_uri() {
|
||||
jwks_uri="${1}"
|
||||
curl -s --request GET \
|
||||
--url "${jwks_uri}" \
|
||||
--header "Authorization: Bearer $(cat /run/secrets/kubernetes.io/serviceaccount/token)" \
|
||||
--insecure
|
||||
}
|
||||
|
||||
# Format JWKS in a way that can be accepted in resource patch.
|
||||
parse_escaped_jwks() {
|
||||
jwks="${1}"
|
||||
echo "${jwks}" | sed 's/"/\\"/g'
|
||||
}
|
||||
|
||||
are_jwks_equal() {
|
||||
jwks1="${1}"
|
||||
jwks2="${2}"
|
||||
test "$(echo "${jwks1}" | base64 -w0)" = "$(echo "${jwks2}" | base64 -w0)"
|
||||
}
|
||||
|
||||
patch_request_authentication_with_escaped_jwks() {
|
||||
jwks_escaped="${1}"
|
||||
curl -s --request PATCH \
|
||||
--url "${RESOURCE_URL}" \
|
||||
--header "Content-Type: application/json-patch+json" \
|
||||
--header "Authorization: Bearer $(cat /run/secrets/kubernetes.io/serviceaccount/token)" \
|
||||
-d '[{ "op": "add", "path": "/spec/jwtRules/0/jwks", "value": "'"${jwks_escaped}"'" }]' \
|
||||
--insecure
|
||||
echo
|
||||
}
|
||||
|
||||
patch_request_authentication_with_jwks_if_required() {
|
||||
echo "Getting RequestAuthentication object."
|
||||
REQUEST_AUTHENTICATION_OBJ="$(get_request_authentication_obj)"
|
||||
|
||||
ISSUER_URL="$(get_issuer_url_from_obj "${REQUEST_AUTHENTICATION_OBJ}")"
|
||||
echo "Issuer Url in RequestAuthentication: ${ISSUER_URL}"
|
||||
|
||||
CURRENT_JWKS_ESCAPED="$(get_current_escaped_jwks_from_obj "${REQUEST_AUTHENTICATION_OBJ}")"
|
||||
printf "Current Jwks (escaped):\n%s\n" "${CURRENT_JWKS_ESCAPED}"
|
||||
|
||||
JWKS_URI="$(get_jwks_uri "${ISSUER_URL}")"
|
||||
echo "Jwks Uri from Well Known OpenID Configuration: ${JWKS_URI}"
|
||||
|
||||
JWKS="$(get_jwks_from_uri "${JWKS_URI}")"
|
||||
JWKS_ESCAPED="$(parse_escaped_jwks "${JWKS}")"
|
||||
printf "JWKS from Well Known OpenID Configuration (escaped): \n%s\n" "${JWKS_ESCAPED}"
|
||||
|
||||
if are_jwks_equal "${JWKS_ESCAPED}" "${CURRENT_JWKS_ESCAPED}"; then
|
||||
echo "JWKS in RequestAuthentication ${REQUEST_AUTHENTICATION_NAME} is configured correctly."
|
||||
else
|
||||
echo "JWKS in RequestAuthentication ${REQUEST_AUTHENTICATION_NAME} needs to be configured."
|
||||
patch_request_authentication_with_escaped_jwks "${JWKS_ESCAPED}"
|
||||
fi
|
||||
}
|
||||
|
||||
verify_jwks_in_request_authentication() {
|
||||
REQUEST_AUTHENTICATION_OBJ="$(get_request_authentication_obj)"
|
||||
ISSUER_URL="$(get_issuer_url_from_obj "${REQUEST_AUTHENTICATION_OBJ}")"
|
||||
CURRENT_JWKS_ESCAPED="$(get_current_escaped_jwks_from_obj "${REQUEST_AUTHENTICATION_OBJ}")"
|
||||
JWKS_URI="$(get_jwks_uri "${ISSUER_URL}")"
|
||||
JWKS="$(get_jwks_from_uri "${JWKS_URI}")"
|
||||
JWKS_ESCAPED="$(parse_escaped_jwks "${JWKS}")"
|
||||
if ! are_jwks_equal "${JWKS_ESCAPED}" "${CURRENT_JWKS_ESCAPED}"; then
|
||||
echo "JWKS not properly configured, exit with error code 1"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
main() {
|
||||
echo "Wait until resource RequestAuthentication ${REQUEST_AUTHENTICATION_NAME} in namespace ${ISTIO_ROOT_NAMESPACE} is ready."
|
||||
wait_for_resource_ready
|
||||
echo "Resource RequestAuthentication ${REQUEST_AUTHENTICATION_NAME} in namespace ${ISTIO_ROOT_NAMESPACE} is ready."
|
||||
|
||||
echo "Patch RequestAuthentication with JWKS if required."
|
||||
patch_request_authentication_with_jwks_if_required
|
||||
|
||||
echo "Wait 5 seconds before verifying RequestAuthentication JWKS configuration."
|
||||
sleep 5
|
||||
|
||||
echo "Verify if RequestAuthentication is properly configured with JWKS..."
|
||||
verify_jwks_in_request_authentication
|
||||
echo "RequestAuthentication is properly configured with JWKS."
|
||||
}
|
||||
|
||||
main
|
||||
+6
@@ -0,0 +1,6 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
|
||||
patches:
|
||||
- path: patches/cm.enable-oauth2-proxy.yaml
|
||||
- path: patches/deployment.jwt-refresh-interval.yaml
|
||||
+36
@@ -0,0 +1,36 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: istio
|
||||
namespace: istio-system
|
||||
data:
|
||||
mesh: |-
|
||||
accessLogFile: /dev/stdout
|
||||
defaultConfig:
|
||||
discoveryAddress: istiod.istio-system.svc:15012
|
||||
proxyMetadata: {}
|
||||
tracing: {}
|
||||
enablePrometheusMerge: true
|
||||
rootNamespace: istio-system
|
||||
tcpKeepalive:
|
||||
interval: 5s
|
||||
probes: 3
|
||||
time: 10s
|
||||
trustDomain: cluster.local
|
||||
extensionProviders:
|
||||
- envoyExtAuthzHttp:
|
||||
headersToDownstreamOnDeny:
|
||||
- content-type
|
||||
- set-cookie
|
||||
headersToUpstreamOnAllow:
|
||||
- authorization
|
||||
- path
|
||||
- x-auth-request-email
|
||||
- x-auth-request-groups
|
||||
- x-auth-request-user
|
||||
includeRequestHeadersInCheck:
|
||||
- authorization
|
||||
- cookie
|
||||
service: oauth2-proxy.oauth2-proxy.svc.cluster.local
|
||||
port: 80
|
||||
name: oauth2-proxy
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: istiod
|
||||
namespace: istio-system
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: discovery
|
||||
env:
|
||||
# Istio will refresh the JWT Public Keys provided with
|
||||
# RequestAuthentication by default every 20 minutes. For deployment
|
||||
# from scratch this is not ideal because there is a high chance that
|
||||
# the istiod will be available before dex is available, triggering
|
||||
# Istio mechanism to use a placeholder jwt until refreshed.
|
||||
- name: PILOT_JWT_PUB_KEY_REFRESH_INTERVAL
|
||||
value: "1m"
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
apiVersion: security.istio.io/v1beta1
|
||||
kind: AuthorizationPolicy
|
||||
metadata:
|
||||
name: istio-ingressgateway-oauth2-proxy
|
||||
namespace: istio-system
|
||||
spec:
|
||||
action: CUSTOM
|
||||
provider:
|
||||
name: oauth2-proxy
|
||||
selector:
|
||||
matchLabels:
|
||||
app: istio-ingressgateway
|
||||
rules:
|
||||
- to:
|
||||
- operation:
|
||||
notPaths:
|
||||
- /favicon*
|
||||
- /webcomponentsjs*
|
||||
- /vendor.bundle.js
|
||||
- /app.bundle.js
|
||||
- /dashboard_lib.bundle.js
|
||||
- /assets*
|
||||
- /app.css
|
||||
+14
@@ -0,0 +1,14 @@
|
||||
apiVersion: security.istio.io/v1beta1
|
||||
kind: AuthorizationPolicy
|
||||
metadata:
|
||||
name: istio-ingressgateway-oauth2-proxy
|
||||
namespace: istio-system
|
||||
spec:
|
||||
action: CUSTOM
|
||||
provider:
|
||||
name: oauth2-proxy
|
||||
selector:
|
||||
matchLabels:
|
||||
app: istio-ingressgateway
|
||||
rules:
|
||||
- {}
|
||||
+10
@@ -0,0 +1,10 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
|
||||
resources:
|
||||
- authorizationpolicy.istio-ingressgateway-oauth2-proxy.yaml
|
||||
|
||||
# If you're running Kubeflow behind CloudFlare, use
|
||||
# authorizationpolicy.istio-ingressgateway-oauth2-proxy-cloudflare.yaml
|
||||
# instead of
|
||||
# authorizationpolicy.istio-ingressgateway-oauth2-proxy.yaml
|
||||
+19
@@ -0,0 +1,19 @@
|
||||
apiVersion: security.istio.io/v1beta1
|
||||
kind: RequestAuthentication
|
||||
metadata:
|
||||
name: dex-jwt
|
||||
namespace: istio-system
|
||||
spec:
|
||||
jwtRules:
|
||||
# forwardOriginalToken must be set to true so the authorization header will
|
||||
# be passed between Kubeflow Components and Istio can configure the
|
||||
# Kubeflow Auth Headers based on this request authorization header.
|
||||
- forwardOriginalToken: true
|
||||
issuer: http://dex.auth.svc.cluster.local:5556/dex
|
||||
# These 5 lines provides integration of istio/oauth2-proxy with
|
||||
# Kubeflow custom auth headers.
|
||||
outputClaimToHeaders:
|
||||
- header: kubeflow-userid
|
||||
claim: email
|
||||
- header: kubeflow-groups
|
||||
claim: groups
|
||||
@@ -0,0 +1,19 @@
|
||||
# Kubernetes M2M Authentication with Istio and RequestAuthentication
|
||||
|
||||
## Overview
|
||||
|
||||
This kustomize component enables M2M (Machine-to-Machine) authentication in Kubernetes, using
|
||||
Istio and the `RequestAuthentication` object. It configures Istio to trust JWTs (JSON Web Tokens)
|
||||
in Authorization Bearer tokens when the JWT issuer matches the one in `RequestAuthentication`. The
|
||||
default setup uses Kubernetes' self-served OIDC issuer with self-signed certificates.
|
||||
|
||||
In Kubernetes clusters managed by platform providers, the OIDC issuer is usually managed by the
|
||||
provider and served behind publicly trusted certificates. In these cases, it's advisable to use
|
||||
the platform-managed Kubernetes OIDC issuer in the `RequestAuthentication` for seamless integration
|
||||
and authentication compliance with the platform's security standards.
|
||||
|
||||
For scenarios where the OIDC issuer is served behind self-signed certificates, the kustomize
|
||||
overlay using this component should include the `common/oidc-client/oauth2-proxy/components/configure-self-signed-kubernetes-oidc-issuer`
|
||||
component. This additional configuration is necessary to handle the self-signed nature of the
|
||||
certificates. This setup is the default in the Kustomize overlay defined in `common/oidc-client/oauth2-proxy/overlays/m2m-self-signed`,
|
||||
which is tailored for environments with self-signed OIDC issuers.
|
||||
+29
@@ -0,0 +1,29 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
|
||||
resources:
|
||||
- requestauthentication.yaml
|
||||
|
||||
configMapGenerator:
|
||||
- name: istio-m2m-params
|
||||
envs:
|
||||
- m2m.env
|
||||
|
||||
replacements:
|
||||
- source:
|
||||
kind: ConfigMap
|
||||
version: v1
|
||||
name: istio-m2m-params
|
||||
fieldPath: data.M2M_TOKEN_ISSUER
|
||||
targets:
|
||||
- fieldPaths:
|
||||
- spec.jwtRules.0.issuer
|
||||
select:
|
||||
group: security.istio.io
|
||||
version: v1beta1
|
||||
kind: RequestAuthentication
|
||||
name: m2m-token-issuer
|
||||
namespace: istio-system
|
||||
|
||||
configurations:
|
||||
- params.yaml
|
||||
@@ -0,0 +1 @@
|
||||
M2M_TOKEN_ISSUER=issuer
|
||||
@@ -0,0 +1,21 @@
|
||||
apiVersion: v1
|
||||
data:
|
||||
M2M_TOKEN_ISSUER: issuer
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: istio-m2m-params-gfd5m8bd92
|
||||
---
|
||||
apiVersion: security.istio.io/v1beta1
|
||||
kind: RequestAuthentication
|
||||
metadata:
|
||||
name: m2m-token-issuer
|
||||
namespace: istio-system
|
||||
spec:
|
||||
jwtRules:
|
||||
- forwardOriginalToken: true
|
||||
issuer: issuer
|
||||
outputClaimToHeaders:
|
||||
- claim: sub
|
||||
header: x-auth-request-user
|
||||
- claim: sub
|
||||
header: kubeflow-userid
|
||||
@@ -0,0 +1,3 @@
|
||||
varReference:
|
||||
- path: spec/jwtRules/issuer
|
||||
kind: RequestAuthentication
|
||||
+14
@@ -0,0 +1,14 @@
|
||||
apiVersion: security.istio.io/v1beta1
|
||||
kind: RequestAuthentication
|
||||
metadata:
|
||||
name: m2m-token-issuer
|
||||
namespace: istio-system
|
||||
spec:
|
||||
jwtRules:
|
||||
- forwardOriginalToken: true
|
||||
issuer: M2M_TOKEN_ISSUER_PLACEHOLDER
|
||||
outputClaimToHeaders:
|
||||
- claim: sub
|
||||
header: x-auth-request-user
|
||||
- claim: email
|
||||
header: kubeflow-userid
|
||||
+19
@@ -0,0 +1,19 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- ../../base
|
||||
|
||||
components:
|
||||
- ../../components/istio-m2m
|
||||
- ../../components/istio-external-auth
|
||||
- ../../components/allow-unauthenticated-issuer-discovery
|
||||
- ../../components/configure-self-signed-kubernetes-oidc-issuer
|
||||
|
||||
configMapGenerator:
|
||||
- name: oauth2-proxy-parameters
|
||||
behavior: merge
|
||||
literals:
|
||||
- ALLOW_SELF_SIGNED_ISSUER=true
|
||||
- ENABLE_M2M_TOKENS=true
|
||||
- EXTRA_JWT_ISSUERS=https://kubernetes.default.svc.cluster.local=https://kubernetes.default.svc.cluster.local
|
||||
@@ -0,0 +1,27 @@
|
||||
# Kustomize Overlay for M2M Authentication Integration with Istio
|
||||
|
||||
## Overview
|
||||
|
||||
This kustomize overlay facilitates the integration of M2M (Machine-to-Machine) authentication
|
||||
with Istio in a Kubernetes environment. It includes components designed to enable effective
|
||||
authentication within the service mesh.
|
||||
|
||||
## Components
|
||||
|
||||
The overlay consists of:
|
||||
|
||||
1. **`istio-m2m`**: Configures Istio to trust JWTs in M2M communication, essential for enabling
|
||||
M2M authentication within the service mesh.
|
||||
|
||||
2. **`component-overwrite-m2m-token-issuer`**: This component is used when the OIDC issuer used
|
||||
by Kubernetes is external to the cluster. It allows for the modification of the default
|
||||
issuer URL in the M2M authentication setup to match the externally defined OIDC issuer. This
|
||||
adjustment is necessary due to kustomize's handling of config map generators, requiring a
|
||||
separate component for proper configuration merging.
|
||||
|
||||
## Usage Scenario
|
||||
|
||||
- **External OIDC Issuer Integration**: In cases where the OIDC Issuer integrated with Kubernetes
|
||||
is defined outside the cluster, `component-overwrite-m2m-token-issuer` is used to update the
|
||||
issuer URL for M2M authentication. This scenario is common in setups where an external OIDC
|
||||
issuer is preferred over the default Kubernetes self-served OIDC issuer.
|
||||
+8
@@ -0,0 +1,8 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
|
||||
configMapGenerator:
|
||||
- name: istio-m2m-params
|
||||
behavior: merge
|
||||
literals:
|
||||
- M2M_TOKEN_ISSUER=https://oidc.eks.region.amazonaws.com/id/1234abcd
|
||||
@@ -0,0 +1,18 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- ../../base
|
||||
|
||||
components:
|
||||
- ../../components/istio-m2m
|
||||
- ../../components/istio-external-auth
|
||||
- component-overwrite-m2m-token-issuer
|
||||
|
||||
configMapGenerator:
|
||||
- name: oauth2-proxy-parameters
|
||||
behavior: merge
|
||||
literals:
|
||||
- FORCE_HTTPS=true
|
||||
- ENABLE_M2M_TOKENS=true
|
||||
- EXTRA_JWT_ISSUERS=https://oidc.eks.region.amazonaws.com/id/1234abcd=https://kubernetes.default.svc
|
||||
@@ -0,0 +1,40 @@
|
||||
apiVersion: networking.istio.io/v1alpha3
|
||||
kind: EnvoyFilter
|
||||
metadata:
|
||||
name: authn-filter
|
||||
spec:
|
||||
workloadSelector:
|
||||
labels:
|
||||
istio: ingressgateway
|
||||
configPatches:
|
||||
- applyTo: HTTP_FILTER
|
||||
match:
|
||||
context: GATEWAY
|
||||
listener:
|
||||
filterChain:
|
||||
filter:
|
||||
name: "envoy.filters.network.http_connection_manager"
|
||||
patch:
|
||||
# For some reason, INSERT_FIRST doesn't work
|
||||
operation: INSERT_BEFORE
|
||||
value:
|
||||
# See: https://www.envoyproxy.io/docs/envoy/v1.17.0/configuration/http/http_filters/ext_authz_filter#config-http-filters-ext-authz
|
||||
name: "envoy.filters.http.ext_authz"
|
||||
typed_config:
|
||||
'@type': type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz
|
||||
http_service:
|
||||
server_uri:
|
||||
uri: http://$(AUTHSERVICE_SERVICE).$(AUTHSERVICE_NAMESPACE).svc.cluster.local
|
||||
cluster: outbound|8080||$(AUTHSERVICE_SERVICE).$(AUTHSERVICE_NAMESPACE).svc.cluster.local
|
||||
timeout: 10s
|
||||
authorization_request:
|
||||
allowed_headers:
|
||||
patterns:
|
||||
# XXX: MUST be lowercase!
|
||||
- exact: "authorization"
|
||||
- exact: "cookie"
|
||||
- exact: "x-auth-token"
|
||||
authorization_response:
|
||||
allowed_upstream_headers:
|
||||
patterns:
|
||||
- exact: "kubeflow-userid"
|
||||
@@ -0,0 +1,47 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- rbac.yaml
|
||||
- service.yaml
|
||||
- statefulset.yaml
|
||||
- envoy-filter.yaml
|
||||
- pvc.yaml
|
||||
|
||||
namespace: istio-system
|
||||
|
||||
configMapGenerator:
|
||||
- name: oidc-authservice-parameters
|
||||
envs:
|
||||
- params.env
|
||||
|
||||
secretGenerator:
|
||||
- name: oidc-authservice-client
|
||||
type: Opaque
|
||||
envs:
|
||||
- secret_params.env
|
||||
|
||||
generatorOptions:
|
||||
disableNameSuffixHash: true
|
||||
|
||||
vars:
|
||||
- name: AUTHSERVICE_NAMESPACE
|
||||
objref:
|
||||
kind: Service
|
||||
name: authservice
|
||||
apiVersion: v1
|
||||
fieldref:
|
||||
fieldpath: metadata.namespace
|
||||
- name: AUTHSERVICE_SERVICE
|
||||
objref:
|
||||
kind: Service
|
||||
name: authservice
|
||||
apiVersion: v1
|
||||
fieldref:
|
||||
fieldpath: metadata.name
|
||||
configurations:
|
||||
- params.yaml
|
||||
images:
|
||||
- name: gcr.io/arrikto/kubeflow/oidc-authservice
|
||||
newName: docker.io/kubeflowmanifestswg/oidc-authservice
|
||||
newTag: e236439
|
||||
@@ -0,0 +1,11 @@
|
||||
OIDC_PROVIDER=http://dex.auth.svc.cluster.local:5556/dex
|
||||
OIDC_AUTH_URL=/dex/auth
|
||||
OIDC_SCOPES=profile email groups
|
||||
AUTHSERVICE_URL_PREFIX=/authservice/
|
||||
SKIP_AUTH_URLS=/dex
|
||||
AFTER_LOGOUT_URL=/
|
||||
USERID_HEADER=kubeflow-userid
|
||||
USERID_PREFIX=
|
||||
USERID_CLAIM=email
|
||||
PORT="8080"
|
||||
STORE_PATH=/var/lib/authservice/data.db
|
||||
@@ -0,0 +1,5 @@
|
||||
varReference:
|
||||
- path: spec/configPatches/patch/value/typed_config/http_service/server_uri/uri
|
||||
kind: EnvoyFilter
|
||||
- path: spec/configPatches/patch/value/typed_config/http_service/server_uri/cluster
|
||||
kind: EnvoyFilter
|
||||
@@ -0,0 +1,10 @@
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: authservice-pvc
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 10Gi
|
||||
@@ -0,0 +1,30 @@
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: authservice
|
||||
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: authn-delegator
|
||||
rules:
|
||||
- apiGroups:
|
||||
- authentication.k8s.io
|
||||
resources:
|
||||
- tokenreviews
|
||||
verbs:
|
||||
- create
|
||||
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: authn-delegators
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: authn-delegator
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: authservice
|
||||
@@ -0,0 +1,2 @@
|
||||
CLIENT_ID=kubeflow-oidc-authservice
|
||||
CLIENT_SECRET=pUBnBOY80SnXgjibTYM9ZWNzY2xreNGQok
|
||||
@@ -0,0 +1,13 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: authservice
|
||||
spec:
|
||||
type: ClusterIP
|
||||
selector:
|
||||
app: authservice
|
||||
ports:
|
||||
- port: 8080
|
||||
name: http-authservice
|
||||
targetPort: http-api
|
||||
publishNotReadyAddresses: true
|
||||
@@ -0,0 +1,43 @@
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: oidc-authservice
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: authservice
|
||||
serviceName: authservice
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
sidecar.istio.io/inject: "false"
|
||||
labels:
|
||||
app: authservice
|
||||
spec:
|
||||
serviceAccountName: authservice
|
||||
containers:
|
||||
- name: authservice
|
||||
image: gcr.io/arrikto/kubeflow/oidc-authservice:e236439
|
||||
imagePullPolicy: Always
|
||||
ports:
|
||||
- name: http-api
|
||||
containerPort: 8080
|
||||
envFrom:
|
||||
- secretRef:
|
||||
name: oidc-authservice-client
|
||||
- configMapRef:
|
||||
name: oidc-authservice-parameters
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /var/lib/authservice
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /
|
||||
port: 8081
|
||||
securityContext:
|
||||
fsGroup: 111
|
||||
volumes:
|
||||
- name: data
|
||||
persistentVolumeClaim:
|
||||
claimName: authservice-pvc
|
||||
+10
@@ -0,0 +1,10 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
bases:
|
||||
- ../../base
|
||||
patchesStrategicMerge:
|
||||
- statefulset.yaml
|
||||
images:
|
||||
- name: busybox
|
||||
newTag: "latest"
|
||||
newName: busybox
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: oidc-authservice
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
initContainers:
|
||||
- name: fix-permission
|
||||
image: busybox
|
||||
command: ['sh', '-c']
|
||||
args: ['chmod -R 777 /var/lib/authservice;']
|
||||
volumeMounts:
|
||||
- mountPath: /var/lib/authservice
|
||||
name: data
|
||||
Reference in New Issue
Block a user