# NeMo Microservices Helm Chart

![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square)

For deployment guide, see [Admin Setup](https://docs.nvidia.com/nemo/microservices/latest/set-up/index.html) in the NeMo Microservices documentation.

## Values

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| customizer.customizerConfig.entityStoreURL | string | `"http://nemo-entity-store:8000"` | Specifies the internal K8s DNS record for the NeMo Entity Store service. |
| customizer.customizerConfig.mlflowURL | string | `""` | Specifies the internal K8s DNS record for the MLflow service. |
| customizer.customizerConfig.models | object | `{"meta/llama-3.1-8b-instruct":{"enabled":true}}` | Specifies the model information to configure with the NeMo Customizer microservice. To find a complete list of models that NeMo Customizer supports, see [Model Catalog](https://docs.nvidia.com/nemo/microservices/latest/fine-tune/models/index.html). |
| customizer.customizerConfig.models."meta/llama-3.1-8b-instruct".enabled | bool | `true` | Specifies whether to use the `meta/llama-3.1-8b-instruct` model for a customization job. |
| customizer.customizerConfig.nemoDataStoreURL | string | `"http://nemo-data-store:3000"` | Specifies the internal K8s DNS record for the NeMo Data Store service. |
| customizer.customizerConfig.training.pvc | object | `{"storageClass":""}` | Specifies where to store the customization job information during training. |
| customizer.enabled | bool | `true` | Specifies whether to install the NeMo Customizer microservice. |
| customizer.modelsStorage | object | `{"accessModes":["ReadWriteOnce"],"enabled":true,"size":"20Gi","storageClassName":""}` | Specifies whether to enable the PVC for storing base models. |
| customizer.modelsStorage.accessModes | list | `["ReadWriteOnce"]` | Specifies the access modes for the PVC. |
| customizer.modelsStorage.size | string | `"20Gi"` | Specifies the size of the PVC. |
| customizer.modelsStorage.storageClassName | string | `""` | Specifies the storage class to use for the PVC. |
| customizer.nemoDataStoreTools | object | `{"imagePullSecret":"nvcrimagepullsecret"}` | Specifies the image pull secret for the NeMo Data Store tools. |
| customizer.postgresql.nameOverride | string | `"customizerdb"` | The name override for the PostgreSQL database. |
| customizer.postgresql.serviceAccount.create | bool | `true` | Specifies whether to create a new service account for PostgreSQL. |
| customizer.postgresql.serviceAccount.name | string | `"customizer-postgresql"` |  |
| customizer.serviceName | string | `"nemo-customizer"` |  |
| customizer.workspaceStorage.enabled | bool | `false` | Specifies whether to enable the PVC for storing a training job's workspace. |
| data-store.enabled | bool | `true` | Specifies whether to install the NeMo Data Store microservice. |
| data-store.objectStore.enabled | bool | `false` | Specifies whether to enable object storage for the NeMo Data Store microservice. |
| data-store.persistence.accessModes | list | `["ReadWriteOnce"]` | The access modes for the persistent volume. |
| data-store.persistence.claimName | string | `"datastore-shared-storage"` | The name of the persistent volume claim. |
| data-store.persistence.enabled | bool | `true` | Specifies whether to enable persistent volume for the NeMo Data Store microservice. |
| data-store.persistence.size | string | `"1Gi"` | The size of the persistent volume. |
| data-store.persistence.storageClass | string | `""` | The storage class to use for the persistent volume. |
| data-store.replicaCount | int | `1` | The number of replicas for the NeMo Data Store microservice deployment. |
| data-store.serviceAccount.create | bool | `true` | Specifies whether to create a service account for the NeMo Data Store microservice. |
| data-store.serviceAccount.name | string | `"gitea"` | The service account name for the NeMo Data Store microservice. |
| data-store.serviceName | string | `"nemo-data-store"` | The service name for the NeMo Data Store microservice. |
| data-store.strategy.type | string | `"Recreate"` | The type of deployment strategy to use for the NeMo Data Store microservice. |
| deployment-management.deployments.entityStoreUrl | string | `"http://nemo-entity-store:8000"` | The URL for the NeMo Entity Store service. |
| deployment-management.deployments.modelSyncPeriod | string | `"30"` | The period in seconds for model synchronization. |
| deployment-management.deployments.nimImagePullSecrets | list | `["nvcrimagepullsecret"]` | The image pull secrets for the NeMo Deployment Management microservice. |
| deployment-management.deployments.nimPeftSource | string | `"http://nemo-entity-store:8000"` | The URL for the NIM PEFT source. |
| deployment-management.enabled | bool | `true` | Specifies whether to install the NeMo Deployment Management microservice. |
| deployment-management.image.pullPolicy | string | `"IfNotPresent"` | The pull policy for the NeMo Deployment Management container image. |
| deployment-management.serviceName | string | `"nemo-deployment-management"` | The service name for the NeMo Deployment Management microservice. |
| dgxc-admission-controller.cloudProviderSpec.dummy | object | `{"type":"dummy"}` | The type of the dummy cloud provider. |
| dgxc-admission-controller.dgxcController.image | string | `"nvcr.io/nvidia/nemo-microservices/dgxc-admission-controller:1.642.0"` | The image for the DGX Cloud Admission Controller. |
| dgxc-admission-controller.enableDGXCRbac | bool | `false` |  |
| dgxc-admission-controller.enableIntegrationsSupport | bool | `false` | Specifies whether to enable integrations support. |
| dgxc-admission-controller.enableLabeler | bool | `false` | Specifies whether to enable the labeler. |
| dgxc-admission-controller.enableNetworkPolicies | bool | `false` | Specifies whether to enable network policies. |
| dgxc-admission-controller.enabled | bool | `false` | Specifies whether to install the DGX Cloud Admission Controller. |
| dgxc-admission-controller.imagePullSecret | string | `"nvcrimagepullsecret"` | The image pull secret for the DGX Cloud Admission Controller. |
| dgxc-admission-controller.kubernetesAffinity.enabled | bool | `false` | Specifies whether to enable Kubernetes affinity. |
| entity-store.appConfig.BASE_URL_DATASTORE | string | `"http://nemo-data-store:3000/v1/hf"` | The base URL for the NeMo Data Store microservice. |
| entity-store.appConfig.BASE_URL_NIM | string | `"http://nemo-nim-proxy:8000"` | The base URL for the NIM service used by the NeMo Entity Store microservice. |
| entity-store.enabled | bool | `true` | Specifies whether to install the NeMo Entity Store microservice. |
| entity-store.postgresql.nameOverride | string | `"entity-storedb"` | The name override for the PostgreSQL database. |
| entity-store.postgresql.serviceAccount.create | bool | `true` | Specifies whether to create a new service account for PostgreSQL. |
| entity-store.postgresql.serviceAccount.name | string | `"entity-store-postgresql"` | The service account name for PostgreSQL. |
| entity-store.serviceName | string | `"nemo-entity-store"` | The service name for the NeMo Entity Store microservice. |
| evaluator.argoWorkflows.controller.serviceAccount.name | string | `"argo-workflows-controller"` | The name of the service account for the Argo Workflows controller. |
| evaluator.argoWorkflows.crds.install | bool | `true` | Specifies whether to install the CRDs for Argo Workflows. |
| evaluator.argoWorkflows.enabled | bool | `true` | Specifies whether to enable the default Argo Workflows integration for the NeMo Evaluator microservice. |
| evaluator.argoWorkflows.server.authModes[0] | string | `"server"` |  |
| evaluator.argoWorkflows.server.secure | bool | `true` | Specifies whether to secure the connection to the Argo Workflows server. |
| evaluator.argoWorkflows.server.serviceAccount.name | string | `"argo-workflows-server"` | The name of the service account for the Argo Workflows server. |
| evaluator.argoWorkflows.server.servicePort | int | `2746` | The service port for the Argo Workflows server. |
| evaluator.argoWorkflows.serviceName | string | `"argo-workflows-server"` | The Argo Workflows service name for the NeMo Evaluator microservice. |
| evaluator.enabled | bool | `true` | Specifies whether to install the NeMo Evaluator microservice. |
| evaluator.external.dataStore | object | `{"endpoint":"http://nemo-data-store:3000/v1/hf"}` | The external URL of the NeMo Data Store microservice. |
| evaluator.milvus.standalone.persistence.persistentVolumeClaim.storageClass | string | `""` | Specifies the storage class to use for the Milvus PVC. |
| evaluator.postgresql.nameOverride | string | `"evaluatordb"` | The name override for the default PostgreSQL database. |
| evaluator.postgresql.primary.networkPolicy.enabled | bool | `false` | Specifies whether to enable the network policy for the PostgreSQL service. |
| evaluator.postgresql.serviceAccount.name | string | `"evaluator-postgresql"` | The name of the service account for PostgreSQL. |
| evaluator.serviceName | string | `"nemo-evaluator"` |  |
| existingImagePullSecret | string | `""` | You can specify an existing Kubernetes image pull secret for pulling images from the container registry. The chart doesn't reference any image pull secret if you leave this empty. |
| existingSecret | string | `""` | You can use an existing Kubernetes secret for pulling images. The chart uses the `ngcAPIKey` value to generate the secret if you leave this empty. |
| global.imagePullSecrets | list | `[{"name":"nvcrimagepullsecret"}]` | The name of the image pull secret to use globally across all services. |
| guardrails.enabled | bool | `true` | Specifies whether to install the NeMo Guardrails microservice. |
| guardrails.env | object | `{"NIM_ENDPOINT_URL":"http://nemo-nim-proxy:8000/v1"}` | The environment variables for the NeMo Guardrails microservice. |
| guardrails.env.NIM_ENDPOINT_URL | string | `"http://nemo-nim-proxy:8000/v1"` | The NIM endpoint URL for the NeMo Guardrails microservice. |
| guardrails.serviceName | string | `"nemo-guardrails"` | The service name for the NeMo Guardrails microservice. |
| imagePullSecrets | list | `[{"name":"nvcrimagepullsecret","password":"YOUR-NGC-API-KEY","registry":"nvcr.io","username":"$$oauthtoken"}]` | List of image pull secrets. Existing secrets override these values if you specify them. Use this only for experimentation when you want to hardcode a secret in your values file. |
| ingress.annotations | object | `{}` | Annotations for the ingress resource. |
| ingress.className | string | `""` | The ingress class to use if your cluster has more than one class. |
| ingress.enabled | bool | `false` | Specifies whether to enable the ingress. |
| ingress.hosts | object | `{"dataStore":{"name":"data-store.test","paths":[{"path":"/","pathType":"Prefix","port":3000,"service":"nemo-data-store"}]},"default":{"name":"","paths":[{"path":"/v1/namespaces","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/projects","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/datasets","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/repos","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/models","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/customization","pathType":"Prefix","port":8000,"service":"nemo-customizer"},{"path":"/v1/evaluation","pathType":"Prefix","port":7331,"service":"nemo-evaluator"},{"path":"/v1/guardrail","pathType":"Prefix","port":7331,"service":"nemo-guardrails"},{"path":"/v1/deployment","pathType":"Prefix","port":8000,"service":"nemo-deployment-management"}]},"nimProxy":{"name":"nim.test","paths":[{"path":"/v1/completions","pathType":"Prefix","port":8000,"service":"nemo-nim-proxy"},{"path":"/v1/chat","pathType":"Prefix","port":8000,"service":"nemo-nim-proxy"},{"path":"/v1/models","pathType":"Prefix","port":8000,"service":"nemo-nim-proxy"}]}}` | A map of hosts and their corresponding paths for the ingress. |
| ingress.hosts.dataStore.name | string | `"data-store.test"` | The host name for the third ingress host for the NeMo Data Store microservice. |
| ingress.hosts.dataStore.paths | list | `[{"path":"/","pathType":"Prefix","port":3000,"service":"nemo-data-store"}]` | The path rules for the third ingress host. |
| ingress.hosts.default.name | string | `""` | The host name for the default ingress host. |
| ingress.hosts.default.paths | list | `[{"path":"/v1/namespaces","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/projects","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/datasets","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/repos","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/models","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/customization","pathType":"Prefix","port":8000,"service":"nemo-customizer"},{"path":"/v1/evaluation","pathType":"Prefix","port":7331,"service":"nemo-evaluator"},{"path":"/v1/guardrail","pathType":"Prefix","port":7331,"service":"nemo-guardrails"},{"path":"/v1/deployment","pathType":"Prefix","port":8000,"service":"nemo-deployment-management"}]` | The path rules for the default ingress host. |
| ingress.hosts.nimProxy.name | string | `"nim.test"` | The host name for the second ingress host for the NIM Proxy microservice. |
| ingress.hosts.nimProxy.paths | list | `[{"path":"/v1/completions","pathType":"Prefix","port":8000,"service":"nemo-nim-proxy"},{"path":"/v1/chat","pathType":"Prefix","port":8000,"service":"nemo-nim-proxy"},{"path":"/v1/models","pathType":"Prefix","port":8000,"service":"nemo-nim-proxy"}]` | The path rules for the second ingress host. |
| ingress.tls | list | `[]` | TLS configurations. |
| nemo-operator.controllerManager.manager.scheduler | string | `"volcano"` | The scheduler to use for the NeMo Operator. Available options: `volcano` and `runai`. |
| nemo-operator.enabled | bool | `true` | Specifies whether to install the NeMo Operator microservice. |
| ngcAPIKey | string | `"YOUR-NGC-API-KEY"` | Your NVIDIA GPU Cloud (NGC) API key authenticates and enables pulling images from the NGC container registry. The existing secret overrides this key if you provide one to the `existingSecret` key. |
| nim-operator.enabled | bool | `true` | Specifies whether to install the NIM Operator. |
| nim-operator.nfd.nodeFeatureRules.deviceID | bool | `false` | Specifies whether to enable device ID feature rules. |
| nim-proxy.enabled | bool | `true` | Specifies whether to install the NIM Proxy microservice. |
| nim-proxy.monitoring.enabled | bool | `false` | Specifies whether to enable monitoring for the NIM Proxy microservice. |
| nim-proxy.serviceName | string | `"nemo-nim-proxy"` | The service name for the NIM Proxy microservice. |
| nim.enabled | bool | `false` | Specifies whether to deploy a NIM for LLM during the Helm installation of the chart. You can deploy a single static NIM by enabling this object and its values. When enabled, the chart deploys `meta/llama-3.1-8b-instruct` as the default NIM. |
| nim.env | list | `[{"name":"NIM_PEFT_SOURCE","value":"http://nemo-entity-store:8000"},{"name":"NIM_PEFT_REFRESH_INTERVAL","value":"30"},{"name":"NIM_MAX_CPU_LORAS","value":"16"},{"name":"NIM_MAX_GPU_LORAS","value":"8"}]` | Environment variables for the NIM service. |
| nim.image.repository | string | `"nvcr.io/nim/meta/llama-3.1-8b-instruct"` |  |
| nim.image.tag | string | `"1.8.3"` |  |
| nim.imagePullSecrets[0].name | string | `"nvcrimagepullsecret"` |  |
| nim.model.name | string | `"meta/llama-3.1-8b-instruct"` | The name of the model to deploy as NIM. |
| nim.model.ngcAPISecret | string | `"ngc-api"` | The NGC API secret for model access. |
| nim.persistence.annotations | object | `{"helm.sh/resource-policy":"keep"}` | Annotations for the PVC. |
| nim.persistence.enabled | bool | `true` | Specifies whether to enable persistence volume claim (PVC) for the NIM service. |
| nim.persistence.storageClass | string | `""` | Specifies the storage class to use for the PVC. |
| nim.resources | object | `{"limits":{"nvidia.com/gpu":1},"requests":{"nvidia.com/gpu":1}}` | Specifies resources for the NIM service. |
| nim.resources.requests | object | `{"nvidia.com/gpu":1}` | Specifies requests for the NIM service. |
| nim.service.labels | object | `{"app.nvidia.com/nim-type":"inference"}` | Labels for the NIM service. |
| nim.statefulSet.enabled | bool | `false` | Specifies whether to enable a stateful set for the NIM service. |
| virtualService | object | A virtual service configuration template. | Specifies whether to enable the virtual service. If you are not using istio and virtualservices, it can be useful to create some virtual services for the NeMo Microservices system. Don't enable unless you use istio. |
| virtualService.additional.data-store | object | `{"entries":{"data-store":{"corsPolicy":{},"match":[{"uri":{"prefix":"/"}}],"route":[{"destination":{"host":"nemo-data-store","port":{"number":3000}}}]}},"gateways":[],"hosts":[]}` | Additional virtual service configurations. |
| virtualService.additional.data-store.entries | object | `{"data-store":{"corsPolicy":{},"match":[{"uri":{"prefix":"/"}}],"route":[{"destination":{"host":"nemo-data-store","port":{"number":3000}}}]}}` | The entries for the virtual NeMo Data Store service. |
| virtualService.additional.data-store.entries.data-store.corsPolicy | object | `{}` | The CORS policy for the virtual NeMo Data Store service. |
| virtualService.additional.data-store.entries.data-store.match | list | `[{"uri":{"prefix":"/"}}]` | The match for the virtual NeMo Data Store service. |
| virtualService.additional.data-store.entries.data-store.route | list | `[{"destination":{"host":"nemo-data-store","port":{"number":3000}}}]` | The route for the virtual NeMo Data Store service. |
| virtualService.additional.data-store.gateways | list | `[]` | The gateways for the virtual NeMo Data Store service. |
| virtualService.additional.data-store.hosts | list | `[]` | The hosts for the virtual NeMo Data Store service. |
| virtualService.additional.nim-proxy.entries.nim-proxy.corsPolicy | object | `{}` | The CORS policy for the virtual NIM Proxy service. |
| virtualService.additional.nim-proxy.entries.nim-proxy.match | list | `[{"uri":{"prefix":"/"}}]` | The match for the virtual NIM Proxy service. |
| virtualService.additional.nim-proxy.entries.nim-proxy.route | list | `[{"destination":{"host":"nemo-nim-proxy","port":{"number":8000}}}]` | The route for the virtual NIM Proxy service. |
| virtualService.additional.nim-proxy.gateways | list | `[]` | The gateways for the virtual NIM Proxy service. |
| virtualService.additional.nim-proxy.hosts | list | `[]` | The hosts for the virtual NIM Proxy service. |
| virtualService.annotations | object | `{}` | Annotations for the virtual service. |
| virtualService.enabled | bool | `false` | Specifies whether to enable the virtual service. |
| virtualService.labels | object | `{}` | Labels for the virtual service. |
| virtualService.main.entries.customizer.corsPolicy | object | `{}` | The CORS policy for the virtual NeMo Customizer service. |
| virtualService.main.entries.customizer.match | list | `[{"uri":{"prefix":"/v1/customization"}}]` | The match for the virtual NeMo Customizer service. |
| virtualService.main.entries.customizer.route | list | `[{"destination":{"host":"nemo-customizer","port":{"number":8000}}}]` | The route for the virtual NeMo Customizer service. |
| virtualService.main.entries.deployment-management.corsPolicy | object | `{}` | The CORS policy for the virtual NeMo Deployment Management service. |
| virtualService.main.entries.deployment-management.match | list | `[{"uri":{"prefix":"/v1/deployment"}}]` | The match for the virtual NeMo Deployment Management service. |
| virtualService.main.entries.deployment-management.route | list | `[{"destination":{"host":"nemo-deployment-management","port":{"number":8000}}}]` | The route for the virtual NeMo Deployment Management service. |
| virtualService.main.entries.entity-store.corsPolicy | object | `{}` | The CORS policy for the virtual NeMo Entity Store service. |
| virtualService.main.entries.entity-store.match | list | `[{"uri":{"prefix":"/v1/namespaces"}},{"uri":{"prefix":"/v1/projects"}},{"uri":{"prefix":"/v1/datasets"}},{"uri":{"prefix":"/v1/repos"}},{"uri":{"prefix":"/v1/models"}}]` | The match for the virtual NeMo Entity Store service. |
| virtualService.main.entries.entity-store.route | list | `[{"destination":{"host":"nemo-entity-store","port":{"number":8000}}}]` | The route for the virtual NeMo Entity Store service. |
| virtualService.main.entries.evaluator.corsPolicy | object | `{}` | The CORS policy for the virtual NeMo Evaluator service. |
| virtualService.main.entries.evaluator.match | list | `[{"uri":{"prefix":"/v1/evaluation"}}]` | The match for the virtual NeMo Evaluator service. |
| virtualService.main.entries.evaluator.route | list | `[{"destination":{"host":"nemo-evaluator","port":{"number":7331}}}]` | The route for the virtual NeMo Evaluator service. |
| virtualService.main.entries.guardrails.corsPolicy | object | `{}` | The CORS policy for the virtual NeMo Guardrails service. |
| virtualService.main.entries.guardrails.match | list | `[{"uri":{"prefix":"/v1/guardrail"}}]` | The match for the virtual NeMo Guardrails service. |
| virtualService.main.entries.guardrails.route | list | `[{"destination":{"host":"nemo-guardrails","port":{"number":7331}}}]` | The route for the virtual NeMo Guardrails service. |
| virtualService.main.gateways | list | `[]` | A list of gateways for the virtual service. |
| virtualService.main.hosts | list | `[]` | A list of hosts for the virtual service. |
| volcano.enabled | bool | `false` | Specifies whether to enable the default Volcano scheduler installation. To learn more, see [Volcano](https://docs.nvidia.com/nemo/microservices/latest/set-up/deploy-as-microservices/customizer.html#volcano). |