You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
ychangkim
f6f3dd53c5
|
3 weeks ago | |
|---|---|---|
| .. | ||
| templates | 3 weeks ago | |
| CUSTOM-README.md | 3 weeks ago | |
| Chart.yaml | 3 weeks ago | |
| README.md | 3 weeks ago | |
| custom-values.yaml | 3 weeks ago | |
| dip-questions.yaml | 3 weeks ago | |
| dip-values.yaml | 3 weeks ago | |
| values.yaml | 3 weeks ago | |
README.md
vLLM Production Stack helm chart
This helm chart lets users deploy multiple serving engines and a router into the Kubernetes cluster.
Key features
- Support running multiple serving engines with multiple different models
- Load the model weights directly from the existing PersistentVolumes
Prerequisites
- A running Kubernetes cluster with GPU. (You can set it up through
minikube: https://minikube.sigs.k8s.io/docs/tutorials/nvidia/) - Helm
Install the helm chart
helm install llmstack . -f values-example.yaml
Uninstall the deployment
run helm uninstall llmstack
Configure the deployments
See helm/values.yaml for mode details.