{"id":513,"date":"2024-06-03T00:29:36","date_gmt":"2024-06-02T23:29:36","guid":{"rendered":"https:\/\/www.alanknipmeyer.phd\/?p=513"},"modified":"2024-06-03T00:30:19","modified_gmt":"2024-06-02T23:30:19","slug":"an-easy-way-to-build-kubernetes-with-gpu-support","status":"publish","type":"post","link":"https:\/\/www.alanknipmeyer.phd\/index.php\/2024\/06\/03\/an-easy-way-to-build-kubernetes-with-gpu-support\/","title":{"rendered":"An easy way to build Kubernetes with GPU Support ?"},"content":{"rendered":"\n

So its been a busy couple of weeks, with writing up and collating information for my first Post Graduate Review (PGR) and actually being going to Bangor for training seminars, but more on that in a few weeks or so when all the sessions are completed !<\/p>\n\n\n\n

Today I’d like to focus on a new to me<\/em> method of running and install Kubernetes on Ubuntu. Many starting with Kubernetes will use, and stick to microk8s<\/a> and it has the ability to use GPU via a single command<\/a>, however I am always keen to try out something new and this week I will focus on kind<\/a><\/p>\n\n\n\n

What is kind and why use it over microk8s \/others ?<\/h2>\n\n\n\n

MicroK8s is excellent, but it can get quite technical, whilst that is not objectionable to me, I like to focus on actually building distributed container images that can work Kubernetes easily, i.e. CI\/CD and MLOps methodologies. What I ‘lose’ on building a ‘production like’ Kubernetes environment, I recoup in time focused on Machine Learning and the ability to use GPU’s easily. For a deeper comparison, I can recommend the page put together by Alperen on ‘Simple Comparison of Lightweight K8S Implementations<\/a>‘. <\/p>\n\n\n\n

Getting started with Kind <\/h2>\n\n\n\n

I stated out with a vanilla install of Ubuntu Server 22.04 and installed the NVIDIA Drivers via the apt repository method. This did put me on a slight older version of CUDA, but sufficient enough to install Docker with GPU support. Documentation for this step is on NVIDIA’s website here<\/a>, I didn’t encounter any issues following the steps provided.<\/p>\n\n\n\n

sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi\nSun Jun  2 22:49:42 2024       \n+-----------------------------------------------------------------------------+\n| NVIDIA-SMI 470.239.06   Driver Version: 470.239.06   CUDA Version: 11.4     |\n|-------------------------------+----------------------+----------------------+\n| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n| Fan  Temp  Perf  Pwr:Usage\/Cap|         Memory-Usage | GPU-Util  Compute M. |\n|                               |                      |               MIG M. |\n|===============================+======================+======================|\n|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N\/A |\n|  0%   48C    P8    14W \/ 184W |      0MiB \/ 12021MiB |      0%      Default |\n|                               |                      |                  N\/A |\n+-------------------------------+----------------------+----------------------+\n                                                                               \n+-----------------------------------------------------------------------------+\n| Processes:                                                                  |\n|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n|        ID   ID                                                   Usage      |\n|=============================================================================|\n|  No running processes found                                                 |\n+-----------------------------------------------------------------------------+<\/code><\/pre>\n\n\n\n
The prerequisite to install Kind is to have Go-lang installed, this was easily done using the Ubuntu package management apt, this page<\/a> was sufficient to install go and without any other dependencies.<\/p>\n\n\n\n
As I was just wanting to get this up and running, I used root for now and will update modify later, but the steps should be the same for a regular user account. Install Kind was as easy os typing a single command, its worth checking to see on the kind repository<\/a> what is the latest version, as of writing its 0.23.0<\/p>\n\n\n\n
PATH=\/root\/go\/bin:$PATH\ngo install sigs.k8s.io\/kind@v0.23.0\nkind create cluster\n<\/code><\/pre>\n\n\n\n
On the ‘dev’ machine I have here it only took a few moments for kind to build all the necessary components to build a fully functioning Kubernetes system.  I tested this out with the standard ‘hello world’ docker containers. Next I wanted to build with GPU support with the NVIDIA runtime support, on a regular Kubernetes cluster, this is quite an involved process, as documented here<\/a>. <\/p>\n\n\n\n
Once I had tested that a ‘standard’ kind Kubernetes install worked correctly, I set about getting the GPU support working. This was far easier than the lengthy process in setting up a Kubernetes cluster on bare-metal with GPU support, I required to install ‘helm’ to load the charts for the GPU. The full installation instructions are provided here<\/a><\/p>\n\n\n\n
kind create cluster --name substratus --config - <<EOF\napiVersion: kind.x-k8s.io\/v1alpha4\nkind: Cluster\nnodes:\n- role: control-plane\n  image: kindest\/node:v1.27.3@sha256:3966ac761ae0136263ffdb6cfd4db23ef8a83cba8a463690e98317add2c9ba72\n  # required for GPU workaround\n  extraMounts:\n    - hostPath: \/dev\/null\n      containerPath: \/var\/run\/nvidia-container-devices\/all\nEOF\n<\/code><\/pre>\n\n\n\n
This process took a few moments and I did run into some issues with “Too Many Open Files”. This was quickly resolved by following the Ubuntu<\/a> and Kubernetes<\/a> fixes for these errors. With a reboot my Kubernetes node was running with GPU support !<\/p>\n\n\n\n
One of my favorite tools I have been introduced to in the last year is k9s, whilst I wont do a deep dive on all the functions it has, I can strongly recommend installing it. I used the ‘snap’ package manager to install k9s<\/p>\n\n\n\n
snap refresh k9s --channel=latest\/stable<\/code><\/pre>\n\n\n\n
It was then super easy to startup k9s and navigate my installation.<\/p>\n\n\n\n
\"\"
K9s running on KIND GPU Node<\/figcaption><\/figure>\n\n\n\n
Testing GPU Support within Kubernetes<\/h2>\n\n\n\n
Whilst my version of CUDA is slightly older, I was able to run a previous version of vectoradd<\/p>\n\n\n\n
kubectl apply -f - << EOF\napiVersion: v1\nkind: Pod\nmetadata:\n  name: cuda-vectoradd\nspec:\n  restartPolicy: OnFailure\n  containers:\n  - name: cuda-vectoradd\n    image: \"nvcr.io\/nvidia\/k8s\/cuda-sample:vectoradd-cuda11.2.1-ubuntu18.04\"\n    resources:\n      limits:\n        nvidia.com\/gpu: 1\nEOF<\/code><\/pre>\n\n\n\n
\"\"
Vector-Add pod running and logs from k9s<\/figcaption><\/figure>\n\n\n\n
Conclusion<\/h2>\n\n\n\n
This was by far the easiest way I have experienced of running Kubernetes with GPU support on Ubuntu, whilst I’m not sure I would use this on a production environment, it works for me as a MLOps\/DevOps engineer that wants to focus on developing Cryptanalysis containers to run Kubernetes, therefore if you want such an environment, I can really recommend it !<\/p>\n\n\n\n
\n