{"id":12962,"date":"2025-12-19T16:36:36","date_gmt":"2025-12-19T07:36:36","guid":{"rendered":"https:\/\/sreake.com\/?p=12962"},"modified":"2026-02-10T16:14:52","modified_gmt":"2026-02-10T07:14:52","slug":"gpu-used-docker-with-nvidia-cuda-image","status":"publish","type":"post","link":"https:\/\/sreake.com\/en\/blog\/gpu-used-docker-with-nvidia-cuda-image\/","title":{"rendered":"Using CUDA in Docker with the nvidia\/cuda image"},"content":{"rendered":"\n<p><em>This blog post is a translation of&nbsp;<a href=\"https:\/\/sreake.com\/blog\/gpu-used-docker-with-nvidia-cuda-image\/\" title=\"\">a Japanese article<\/a>&nbsp;posted on March 27th, 2025.<\/em><\/p>\n\n\n\n<p>Hello! This is Kakutani from the Sreake application team.<\/p>\n\n\n\n<p>Recently, the rise of Machine Learning, Deep Learning and Generative AI made GPUs more important than ever. Large-scale models like Stable Diffusion and ChatGPT require massive computational resources, and using NVIDIA&#8217;s CUDA allows you to fully unlock the power of GPUs!<\/p>\n\n\n\n<p>When developing GPU-enabled applications, using Docker containers makes setting up and managing development environments significantly easier. In this article, we will talk about the <code>nvidia\/cuda<\/code> Docker image, and how it allows us to use CUDA in Docker containers.<\/p>\n\n\n\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_75 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/sreake.com\/en\/blog\/gpu-used-docker-with-nvidia-cuda-image\/#What_is_nvidiacuda\" >What is nvidia\/cuda?<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/sreake.com\/en\/blog\/gpu-used-docker-with-nvidia-cuda-image\/#About_CUDA\" >About CUDA<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/sreake.com\/en\/blog\/gpu-used-docker-with-nvidia-cuda-image\/#Some_CUDA_Use_Cases\" >Some CUDA Use Cases<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/sreake.com\/en\/blog\/gpu-used-docker-with-nvidia-cuda-image\/#Technologies_used_in_nvidiacuda\" >Technologies used in nvidia\/cuda<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/sreake.com\/en\/blog\/gpu-used-docker-with-nvidia-cuda-image\/#Using_GPUs_inside_Docker\" >Using GPUs inside Docker<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/sreake.com\/en\/blog\/gpu-used-docker-with-nvidia-cuda-image\/#Our_Environment\" >Our Environment<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/sreake.com\/en\/blog\/gpu-used-docker-with-nvidia-cuda-image\/#Installing_NVIDIA_Container_Toolkit\" >Installing NVIDIA Container Toolkit<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/sreake.com\/en\/blog\/gpu-used-docker-with-nvidia-cuda-image\/#Our_Dockerfile\" >Our Dockerfile<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/sreake.com\/en\/blog\/gpu-used-docker-with-nvidia-cuda-image\/#Configuring_Docker_Compose\" >Configuring Docker Compose<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/sreake.com\/en\/blog\/gpu-used-docker-with-nvidia-cuda-image\/#Starting_the_Container\" >Starting the Container<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/sreake.com\/en\/blog\/gpu-used-docker-with-nvidia-cuda-image\/#Managing_GPU_Resources\" >Managing GPU Resources<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/sreake.com\/en\/blog\/gpu-used-docker-with-nvidia-cuda-image\/#Measuring_GPU_Usage\" >Measuring GPU Usage<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/sreake.com\/en\/blog\/gpu-used-docker-with-nvidia-cuda-image\/#Summary\" >Summary<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"What_is_nvidiacuda\"><\/span>What is nvidia\/cuda?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p><code>nvidia\/cuda<\/code> is a Docker image officially provided by NVIDIA, designed to easily set up a CUDA environment without having to install anything directly on the host machine.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"About_CUDA\"><\/span>About CUDA<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>What exactly is CUDA, by the way?<\/p>\n\n\n\n<p>CUDA (short for Compute Unified Device Architecture) is a parallel computing platform and programming model developed by NVIDIA. It lets us use GPU resources (GPGPU) for general-purpose processing, all in a C\/C++ environment.<\/p>\n\n\n\n<p>CUDA works by offloading calculations to the GPU, which makes it particularly great at parallel processing tasks (image processing, machine learning and scientific computing). Its speed makes it very attractive, which is probably why it is very well-supported by major machine learning frameworks such as PyTorch and TensorFlow.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Some_CUDA_Use_Cases\"><\/span>Some CUDA Use Cases<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Generative AI:<\/strong> Inference and training of large-scale models like Stable Diffusion, GPT, BERT, and Llama.<\/li>\n\n\n\n<li><strong>Machine Learning &amp; Deep Learning:<\/strong> High-speed execution of frameworks like TensorFlow, PyTorch, and JAX on GPUs.<\/li>\n\n\n\n<li><strong>HPC (High-Performance Computing):<\/strong> Computational processing for scientific simulations and calculations.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Technologies_used_in_nvidiacuda\"><\/span>Technologies used in nvidia\/cuda<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p><code>nvidia\/cuda<\/code> combines various technologies alongside CUDA to efficiently utilize GPU computational resources.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>cuDNN (CUDA Deep Neural Network library)<\/strong>\n<ul class=\"wp-block-list\">\n<li>A library optimized for deep learning.<\/li>\n\n\n\n<li>Optimizes calculations for convolutional layers, pooling layers, and activation functions to achieve high speed.<\/li>\n\n\n\n<li>Improves efficiency particularly for CNN (Convolutional Neural Network) training and inference.<\/li>\n\n\n\n<li>Frequently used in Generative AI for training deep learning models like image generation and speech synthesis.<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li><strong>TensorRT<\/strong>\n<ul class=\"wp-block-list\">\n<li>An NVIDIA library for optimizing inference processing.<\/li>\n\n\n\n<li>Performs model quantization and optimization to improve inference speed.<\/li>\n\n\n\n<li>Ideal for Generative AI applications requiring low latency and high throughput.<\/li>\n\n\n\n<li>Used during inference for Large Language Models (LLMs) like GPT.<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li><strong>cuBLAS (CUDA Basic Linear Algebra Subprograms)<\/strong>\n<ul class=\"wp-block-list\">\n<li>A library for accelerating linear algebra operations.<\/li>\n\n\n\n<li>Optimizes basic linear algebra operations such as matrix calculations, vector operations, and matrix decomposition.<\/li>\n\n\n\n<li>Used in Generative AI for model parameter updates and calculations during inference.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Using_GPUs_inside_Docker\"><\/span>Using GPUs inside Docker<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>In this section, we will configure <a href=\"https:\/\/github.com\/comfyanonymous\/ComfyUI\" target=\"_blank\" rel=\"noreferrer noopener\">ComfyUI<\/a> to run inside a container.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Our_Environment\"><\/span>Our Environment<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>We are using EC2 with a <code>g5.2xlarge<\/code> instance type.<\/p>\n\n\n\n<pre class=\"wp-block-code code-block\"><code># OS version\n$ lsb_release -a\nNo LSB modules are available.\nDistributor ID: Ubuntu\nDescription:    Ubuntu 22.04.5 LTS\nRelease:        22.04\nCodename:       jammy\n\n# CPU info\n$ sudo lshw -class processor\n       description: CPU\n       product: AMD EPYC 7R32\n       vendor: Advanced Micro Devices &#91;AMD]\n       physical id: 4\n       bus info: cpu@0\n       version: 23.49.0\n       slot: CPU 0\n       size: 2800MHz\n       capacity: 3300MHz\n       width: 64 bits\n       clock: 100MHz\n       capabilities: lm fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp x86-64 constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch topoext ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr rdpru wbnoinvd arat npt nrip_save rdpid\n       configuration: cores=4 enabledcores=4 microcode=137367679 threads=8\n\n# GPU info\n$ lspci | grep -i nvidia\n00:1e.0 3D controller: NVIDIA Corporation GA102GL &#91;A10G] (rev a1)\n\n$ lspci -s 00:1e.0 -v\n00:1e.0 3D controller: NVIDIA Corporation GA102GL &#91;A10G] (rev a1)\n        Subsystem: NVIDIA Corporation GA102GL &#91;A10G]\n        Physical Slot: 30\n        Flags: bus master, fast devsel, latency 0, IRQ 10\n        Memory at c0000000 (32-bit, non-prefetchable) &#91;size=16M]\n        Memory at 1000000000 (64-bit, prefetchable) &#91;size=32G]\n        Memory at 840000000 (64-bit, prefetchable) &#91;size=32M]\n        Capabilities: &lt;access denied&gt;\n        Kernel driver in use: nvidia\n        Kernel modules: nvidiafb, nvidia_drm, nvidia\n\n# GPU usage stats\n$ nvidia-smi                                                                   (base) \nTue Mar  4 18:04:47 2025       \n+---------------------------------------------------------------------------------------+\n| NVIDIA-SMI 535.183.01             Driver Version: 535.183.01   CUDA Version: 12.2     |\n|-----------------------------------------+----------------------+----------------------+\n| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n| Fan  Temp   Perf          Pwr:Usage\/Cap |         Memory-Usage | GPU-Util  Compute M. |\n|                                         |                      |               MIG M. |\n|=========================================+======================+======================|\n|   0  NVIDIA A10G                    Off | 00000000:00:1E.0 Off |                    0 |\n|  0%   17C    P8                 9W \/ 300W |      0MiB \/ 23028MiB |      0%      Default |\n|                                         |                      |                  N\/A |\n+-----------------------------------------+----------------------+----------------------+\n                                                                                      \n+---------------------------------------------------------------------------------------+\n| Processes:                                                                            |\n|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n|        ID   ID                                                             Usage      |\n|=======================================================================================|\n|  No running processes found                                                           |\n+---------------------------------------------------------------------------------------+<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Installing_NVIDIA_Container_Toolkit\"><\/span>Installing NVIDIA Container Toolkit<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>The NVIDIA Container Toolkit is a set of tools enabling the use of NVIDIA GPUs within Docker containers. We&#8217;ll install it using Ubuntu&#8217;s package manager.<\/p>\n\n\n\n<pre class=\"wp-block-code code-block\"><code>sudo apt update\nsudo apt install -y nvidia-container-toolkit\nsudo systemctl restart docker<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Our_Dockerfile\"><\/span>Our Dockerfile<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>Here&#8217;s the Dockerfile we&#8217;ll use to run ComfyUI. We set the host to <code>0.0.0.0<\/code> to connect to ComfyUI from outside.<\/p>\n\n\n\n<pre class=\"wp-block-code code-block\"><code>FROM nvidia\/cuda:12.1.0-devel-ubuntu22.04\n\nRUN apt-get update &amp;&amp; apt-get install -y \\\n    python3 \\\n    python3-pip \\\n    git \\\n    &amp;&amp; rm -rf \/var\/lib\/apt\/lists\/*\n\nWORKDIR \/app\nRUN git clone https:\/\/github.com\/comfyanonymous\/ComfyUI.git ComfyUI &amp;&amp; \\\n    cd ComfyUI &amp;&amp; \\\n    pip3 install -r requirements.txt\n\nCMD &#91;\"python3\", \"\/app\/ComfyUI\/main.py\", \"--listen\", \"0.0.0.0\", \"--port\", \"8188\"]<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Configuring_Docker_Compose\"><\/span>Configuring Docker Compose<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>Here&#8217;s the Compose specification we&#8217;ll use to set everything together.<\/p>\n\n\n\n<pre class=\"wp-block-code code-block\"><code>services:\n  cuda-container-comfyui:\n    build:\n      context: .\n      dockerfile: Dockerfile\n    ports:\n      - \"8188:8188\"\n    deploy:\n      resources:\n        # GPU\u30c7\u30d0\u30a4\u30b9\u306e\u4e88\u7d04\u8a2d\u5b9a\n        reservations:\n          devices:\n             # NVIDIA\u30c9\u30e9\u30a4\u30d0\u3092\u4f7f\u7528\u3059\u308b\u30c7\u30d0\u30a4\u30b9\u306e\u8a2d\u5b9a\n            - driver: nvidia\n              device_ids: &#91;\"0\"]\n              capabilities: &#91;all]<\/code><\/pre>\n\n\n\n<p>We use the <code>capabilities<\/code> field to allow access to all GPU features (<code>all<\/code>), but you can be more granular with this setting:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>gpu<\/code>: For basic GPU resources<\/li>\n\n\n\n<li><code>compute<\/code>: For computing and math capabilities<\/li>\n\n\n\n<li><code>display<\/code>: For display (GUI) features<\/li>\n\n\n\n<li><code>video<\/code>: For video processing, encoding, decoding and streaming<\/li>\n\n\n\n<li><code>graphics<\/code>: For graphics processing and 3D rendering<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Starting_the_Container\"><\/span>Starting the Container<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>We can now start the container!<\/p>\n\n\n\n<pre class=\"wp-block-code code-block\"><code>$ docker compose up --build<\/code><\/pre>\n\n\n\n<p>Using the <code>nvidia-smi<\/code> command, we can see our <code>python<\/code> process is using GPU resources:<\/p>\n\n\n\n<pre class=\"wp-block-code code-block\"><code>$ nvidia-smi\n\n+---------------------------------------------------------------------------------------+\n| NVIDIA-SMI 535.183.01             Driver Version: 535.183.01   CUDA Version: 12.2     |\n|-----------------------------------------+----------------------+----------------------+\n| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n| Fan  Temp   Perf          Pwr:Usage\/Cap |         Memory-Usage | GPU-Util  Compute M. |\n|                                         |                      |               MIG M. |\n|=========================================+======================+======================|\n|   0  NVIDIA A10G                    Off | 00000000:00:1E.0 Off |                    0 |\n|  0%   25C    P0             59W \/ 300W |    256MiB \/ 23028MiB |      0%      Default |\n|                                         |                      |                  N\/A |\n+-----------------------------------------+----------------------+----------------------+\n                                                                                      \n+---------------------------------------------------------------------------------------+\n| Processes:                                                                            |\n|  GPU   GI   CI        PID   Type   Process name                            GPU coMemory |\n|        ID   ID                                                             Usage      |\n|=======================================================================================|\n|    0   N\/A  N\/A      34674       C   python3                                   248MiB |\n+---------------------------------------------------------------------------------------+\n<\/code><\/pre>\n\n\n\n<p>And you can now access ComfyUI!<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"604\" src=\"https:\/\/sreake.com\/wp-content\/uploads\/2025\/12\/image-3-1024x604.png\" alt=\"\" class=\"wp-image-12957\" srcset=\"https:\/\/sreake.com\/wp-content\/uploads\/2025\/12\/image-3-1024x604.png 1024w, https:\/\/sreake.com\/wp-content\/uploads\/2025\/12\/image-3-300x177.png 300w, https:\/\/sreake.com\/wp-content\/uploads\/2025\/12\/image-3-768x453.png 768w, https:\/\/sreake.com\/wp-content\/uploads\/2025\/12\/image-3-1536x906.png 1536w, https:\/\/sreake.com\/wp-content\/uploads\/2025\/12\/image-3.png 1634w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Managing_GPU_Resources\"><\/span>Managing GPU Resources<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>The current environment has only one GPU installed. If you have multiple GPUs, you can specify which GPU to allocate with the <code>device_ids<\/code> field.<\/p>\n\n\n\n<pre class=\"wp-block-code code-block\"><code>services:\n  cuda-container-comfyui:\n    build:\n      context: .\n      dockerfile: Dockerfile\n    ports:\n      - \"8188:8188\"\n    deploy:\n      resources:\n        reservations:\n          devices:\n            - driver: nvidia\n              device_ids: &#91;\"0\", \"1\"]  # Use the first and second GPUs in your environment\n              capabilities: &#91;all]<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Measuring_GPU_Usage\"><\/span>Measuring GPU Usage<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>While AWS CloudWatch offers GPU monitoring, you can also execute the <code>nvidia-smi dmon<\/code> command to monitor GPU usage in real-time.<\/p>\n\n\n\n<pre class=\"wp-block-code code-block\"><code>$ nvidia-smi dmon\n\n# gpu    pwr  gtemp  mtemp     sm    mem    enc    dec    jpg    ofa   mclk   pclk \n# Idx      W      C      C      %      %      %      %      %      %    MHz    MHz \n    0     58     29      -      0      0      0      0      0      0   6250   1710 \n    0     58     29      -      0      0      0      0      0      0   6250   1710 \n    0     58     29      -      0      0      0      0      0      0   6250   1710 \n    ...\n<\/code><\/pre>\n\n\n\n<p>Here&#8217;s an overview of available metrics:<\/p>\n\n\n\n<figure class=\"wp-block-table is-style-stripes\"><table><thead><tr><td><strong>Item<\/strong><\/td><td><strong>Description<\/strong><\/td><\/tr><\/thead><tbody><tr><td><strong>gpu (Idx<\/strong>)<\/td><td>The GPU used (starting from 0)<\/td><\/tr><tr><td><strong>pwr (W)<\/strong><\/td><td>The current power consumption (Watt). Increases under high load.<\/td><\/tr><tr><td><strong>gtemp (C)<\/strong><\/td><td>The GPU Temperature (\u2103). Thermal throttling occurs if it becomes too hot.<\/td><\/tr><tr><td><strong>mtemp (C)<\/strong><\/td><td>The Memory Temperature (\u2103). Shows up as <code>-<\/code> if cannot be retrieved.<\/td><\/tr><tr><td><strong>sm (%)<\/strong><\/td><td>The Streaming Multiprocessors (SM) usage rate.<\/td><\/tr><tr><td><strong>mem (%)<\/strong><\/td><td>The GPU memory usage rate. Indicates VRAM usage.<\/td><\/tr><tr><td><strong>enc (%)<\/strong><\/td><td>The Hardware Encoder (NVENC) usage rate. Increases during video encoding.<\/td><\/tr><tr><td><strong>dec (%)<\/strong><\/td><td>The Hardware Decoder (NVDEC) usage rate. Increases during video decoding.<\/td><\/tr><tr><td><strong>jpg (%)<\/strong><\/td><td>The JPEG Engine usage rate. Related to image processing.<\/td><\/tr><tr><td><strong>ofa (%)<\/strong><\/td><td>The Optical Flow Accelerator usage rate. Used in machine learning and video processing.<\/td><\/tr><tr><td><strong>mclk (MHz)<\/strong><\/td><td>The Memory Clock (MHz).<\/td><\/tr><tr><td><strong>pclk (MHz)<\/strong><\/td><td>The GPU Core Clock (MHz).<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Summary\"><\/span>Summary<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Using the <code>nvidia\/cuda<\/code> image made using CUDA in Docker a piece of cake!<\/p>\n\n\n\n<p>I hope this article will be helpful when trying to build GPU-aware applications. CUDA is a must when working in Generative AI and Deep Learning, and it&#8217;s really handy when your development environment is built from the ground up to support GPU usecases.<\/p>\n\n\n\n<p>Happy coding!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This blog post is a translation of&nbsp;a Japanese article&nbsp;posted on March 27th, 2025. Hello! This is Kakutani from the Sreake application team. Recently, the rise of Machine Learning, Deep Learning and Generative AI made GPUs more important than ever. Large-scale models like Stable Diffusion and ChatGPT require massive computational resources, and using NVIDIA&#8217;s CUDA allows [&hellip;]<\/p>\n","protected":false},"author":45,"featured_media":12958,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_locale":"en_US","_original_post":"https:\/\/sreake.com\/?p=10686","footnotes":""},"categories":[17],"tags":[23],"class_list":["post-12962","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog","tag-enginner-blog","en-US"],"acf":[],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/sreake.com\/wp-json\/wp\/v2\/posts\/12962","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sreake.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sreake.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sreake.com\/wp-json\/wp\/v2\/users\/45"}],"replies":[{"embeddable":true,"href":"https:\/\/sreake.com\/wp-json\/wp\/v2\/comments?post=12962"}],"version-history":[{"count":2,"href":"https:\/\/sreake.com\/wp-json\/wp\/v2\/posts\/12962\/revisions"}],"predecessor-version":[{"id":12966,"href":"https:\/\/sreake.com\/wp-json\/wp\/v2\/posts\/12962\/revisions\/12966"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/sreake.com\/wp-json\/wp\/v2\/media\/12958"}],"wp:attachment":[{"href":"https:\/\/sreake.com\/wp-json\/wp\/v2\/media?parent=12962"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sreake.com\/wp-json\/wp\/v2\/categories?post=12962"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sreake.com\/wp-json\/wp\/v2\/tags?post=12962"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}