Deploying an AI Application Using Docker Model Runner

Find out what the recently launched Docker Model Runner is and how to deploy an AI application using it.

- 

Docker has just launched Docker Model Runner to help developers run or test LLM-based applications locally by pulling the model running with the Docker Desktop (minimum version required is 4.40) on localhost. This beta feature must be enabled in the Docker Desktop settings. We can pull different models like ai/smollm2, ai/deepseek-r1-distill-llama, ai/llama3.2, etc, from the Docker model registry at https://docs.docker.com/ai/model-runner/, as well as any of the required models on localhost as per our requirements. Once the Docker Model Runner is enabled we can work with the Docker model using the commands shown in Figure 1.