Add blog post on Docker Model Runner and its engineering implications

- Introduced a comprehensive overview of Docker Model Runner's features and benefits for local AI development.
- Highlighted key engineering takeaways including OCI compliance, performance optimizations, and ecosystem integrations.
- Discussed future developments and the potential impact of Docker Model Runner on standard engineering practices in AI.

Signed-off-by: Lee Calcote <lee.calcote@layer5.io>

Author: leecalcote

src/collections/blog/2025/04-02-docker-model-runner-oci/post.mdx

@@ -19,13 +19,14 @@ import { BlogWrapper } from "../../Blog.style.js";
 import { Link } from "gatsby";
 
 <BlogWrapper>
-In our previous post, we introduced Docker Model Runner as a promising new toolkit for simplifying local AI development. Now, let's delve into one of its foundational—and perhaps most strategically significant—aspects: its deep reliance on the Open Container Initiative (OCI) standard for managing AI models.  
+In our [previous post](https://layer5.io/blog/docker/docker-model-runner), we introduced Docker Model Runner as a promising new toolkit for simplifying local AI development. Now, let's delve into one of its foundational—and perhaps most strategically significant—aspects: its deep reliance on the Open Container Initiative (OCI) standard for managing AI models.
 
 If you've wrestled with AI models, you know the "messy landscape" of model distribution. Models often arrive as loose files, tucked behind proprietary download tools, or lacking clear versioning. This fragmentation makes standardization, reproducibility, and integration into automated workflows a real headache for engineers. Docker Model Runner aims to bring order to this chaos by treating AI models as OCI artifacts, and this decision has profound implications for how you, as an engineer, can manage the entire lifecycle of your AI models.
 
 ## **OCI: More Than Just docker model pull**
 
-You might see docker model pull ai/llama3.2:1B-Q8\_0 and think it's just a convenient way to download models. But packaging models as OCI artifacts is a strategic move by Docker that goes far deeper. It aligns AI model management with the mature, robust ecosystem already built around OCI for container images.  
+You might see docker model pull `ai/llama3.2:1B-Q8_0` and think it's just a convenient way to download models. But packaging models as OCI artifacts is a strategic move by Docker that goes far deeper. It aligns AI model management with the mature, robust ecosystem already built around OCI for container images.  
+
 Essentially, Docker is working to make AI models **first-class citizens within the Docker ecosystem**. This means the same trusted registries and workflows you use for your application containers can now, in principle, be applied to your AI models. Imagine the possibilities:
 
 * **Unified Workflows:** Manage, version, and distribute your AI models using the same tools and processes you already use for your containerized applications. No more separate, bespoke systems for model management.  

…4-09-docker-model-runner-/hero-image.png …-model-runner-host-native/hero-image.pngsrc/collections/blog/2025/04-09-docker-model-runner-/hero-image.png renamed to src/collections/blog/2025/04-09-docker-model-runner-host-native/hero-image.png src/collections/blog/2025/04-09-docker-model-runner-/hero-image.png renamed to src/collections/blog/2025/04-09-docker-model-runner-host-native/hero-image.png

@@ -0,0 +1,67 @@
+---
+title: "Docker Model Runner"
+subtitle: "API Architecture, OpenAI Compatibility, and Connection Strategies"
+date: 2025-04-15 10:30:05 -0530
+author: Lee Calcote
+thumbnail: ./hero-image.png
+darkthumbnail: ./hero-image.png
+category: "Docker"
+# description: "Git command line aliases and git shortcuts"
+tags:
+  - docker
+  - ai
+type: Blog
+resource: true
+published: true
+---
+
+import { BlogWrapper } from "../../Blog.style.js";
+import { Link } from "gatsby";
+
+<BlogWrapper>
+
+In our last [post in this series](/blog/category/docker), explored Docker Model Runner's OCI-based model management and its performance-centric execution model, we now turn our attention to another critical area for engineers: its **API architecture and connectivity options**. How do your applications actually *talk* to the models running locally via Model Runner? The answer lies in a thoughtfully designed API layer, with OpenAI compatibility at its core, and flexible connection methods to suit diverse development scenarios.  
+
+For engineers, a well-defined and accessible API is paramount. It dictates the ease of integration, the reusability of existing code, and the overall developer experience when building AI-powered applications.
+
+## **The Heart of the Engine: llama.cpp and a Pluggable Future**
+
+In its initial Beta release, Docker Model Runner's inference capabilities are powered by an integrated engine built on llama.cpp. This open-source project is renowned for its efficient execution of LLMs across various hardware, making it a solid foundation for local inference.  
+
+When you interact with Model Runner, you're essentially communicating with this llama.cpp-based server, which runs as a native host process. The API paths often reflect this underlying engine, for example, with endpoints structured under /engines/llama.cpp/v1/... or a more generalized `/engines/v1/...`.  
+While llama.cpp provides a robust initial backbone, the API path structure (e.g., `/engines/...`) hints at a potentially pluggable architecture. This is a common design pattern that could allow Docker to integrate other inference engines or model serving technologies in the future. This foresight means Model Runner could evolve to support a wider array of model types, quantization methods, or hardware acceleration frameworks without requiring a fundamental redesign of its API interaction model.
+
+## **The "Superpower": OpenAI-Compatible API**
+
+Perhaps the most strategically significant aspect of Model Runner's API is its **OpenAI compatibility**. This is a game-changer for several reasons:
+
+1. **Leverage Existing SDKs and Tools:** Engineers can use their existing OpenAI SDKs (Python, Node.js, etc.) and a vast ecosystem of compatible tools like LangChain or LlamaIndex with minimal, if any, code changes. This dramatically lowers the barrier to adoption.  
+2. **Simplified Migration:** If you've been developing against OpenAI's cloud APIs, transitioning to local models with Model Runner can often be as simple as changing the baseURL in your client configuration. This seamless switch accelerates local development and testing.  
+3. **Reduced Learning Curve:** There's no need to learn a new, proprietary API. The familiar OpenAI request/response structures for tasks like chat completions (`/chat/completions`) or embeddings (`/embeddings`) remain consistent. 
+
+This adherence to a de facto industry standard API is a deliberate choice by Docker to maximize interoperability and ease of integration, allowing developers to focus on application logic rather than wrestling with new API paradigms.
+
+## **Connecting Your Applications: A Multi-Pronged Approach**
+
+Docker Model Runner offers several ways for your applications and tools to connect to the local inference engine, providing flexibility for different development setups:
+
+1. **Internal DNS for Containerized Applications (model-runner.docker.internal):**  
+   * **How it works:** For applications running as Docker containers themselves (e.g., a backend API service), Model Runner provides a stable internal DNS name: http://model-runner.docker.internal.  
+   * **Benefit for Engineers:** This is incredibly convenient. Your containerized service can simply target this DNS name to reach the Model Runner API, without needing to know the host's IP address or worry about dynamic port mappings. It simplifies network configuration within your Docker environment.  
+   * **Endpoint Example:** http://model-runner.docker.internal/engines/v1/chat/completions  
+2. **Host TCP Port for Direct Access:**  
+   * **How it works:** You can configure Model Runner to listen on a specific TCP port on your host machine. This is typically done via a Docker Desktop setting or a command like docker desktop enable model-runner \--tcp \<port\> (e.g., port 12434).  
+   * **Benefit for Engineers:** This allows applications running directly on your host (outside of Docker containers)—such as IDEs, local scripts, or standalone Java applications using Spring AI—to connect to the Model Runner.  
+   * **Endpoint Example:** http://localhost:12434/engines/v1/chat/completions  
+3. **Docker Socket (Advanced/CLI Use):**  
+   * **How it works:** For direct interactions via the Docker API or for certain CLI scripting scenarios, the Docker socket (/var/run/docker.sock on Linux/macOS) can be used. API calls through the socket might have a specific path prefix (e.g., `/exp/vDD4.40/...` as seen in early versions).  
+   * **Benefit for Engineers:** This offers a lower-level interface, useful for automation scripts or tools that integrate deeply with the Docker daemon.
+
+This multi-faceted approach to connectivity ensures that whether your application is containerized, running natively on the host, or interacting via CLI tools, there's a clear and supported path to communicate with the local AI models managed by Docker Model Runner.  
+
+Understanding these API mechanics and connection options is crucial for effectively integrating Docker Model Runner into your development workflows. It allows you to choose the most appropriate method for your specific application architecture and leverage the power of local AI models with ease.  
+In our next post, we'll explore how Docker Model Runner integrates with Docker Compose, enabling the orchestration of complex, multi-service AI applications locally.  
+
+*This blog post is based on information about Docker Model Runner, a Beta feature. Features, commands, and APIs are subject to change.*
+
+</BlogWrapper>

src/collections/blog/2025/04-09-docker-model-runner-host-native/post.mdx

@@ -20,13 +20,15 @@ import { Link } from "gatsby";
 
 <BlogWrapper>
 
-In our previous discussions, we've explored Docker Model Runner's role in simplifying local AI development and its strategic use of OCI artifacts for model management. Now, we peel back another layer to examine a critical aspect for any engineer working with Large Language Models (LLMs): **performance**. How does Docker Model Runner achieve the responsiveness needed for an efficient local development loop? The answers lie in its architectural choices, particularly its embrace of host-native execution and direct GPU access.  
+In our series on [Docker Model Runner](/blog/category/docker), we've explored Docker Model Runner's role in simplifying local AI development and its strategic use of OCI artifacts for model management. Now, we peel back another layer to examine a critical aspect for any engineer working with Large Language Models (LLMs): **performance**. How does Docker Model Runner achieve the responsiveness needed for an efficient local development loop? The answers lie in its architectural choices, particularly its embrace of host-native execution and direct GPU access.  
+
 For engineers, "local" often implies a trade-off: convenience versus raw power. Docker Model Runner attempts to bridge this gap, and understanding its performance model is key to leveraging it effectively.
 
 ## **The Architectural Pivot: Why docker model run Isn't docker container run**
 
 One of the most crucial, and perhaps initially counter-intuitive, aspects of Docker Model Runner is how it executes AI models. Seasoned Docker users might expect docker model run some-model to spin up an isolated Docker container housing the model and its inference engine. However, Model Runner takes a more direct path to prioritize local performance.  
-As detailed in multiple technical breakdowns and official documentation, when you execute docker model run:
+
+As detailed in multiple technical breakdowns and official documentation, when you execute `docker model run`:
 
 * **No Traditional Container for Inference:** The command doesn't launch a standard Docker container for the core inference task.  
 * **Host-Native Inference Server:** Instead, it interacts with an inference server (initially built on the efficient llama.cpp engine) that runs as a **native process directly on your host machine**. This server is managed as part of Docker Desktop or the Model Runner plugin.
@@ -58,6 +60,7 @@ This combination of on-demand loading and inactivity-based unloading helps balan
 The decision to run the inference engine as a host-native process is a clear trade-off: Docker is prioritizing local inference speed and direct hardware access over the complete process isolation typically provided by containers *for the inference step itself*. While the applications *using* the model can still be containerized and benefit from Docker's isolation, the model execution core operates closer to the metal.  
 This architectural choice highlights Docker's commitment to making the local AI development experience as smooth and fast as possible, even if it means deviating slightly from its traditional container-centric execution model for this specific, performance-sensitive component.  
 Understanding this performance architecture—host-native execution, direct GPU access, and smart resource management—allows engineers to better anticipate Model Runner's behavior, optimize their local AI workflows, and appreciate the engineering decisions aimed at making local LLM development more practical and efficient.  
+
 In our next post, we'll explore the API architecture of Docker Model Runner, focusing on its OpenAI compatibility and the various ways you can connect your applications to the local inference engine.  
 
 *This blog post is based on information about Docker Model Runner, a Beta feature. Features, commands, and APIs are subject to change.*

src/collections/blog/2025/04-15-docker-model-runner-api-architecture/hero-image.png

@@ -0,0 +1,88 @@
+---
+title: "Docker Model Runner & Compose"
+subtitle: "Orchestrating Multi-Service AI Applications Locally"
+date: 2025-04-24 10:30:05 -0530
+author: Lee Calcote
+thumbnail: ./hero-image.png
+darkthumbnail: ./hero-image.png
+category: "Docker"
+# description: "Git command line aliases and git shortcuts"
+tags:
+  - docker
+  - ai
+type: Blog
+resource: true
+published: true
+---
+
+import { BlogWrapper } from "../../Blog.style.js";
+import { Link } from "gatsby";
+
+<BlogWrapper>
+
+So far in our [series on Docker Model Runner](/blog/category/docker), we've dissected its OCI-based model management, its performance-optimized execution architecture, and its OpenAI-compatible API. Now, we explore a feature that truly elevates its utility for engineers building complex systems: **deep integration with Docker Compose via a novel provider service type.**  
+
+For engineers, Docker Compose is the go-to tool for defining and running multi-container Docker applications. The introduction of the provider service type specifically for Model Runner bridges the gap between local AI model execution and the broader application stack, allowing you to define and manage AI models as integral components of your local development environment declaratively.
+
+## **Beyond CLI: Models as First-Class Services in Compose**
+
+While docker model run is handy for quick tests, real-world applications often involve multiple interacting services—a web frontend, a backend API, a database, and now, an AI model. Docker Model Runner's Compose integration allows you to define the AI model itself as a service within your `docker-compose.yml` file.  
+
+The key innovation here is the provider attribute within a service definition. Here's a conceptual example based on Docker's documentation:
+
+```yaml
+services:  
+  model\_provider\_service: \# You can name this service as you like  
+    provider:  
+      type: model        \# Specifies this is a model provider  
+      image: ai/llama3.2:1B-Q8\_0 \# The OCI image for the model  
+    \# No 'build' or 'image' directives here in the traditional sense for the provider
+
+  my\_app\_service:  
+    build: ./app  
+    ports:  
+      \- "8080:80"  
+    depends\_on:  
+      \- model\_provider\_service \# Ensures model is ready before the app starts  
+    environment:  
+      \# Environment variables will be injected here (see below)  
+      MODEL\_NAME: ${MODEL\_PROVIDER\_SERVICE\_MODEL}  
+      MODEL\_URL: ${MODEL\_PROVIDER\_SERVICE\_URL}
+```
+
+In this setup:
+
+* model\_provider\_service doesn't run a traditional container in the same way my\_app\_service does. Instead, it instructs Docker Compose to leverage Docker Model Runner.  
+* Docker Model Runner, when processing this provider service, will ensure the specified image (the AI model) is pulled and made available via its host-native inference engine.
+
+## **Automatic Model Provisioning and Service Discovery**
+
+This Compose integration brings significant benefits for engineers:
+
+1. **Declarative Model Dependencies:**  
+   * You declare your AI model dependency directly in your docker-compose.yml. Docker Model Runner handles the provisioning (pulling and preparing the model if needed) when you run docker compose up.  
+   * This is a stark improvement over manual docker model run commands or custom scripts to manage model lifecycle alongside your application stack.  
+2. **Automated Service Discovery via Environment Variables:**  
+   * This is a crucial feature for seamless integration. When my\_app\_service starts (after model\_provider\_service is ready), Docker Compose automatically injects environment variables into my\_app\_service.  
+   * These variables typically follow the pattern: PROVIDER\_SERVICE\_NAME\_MODEL and PROVIDER\_SERVICE\_NAME\_URL.  
+     * MODEL\_PROVIDER\_SERVICE\_MODEL: Contains the name/tag of the model being served (e.g., ai/llama3.2:1B-Q8\_0).  
+     * MODEL\_PROVIDER\_SERVICE\_URL: Provides the URL your application should use to access the Model Runner's API endpoint for this model. This would often point to the internal DNS http://model-runner.docker.internal or a host-accessible TCP port if configured.  
+   * Your application code can then dynamically use these environment variables to configure its AI client, making the connection to the local model effortless and portable.  
+3. **Simplified depends\_on for Startup Order:**  
+   * Using depends\_on ensures that your application services only start after Model Runner has signaled that the model provider is ready. This prevents your application from trying to connect to a model that isn't yet available.
+
+## **Engineering Benefits for Complex AI Applications**
+
+This declarative, integrated approach offers tangible advantages:
+
+* **Reproducible AI Development Environments:** Your entire local stack, including the specific AI model version, is defined in code (docker-compose.yml), making it easy to share, version control, and ensure consistency across development team members.  
+* **Simplified Onboarding:** New developers can get a complex AI-powered application stack running locally with a single docker compose up command.  
+* **Streamlined Local Testing of AI Features:** Test end-to-end flows involving your application logic and AI model interactions in a fully integrated local environment that mirrors how services would interact.  
+* **Foundation for Local MLOps Loops:** While focused on local development, this pattern lays a conceptual foundation for how AI models can be treated as manageable dependencies within larger application architectures, aligning with MLOps principles.
+
+By treating AI models as discoverable services managed by Compose, Docker Model Runner significantly lowers the barrier to building and iterating on sophisticated multi-service applications that leverage local AI capabilities. This moves beyond simply running a model in isolation to truly integrating AI into your development workflow.  
+Next up, we'll explore how Docker Model Runner specifically caters to Java developers through its integration with frameworks like Spring AI, further simplifying the adoption of local AI.  
+
+*This blog post is based on information about Docker Model Runner, a Beta feature. Features, commands, and APIs are subject to change.*
+
+</BlogWrapper>