You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/collections/blog/2025/05-14-docker-model-runner-spring/post.mdx
+12-5Lines changed: 12 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -38,7 +38,8 @@ Spring AI supports various AI model providers, including commercial cloud servic
38
38
When Docker Model Runner is active and serving a model (e.g., Llama 3, Gemma) with its API endpoint accessible (typically http://localhost:12434 or http://model-runner.docker.internal if accessed from another container), Spring AI can be configured to point to it.
39
39
Here's how a Java engineer benefits:
40
40
41
-
1. Simplified Configuration in Spring Boot:
41
+
1.**Simplified Configuration in Spring Boot**
42
+
42
43
Spring AI's autoconfiguration can often detect and set up the necessary beans to interact with an OpenAI-compatible endpoint. For Docker Model Runner, this typically involves setting a few properties in your application.properties or application.yml file:
43
44
44
45
```java
@@ -52,9 +53,12 @@ Here's how a Java engineer benefits:
52
53
\# Potentially disable API key ifDMR doesn't require it strictly for local
53
54
```
54
55
55
-
(Note: The exact property names and structure might vary slightly based on the Spring AI version and whether you're configuring a generic OpenAI client or a more specific Ollama-like client type ifSpringAI introduces more direct DMR support.)
56
-
2.LeveragingSpringAI's ChatClient and EmbeddingClient:
56
+
_(Note: The exact property names and structure might vary slightly based on the Spring AI version and whether you're configuring a generic OpenAI client or a more specific Ollama-like client type ifSpringAI introduces more direct DMR support.)_
57
+
58
+
2.**LeveragingSpringAI's ChatClient and EmbeddingClient**
59
+
57
60
Once configured, developers can inject and use Spring AI's standard clients without needing to know that the underlying provider is DockerModelRunner.
61
+
58
62
```java
59
63
importorg.springframework.ai.chat.ChatClient;
60
64
importorg.springframework.ai.chat.prompt.Prompt;
@@ -77,10 +81,13 @@ Here's how a Java engineer benefits:
77
81
}
78
82
}
79
83
```
84
+
80
85
This code remains the same whether Spring AI is talking to OpenAI's cloud API, a self-hosted Ollama instance, or Docker Model Runner serving a local model. This portability is a huge win.
81
-
3. Seamless Local Development and Testing:
86
+
87
+
3.**Seamless Local Development and Testing**
82
88
Engineers can develop and test AI-driven features entirely locally using their preferred Java tools and the Spring framework. Docker Model Runner handles the model serving, and Spring AI provides the clean Java interface. This speeds up iteration cycles and reduces reliance on potentially costly cloud APIs during development.
83
-
4. Consistency with Production (Potentially):
89
+
90
+
4.**Consistency with Production (Potentially)**
84
91
While Docker Model Runner is primarily for local development, the abstraction provided by Spring AI means that switching to a production-grade, potentially cloud-hosted model provider for deployment can be achieved mainly through configuration changes, without altering the core application logic.
85
92
86
93
## **The Bigger Picture: Local AI in Enterprise Java**
0 commit comments