You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/manuals/compose/how-tos/model-runner.md
+66-19Lines changed: 66 additions & 19 deletions
Original file line number
Diff line number
Diff line change
@@ -17,14 +17,73 @@ This lets you define and run AI-powered applications alongside your other servic
17
17
18
18
## Prerequisites
19
19
20
-
- Docker Compose v2.35 or later
21
-
- Docker Desktop 4.41 or later
20
+
- Docker Compose v2.38 or later
21
+
- Docker Desktop 4.43 or later
22
22
- Docker Desktop for Mac with Apple Silicon or Docker Desktop for Windows with NVIDIA GPU
23
23
-[Docker Model Runner enabled in Docker Desktop](/manuals/ai/model-runner.md#enable-docker-model-runner)
24
24
25
-
## Provider services
25
+
## Use `models` definition
26
26
27
-
Compose introduces a new service type called `provider` that allows you to declare platform capabilities required by your application. For AI models, you can use the `model` type to declare model dependencies.
27
+
The `models` top-level element in the Compose file lets you define AI models to be used by your application.
28
+
Compose will use Docker Model Runner as the model runtime.
29
+
30
+
The following example shows how to provide the minimal configuration to use a model within your Compose application:
31
+
32
+
```yaml
33
+
services:
34
+
my-chat-app:
35
+
image: my-chat-app
36
+
models:
37
+
- smollm2
38
+
39
+
models:
40
+
smollm2:
41
+
image: ai/smollm2
42
+
```
43
+
44
+
### How it works
45
+
46
+
During the `docker compose up` process, Docker Model Runner automatically pulls and runs the specified model.
47
+
It also sends Compose the model tag name and the URL to access the model runner.
48
+
49
+
This information is then passed to services which declare a dependency on the model provider.
50
+
In the example above, the `my-chat-app` service receives 2 environment variables prefixed by the service name:
51
+
- `SMOLLM2_ENDPOINT`with the URL to access the model
52
+
- `SMOLLM2_MODEL`with the model name
53
+
54
+
This lets the `my-chat-app` service to interact with the model and use it for its own purposes.
55
+
56
+
57
+
### Customizing environment variables
58
+
59
+
You can customize the environment variable names which will be passed to your service container using the long syntax:
60
+
61
+
```yaml
62
+
services:
63
+
my-chat-app:
64
+
image: my-chat-app
65
+
models:
66
+
smollm2:
67
+
endpoint_var: AI_MODEL_URL
68
+
model_var: AI_MODEL_NAME
69
+
70
+
models:
71
+
smollm2:
72
+
image: ai/smollm2
73
+
```
74
+
75
+
With this configuration, your `my-chat-app` service will receive:
76
+
- `AI_MODEL_URL`with the URL to access the model
77
+
- `AI_MODEL_NAME`with the model name
78
+
79
+
This allows you to use more descriptive variable names that match your application's expectations.
80
+
81
+
82
+
## Alternative configuration with Provider services
83
+
> [!NOTE]
84
+
> Now that Compose supports a `models` top-level element, we recommend using it instead of the provider services approach.
85
+
86
+
Compose introduced a new service type called `provider` that allows you to declare platform capabilities required by your application. For AI models, you can use the `model` type to declare model dependencies.
28
87
29
88
Here's an example of how to define a model provider:
30
89
@@ -45,21 +104,9 @@ services:
45
104
Notice the dedicated `provider` attribute in the `ai_runner` service.
46
105
This attribute specifies that the service is a model provider and lets you define options such as the name of the model to be used.
47
106
48
-
There is also a `depends_on` attribute in the `chat` service.
49
-
This attribute specifies that the `chat` service depends on the `ai_runner` service.
50
-
This means that the `ai_runner` service will be started before the `chat` service to allow injection of model information to the `chat` service.
51
-
52
-
## How it works
53
-
54
-
During the `docker compose up` process, Docker Model Runner automatically pulls and runs the specified model.
55
-
It also sends Compose the model tag name and the URL to access the model runner.
56
-
57
-
This information is then passed to services which declare a dependency on the model provider.
58
-
In the example above, the `chat` service receives 2 environment variables prefixed by the service name:
59
-
- `AI_RUNNER_URL`with the URL to access the model runner
60
-
- `AI_RUNNER_MODEL`with the model name which could be passed with the URL to request the model.
61
-
62
-
This lets the `chat` service to interact with the model and use it for its own purposes.
107
+
There is also a `depends_on` attribute in the `my-chat-app` service.
108
+
This attribute specifies that the `my-chat-app` service depends on the `ai_runner` service.
109
+
This means that the `ai_runner` service will be started before the `my-chat-app` service to allow injection of model information to the `my-chat-app` service.
0 commit comments