Skip to content

Commit f5ad3ee

Browse files
EugeneLightsOnEugeneLightsOn
authored
Azure Container App one click deployment (#165)
* Azure HPA one click deployment - DRAFT * Azure HPA one click deployment - DRAFT * Azure HPA one click deployment - DRAFT * Azure HPA one click deployment - DRAFT * Azure HPA one click deployment - DRAFT * Azure HPA one click deployment - DRAFT * Azure HPA one click deployment - DRAFT * Azure HPA one click deployment - DRAFT * Azure Container App one click deployment
1 parent d089057 commit f5ad3ee

File tree

4 files changed

+287
-6
lines changed

4 files changed

+287
-6
lines changed

‎.github/workflows/build_and_push_docker_proxy.yml

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
#
2+
name: Create and publish single container image with nginx proxy
3+
4+
on:
5+
release:
6+
types: [published]
7+
workflow_dispatch:
8+
inputs:
9+
tags:
10+
description: "tags to attach to image"
11+
required: false
12+
type: string
13+
14+
env:
15+
REGISTRY: ghcr.io
16+
IMAGE_NAME: cohere-toolkit-proxy
17+
18+
jobs:
19+
build-and-push-image:
20+
runs-on: ubuntu-latest
21+
permissions:
22+
contents: read
23+
packages: write
24+
#
25+
steps:
26+
- name: Checkout repository
27+
uses: actions/checkout@v4
28+
- name: Log in to the Container registry
29+
uses: docker/login-action@v3
30+
with:
31+
registry: ${{ env.REGISTRY }}
32+
username: ${{ github.actor }}
33+
password: ${{ secrets.GITHUB_TOKEN }}
34+
35+
- name: Extract metadata (tags, labels) for Docker
36+
id: meta
37+
uses: docker/metadata-action@v4
38+
with:
39+
images: ${{ env.REGISTRY }}/cohere-ai/${{ env.IMAGE_NAME }}
40+
- name: Build and push Docker image
41+
uses: docker/build-push-action@v5
42+
with:
43+
context: .
44+
push: true
45+
tags: ${{ steps.meta.outputs.tags }}
46+
file: ./Dockerfile
47+
- name: Test the Docker image
48+
run: |
49+
docker run -d -p 8000:8000 $REGISTRY/cohere-ai/$IMAGE_NAME
50+
sleep 30
51+
status_code=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:8000/health)
52+
if [ $status_code -eq 200 ]; then
53+
echo "Backend started successfully"
54+
else
55+
echo "Backend failed to start"
56+
exit 1
57+
fi

‎Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ COPY pyproject.toml poetry.lock ./
3636
# Install dependencies
3737
RUN pip3 install --no-cache-dir poetry==1.6.1 \
3838
&& poetry config installer.max-workers 10 \
39-
&& poetry install --without setup \
39+
&& poetry install \
4040
&& (poetry cache clear --all --no-interaction PyPI || true) \
4141
&& (poetry cache clear --all --no-interaction _default_cache || true)
4242

‎azuredeploy.hpa.json

Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
{
2+
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
3+
"contentVersion": "1.0.0.0",
4+
"resources": [
5+
{
6+
"type": "Microsoft.OperationalInsights/workspaces",
7+
"apiVersion": "2022-10-01",
8+
"name": "[parameters('containerAppLogAnalyticsName')]",
9+
"location": "[parameters('location')]",
10+
"properties": {
11+
"sku": {
12+
"name": "PerGB2018"
13+
}
14+
}
15+
},
16+
{
17+
"type": "Microsoft.App/managedEnvironments",
18+
"apiVersion": "2023-11-02-preview",
19+
"name": "[parameters('containerAppEnvName')]",
20+
"location": "[parameters('location')]",
21+
"sku": {
22+
"name": "Consumption"
23+
},
24+
"properties": {
25+
"appLogsConfiguration": {
26+
"destination": "log-analytics",
27+
"logAnalyticsConfiguration": {
28+
"customerId": "[reference(resourceId('Microsoft.OperationalInsights/workspaces', parameters('containerAppLogAnalyticsName'))).customerId]",
29+
"sharedKey": "[listKeys(resourceId('Microsoft.OperationalInsights/workspaces', parameters('containerAppLogAnalyticsName')), '2022-10-01').primarySharedKey]"
30+
}
31+
}
32+
},
33+
"dependsOn": [
34+
"[resourceId('Microsoft.OperationalInsights/workspaces', parameters('containerAppLogAnalyticsName'))]"
35+
]
36+
},
37+
{
38+
"type": "Microsoft.App/containerApps",
39+
"apiVersion": "2022-06-01-preview",
40+
"name": "[parameters('containerAppName')]",
41+
"location": "[parameters('location')]",
42+
"properties": {
43+
"managedEnvironmentId": "[resourceId('Microsoft.App/managedEnvironments', parameters('containerAppEnvName'))]",
44+
"configuration": {
45+
"ingress": {
46+
"external": true,
47+
"targetPort": "[parameters('targetPort')]",
48+
"allowInsecure": false,
49+
"traffic": [
50+
{
51+
"latestRevision": true,
52+
"weight": 100
53+
}
54+
]
55+
}
56+
},
57+
"template": {
58+
"containers": [
59+
{
60+
"name": "[parameters('containerAppName')]",
61+
"image": "[parameters('image')]",
62+
"resources": {
63+
"cpu": "[json(parameters('cpuCore'))]",
64+
"memory": "[format('{0}Gi', parameters('memorySize'))]"
65+
},
66+
"env": [
67+
{
68+
"name": "DATABASE_URL",
69+
"value": "[parameters('databaseUrl')]"
70+
},
71+
{
72+
"name": "COHERE_API_KEY",
73+
"value": "[parameters('cohereApiKey')]"
74+
}
75+
]
76+
}
77+
],
78+
"scale": {
79+
"minReplicas": "[parameters('minReplicas')]",
80+
"maxReplicas": "[parameters('maxReplicas')]",
81+
"rules": [
82+
{
83+
"name": "http-scale-rule",
84+
"http": {
85+
"metadata": {
86+
"concurrentRequests": "100"
87+
}
88+
}
89+
}
90+
]
91+
}
92+
}
93+
},
94+
"dependsOn": [
95+
"[resourceId('Microsoft.App/managedEnvironments', parameters('containerAppEnvName'))]"
96+
]
97+
}
98+
],
99+
"parameters": {
100+
"containerAppName": {
101+
"type": "string",
102+
"defaultValue": "[format('toolkit-app-{0}', uniqueString(resourceGroup().id))]",
103+
"metadata": {
104+
"description": "Specifies the name of the container app."
105+
}
106+
},
107+
"containerAppEnvName": {
108+
"type": "string",
109+
"defaultValue": "[format('toolkit-env-{0}', uniqueString(resourceGroup().id))]",
110+
"metadata": {
111+
"description": "Specifies the name of the container app environment."
112+
}
113+
},
114+
"containerAppLogAnalyticsName": {
115+
"type": "string",
116+
"defaultValue": "[format('toolkit-log-{0}', uniqueString(resourceGroup().id))]",
117+
"metadata": {
118+
"description": "Specifies the name of the log analytics workspace."
119+
}
120+
},
121+
"location": {
122+
"type": "string",
123+
"defaultValue": "[resourceGroup().location]",
124+
"metadata": {
125+
"description": "Specifies the location for all resources."
126+
}
127+
},
128+
"targetPort": {
129+
"type": "int",
130+
"defaultValue": 4000,
131+
"metadata": {
132+
"description": "Specifies the container port."
133+
}
134+
},
135+
"cpuCore": {
136+
"type": "string",
137+
"defaultValue": "2",
138+
"allowedValues": [
139+
"0.25",
140+
"0.5",
141+
"0.75",
142+
"1",
143+
"1.25",
144+
"1.5",
145+
"1.75",
146+
"2"
147+
],
148+
"metadata": {
149+
"description": "Number of CPU cores the container can use. Can be with a maximum of two decimals."
150+
}
151+
},
152+
"memorySize": {
153+
"type": "string",
154+
"defaultValue": "4",
155+
"allowedValues": [
156+
"0.5",
157+
"1",
158+
"1.5",
159+
"2",
160+
"3",
161+
"3.5",
162+
"4"
163+
],
164+
"metadata": {
165+
"description": "Amount of memory (in gibibytes, GiB) allocated to the container up to 4GiB. Can be with a maximum of two decimals. Ratio with CPU cores must be equal to 2."
166+
}
167+
},
168+
"minReplicas": {
169+
"type": "int",
170+
"defaultValue": 1,
171+
"maxValue": 2,
172+
"minValue": 0,
173+
"metadata": {
174+
"description": "Minimum number of replicas that will be deployed"
175+
}
176+
},
177+
"maxReplicas": {
178+
"type": "int",
179+
"defaultValue": 1,
180+
"maxValue": 2,
181+
"minValue": 0,
182+
"metadata": {
183+
"description": "Maximum number of replicas that will be deployed"
184+
}
185+
},
186+
"image": {
187+
"type": "string",
188+
"defaultValue": "ghcr.io/cohere-ai/cohere-toolkit-proxy",
189+
"metadata": {
190+
"description": "Docker image to deploy"
191+
}
192+
},
193+
"databaseUrl": {
194+
"type": "string",
195+
"metadata": {
196+
"description": "Your Database connection URL"
197+
}
198+
},
199+
"cohereApiKey": {
200+
"type": "string",
201+
"metadata": {
202+
"description": "Your Cohere API key for the application"
203+
}
204+
}
205+
}
206+
}

‎docs/service_deployments.md

Lines changed: 23 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,11 +7,29 @@ Looking to serve your application in production? Deploy the Toolkit to your pref
77
- [AWS ECS Fargate Deployment](deployment_guides/aws_ecs_single_container.md): Deploy the Toolkit single container to AWS ECS(Fargate).
88
- [AWS ECS EC2 Deployment](deployment_guides/aws_ecs_single_container_ec2.md): Deploy the Toolkit single container to AWS ECS(EC2).
99
- [Google Cloud Platform](deployment_guides/gcp_deployment.md): Help setup your Cloud SQL instance, then build, push and deploy backend+frontend containers to Cloud Run.
10-
- Deploying to Azure. You can deploy Toolkit with one click to Microsoft Azure Platform: [<img src="https://aka.ms/deploytoazurebutton" height="24px">](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fcohere-ai%2Fcohere-toolkit%2Fmain%2Fazuredeploy.json). This deployment type uses Azure Container Instances to host the Toolkit. After your deployment is complete click "Go to resource" button.
11-
1) Check the logs to see if the container is running successfully:
10+
- [One Click Deploy to GCP](deployment_guides/gcp_one_click_deployment.md): Help setup your container to Cloud Run.
11+
- Deploying to Azure Container Instance. You can deploy Toolkit with one click to Microsoft Azure Platform: [<img src="https://aka.ms/deploytoazurebutton" height="24px">](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fcohere-ai%2Fcohere-toolkit%2Fmain%2Fazuredeploy.json).
12+
This deployment type uses Azure Container Instances to host the Toolkit. After your deployment is complete click "Go to resource" button.
13+
- Check the logs to see if the container is running successfully:
1214
- click on the "Containers" button on the left side of the screen
1315
- click on the container name
1416
- click on "Logs" tab to see the logs
15-
2) Navigate to the "Overview" tab to see the FQDN of the container instance
16-
3) Open the \<FQDN\>:4000 in your browser to access the Toolkit
17-
- [One Click Deploy to GCP](deployment_guides/gcp_one_click_deployment.md): Help setup your container to Cloud Run.
17+
- Navigate to the "Overview" tab to see the FQDN of the container instance
18+
- Open the \<FQDN\>:4000 in your browser to access the Toolkit
19+
- Deploying to Azure Cloud App. You can deploy Toolkit with one click to Microsoft Azure Platform: [<img src="https://aka.ms/deploytoazurebutton" height="24px">](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fcohere-ai%2Fcohere-toolkit%2Feugene%2FEXT2-69_azure_hpa%2Fazuredeploy.hpa.json).
20+
This deployment type uses Azure Container App to host the Toolkit. Follow these steps to deploy the Toolkit:
21+
- Select your subscription and resource group. If you don't have a resource group, create a new one.
22+
- Enter the connection string of the format `postgresql+psycopg2://USERNAME:PASSWORD@HOST:PORT/DB_NAME`.
23+
The `HOST` value here is the Public IP address or DNS name of your provisioned PostgreSQL database, and the default `PORT` is 5432.
24+
Make sure to use the username and password pair you set when creating your SQL instance. For example, `postgresql+psycopg2://myuser:mypassword@<your-db-public-ip-address>:5432/toolkit`.
25+
- Enter your Cohere API key.
26+
- Click "Review + create" and then "Create" to deploy the Toolkit.
27+
- After the deployment is complete, click on the "Go to resource group" button.
28+
- Click on the Toolkit container app.
29+
- Click on the "Overview" tab to see the "Application Url" of the container app.
30+
- Navigate to the "Application Url" to access the Toolkit.
31+
32+
To scale the Toolkit, you can enable the Automatic Horizontal Scaling by following this [tutorial](https://learn.microsoft.com/en-us/azure/container-apps/tutorial-scaling).
33+
34+
35+

0 commit comments

Comments
 (0)