Publish ASP.NET Core 5 Application with Multiple Ingress Controllers in Azure Kubernetes Service (AKS) - Build and Deploy - PART 1
1. Overall Architecture overview, components and tech stack
Front-End: Angular 11
Back-End: ASP.NET Core 5 (API)
Azure AKS (K8S ): 1.19
Azure ACR
2. Create an AKS cluster
I will skip this step, creation of the AKS cluster pretty straight forward on Azure, just follow the instructions:
3. Deploy ASP.NET Core API
Create Image
First, we have to add Dockerfile to our solution, I'm doing this from VS2019:
Here is the catch, VS2019 will add Dockers file to your Web Project, not the solution location. In case you have one project it will work fine, but in case you have multiple projects like I do, the docker build command will fail. In order to fix this you have to use the following docker build mentioned files and project locations explicitly:
Push Image to Azure Container Registry (ACR)
Next, don't forget to log in to Azure ACR instance: docker login xxx.azurecr.io
Qa namespace:
Now let's prepare K8S manifests for all components above. You can use one yml for all definitions, I would suggest splitting per kind cause some manifests you will need to run just ones (like namespace / secrets, configMaps) others every build, like deployment.
And now we can tag and push the Image to Azure ACR:
docker tag epiqims-apis xxx.azurecr.io/epiqims-apis:dev
docker push xxx.azurecr.io/epiqims-apis:dev
Please pay attention xxx.azurecr.io in the tag commend must be your ACR Login Server [you can find it in Azure Portal ACR Overview Dashboard]
Create Kubernetes (AKS) Deployment Manifests (yml)
We will implement the following architecture in our AKS cluster (two environments Dev and Qa)
I assumed you created AKS cluster on Azure already, if you need guidelines on how to do it, take a look here: Create AKS on Azure
We will isolate our environments by namespace (there are few more approaches to manage environments this one is the cheapest one)
Ingress Namespace:
Dev namespace:
Qa namespace:
Now let's prepare K8S manifests for all components above. You can use one yml for all definitions, I would suggest splitting per kind cause some manifests you will need to run just ones (like namespace / secrets, configMaps) others every build, like deployment.
Dev:
Namespace:
apiVersion: v1
kind: Namespace
metadata:
name: coreapi
Deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: webapis
namespace: coreapi
spec:
replicas: 1
selector:
matchLabels:
app: epiqims-webapi
template:
metadata:
labels:
app: epiqims-webapi
spec:
containers:
- name: webapi
image: 'xxx.azurecr.io/epiqims-apis:##BUILD_ID##'
imagePullPolicy: Always
ports:
- containerPort: 80
protocol: TCP
env:
- name: "ASPNETCORE_ENVIRONMENT"
value: "Development"
- name: "ASPNETCORE_FORWARDEDHEADERS_ENABLED"
value: "true"
- name: "ApiVersion"
value: "##BUILD_ID##"
- name: psDatabase
valueFrom:
secretKeyRef:
name: secret-app-settings
key: psDatabase
- name: azureSignalRConnectionString
valueFrom:
secretKeyRef:
name: secret-app-settings
key: azureSignalRConnectionString
livenessProbe:
httpGet:
path: /api/ping
port: 80
initialDelaySeconds: 15
timeoutSeconds: 1
periodSeconds: 15
readinessProbe:
httpGet:
path: /api/ping
port: 80
initialDelaySeconds: 5
timeoutSeconds: 1
periodSeconds: 15
Secret:
apiVersion: v1
kind: Secret
metadata:
name: secret-app-settings
namespace: coreapi
type: Opaque
data:
psDatabase: base64 encrypted connection string
azureSignalRConnectionString: base64 encrypted connectionstring
psDatabase and SignalR connection string (I'm using Azure SignaR service: Azure SignalR) I moved from appsettings.json to secret just not expose it
You can use an online tool to encode/decode base64: https://www.base64decode.org/
Service:
apiVersion: v1
kind: Service
metadata:
name: webapi
namespace: coreapi
spec:
type: ClusterIP
ports:
- name: http
port: 80
targetPort: 80
- name: https
port: 443
targetPort: 80
selector:
app: epiqims-webapi
Ingress:
Ingress purpose:
- Routing
- TLS Offload
Now, let's deploy Ingress Operator before adding Ingress to our cluster:
helm install nginx-ingress ingress-nginx/ingress-nginx \
--namespace ingress-basic \
--set controller.replicaCount=2 \
--set controller.nodeSelector."beta\.kubernetes\.io/os"=linux \
--set defaultBackend.nodeSelector."beta\.kubernetes\.io/os"=linux \
--set controller.admissionWebhooks.patch.nodeSelector."beta\.kubernetes\.io/os"=linux
* I assume you have HELM installed on your machine
! There are multiple types of ingress controllers we can use on Azure (one of them Azure Gateway Ingress Controller: AGIC), I used for this example K8S ingress controller.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: dev-webapi-ingress
annotations:
kubernetes.io/ingress.class: nginx-dev
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/whitelist-source-range: '192.168.65.3/32'
namespace: coreapi
spec:
tls:
- hosts:
- host.cloudapp.azure.com
# secretName: wepapi-secret
rules:
- host: host.cloudapp.azure.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: webapi
port:
number: 80
Please pay attention TLS currently comment out we will deal with a custom domain and TLS cert later.
I hope you mention kubernetes.io/ingress.class: nginx-dev, we need to create two different controllers with two different types, K8S doesn't allow two different controllers with the same class and the same paths in the same cluster - which make sense :)
We almost ready to run all ymls and deploy the Dev environment, just check out we have ##BUILD_ID##
in our deployment yml - we will need it to build CICD later, for now just substitute it with the build tag you used in
docker tag and docker push in the previous step.
Ok, run all ymls - we suppose to have our asp.net Core app up and running:
http://epiq-ims-api-dev.canadacentral.cloudapp.azure.com/api/service
To deploy Qa environment use the same ymls, just change the following:
apiVersion: v1
kind: Namespace
metadata:
name: qa-coreapi
- name: "ASPNETCORE_ENVIRONMENT" value: "Qa"
- name: "ASPNETCORE_ENVIRONMENT"
value: "Qa"
and all the services namespaces change to qa-coreapi
ASPNETCORE_ENVIROENMENT very important since the asp.net core application will choose appsettings.json depends on the environment variable we deployed in deployment.yml:
Start / Pause AKS cluster - automation script
AKS not a cheap asset, if you like to save some $$$, you can stop your cluster running at night and resume on the morning - your call.
There is new functionality Microsft AKs provivides - stop / resume cluster with the script:
• az extension add --name aks-preview
• az extension update --name aks-preview
• az feature register --namespace "Microsoft.ContainerService" --name "StartStopPreview"
• az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/StartStopPreview')].{Name:name,State:properties.state}"
• az provider register --namespace Microsoft.ContainerService
• az aks stop --name clustername --resource-group rgname
• az extension update --name aks-preview
• az feature register --namespace "Microsoft.ContainerService" --name "StartStopPreview"
• az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/StartStopPreview')].{Name:name,State:properties.state}"
• az provider register --namespace Microsoft.ContainerService
• az aks stop --name clustername --resource-group rgname
- az aks start --name clustername --resource-group rgname
Tip: Parameters management in appsettings: configuration.GetValue vs configuration.GetConnectionString("psDatabase");
In your Asp.Net Core don't use:
configuration.GetConnectionString("psDatabase");
but use:
configuration.GetValue<string>("psDatabase");
then you will avoid problems to fetch config from K8S secrets.
Comments
Post a Comment