properly reflect the changes required by the argo applicationSet
This commit is contained in:
60
README.md
60
README.md
@@ -1,50 +1,30 @@
|
||||
# Backend deployment
|
||||
# AnyWay backend deployment
|
||||
|
||||
The container is built using the Dockerfile and deployed to our own kubernetes cluster.
|
||||
|
||||
Both steps are handled by the CI/CD pipeline:
|
||||
- First the container is built in the workflow `workflow_build-image.yaml` and pushed to the (local) container registry.
|
||||
- Then the deployment is handled in the workflow `workflow_deploy-container.yaml` which deploys the container to the kubernetes cluster. Depending on the branch, the deployment is done to the staging or production environment.
|
||||
The backend is a containerized fastapi application built in [https://git.kluster.moll.re/anydev/anyway](https://git.kluster.moll.re/anydev/anyway). The container is deployed along some helper services on our own kubernetes cluster.
|
||||
|
||||
|
||||
## Overview
|
||||
The deployment comes in two flavors - `staging` and `production`. The setup is straightforward and mostly identical between the two environments: most configuration lies in the `base/` folder and only the environment-specific adjustments are in the `overlays/` folders.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
For the deployment of the **backend** application, the following prerequisites need to be met:
|
||||
|
||||
- On the cluster side: a Kubernetes cluster with the following components:
|
||||
- ingress controller
|
||||
- storage class
|
||||
- On the local side: for development and local testing:
|
||||
- `kubectl` installed
|
||||
- `kustomize` (usually bundled with `kubectl`)
|
||||
The deployment is in principle fully functional as-is, but the main benefits - automatic deployment, provisioning and updates - are only available when levering argoCD. In the recommended setup, the manifests are never applied manually, but are instead referenced in the [app of apps](https://git.kluster.moll.re/anydev/anydev-app-of-apps). This handles both the deployment of a stable `production` version and the automatic deployment of `staging` versions for each pull request.
|
||||
|
||||
|
||||
## Manual deployment
|
||||
## Cluster requirements
|
||||
|
||||
#### Initial deployment
|
||||
To deploy the backend application, follow these steps:
|
||||
|
||||
1. Clone this repository
|
||||
1. Apply the desired overly: `kubectl apply -k <path-to-overlay>`
|
||||
|
||||
#### Rolling updates
|
||||
Since the deployment uses a single tag (along with the `always` pull policy) for the backend application, we simply need to trigger a rolling update which will pull the latest image from the registry. To do this, run the following command:
|
||||
|
||||
```bash
|
||||
kubectl -n <namespace> rollout restart deployment/anyway-backend
|
||||
```
|
||||
For the deployment of the **backend** application, the following prerequisites need to be met on the kubernetes cluster:
|
||||
- ingress controller (assumed to be `traefik` in the manifests but easily adaptable)
|
||||
- a storage class (assumed to be `nfs-client` in the manifests but easily adaptable)
|
||||
- argoCD (for automatic deployment)
|
||||
- (upcoming) sealed secrets controller (for managing secrets)
|
||||
|
||||
|
||||
## Automated deployment
|
||||
### CI/CD and credentials
|
||||
For the deployment to work, the CI runner needs to authenticate against the kubernetes cluster. This is done by creating a service account in the cluster and providing the CI runner with the necessary credentials through a kubeconfig file. This file is stored as a repository secret `KUBE_CONFIG` and is used by the `kubectl` command in the CI pipeline.
|
||||
|
||||
The RBAC configuration for the service included for your reference in the file `rbac.yaml`. To use it, extract the token from the secret `deployment-token-secret`.
|
||||
Then fill in the token and service account name into the kubeconfig.
|
||||
|
||||
|
||||
### Deployment environments
|
||||
## Deployments
|
||||
The deployment is done to two environments:
|
||||
- Staging: All builds from forks and pull requests are deployed to the staging environment. This is done to test the changes before merging them to the main branch.
|
||||
- Production: Only builds from the main branch are deployed to the production environment. This is the live environment that is used by the users. The main branch is protected and can only be merged to through pull requests.
|
||||
|
||||
|
||||
### Production
|
||||
Only tagged builds from the `main` branch are deployed to the production environment. This is the live environment that is used by the users. The main branch is protected and can only be merged to through pull requests. This is handled as a simple argoCD application that points to the `overlays/production` folder which references the latest `semver` tagged version of the backend.
|
||||
|
||||
|
||||
### Staging
|
||||
All builds from forks and pull requests are deployed to a seperate namespace. The crucial part is that this is handled as an argoCD `applicationSet` that automatically creates a new application for each pull request, adding a further modification to the `overlays/staging` patches. This way, each pull request gets its own isolated deployment that can be tested and validated before merging to main. The staging deployments are automatically removed when the pull request is closed.
|
||||
|
||||
@@ -22,7 +22,7 @@ spec:
|
||||
- containerPort: 8000
|
||||
env:
|
||||
- name: MEMCACHED_HOST_PATH
|
||||
value: "memcached:11211"
|
||||
value: "memcached" # this value may be changed by kustomize nameReference transformer
|
||||
- name: NUM_WORKERS
|
||||
value: "3"
|
||||
- name: OSM_CACHE_DIR
|
||||
@@ -44,4 +44,3 @@ spec:
|
||||
persistentVolumeClaim:
|
||||
claimName: osm-cache
|
||||
|
||||
|
||||
@@ -8,3 +8,7 @@ resources:
|
||||
- service.yaml
|
||||
- ingress.yaml
|
||||
- memcached/
|
||||
|
||||
configurations:
|
||||
# allow nameReference to work with different mentions of the same resource as well
|
||||
- name_reference.yaml
|
||||
|
||||
13
base/name_reference.yaml
Normal file
13
base/name_reference.yaml
Normal file
@@ -0,0 +1,13 @@
|
||||
nameReference:
|
||||
# Tie target Service metadata.name to IngressRoute's spec.routes.services.name
|
||||
# Once Service name is changed, the IngressRoute referrerd service name will be changed as well.
|
||||
- kind: Service
|
||||
fieldSpecs:
|
||||
- kind: IngressRoute
|
||||
path: spec/routes/services/name
|
||||
|
||||
# same for the memcached service: since separate overlays may change its name the env variable in the deployment should be updated accordingly
|
||||
- kind: Service
|
||||
fieldSpecs:
|
||||
- kind: Deployment
|
||||
path: spec/template/spec/containers/env/value
|
||||
@@ -5,7 +5,6 @@ resources:
|
||||
- ../../base
|
||||
|
||||
namespace: anydev-anyway-backend-prod
|
||||
# nameSuffix: -prod
|
||||
labels:
|
||||
- includeSelectors: true
|
||||
pairs:
|
||||
@@ -21,7 +20,3 @@ patches:
|
||||
target:
|
||||
kind: IngressRoute
|
||||
name: anyway-ingress
|
||||
- path: patch-deployment-memcached-address.yaml
|
||||
target:
|
||||
kind: Deployment
|
||||
name: anyway-backend
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/env/0/value
|
||||
value: "memcached-prod:11211"
|
||||
@@ -1,6 +1,3 @@
|
||||
- op: replace
|
||||
path: /spec/routes/0/match
|
||||
value: Host(`anyway.anydev.info`) || Host(`anyway.kluster.moll.re`)
|
||||
- op: replace
|
||||
path: /spec/routes/0/services/0/name
|
||||
value: anyway-backend-prod
|
||||
value: Host(`anyway.anydev.info`) || Host(`anyway.kluster.moll.re`)
|
||||
|
||||
@@ -4,8 +4,9 @@ kind: Kustomization
|
||||
resources:
|
||||
- ../../base
|
||||
|
||||
namespace: anyway-backend
|
||||
nameSuffix: -stg
|
||||
namespace: anydev-anyway-backend-stg
|
||||
# nameSuffix: "" the suffix is added by argocd when deploying multiple instances through an applicationSet
|
||||
|
||||
labels:
|
||||
- includeSelectors: true
|
||||
pairs:
|
||||
@@ -22,12 +23,7 @@ patches:
|
||||
target:
|
||||
kind: IngressRoute
|
||||
name: anyway-ingress
|
||||
- path: patch-deployment-memcached-address.yaml
|
||||
target:
|
||||
kind: Deployment
|
||||
name: anyway-backend
|
||||
- path: patch-deployment.yaml
|
||||
target:
|
||||
kind: Deployment
|
||||
name: anyway-backend
|
||||
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/env/0/value
|
||||
value: "memcached-stg:11211"
|
||||
@@ -1,6 +1,3 @@
|
||||
- op: replace #action
|
||||
- op: replace #action
|
||||
path: /spec/routes/0/match
|
||||
value: Host(`anyway-stg.anydev.info`) || Host(`anyway-stg.kluster.moll.re`)
|
||||
- op: replace
|
||||
path: /spec/routes/0/services/0/name
|
||||
value: anyway-backend-stg
|
||||
value: Host(`anyway-stg.anydev.info`) || Host(`anyway-stg.kluster.moll.re`)
|
||||
|
||||
61
rbac.yaml
61
rbac.yaml
@@ -1,61 +0,0 @@
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
namespace: anyway-backend
|
||||
name: deployment-role
|
||||
rules:
|
||||
- apiGroups:
|
||||
- ""
|
||||
- "apps"
|
||||
- "traefik.io"
|
||||
resources:
|
||||
- pods
|
||||
- services
|
||||
- deployments
|
||||
- ingressroutes
|
||||
- persistentvolumeclaims
|
||||
verbs:
|
||||
- create
|
||||
- delete
|
||||
- deletecollection
|
||||
- get
|
||||
- list
|
||||
- patch
|
||||
- update
|
||||
- watch
|
||||
|
||||
---
|
||||
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: deployment-rolebinding
|
||||
namespace: anyway-backend
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: deployment-sa
|
||||
namespace: anyway-backend
|
||||
roleRef:
|
||||
kind: Role
|
||||
name: deployment-role
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
|
||||
---
|
||||
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: deployment-sa
|
||||
namespace: anyway-backend
|
||||
automountServiceAccountToken: false
|
||||
|
||||
---
|
||||
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
namespace: anyway-backend
|
||||
name: deployment-token-secret
|
||||
annotations:
|
||||
kubernetes.io/service-account.name: deployment-sa
|
||||
type: kubernetes.io/service-account-token
|
||||
Reference in New Issue
Block a user