CICD 스터디 3주차 두번째

Jenkins CI + Argo CD + K8S(Kind)

Argo - 공홈 & CD Docs & Rollouts Docs & Blog

Argo Project

https://github.com/argoproj/

blog.argoproj.io

Architecture

Argo CD 설치 및 기본 설정 - helm_chart

# 네임스페이스 생성 및 파라미터 파일 작성
kubectl create ns argocd
cat <<EOF > argocd-values.yaml
dex:
  enabled: false

server:
  service:
    type: NodePort
    nodePortHttps: 30002
EOF

# 설치
helm repo add argo https://argoproj.github.io/argo-helm
helm install argocd argo/argo-cd --version 7.7.10 -f argocd-values.yaml --namespace argocd

# 확인
kubectl get pod,svc,ep -n argocd
kubectl get crd | grep argo
applications.argoproj.io                     2024-04-14T08:12:16Z
applicationsets.argoproj.io                  2024-04-14T08:12:17Z
appprojects.argoproj.io                      2024-04-14T08:12:16Z

# 최초 접속 암호 확인
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d ;echo
PCdOlwZT8c4naBWK

# Argo CD 웹 접속 주소 확인 : 초기 암호 입력 (admin 계정)
open "https://127.0.0.1:30002" # macOS
## Windows OS경우 직접 웹 브라우저에서 https://127.0.0.1:30002 접속

확인

ops-deploy Repo 등록 : Settings → Repositories → CONNECT REPO 클릭
- connection method : VIA HTTPS
- Type : git
- Project : default
- Repo URL : http://192.168.254.124:3000/devops/ops-deploy
- Username : devops
- Password : <Gogs 토큰>

확인

(기초) helm chart 를 통한 배포 실습

#
mkdir nginx-chart
cd nginx-chart

mkdir templates

cat > templates/configmap.yaml <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ .Release.Name }}
data:
  index.html: |
{{ .Values.indexHtml | indent 4 }}
EOF

cat > templates/deployment.yaml <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Release.Name }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      app: {{ .Release.Name }}
  template:
    metadata:
      labels:
        app: {{ .Release.Name }}
    spec:
      containers:
      - name: nginx
        image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
        ports:
        - containerPort: 80
        volumeMounts:
        - name: index-html
          mountPath: /usr/share/nginx/html/index.html
          subPath: index.html
      volumes:
      - name: index-html
        configMap:
          name: {{ .Release.Name }}
EOF

cat > templates/service.yaml <<EOF
apiVersion: v1
kind: Service
metadata:
  name: {{ .Release.Name }}
spec:
  selector:
    app: {{ .Release.Name }}
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
    nodePort: 30000
  type: NodePort
EOF

cat > values.yaml <<EOF
indexHtml: |
  <!DOCTYPE html>
  <html>
  <head>
    <title>Welcome to Nginx!</title>
  </head>
  <body>
    <h1>Hello, Kubernetes!</h1>
    <p>Nginx version 1.26.1</p>
  </body>
  </html>

image:
  repository: nginx
  tag: 1.26.1

replicaCount: 1
EOF

cat > Chart.yaml <<EOF
apiVersion: v2
name: nginx-chart
description: A Helm chart for deploying Nginx with custom index.html
type: application
version: 1.0.0
appVersion: "1.26.1"
EOF

# 이전 timeserver/service(nodeport) 삭제
kubectl delete deploy,svc --all

# 직접 배포 해보기
helm install dev-nginx . -f values.yaml
helm list
kubectl get deploy,svc,ep,cm dev-nginx -owide

#
curl http://127.0.0.1:30000
curl -s http://127.0.0.1:30000 | grep version
open http://127.0.0.1:30000


# value 값 변경 후 적용 해보기 : version/tag, replicaCount
cat > values.yaml <<EOF
indexHtml: |
  <!DOCTYPE html>
  <html>
  <head>
    <title>Welcome to Nginx!</title>
  </head>
  <body>
    <h1>Hello, Kubernetes!</h1>
    <p>Nginx version 1.26.2</p>
  </body>
  </html>

image:
  repository: nginx
  tag: 1.26.2

replicaCount: 2
EOF

# helm chart 업그레이드 적용
helm upgrade dev-nginx . -f values.yaml

# 확인
helm list
kubectl get deploy,svc,ep,cm dev-nginx -owide
curl http://127.0.0.1:30000
curl -s http://127.0.0.1:30000 | grep version
open http://127.0.0.1:30000

# 확인 후 삭제
helm uninstall dev-nginx

Repo(ops-deploy) 에 nginx helm chart 를 Argo CD를 통한 배포 1

#
cd ~/cicd-labs
git clone http://192.168.254.124:3000/devops/ops-deploy.git
cd ops-deploy

#
git config user.name "devops"
git config user.email "a@a.com"
git config init.defaultBranch main
git config credential.helper store

#
VERSION=1.26.1
mkdir nginx-chart
mkdir nginx-chart/templates

cat > nginx-chart/VERSION <<EOF
$VERSION
EOF

cat > nginx-chart/templates/configmap.yaml <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ .Release.Name }}
data:
  index.html: |
{{ .Values.indexHtml | indent 4 }}
EOF

cat > nginx-chart/templates/deployment.yaml <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Release.Name }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      app: {{ .Release.Name }}
  template:
    metadata:
      labels:
        app: {{ .Release.Name }}
    spec:
      containers:
      - name: nginx
        image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
        ports:
        - containerPort: 80
        volumeMounts:
        - name: index-html
          mountPath: /usr/share/nginx/html/index.html
          subPath: index.html
      volumes:
      - name: index-html
        configMap:
          name: {{ .Release.Name }}
EOF

cat > nginx-chart/templates/service.yaml <<EOF
apiVersion: v1
kind: Service
metadata:
  name: {{ .Release.Name }}
spec:
  selector:
    app: {{ .Release.Name }}
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
    nodePort: 30000
  type: NodePort
EOF

cat > nginx-chart/values-dev.yaml <<EOF
indexHtml: |
  <!DOCTYPE html>
  <html>
  <head>
    <title>Welcome to Nginx!</title>
  </head>
  <body>
    <h1>Hello, Kubernetes!</h1>
    <p>DEV : Nginx version $VERSION</p>
  </body>
  </html>

image:
  repository: nginx
  tag: $VERSION

replicaCount: 1
EOF

cat > nginx-chart/values-prd.yaml <<EOF
indexHtml: |
  <!DOCTYPE html>
  <html>
  <head>
    <title>Welcome to Nginx!</title>
  </head>
  <body>
    <h1>Hello, Kubernetes!</h1>
    <p>PRD : Nginx version $VERSION</p>
  </body>
  </html>

image:
  repository: nginx
  tag: $VERSION

replicaCount: 2
EOF

cat > nginx-chart/Chart.yaml <<EOF
apiVersion: v2
name: nginx-chart
description: A Helm chart for deploying Nginx with custom index.html
type: application
version: 1.0.0
appVersion: "$VERSION"
EOF

tree nginx-chart
nginx-chart
├── Chart.yaml
├── VERSION
├── templates
│   ├── configmap.yaml
│   ├── deployment.yaml
│   └── service.yaml
├── values-dev.yaml
└── values-prd.yaml

#
git status && git add . && git commit -m "Add nginx helm chart" && git push -u origin main

Argo CD에 App 등록 : Application → NEW APP
- GENERAL
  - App Name : dev-nginx
  - Project Name : default
  - SYNC POLICY : Manual
  - SYNC OPTIONS : AUTO-CREATE NAMESPACE(Check)
- Source
  - Repo URL : 설정되어 있는 것 선택
  - Revision : HEAD
  - PATH : nginx-chart
- DESTINATION
  - Cluster URL : <기본값>
  - NAMESPACE : dev-nginx
- HELM
  - Values files : values-dev.yaml

확인

#
kubectl get applications -n argocd
NAME        SYNC STATUS   HEALTH STATUS
dev-nginx   OutOfSync     Missing

kubectl describe applications -n argocd dev-nginx

# 반복 접속 시도
while true; do curl -s --connect-timeout 1 http://127.0.0.1:30000 ; date ; echo "------------" ; sleep 1 ; done

SYNC 클릭 으로 반영 확인

# 아래 처럼 yaml 로 APP 생성 가능
kubectl get applications -n argocd
kubectl get applications -n argocd -o yaml | kubectl neat

# 배포 확인
kubectl get all -n dev-nginx -o wide

코드 수정 후 반영 확인

#
VERSION=1.26.2

cat > nginx-chart/VERSION <<EOF
$VERSION
EOF

cat > nginx-chart/values-dev.yaml <<EOF
indexHtml: |
  <!DOCTYPE html>
  <html>
  <head>
    <title>Welcome to Nginx!</title>
  </head>
  <body>
    <h1>Hello, Kubernetes!</h1>
    <p>DEV : Nginx version $VERSION</p>
  </body>
  </html>

image:
  repository: nginx
  tag: $VERSION

replicaCount: 2
EOF

cat > nginx-chart/values-prd.yaml <<EOF
indexHtml: |
  <!DOCTYPE html>
  <html>
  <head>
    <title>Welcome to Nginx!</title>
  </head>
  <body>
    <h1>Hello, Kubernetes!</h1>
    <p>PRD : Nginx version $VERSION</p>
  </body>
  </html>

image:
  repository: nginx
  tag: $VERSION

replicaCount: 2
EOF

#
git status && git add . && git commit -m "Update nginx version $(cat nginx-chart/VERSION)" && git push -u origin main

리프레시 후 확인

SYNC 클릭 → SYNCHRONIZE 클릭

확인

Argo CD 웹에서 App 삭제

watch -d kubectl get all -n dev-nginx -o wide

Repo(ops-deploy) 에 nginx helm chart 를 Argo CD를 통한 배포 2 : ArgoCD Declarative Setup - ArgoCD 애플리케이션 자체를 yaml로 생성

##dev-nginx App 생성 및 Auto SYNC
#
cat <<EOF | kubectl apply -f -
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: dev-nginx
  namespace: argocd
  finalizers:
  - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    helm:
      valueFiles:
      - values-dev.yaml
    path: nginx-chart
    repoURL: http://192.168.254.124:3000/devops/ops-deploy
    targetRevision: HEAD
  syncPolicy:
    automated:
      prune: true
    syncOptions:
    - CreateNamespace=true
  destination:
    namespace: dev-nginx
    server: https://kubernetes.default.svc
EOF

#
kubectl get applications -n argocd dev-nginx
kubectl get applications -n argocd dev-nginx -o yaml | kubectl neat
kubectl describe applications -n argocd dev-nginx
kubectl get pod,svc,ep,cm -n dev-nginx

#
curl http://127.0.0.1:30000
open http://127.0.0.1:30000

# Argo CD App 삭제
kubectl delete applications -n argocd dev-nginx

확인

prd-nginx App 생성 및 Auto SYNC

#
cat <<EOF | kubectl apply -f -
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: prd-nginx
  namespace: argocd
  finalizers:
  - resources-finalizer.argocd.argoproj.io
spec:
  destination:
    namespace: prd-nginx
    server: https://kubernetes.default.svc
  project: default
  source:
    helm:
      valueFiles:
      - values-prd.yaml
    path: nginx-chart
    repoURL: http://192.168.254.124:3000/devops/ops-deploy
    targetRevision: HEAD
  syncPolicy:
    automated:
      prune: true
    syncOptions:
    - CreateNamespace=true
EOF

#
kubectl get applications -n argocd prd-nginx
kubectl describe applications -n argocd prd-nginx
kubectl get pod,svc,ep,cm -n prd-nginx

#
curl http://127.0.0.1:30000
open http://127.0.0.1:30000

# Argo CD App 삭제
kubectl delete applications -n argocd prd-nginx

Full CI/CD 구성도

ops-deploy Repo 코드 작업

#
cd ops-deploy
mkdir dev-app

# 도커 계정 정보
DHUSER=<도커 허브 계정>
DHUSER=gasida

# 버전 정보 
VERSION=0.0.1

#
cat > dev-app/VERSION <<EOF
$VERSION
EOF

cat > dev-app/timeserver.yaml <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: timeserver
spec:
  replicas: 2
  selector:
    matchLabels:
      pod: timeserver-pod
  template:
    metadata:
      labels:
        pod: timeserver-pod
    spec:
      containers:
      - name: timeserver-container
        image: docker.io/$DHUSER/dev-app:$VERSION
      imagePullSecrets:
      - name: dockerhub-secret
EOF

cat > dev-app/service.yaml <<EOF
apiVersion: v1
kind: Service
metadata:
  name: timeserver
spec:
  selector:
    pod: timeserver-pod
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
    nodePort: 30000
  type: NodePort
EOF

#
git status && git add . && git commit -m "Add dev-app deployment yaml" && git push -u origin main

Argo CD app 생성

#
cat <<EOF | kubectl apply -f -
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: timeserver
  namespace: argocd
  finalizers:
  - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    path: dev-app
    repoURL: http://192.168.254.124:3000/devops/ops-deploy
    targetRevision: HEAD
  syncPolicy:
    automated:
      prune: true
    syncOptions:
    - CreateNamespace=true
  destination:
    namespace: default
    server: https://kubernetes.default.svc
EOF

#
kubectl get applications -n argocd timeserver
kubectl get applications -n argocd timeserver -o yaml | kubectl neat
kubectl describe applications -n argocd timeserver
kubectl get deploy,rs,pod
kubectl get svc,ep timeserver

#
curl http://127.0.0.1:30000
open http://127.0.0.1:30000

dev-app Repo 코드 작업

dev-app Repo에 VERSION 업데이트 시 → ops-deploy Repo 에 dev-app 에 파일에 버전 정보 업데이트 작업 추가
1. 기존 버전 정보는 VERSION 파일 내에 정보를 가져와서 변수 지정 : OLDVER=$(cat dev-app/VERSION)
2. 신규 버전 정보는 environment 도커 태그 정보를 가져와서 변수 지정 : NEWVER=$(echo ${DOCKER_TAG})
3. 이후 sed 로 ops-deploy Repo 에 dev-app/VERSION, timeserver.yaml 2개 파일에 ‘기존 버전’ → ‘신규 버전’으로 값 변경
4. 이후 ops-deploy Repo 에 git push ⇒ Argo CD app 가 최대 3분 사이에 변경 확인 후 AutoSync 로 신규 버전 업데이트 진행

아래는 dev-app 에 위치한 Jenkinsfile 로 젠킨스에 SCM-Pipeline(SCM:git) 으로 사용되고 있는 파일을 수정해서 실습에 사용

pipeline {
    agent any
    environment {
        DOCKER_IMAGE = 'gasida/dev-app' // Docker 이미지 이름
        GOGSCRD = credentials('gogs-crd')
    }
    stages {
        stage('dev-app Checkout') {
            steps {
                 git branch: 'main',
                 url: 'http://192.168.254.124:3000/devops/dev-app.git',  // Git에서 코드 체크아웃
                 credentialsId: 'gogs-crd'  // Credentials ID
            }
        }
        stage('Read VERSION') {
            steps {
                script {
                    // VERSION 파일 읽기
                    def version = readFile('VERSION').trim()
                    echo "Version found: ${version}"
                    // 환경 변수 설정
                    env.DOCKER_TAG = version
                }
            }
        }
        stage('Docker Build and Push') {
            steps {
                script {
                    docker.withRegistry('https://index.docker.io/v1/', 'dockerhub-crd') {
                        // DOCKER_TAG 사용
                        def appImage = docker.build("${DOCKER_IMAGE}:${DOCKER_TAG}")
                        appImage.push()
                        appImage.push("latest")
                    }
                }
            }
        }
        stage('ops-deploy Checkout') {
            steps {
                 git branch: 'main',
                 url: 'http://192.168.254.124:3000/devops/ops-deploy.git',  // Git에서 코드 체크아웃
                 credentialsId: 'gogs-crd'  // Credentials ID
            }
        }
        stage('ops-deploy version update push') {
            steps {
                sh '''
                OLDVER=$(cat dev-app/VERSION)
                NEWVER=$(echo ${DOCKER_TAG})
                sed -i -e "s/$OLDVER/$NEWVER/" dev-app/timeserver.yaml
                sed -i -e "s/$OLDVER/$NEWVER/" dev-app/VERSION
                git add ./dev-app
                git config user.name "devops"
                git config user.email "a@a.com"
                git commit -m "version update ${DOCKER_TAG}"
                git push http://${GOGSCRD_USR}:${GOGSCRD_PSW}@192.168.254.124:3000/devops/ops-deploy.git
                '''
            }
        }
    }
    post {
        success {
            echo "Docker image ${DOCKER_IMAGE}:${DOCKER_TAG} has been built and pushed successfully!"
        }
        failure {
            echo "Pipeline failed. Please check the logs."
        }
    }
}

아래는 dev-app (Repo) 에서 git push 수행

# VERSION 파일 수정 : 0.0.3
# server.py 파일 수정 : 0.0.3

# git push : VERSION, server.py, Jenkinsfile
git add . && git commit -m "VERSION $(cat VERSION) Changed" && git push -u origin main

동작 확인

dev-app Repo 에서 한번 더 버전 업데이트 수행

# VERSION 파일 수정 : 0.0.4
# server.py 파일 수정 : 0.0.4

# git push : VERSION, server.py, Jenkinsfile
git add . && git commit -m "VERSION $(cat VERSION) Changed" && git push -u origin main

# VERSION 파일 수정 : 0.0.5
# server.py 파일 수정 : 0.0.5

# git push : VERSION, server.py, Jenkinsfile
git add . && git commit -m "VERSION $(cat VERSION) Changed" && git push -u origin main

Argo Rollout + K8S(Kind)

아키텍처 - Docs

Argo Rollouts 설치 및 Sample 테스트 - Docs

Basic Usage - Argo Rollouts - Kubernetes Progressive Delivery Controller

Getting Started This guide will demonstrate various concepts and features of Argo Rollouts by going through deployment, upgrade, promotion, and abortion of a Rollout. Requirements Kubernetes cluster with argo-rollouts controller installed (see install guid

argoproj.github.io

# 네임스페이스 생성 및 파라미터 파일 작성
cd $PWD

kubectl create ns argo-rollouts
cat <<EOT > argorollouts-values.yaml
dashboard:
  enabled: true
  service:
    type: NodePort
    nodePort: 30003
EOT

# 설치
helm install argo-rollouts argo/argo-rollouts --version 2.35.1 -f argorollouts-values.yaml --namespace argo-rollouts

# 확인
kubectl get all -n argo-rollouts
kubectl get crds

# Argo rollouts 대시보드 접속 주소 확인
echo "http://127.0.0.1:30003"
open "http://127.0.0.1:30003"

Deploying a Rollout

spec:
  replicas: 5
  strategy:
    canary:
      steps:
      - setWeight: 20
      - pause: {}
      - setWeight: 40
      - pause: {duration: 10}
      - setWeight: 60
      - pause: {duration: 10}
      - setWeight: 80
      - pause: {duration: 10}

# Run the following command to deploy the initial Rollout and Service:
kubectl apply -f https://raw.githubusercontent.com/argoproj/argo-rollouts/master/docs/getting-started/basic/rollout.yaml
kubectl apply -f https://raw.githubusercontent.com/argoproj/argo-rollouts/master/docs/getting-started/basic/service.yaml

# 확인
kubectl get rollout
kubectl describe rollout

kubectl get pod -l app=rollouts-demo
kubectl get svc,ep rollouts-demo
kubectl get rollouts rollouts-demo -o json | grep rollouts-demo
...
   "image": "argoproj/rollouts-demo:blue"
...

# Run the following command to update the rollouts-demo Rollout with the "yellow" version of the container:
kubectl argo rollouts set image rollouts-demo rollouts-demo=argoproj/rollouts-demo:yellow
kubectl edit rollouts rollouts-demo
..
     - image: argoproj/rollouts-demo:yellow
...

# 파드 label 정보 확인
watch -d kubectl get pod -l app=rollouts-demo -owide --show-labels

그리고 전체 배포

그러면 순차적으로 진행됩니다.

'study > CICD' 카테고리의 다른 글

CICD 스터디 3주차 첫번째 (0)	2024.12.22
CICD 스터디 2주차 (1)	2024.12.15
CICD 스터디 1주차 두번째 (0)	2024.12.07
CICD 스터디 1주차 첫번째 (1)	2024.12.06

덴고의 공부방

CICD 스터디 3주차 두번째

Jenkins CI + Argo CD + K8S(Kind)

Full CI/CD 구성도

'study > CICD' 카테고리의 다른 글

티스토리툴바

CICD 스터디 3주차 두번째

Jenkins CI + Argo CD + K8S(Kind)

Full CI/CD 구성도

'study > CICD' 카테고리의 다른 글

'study/CICD' Related Articles

티스토리툴바