Skip to main content

Quest For CICD - Jenkins

·4 mins
Homelab CICD Jenkins Automation FOSS Selfhosted
Omar Amin
Author
Omar Amin
Loves boxing, FOSS and Selfhosting
Table of Contents
The Quest for CICD - This article is part of a series.
Part 2: This Article

User Interface

I was fine with the user interface. It’s relatively easy to understand when creating pipelines and executing them from the web page.

Jenkins Main Interface
Here’s what your pipeline looks like visualised as stages:
Jenkins Pipeline Console
As I mentionerd in my last post, the console output was fine for reviewing any issues, but some of error messages I got back were quite obtuse.
Jenkins Pipeline Console

Pipeline Syntax - Jenkinsfile

Jenkins has it’s own syntax unlike some other platforms that use plain YAML or TOML. It had a bit of a steep learning curve for me:

pipeline {
    agent { label 'dockeragentdnid'}

    stages {
        stage('Clone repository') { 
            steps { 
                script{
                cleanWs()                    
                checkout scm
                }
            }
        }
        stage ('BuildJekyll') {
            steps {
                script {

                    sh 'docker run -i --rm -v <DOCKERPATH>/jenkinsagent/workspace/Gitea_CICD_omaraminme_main:/srv/jekyll jekyll/builder:latest jekyll build'
    
                }
            }
        }
        stage('BuildandDeploy') {
            steps {
                script{
                    app = docker.build("<CONTAINER_REPOSITORY_URL>/omaraminme")
                    docker.withRegistry('<CONTAINER_REPOSITORY_URL>', '<REGISTRY_PASSWORD>') {
                        app.push("${env.BUILD_NUMBER}")
                        app.push("latest")                 
                    }
                }
            }
        }
        stage('Test'){
            steps {
                script {
                    if (fileExists('_site/index.html')) {
                        echo "Jekyll Site Built"
                    }
                }
            }
        }
    }
    post {
        success {
            sh 'sshpass <SSH_CREDENTIALS> "cd <DOCKERPATH> && docker pull <CONTAINER_REPOSITORY_URL>/omaraminme && docker compose up -d omaraminme"'
            sh 'curl -s -X POST https://api.telegram.org/<TELEGRAM_SECRET>/sendMessage -d chat_id=<TELEGRAM_CHAT> -d text="omaraminme build success"'
            emailext to: "<EMAIL>",
            subject: "jenkins build:${currentBuild.currentResult}: ${env.JOB_NAME}",
            body: "Success!"
        }
        failure{
            sh 'curl -s -X POST https://api.telegram.org/<TELEGRAM_SECRET>/sendMessage -d chat_id=<TELEGRAM_CHAT> -d text="omaraminme build failed"'
            emailext to: "<EMAIL>",
            subject: "jenkins build:${currentBuild.currentResult}: ${env.JOB_NAME}",
            body: "${currentBuild.currentResult}: Job ${env.JOB_NAME}\nMore Info can be found here: ${env.BUILD_URL}"
        }        
        cleanup {
            /* clean up our workspace */
            deleteDir()
            /* clean up tmp directory */
            dir("${workspace}@tmp") {
                deleteDir()
            }
            /* clean up script directory */
            dir("${workspace}@script") {
                deleteDir()
            }
         }
    }
}
I’ve replaced some confidential info with <> blocks.

What happens when Jenkins receives a webhook from Gitea is that it spins up the agent defined in the Jenkinsfile and runs through the stages. My pipeline consists of 4 stages and 2 post build actions. I’ll try to describe what’s going on in a bit more detail:

Pre-Build Stage

The only pre-build task I really used was specifing a Docker in Docker (DinD) agent with this code agent { label 'dockeragentdnid'} (I described the setup of this here).

Stage 1 - Clone Repository

        stage('Clone repository') { 
            steps { 
                script{
                cleanWs()                    
                checkout scm
                }
            }
        }

This code block clones the repository from my gitea instance into the workspace directory ON THE DOCKER SERVER. That’s right, in the config of the agent, I had to set up a mount from the docker server into the DinD agent. This was the only way I figured out how to persist files accross the different containers.

checkout scm is smart enough to understand cloning the correct repository - i.e. the one that generated the webhook to kick off this pipeline.

Stage 2 - Build the Jekyll Site

        stage ('BuildJekyll') {
            steps {
                script {

                    sh 'docker run -i --rm -v <DOCKERPATH>/jenkinsagent/workspace/Gitea_CICD_omaraminme_main:/srv/jekyll jekyll/builder:latest jekyll build'
    
                }
            }
        }

This block tells jenkins to pull and run the jekyll/builder image with the command jekyll build The important thing to note here is that we need to mount the workspace directory into the jekyll container and that the path to this is on the docker server machine, not a path within the DinD container. This really bad block diagram shows what I am trying to explain:

DinD Block Diagram

Stage 3 - Build and Push to Registry

        stage('BuildandDeploy') {
            steps {
                script{
                    app = docker.build("<CONTAINER_REPOSITORY_URL>/omaraminme")
                    docker.withRegistry('<CONTAINER_REPOSITORY_URL>', '<REGISTRY_PASSWORD>') {
                        app.push("${env.BUILD_NUMBER}")
                        app.push("latest")                 
                    }
                }
            }
        }

This stage builds whatever Dockerfile is present in the workspace directory, tags the image and pushes it to my Gitea container repository.

Stage 4 - Testing

        stage('Test'){
            steps {
                script {
                    if (fileExists('_site/index.html')) {
                        echo "Jekyll Site Built"
                    }
                }
            }
        }

A simple test to see if the index.html exists. This means Jekyll has built something

Post Build Tasks

        success {
            sh 'sshpass <SSH_CREDENTIALS> "cd <DOCKERPATH> && docker pull <CONTAINER_REPOSITORY_URL>/omaraminme && docker compose up -d omaraminme"'
            sh 'curl -s -X POST https://api.telegram.org/<TELEGRAM_SECRET>/sendMessage -d chat_id=<TELEGRAM_CHAT> -d text="omaraminme build success"'
            emailext to: "<EMAIL>",
            subject: "jenkins build:${currentBuild.currentResult}: ${env.JOB_NAME}",
            body: "Success!"
        }
        failure{
            sh 'curl -s -X POST https://api.telegram.org/<TELEGRAM_SECRET>/sendMessage -d chat_id=<TELEGRAM_CHAT> -d text="omaraminme build failed"'
            emailext to: "<EMAIL>",
            subject: "jenkins build:${currentBuild.currentResult}: ${env.JOB_NAME}",
            body: "${currentBuild.currentResult}: Job ${env.JOB_NAME}\nMore Info can be found here: ${env.BUILD_URL}"
        }        

This is a conditional block that emails me and send me a telegram message on success or fail of the build

        cleanup {
            /* clean up our workspace */
            deleteDir()
            /* clean up tmp directory */
            dir("${workspace}@tmp") {
                deleteDir()
            }
            /* clean up script directory */
            dir("${workspace}@script") {
                deleteDir()
            }
         }

Since the workspace directory is persistant on the docker server, I have to clean it up to make sure old files don’t interfere with consequent builds.

The Quest for CICD - This article is part of a series.
Part 2: This Article

Related

My Quest for CICD
·10 mins
Homelab CICD WoodpeckerCI Jenkins ConcourseCI Automation FOSS Selfhosted
Summary of the three CICD platforms I have tried - Jenkins, ConcourseCI and WoodpeckerCI