Quest For CICD - ConcourseCI

Table of Contents

The Quest for CICD - This article is part of a series.

Part 3: This Article

User Interface

I really like the concourse UI. It’s really simple and easy to understand and show the right amount of info to get you going with developing pipelines.

I particularly like the way it graphs out your pipelines so that complex ones are easy to follow.

Here we see my bild pipeline consists of 1 imput task (a git clone from omaraminme) and 2 output tasks (omaraminimage that builds the docker image and dockerserverssh that logins into my docker server and pulls + restarts the site image).

The output through the UI is useful as well. It shows console output for each of the tasks in the pipeline:

Pipeline Syntax - YAML

ConcourseCI uses a pretty simele YAML based syntax. The main thing to get your head around is that everything is a resource with an input and an output.

resource_types:
- name: ssh
  type: docker-image
  source:
    repository: quay.io/henry40408/concourse-ssh-resource
  
resources:
- name: omaraminme
  type: git
  check_every: never
  webhook_token: buildomaraminme
  icon: github
  source:
    uri: <GIT_REPOSITORY_url>/omaraminme.git
    username: <GITEA_USERNAME>
    password: <GITEA_PASSWORD>

- name: omaraminimage
  type: registry-image
  icon: docker
  source:
    repository: <CONTAINER_REPOSITORY>omaraminme
    tag: latest
    username: <GITEA_USERNAME>
    password: <GITEA_PASSWORD>

- name: dockerserverssh
  type: ssh
  source:
    host: dockerserver.homelan
    user: <DOCKER_SERVER_USERNAME>
    password: <DOCKER_SERVER_PASSWORD>

jobs:
- name: build
  plan:
  - get: omaraminme
    trigger: true
  - task: buildsite
    config:
      platform: linux
      inputs:
      - name: omaraminme      
      outputs:
      - name: omaraminme
      image_resource:
        type: registry-image
        source: 
          repository: jekyll/builder
      run:
        path: sh
        args:
        - -exc
        - |
          ls -lh
          chown jekyll:jekyll omaraminme
          cd omaraminme
          jekyll build          
  - task: builddockerimage
    privileged: true # oci-build-task must run in a privileged container
    config:
      platform: linux
      image_resource:
        type: registry-image
        source:
          repository: concourse/oci-build-task
      inputs:
      - name: omaraminme
        path: .
      outputs:
      - name: image       
      run: # binary used to build the image
        path: build
  - put: omaraminimage
    params:
      image: image/image.tar
  - put: dockerserverssh
    params:
      interpreter: /bin/sh
      script: |
        cd /Docker
        docker compose pull omaraminme
        docker compose up -d omaraminme

I’ve replaced some confidential info with <> blocks.

My pipeline file consists of the following:

A pre-build stage where I define the resources I’ll be using
A jobs: block where the pipeline is defined. It consists of 2 tasks: and 2 put: steps

Pre-Build Stage

Here we define the resources we are going to use. You can use built in resources that are defined in the resources: block and you can define your own type of resources in the resource_types: block. What’s important to note is that all resources are docker images that will be pulled and run as part of the pipeline.

resource_types:
- name: ssh
  type: docker-image
  source:
    repository: quay.io/henry40408/concourse-ssh-resource
  
resources:
- name: omaraminme
  type: git
  check_every: never
  webhook_token: buildomaraminme
  icon: github
  source:
    uri: <GIT_REPOSITORY_url>/omaraminme.git
    username: <GITEA_USERNAME>
    password: <GITEA_PASSWORD>

- name: omaraminimage
  type: registry-image
  icon: docker
  source:
    repository: <CONTAINER_REPOSITORY>omaraminme
    tag: latest
    username: <GITEA_USERNAME>
    password: <GITEA_PASSWORD>

- name: dockerserverssh
  type: ssh
  source:
    host: dockerserver.homelan
    user: <DOCKER_SERVER_USERNAME>
    password: <DOCKER_SERVER_PASSWORD>

I’ve replaced some confidential info with <> blocks.

I’m defining one new resource type with the name ssh. This is a docker image that will allow me to ssh into other machines.

I’m also using 2 built in resources:

A git repository named omaraminme - This is my site git repo
A docker image named omaraminimage - This is what I will build and push into my container repository

The dockerserverssh resource is an instanciation of the ssh resource type I defined earlier.

Task 1 - Build the Jekyll Site

In the jobs: block we define a plan. A plan is a sequential list of tasks that will get run. The first task defined is to build the jekyll site. I have set trigger=true to allow this job to be triggered by the Gitea webhook.

  - task: buildsite
    config:
      platform: linux
      inputs:
      - name: omaraminme      
      outputs:
      - name: omaraminme
      image_resource:
        type: registry-image
        source: 
          repository: jekyll/builder
      run:
        path: sh
        args:
        - -exc
        - |
          ls -lh
          chown jekyll:jekyll omaraminme
          cd omaraminme
          jekyll build

For each task in a ConcourseCI pipeline, we can define a config: block that configures the inputs, outputs and resources used by that task. Here we have said:

The task will run on a linux platform
We have a git repository resource named omaraminme as an input (defined in the Pre-Build Stage)
We will output the same git repository (after we have build the Jekyll site)
We will use the jekyll/builder docker image to run the steps of this task

The run: block then defines the shell commands that will be run to build the Jekyll site.

In essance, this task is cloning the git repository that contains all of the Jekyll markdown files and Dockerfile that are required. Running ‘jekyll build` against them and outputting the repository with the build Jekyll files (in the _site directory) and Dockerfile.

Task 2 - Build the Docker Image

  - task: builddockerimage
    privileged: true # oci-build-task must run in a privileged container
    config:
      platform: linux
      image_resource:
        type: registry-image
        source:
          repository: concourse/oci-build-task
      inputs:
      - name: omaraminme
        path: .
      outputs:
      - name: image       
      run: # binary used to build the image
        path: build

The config: block for this task is defined as:

The task will run on a linux platform
The resource used will be the concourse/oci-build-task docker image
It will take the omaraminme git repo as an input (which includes the Dockerfile)
The output will be a docker image

The run: block states that the build command from the docker image will be executed.

Put Step 1 - Push the Docker Image to the Repository

  - put: omaraminimage
    params:
      image: image/image.tar

This put task takes the image output of the builddockerimage task and pushes it to the omaraminimage resource (which is an image in a container repository)

Put Step 2 - SSH into the Docker Server

  - put: dockerserverssh
    params:
      interpreter: /bin/sh
      script: |
        cd Docker
        docker compose pull omaraminme
        docker compose up -d omaraminme