A C++ GitHub Template Repository utilizing CircleCI, CMAKE, Docker and Doxygen

Welcome back to a new post at thoughts-on-coding.com. This time I would like to give an introduction into the structure of my customized GitHub project template for C++. This C++ project template is consisting of a dummy library and an application that is build by cmake. The project is utilizing a debian based and customized docker image to run continuous integration builds (cmake) and executes tests (ctest) on CircleCI.

The dependencies of the project are resolved via vcpkg which can be customized for each deriving project. Additionally, the project is also generating doxygen based code documentation which is then published to the gh-pages branch. As always the repository can be found and forked at GitHub and is available as v1.1.0.

Principal Project Structure

The project structure is rather simple. It contains

  • An app folder with the actual main function, and a lib folder with all project-related libraries. Both are build by cmake.
  • A buildutils directory which holds the python script necessary to convert ctest results into JUnit from.
  • An images folder containing all project-related images
  • The CircleCI configuration file .circleci/config.yml
  • A Dockerfile as which is used to build the docker image necessary to build the project on CircleCI
  • A Doxyfile which configures the documentation generation with doxygen
Project tree structure with app, lib, build, buildutils, docs and images directories
Project principal directory and file structure

CircleCI Configuration

Let’s start with the CircleCI configuration file .circleci/config.yml. At first, an executor (1) using a customized docker image (2) is defined. The executor allows the definition of the environment a specific job in CircleCI is running in. Instead of defining a docker image for each job, an executor can be reused. The project is build upon two jobs, build and test. Each job contains several steps that have to be fulfilled to make a job successful. The build job is getting the sources from GitHub via the checkout command. Afterward, the project dependencies are setup by installing them with vcpkg (3). Then the project tooling is setup (4) and the build process gets started (5). At the end of the build job, it is necessary to persist (6) the resulting artifacts with a workspace to use the build results in the following test job. This is necessary because the jobs are independent of each other due to the volatile nature of docker.

First, the test job is starting by loading the persisted workspace (7) into the job. The actual invocation of ctest (8) might appear a bit strange because of the || true. The reason is that ctest always returns if a test has been successful or not, but in case a test is failing also the CircleCI job would fail which then would prevent the following job steps. At the end of the job step the test results generated by ctest need to be converted (9) into a JUnit XML conform structure before the test results get published (10).

The workflow defined at the end of the .circleci/config.yml file is defining the actual workflow the build process is following. In this case, the build job is invoked first, followed by the test job which is depending on a successful build job.

version: 2.1
executors: #(1)
image: dockerben/cpptemplate:latest #(2)
executor: exectr #(1)
name: Install vcpkg dependencies
command: ./../../vcpkg/vcpkg install fmt doctest #(3)
name: Create build directories
command: |
mkdir -p build
mv buildutils build/buildutils
name: Setup cmake and build artifacts
command: |
cd build
cmake -DCMAKE_TOOLCHAIN_FILE=/vcpkg/scripts/buildsystems/vcpkg.cmake .. #(4)
cmake –build . –config Release #(5)
persist_to_workspace: #(6)
root: .
paths: build
executor: exectr #(1)
attach_workspace: #(7)
at: .
name: Create test directory
command: |
cd build
mkdir -p Test
name: Execute Tests
command: |
cd build
ctest –no-compress-output -T Test || true #(8)
name: Transform test results into JUnit conform notation
command: |
python3 build/buildutils/ctest2JUnit.py build build/buildutils/CTest2JUnit.xsl > build/Test/results.xml #(9)
store_test_results: #(10)
path: build/Test
version: 2
build #(11)
test: #(12)
view raw config.yml hosted with ❤ by GitHub
Circleci screenshot with a successful workflow
Example CircleCI workflow

With the CircleCI CLI there is also a nice tool to test our CircleCI configuration by sudo circleci local execute or sudo circleci local execute JOBNAME for a specific job to run. The only drawback is that it’s not supporting workflows right now, but still, it’s enough to test the configuration without nee to commit, maybe broken, code to our repositories.

Example job invocation of the circleci CLI

Custom Docker Image

CircleCI is using docker images to provide the build environment and therefore it can be completely customized to our needs. Unfortunately, they don’t provide a C++ ready pre-build docker image yet so we have to define our own. If you’re not already familiar with the docker basics you should have a look at the docker overview.

To build a custom docker image we need to define a, rather simple, Dockerfile which starts with a base image (1) where we want to derive from. To build our C++ project we need to install several tools and libraries (2), such as gcc, clang and so on. Additional python is necessary to convert the ctest results into JUnit XML. To resolve the project dependencies we are installing vcpkg (3) and clean up afterward (4) to reduce the resulting docker image size. With 649MB it’s anyway quite a big image and there might be tricks to reduce it’s size further which I’m not aware of right now.

FROM debian:stable-slim #(1)
LABEL maintainer="ben.mahr@gmail.com" \
description="Image which consists of C++ related build tools." \
RUN apt-get update -y && \
apt-get install -y –no-install-recommends \ #(2)
git \
openssh-server \
curl \
gcc \
g++ \
clang \
build-essential \
cmake \
unzip \
tar \
gzip \
sudo \
python3 \
python3-defusedxml \
python3-lxml \
libssl-dev \
libffi-dev \
ca-certificates && \
apt-get autoclean && \
apt-get autoremove && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
ENV CMAKE_TOOLCHAIN_FILE ${VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake
RUN git clone https://github.com/microsoft/vcpkg.git && \ #(3)
.${VCPKG_ROOT}/bootstrap-vcpkg.sh -disableMetrics && \
sudo .${VCPKG_ROOT}/vcpkg integrate install && \
rm -rf ${VCPKG_ROOT}/buildtrees/* && \ #(4)
rm -rf ${VCPKG_ROOT}/downloads/* #(4)
view raw Dockerfile hosted with ❤ by GitHub

The image can then be build (make sure you’re at the same directory as the Dockerfile is) by sudo docker build -t IMAGENAME ., afterward the resulting image needs to be tagged by sudo docker tag IMAGENAME:TAG DOCKERHUBUSERNAME/IMAGENAME:TAG and then can be pushed to dockerhub by sudo docker push DOCKERHUBUSERNAME/IMAGENAME:TAG.

Converting Test Results

Because ctest results are not JUnit XML conform, which is a requirement by CircleCI, we have to convert them which is done by a python script and an xsl transformation (ref. post by sakra). Both can be found within the buildutils folder. The python script is using the ElementTree API of the xml processing package lxml to process the xslt file. The script starts by resolving the test results folder, which contains the Test.xml, by reading the first line of the TAG file (1). In my case e.g. the folder name, derived from the timestamp the tests have been invoked, is called 20200419-1806. Afterward, the Test.xml file gets parsed (2) and the xslt file is then transformed into an XML tree pointing to its root by the etree.XML() function (3) which is then converted into an XSLT object (4) which transforms the Test.xml (5) into JUnit XML.

from lxml import etree
import io
import sys
TAGfile = open(sys.argv[1]+"/Testing/TAG", 'r') #(1)
dirname = TAGfile.readline().strip()
xslfile = open(sys.argv[2], 'r')
xslcontent = xslfile.read()
xmldoc = etree.parse(sys.argv[1]+"/Testing/"+dirname+"/Test.xml") #(2)
xslt_root = etree.XML(xslcontent) #(3)
transform = etree.XSLT(xslt_root) #(4)
result_tree = transform(xmldoc) #(5)
view raw ctest2JUnit.py hosted with ❤ by GitHub

Documentation Generation and Deployment

For the generation and deployment of the documentation to a gh-pages branch, which is published by GitHub, we use GitHub-Actions. The configuration .github/workflows/doxygen.yml of the GitHub-Action starts by defining the events (1) which will invoke the action. In our case, we want to run the action when someone is pushing or doing a pull request to the master branch or a change happens to a Pages-enabled branch. Then the doxygen-action is loaded (2) which simply needs a path to a Doxyfile (3). The Doxyfile itself is configuring, besides many other settings, the output format (in our case just HTML) GENERATE_HTML = YES, the output path HTML_OUTPUT = docs, and the path to related images IMAGE_PATH = images. It is clear that the docker configuration needs to be customized for each project. By annotating the README.md with {#mainpage} we can use this file as a start page of the documentation. The final step is loading the actions-gh-pages (4) action which is deploying to a gh-pages branch whatever can be found inside the publish_dir (5). To avoid the automatic build invocation, by CircleCI, of the gh-pages branch we add a [ci skip] statement at each commit message (6). Otherwise, we would always have a broken build caused by the gh-pages branch.

name: Documentation
on: #(1)
runs-on: ubuntu-latest
uses: actions/checkout@v1
name: Build documentation
uses: mattnotmitt/doxygen-action@v1 #(2)
doxyfile-path: ./Doxyfile #(3)
name: Deploy documentation
uses: peaceiris/actions-gh-pages@v3 #(4)
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./docs #(5)
commit_message: '[ci skip] ${{ github.event.head_commit.message }}' #(6)
view raw doxygen.yml hosted with ❤ by GitHub
Circleci screenshot with a master and skipped branch
A branch (without a config.yml) can be ignored by adding [ci skip] to the git commit text


With GitHub template repositories and a bit of customized tooling, including a CI pipeline and automated documentation generation and deployment, we have a good and easy to extend starting point for many C++ projects. Although GitHub template repositories can be used for all certain kinds of projects, it is extremely helpful for C++ projects which are mostly a bit more complex in their setup.

Did you like the post?
What are your thoughts? Did you like the post?
Feel free to comment and share this post.

Success! You're on the list.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.