In this post, I’m going to share my current CI/CD configuration for jekyll.is/land, which leverages GitLab Auto DevOps and rssh. Basically, a GitLab runner builds the site via jekyll build and scp’s the _site directory to the web server via a system user secured with the rssh shell.

Configuring GitLab Auto DevOps

The first thing to do is create a .gitlab-ci.yml file, which defines our deployment pattern. The GitLab runner runs a Docker image. For ease of use, I use the jekyll/builder image (from Jordon Bedwell) which comes with Jekyll and SSH tools pre-installed. This image is based on Alpine Linux, using the apk package management tool, so for good measures I’ll run apk update before we get started.

# .gitlab-ci.yml
image: jekyll/builder

before_script:
  - apk update
# ...

The first thing to do in the deploy stage is to move into the Jekyll directory and execute the build command.

# ...
deploy:
  stage: deploy
  script:
    - echo "Building Jekyll site..."
    - cd land
    - jekyll build
# ...

Next, the runner must scp the _site directory, containing the public web files, into the web server. To do this, a system user is made on the web server and configured for public key authentication (we’ll get to this in a bit). The private key is stored in the $SSH_PRIVATE_KEY environment variable on the GitLab Runner. Additionally, the web server’s system user, IP address, SSH port, and web directory are stored in $USER, $ADDRESS, $PORT, and $DIRECTORY, respectively. GitLab environment variables can be configured under Settings -> CI/CD -> Environment Variables.

First, the script makes the private key file, then echo’s the environment variable into the file and sets the permissions to 400. Next, it recursively scp’s the _site directory to the server and, for good measures, deletes the private key from the runner.

# ...
    - echo "Creating SSH private key..."
    - touch ~/id_rsa
    - echo "$SSH_PRIVATE_KEY" | tr -d '\r' >> ~/id_rsa
    - chmod 400 ~/id_rsa
    - echo "Copying public files with SCP..."
    - scp -r -i ~/id_rsa -o StrictHostKeyChecking=no -P $PORT _site/* $USER@$ADDRESS:$DIRECTORY
    - echo "Removing SSH private key..."
    - rm ~/id_rsa
# ...

The -o StrictHostKeyChecking=no option may look interesting. After running into authentication issues without it, I found this Stack Exchange post that exhibited my same issue. It appears that because the connecting server - the GitLab Runner - is not static, the authentication fails because it is an unknown server, or different from the last server that authenticated. Enabling this option skips this kind of check.

More information on using SSH keys with GitLab CI/CD can be found on their website.

The last thing to do with GitLab Auto DevOps is to finish the .gitlab-ci.yml file by defining the environments and branches it applies to.

# ...
  environment:
    name: island-ark
    url: https://jekyll.is/land
  only:
    - master

Configuring the web server and rssh

Because the system user on the web server is only ever going to receive files into a directory, I wanted to limit what it’s allowed to do, and if possible, restrict its login shell. I found this Stack Exchange post which asks “Do you need a shell for SCP?” The short answer is yes - but there are shells that limit the user to only SCP methods. rssh and scponly are two solutions that accomplish this. I choose rssh.

The first step is to download and install the rssh source on the web server, which is done with your favorite GNU tools - .configure make and sudo make install.

Next, make the system user. The system user needs to have a home directory (for the ssh key) and has its shell set to rssh, which in my case is located at /usr/local/bin/rssh. I’ve also added the -r option which defines the user as a system user. The following command does all of this:

mkdir /home/username
sudo useradd -r -d /home/username -s /usr/local/bin/rssh username

To make the SSH keypair, move into the new user’s directory and create the keypair with the -f <keyname> option so as to not overwrite an existing keypair. Move or copy the public key into the .ssh/authorized_keys file.

cd /home/username
sudo ssh-keygen -f key
sudo mkdir .ssh
sudo mv key.pub .ssh/authorized_keys

Next, sudo cat key and copy the private key into a GitLab environment variable. You may optionally delete the key file from the server, or back it up into the .ssh directory or another safe location. Remember that this keypair can be recreated later!

Finally, make sure that the new system user owns its own directory (including its new key files) and the web directory.

sudo chown -r username:username /home/username
sudo chown -r username:username <path-to-web-directory>

If all is configured, pushing your Jekyll source directory should trigger the deployment script and copy the built site to the web server!

A note on security

No doubt is it a little scary to put a private key in the GitLab cloud, even as an environment variable. There is some insight from a Reddit user as to how these variables are internally stored, which seems to be reasonably secure.

However, the great thing about keypairs is how easily they can be recreated and redeployed.

The bigger security issue, in my opinion, lies in the rssh program, which is unmaintained. It is important to do your due diligence as an rssh user - read the security page and join the mailing list. rssh covers a variety of programs (scp, sftp, rdist, rsync, and cvs) and old dogs turn up new bugs so it is important to stay on top of vulnerabilities and know when to manually update your program.

Thanks for reading!