This guide is a successor to Alex Ellis's guide to deploying your applications to Civo with GitOps. It will use some of the same terms and similar techniques, but it will repeat all key information so that this post can act as a stand-alone guide. All code used in this post can be found here.

What is GitOps?

GitOps as a term was first coined by Alexis Richardson, the CEO of Weaveworks.

Alexis describes GitOps as the following:

Describe the desired state of the whole system using a declarative specification for each environment.

An alternative short description of GitOps is "Operations by Pull Request". The idea is that all configuration specifications are stored in a repository, and can be deployed at the push of a button. GitHub Actions are, as of November 13 2019, in general availability and accessible to all users. This means we can use them and the Civo API Python library to construct a static site GitOps work flow.

This post will cover requirements for deploying your application in this way, the set-up of the deployment, and the use of GitHub Actions as triggers for deployment updates based on specifications in the repository.

Requirements

Apply GitOps in Civo using a Python library

There is a Python library to interact with Civo available here. This library can be used to manage your infrastructure on the platform including: Instances (Virtual Machines), Firewalls, Load balancers, Snapshots, and more.

To keep this demonstration simple, these are the steps we will follow:

  • Download the Civo Python library for a dry run demonstration.
  • Create an Instance in your Civo account.
  • Write the logic in Python using the Civo library. We will not be running GitOps as a separate daemon to keep the demonstration simple and quick to follow. A separate daemon would require additional infrastructure.
  • We will deploy a static website and use Nginx to host it. GitOps tends to involve orchestrating Kubernetes, but, again, for simplicity, we will be deploying a static site.

A quick look to the Python library

In our case, the library will be used directly when called by GitHub Actions. But to test we can install it in an environment, or directly on your system in this way, assuming you have Python installed:

pip install civo

To be able to use it we must have our API token in our environment variables, or pass it directly to the library substituting our key for the 'token' below:

from civo import Civo
from os.path import expanduser
​
civo = Civo('token')
home = expanduser("~/.ssh/")
ssh_file = open('{}id_dsa.pub'.format(home)).read()
​
# you can filter the result
size_id = civo.size.search(filter='name:g2.xsmall')[0]['name']
template = civo.templates.search(filter='code:debian-stretch')[0]['id']
​
civo.ssh.create(name='default', public_key=ssh_file)
ssh_id = civo.ssh.search(filter='name:default')[0]['id']
civo.instances.create(hostname='text.example.com', size=size_id,
                      region='lon1', template_id=template,
                      public_ip='true', ssh_key=ssh_id)

This is an example of how to add our SSH keys to the tool, and then create an instance that uses that key. We will be using a variant of this code later to run as GitOps.

The Fun Part

Add secrets to Github and SSH key to Civo

The first thing, if you have not already added one, will be to add our SSH key to Civo either through the web UI, the Civo CLI tool, or using the Python library.

Then we need to go to GitHub and create a new Repository. In my case I called my repository gitops-civo. Once in the repository, under Settings > Secrets we will add two keys, called CIVO_TOKEN and SSHKEYPRIVATE. This will allow us to connect from either the Python library that is triggered through GitHub Actions, or through SSH to our instance.

CIVO_TOKEN is your API key, obtained from the API page. SSHKEYPRIVATE is the SSH key you have generated as a base64 encoded string, obtained by running the command base64 ~/.ssh/id_rsa. Simply paste this string into the Secret field.

Add static site content to repository

We will need some content to publish onto our site. Let's create a webroot/ directory, inside which we will create index.html with the following contents: html <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>GitOps</title> </head> <body style="background-color: darkgray; text-align: center"> <h4>Hello from GitOps with Github Actions</h4> </body> </html> Push the contents to your repository, making sure you end up with webroot/index.html and move on to create the workflow.

Create the workflow in github

To create our workflow, the first thing will be to go to the GitHub Actions page to create your own action. Create a new Action, call it gitops.yml and paste in this content:

name: GitOps

on:
  push:
    branches:
    - master

jobs:
  build:
    runs-on: ubuntu-latest   
    steps:
    - uses: actions/checkout@v1
    - name: Setup Python for use with Actions
      uses: actions/setup-python@v1.0.0
    - name: Install client-python from Civo
      run: |
        pip3 install civo
        pip3 install fabric
    - name: Add ssh private key to the server
      run: |
        mkdir -p $HOME/.ssh/
        chmod 700 $HOME/.ssh/
        echo -n "${{ secrets.SSH_KEY_PRIVATE }}" | base64 --decode > $HOME/.ssh/id_rsa
        chmod 600 $HOME/.ssh/id_rsa
        stat $HOME/.ssh/id_rsa
    - name: Compact webroot
      run: tar -vczf webroot.gz webroot/
    - name: Run check script
      env:
        CIVO_TOKEN: ${{ secrets.CIVO_TOKEN }}
      run: python check.py

It will look like this: GitOps Actions

Click on "Start commit" and save the changes of your new action.

Now I am going to explain more or less line by line how this file works:

name: GitOps

on:
  push:
    branches:
    - master

The first is the name of our action, the second part is when this action will be executed, which in this case is when a push is made to the master branch. This could be modified and done when a release is made, or, if our app was written in a language like Golang for example, after compiling it we could send this action to run and deploy our app.

jobs:
  build:
    runs-on: ubuntu-latest   
    steps:
    - uses: actions/checkout@v1
    - name: Setup Python for use with Actions
      uses: actions/setup-python@v1.0.0
    - name: Install client-python from Civo
      run: |
        pip3 install civo
        pip3 install fabric
    - name: Add ssh private key to the server
      run: |
        mkdir -p $HOME/.ssh/
        chmod 700 $HOME/.ssh/
        echo -n "${{ secrets.SSH_KEY_PRIVATE }}" | base64 --decode > $HOME/.ssh/id_rsa
        chmod 600 $HOME/.ssh/id_rsa
        stat $HOME/.ssh/id_rsa
    - name: Compact webroot
      run: tar vczf webroot.gz webroot
    - name: Run check script
      env:
        CIVO_TOKEN: ${{ secrets.CIVO_TOKEN }}
      run: python check.py

The above job consists of several steps that can be defined by a user themselves, or use ones predefined by GitHub. In this case we use a mix of both. The first is checkout@v1 which is defined by GitHub and is to checkout the project for the intance that is created. The second one, setup-python@v1.0.0 is also from GitHub, and installs Python in the created instance.

Now come the ones created by me for this guide. The first thing is to install the libraries civo and fabric to be able to connect via SSH to the instance created in Civo's cloud.

The next part creates our SSH key inside the instance using the GitHub Secrets, and the third step is to compact our webroot.

Finally, it is time to create and run a Python script to do the magic and the fun part, running check.py.

We will need to create check.py with the following content, which I will also explain. If you tried the code in the above section on your local machine, this will look familiar:

import time
from fabric import Connection
from civo import Civo

hostname_default = 'gitops-civo.example.com' # Change this to the hostname of your choice

civo = Civo()
size_id = civo.size.search(filter='name:g2.xsmall')[0]['name']
template = civo.templates.search(filter='code:debian-stretch')[0]['id']
search_hostname = civo.instances.search(filter='hostname:{}'.format(hostname_default))
ssh_id = civo.ssh.search(filter='name:alejandrojnm')[0]['id'] # Change the part after name: to your SSH Key name

if not search_hostname:
    instance = civo.instances.create(hostname=hostname_default, size=size_id, region='lon1', template_id=template,
    public_ip='true', ssh_key_id=ssh_id)
    status = instance['status']

    while status != 'ACTIVE':
        status = civo.instances.search(filter='hostname:{}'.format(hostname_default))[0]['status']
        time.sleep(10)
    # add this because a new instance needs to start the ssh-daemon
    time.sleep(20)


ip_server = civo.instances.search(filter='hostname:{}'.format(hostname_default))[0]['public_ip']
username = 'admin'

c = Connection('{}@{}'.format(username, ip_server))
result = c.put('webroot.gz', remote='/tmp')
print("Uploaded {0.local} to {0.remote}".format(result))
c.sudo('apt update')
c.sudo('apt install -qy nginx')
c.sudo('rm -rf /var/www/html/*')
c.sudo('tar -C /var/www/html/ -xzvf /tmp/webroot.gz')

I think Python is one of the languages that when you see the code, it will be fairly clear, but in this case we will do it, for the benefit of those who are not as familiar with the language.

The first thing is to import the libraries necessary for our case:

import time
from fabric import Connection
from civo import Civo

Second, we declare the main variables to use:

hostname_default = 'gitops-civo.example.com'

civo = Civo()
size_id = civo.size.search(filter='name:g2.xsmall')[0]['name']
template = civo.templates.search(filter='code:debian-stretch')[0]['id']
search_hostname = civo.instances.search(filter='hostname:{}'.format(hostname_default))
ssh_id = civo.ssh.search(filter='name:alejandrojnm')[0]['id']

hostname_default is the name of our instance, which you can call what you want to. Then we create the class Civo(). We create several variables such as the size of the instance to use, the template, then we check if there is an instance already created with the name we created above. Finally we look for the SSH keys in our Civo cloud account, to be able to assign the key to the instance we are going to create.

Then we have this:

if not search_hostname:
    instance = civo.instances.create(hostname=hostname_default, size=size_id, region='lon1', template_id=template,
    public_ip='true', ssh_key_id=ssh_id)
    status = instance['status']

    while status != 'ACTIVE':
        status = civo.instances.search(filter='hostname:{}'.format(hostname_default))[0]['status']
        time.sleep(10)

The above says that if there is not an instance with the chosen name, we create one, and wait until Civo responds that the instance is ACTIVE (this gets asked every 10 seconds, so the time.sleep(10))

And now the final part of the script:

# add this because the instance is not ready to handle ssh connection so fast
time.sleep(20)
ip_server = civo.instances.search(filter='hostname:{}'.format(hostname_default))[0]['public_ip']
username = 'admin'

c = Connection('{}@{}'.format(username, ip_server))
result = c.put('webroot.gz', remote='/tmp')
print("Uploaded {0.local} to {0.remote}".format(result))
c.sudo('apt update')
c.sudo('apt install -qy nginx')
c.sudo('rm -rf /var/www/html/*')
c.sudo('tar -C /var/www/html/ -xzvf /tmp/webroot.gz')

The first line is because the instance takes a little time from booting to start responding to SSH. After that, we get the IP address of the instance. In my case the username is admin because my instance is Debian. Then we open a connection with fabric to the IP of the instance, copy the compacted webroot to /tmp, send to install nginx, delete the contents of /var/www/html/ and unpack webroot.gz in that directory.

Conclusion

Your site should now be visible at the IP address of your instance, in the directory webroot/. Any changes you make to files in the webroot directory in your local repository will be reflected on the remote host as soon as the Action completes after you push changes. This will allow you to deploy changes onto your site from anywhere with git access.

And well, this is it! Though it is a very basic example of how to use the Python library of Civo to run GitOps, I hope you find this option for deployments interesting and inspiring. It could be easily made more complex, such as by employing Kubernetes through the world's first managed k3s offering from Civo.