Github Automation with API Gateways and Lambda

Development

Github Automation with API Gateways and Lambda

by Nicholas Anderson

24 Oct 2017 · 10 min read

Sick and tired of doing things that should be automated but aren't?

As a recent addition to the Inlight team, I have worked on several projects and observed a number of processes. Often repetitive, not very time consuming and prone to be forgotten, it made me think of automation and how it could benefit the team. One of my first goals was to automate Github repo tasks with the goal of providing a solid project foundation as well as greater coherency between projects.

Our Goal

Whenever a Github repository is created there are a number of tasks that are required to complete. If forgotten, the repository is at risk of error. Automating these tasks would save time and improve consistency between repos, increasing reliability and decreasing the chance for errors. This line of thought has allowed us to explore further opportunities for automation in an effort to save time and reduce error.

The GitHub API

Scouting out GitHub’s API and finding out what type of settings we can automate was essential. GitHub’s extensive API and perfect timing for the full release of a branch protection API, I found out that they had support calls for everything we need. This included; editing the protection on a branch, creating labels and editing base repo settings such as the type of merges that are permitted. While looking at that I took a look at GitHub’s web hooks and saw that they can send us specific events when they occur.

During this period, I also observed our current repos and found inconsistencies such as unprotected master branches and voluntary reviews of code before merge into a master. Making these reviews voluntary instead of compulsory means the codebase might have inconsistencies when merged with the master.

The framework

After the research, I decided what platform we were going to use. I figured that we’d need a way for GitHub to notify us, meaning we needed an API gateway. For this reason, we have decided to create an API gateway using node.js with express.js to get an easy and fast gateway up and running. After setting that up and configuring the application to our liking, we used apex/up as the serverless framework to deploy the application straight to the hosting with AWS Lambda.

Setting up the node application is as simple as installing express.js and then creating a server off of that. Here’s some code to get you started

        const express = require('express');
const PORT = process.env.PORT;
const app = express();
const server = require('http').Server(app);
server.listen(PORT, function() {
  console.log('API running on port', PORT);
});
      

I suggest you have a read of express.js and its functionality if you haven’t already. It’s super lightweight and easy to use. Being new to backend development and still constantly learning heaps of new things, this was the second time i’ve ever used node.js and was still learning about how to use node, it was surprisingly easy to learn when creating a fun, but extremely useful application like this.

A tricky bit for me during this process of creating the framework was trying to figure out how to decrypt the data that GitHub sends to you at the same time that you want to run some commands. I do suggest enabling a secret key this to secure your application from any outsiders using it. Since I hadn’t done much cryptography in my time with node it was all about finding the libraries it has that can do what we need it to do. A great hint here is to ensure that you use

crypto.timingSafeEqual(new Buffer(hash), calculated));

This allows you to check your hash vs the one that GitHub sent without leaking any timing information that a hacker may use to find one of the values.

Why an API Gateway and Lambda?

AWS Lambda was chosen due to the nature of the API potentially not being used for days and still being readily available. Lambda allows us to keep the API gateway up and running without charging us until it is used and only per use, which is cost efficient - because how many GitHub repos do you really make? The gateway allows us to listen for GitHub whenever a repo is created so that we can then run our calls to GitHub’s API.

Basically, the gateway allows GitHub to communicate with us, and Lambda is an extremely cheap way for us to host our code which will respond to GitHub’s requests. The image to the side demonstrates the basic flow of what we are trying to accomplish here - receiving a call to our API from GitHub and then our Lambda application sending back data to GitHub using the data received from GitHub.

The automation

By this time, we had created an API gateway which received data from GitHub webhook and then with that data, automatically created labels, set the protection of a branch and set repo settings including the collaborators. Firstly, we created the endpoint `github` where GitHub will post its event to us along with the data

        app.post('/github', githubPost);

function githubPost(req, res) {
 // Bail early if this isn't an action we're interested in (e.g. created).
 if(req.body.action !== 'created') {
   console.log('Did not get a created event, returning 200');
   return res.sendStatus(200);
 }

 // Bail early if the hashes don't match (someone else may be spoofing Github).
 if (!verifyHash(req.headers['x-hub-signature'], JSON.stringify(req.body))) {
   console.log('Hash did not match, returning 400');
   return res.sendStatus(400);
 }
}
      

githubPost is a function which will check whether the GitHub event is a `created` event - and if it is it will continue on with its process of checking the hash.

After creating that function, use the data sent from GitHub to run your own requests to GitHub’s API. Don’t forget when setting this up with GitHub that you should set a secret which sends through a header with the data sent which is encrypted. The following is the verifyHash function, which allows you to check the hash sent and compare it to the computed hash with your secret.

        function verifyHash(hash, data) {
 const hmac = crypto.createHmac('sha1', process.env.GITHUB_SECRET_KEY);
 const calculated = new Buffer('sha1=' + hmac.update(data).digest('hex'));

 try {
   if (crypto.timingSafeEqual(new Buffer(hash), calculated)) {
     return true;
   }
 } catch (err) {
   return false;
 }
 return false;
}
      

What’s next?

This API Gateway is completely expandable and can be set to do all sorts of things. This API is just a stepping stone to getting more processes automated throughout all platforms that we use. Being able to eliminate inconsistencies like the ones discussed in the blog will provide us benefit in future projects and ensuring the integrity of the code that is in any codebase.

Some references, for you.

GitHub API

AWS Lambda

Serverless Computing

Express.js

apex/up

About the author

Nicholas Anderson

Student Developer

Nicholas Anderson | Student Developer

Nicholas is currently a Student Developer who is here at Inlight for his placement for Swinburne University of Technology. Nicholas enjoys developing and has only more recently been introduced to web development during his time at Inlight. He has been a responsible for development in projects including MCRI, Red Island and the Inlight website.

Connect with Nicholas on LinkedIn

All Articles

Development