I recently came across the problem that I had some long running GitHub Actions that needed only a few resources. So I figured that this workload would be AWS free-tier eligible. To save some money I looked for a way to use some self-hosted GitHub runners.
Luckily, there is already an open source solution for that: https://github.com/machulav/ec2-github-runner.
It allows you to use EC2 instances as runners. It creates a new EC2 instance, registers it as runner and forces the GitHub job to that runner. Afterwards it de-registers the runner and terminates the EC2 instance.
Volodymyr Machula, the creator of the e2-github-runner action already provides a comprehensive description how to set up the system. However, I want to add some details and provide Terraform code to setup the necessary AWS resources.
How do Self-Hosted Runners Work?
If you used GitHub Actions before you know the GitHub-hosted runners. You write jobs and specify with runs-on a base image name which determines to which runner a job is delegated. Those jobs will run on a GitHub server and you are billed per minute. But you if you have registered your own runners, the runs-on can also select labels of your self-hosted runners.
GitHub provides the runner software that they use so that we can run it on our own machines. The runner software receives the jobs from GitHub and takes care of things like secret management. It will also update itself regularly automatically.
The runner has two basic commands: ./config.sh which registers the runner at GitHub for a repository or organisation. You can provide a list of labels, based on which jobs are matched to the runner. And ./run.sh which starts the runner.
You have to provide a bash environment in which the runner is started. That can be in a Docker container or native on a virtual machine. I will show you how to run it on a virtual machine from AWS.
Setting Up AWS Resources with Terraform
If I create AWS resources I always try to use Terraform as much as possible. It should be mandatory if you work in a team, so that everyone can easily comprehend when which resources were created for a specific purpose. I assume that already know a little bit about Terraform, if not go check out this tutorial.
Here you define the default Terraform configurations for AWS:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 4.16"
}
required_version = "~> 1.3"
}
provider "aws" {
region = "eu-central-1"
}
You need a user that can manage EC2 instances: create, configure and terminate them. The script first creates the user and then attaches an IAM policy to it. Lastly it defines the outputs for the access key id and secret key.
resource "aws_iam_user" "github-runner-manager" {
name = "github-runner-manager"
tags = {
CreatedBy = "terraform"
}
}
data "aws_iam_policy_document" "manage-ec2-instances" {
statement {
effect = "Allow"
actions = [
"ec2:RunInstances",
"ec2:TerminateInstances",
"ec2:DescribeInstances",
"ec2:DescribeInstanceStatus",
]
resources = ["*"]
}
}
resource "aws_iam_policy" "manage-ec2-instances" {
policy = data.aws_iam_policy_document.manage-ec2-instances.json
}
resource "aws_iam_user_policy_attachment" "manage-ec2-instances" {
policy_arn = aws_iam_policy.manage-ec2-instances.arn
user = aws_iam_user.github-runner-manager.name
}
resource "aws_iam_access_key" "github-runner-manager-access-key" {
user = aws_iam_user.github-runner-manager.name
}
output "github-runner-manager-access-key-id" {
description = "AWS access key id for the github-runner-manager"
value = aws_iam_access_key.github-runner-manager-access-key.id
}
output "github-runner-manager-access-key-secret" {
description = "AWS access key secret for the github-runner-manager"
value = nonsensitive(aws_iam_access_key.github-runner-manager-access-key.secret)
}
Your EC2 instances will need network connectivity and have to access the internet, e.g. GitHub to register themselves as runners. For that you need a virtual private cloud (VPC). This VPC has a subnet in which the instances will live. To connect to the internet from the subnet it needs an internet gateway and a configured routing table. Each VPC has a security group the controls what traffic can enter or leave the VPC. We will only allow incoming ssh traffic (as debugging option) and outgoing HTTPS traffic. If you plan to run jobs that also need to download resources from HTTP or FTP servers you have to whitelist them here.
You will need the name of the security group and the subnet for the configuration of the instances later-on, so we print them as output of Terraform.
resource "aws_vpc" "runner-vpc" {
cidr_block = "10.0.0.0/16"
tags = {
CreatedBy = "terraform"
}
}
resource "aws_subnet" "runner-subnet" {
vpc_id = aws_vpc.runner-vpc.id
cidr_block = "10.0.0.0/16"
map_public_ip_on_launch = true
}
resource "aws_route_table" "runner-vpc-route-table" {
vpc_id = aws_vpc.runner-vpc.id
# target local for 10.0.0.0/16 is created implicitly
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.runner-subnet-igw.id
}
tags = {
CreatedBy = "terraform"
}
}
resource "aws_internet_gateway" "runner-subnet-igw" {
vpc_id = aws_vpc.runner-vpc.id
tags = {
CreatedBy = "terraform"
}
}
resource "aws_security_group" "runner-sg" {
name = "runner-sg"
description = "Allow all outbound traffic and restrict inbound traffic"
vpc_id = aws_vpc.runner-vpc.id
ingress {
description = "Allow inbound traffic from SSH"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
ipv6_cidr_blocks = ["::/0"]
}
egress {
description = "Allow outbound traffic to HTTPS"
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
ipv6_cidr_blocks = ["::/0"]
}
}
output "github-runner-subnet-id" {
description = "Subnet id for the github-runner"
value = aws_subnet.runner-subnet.id
}
output "github-runner-security-group-id" {
description = "Security group id for the github-runner"
value = aws_security_group.runner-sg.id
}
Apply the configuration with terraform apply and copy the output values, you will need them later.
Creating the AMI Image and GitHub Token
Two more steps are needed: Creating the Amazon Machine Image (AMI) and a GitHub token. The image defines the initial state of the virtual machine that hosts the runner. I haven't figured out a way to create images via Terraform yet, if you find one please let me know. The steps to manually create it are already well covered in the repo of the ec2-github-runner. Same applies for the instructions to create a GitHub personal access token that allows the runner to register itself for your repository.
Configure Your GitHub Action to Run on Your Own Runners
Configure the GitHub Action in two steps: For good security we provide the AWS access and GitHub access tokens as secrets. So you create three secrets for the repository (or your account/organisation if you want):
- AWS_ACCESS_KEY_ID
- AWS_SECRET_ACCESS_KEY
- GH_MACHINE_PAT
Lastly, the workflow yaml that calls the ec2-github-runner action. Your actual task is sandwiched by two jobs that create and delete the EC2 instance. You always have to add both of them. If you forget the removal, the EC2 instance will run indefinitely. No worries, if your build tasks fails, the clean up job will run anyway.
Insert the id of the AMI, subnet and security-group that you created earlier. I marked a few fields that you can additionally change based your needs: the AWS region and the instance type.
name: do-the-job
on: push
jobs:
start-runner:
name: Start self-hosted EC2 runner
runs-on: ubuntu-latest
outputs:
label: ${{ steps.start-ec2-runner.outputs.label }}
ec2-instance-id: ${{ steps.start-ec2-runner.outputs.ec2-instance-id }}
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: eu-central-1 # <---------------- change to your region
- name: Start EC2 runner
id: start-ec2-runner
uses: machulav/ec2-github-runner@v2
with:
mode: start
github-token: ${{ secrets.GH_MACHINE_PAT}}
ec2-image-id: ami-123456789 # <------------- insert ami id here
ec2-instance-type: t3.nano # <-------------- optionally change type of instance
subnet-id: subnet-081931dde4dee39bd # <----- insert your subnet id here
security-group-id: sg-029fab7c4b2b22bf4 # <- insert your security group id here
do-the-job:
name: Do the job on the runner
needs: start-runner # required to start the main job when the runner is ready
runs-on: ${{ needs.start-runner.outputs.label }} # run the job on the newly created runner
steps:
- name: Hello World
run: echo 'Hello World!'
stop-runner:
name: Stop self-hosted EC2 runner
needs:
- start-runner # required to get output from the start-runner job
- do-the-job # required to wait when the main job is done
runs-on: ubuntu-latest
if: ${{ always() }} # required to stop the runner even if the error happened in the previous jobs
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: eu-central-1
- name: Stop EC2 runner
uses: machulav/ec2-github-runner@v2
with:
mode: stop
github-token: ${{ secrets.GH_MACHINE_PAT }}
label: ${{ needs.start-runner.outputs.label }}
ec2-instance-id: ${{ needs.start-runner.outputs.ec2-instance-id }}
Conclusion
In this post I showed how to create on-demand self-hosted GitHub runners that allow you to use AWS free-tier instances for your build jobs. For better team collaboration we set up the AWS resources with Terraform.
Do you need Kubernetes or Go Experts?
We are a Software Agency from Stuttgart, Germany and can support you on your journey to deliver outstanding user experiences and resilient infrastructure. Reach out to us.
Something Else?
Make sure to check out our build tool for large multi-language applications: https://bob.build