Push, Deploy, Repeat

Published: 2024-10-27 | By: Tyler Talaga Author

Intro

Who enjoys the tedium of SSHing into a server, pulling the latest code, and crossing their fingers that nothing breaks? I don't, and I bet you don't either. Imagine if every time you pushed to GitHub, your server updated itself—no manual labor with nothing to forget.

In this guide, we're going to make that dream come true. We'll set up GitHub webhooks to automatically deploy updates, using Flask to handle the magic behind the scenes. The goal? No more late-night deployment mistakes or forgotten updates. Just smooth, reliable automation—so you have more time for the important things.

Automation Magic

The Lowdown on Webhooks

First things first, what's a webhook? Think of it as GitHub's way of nudging your server and saying, "Hey buddy, new code just arrived. Up and at 'em!"

By setting up a webhook, we can have GitHub automatically notify our server whenever we push changes to the main branch. Our server then pulls the latest code, and boom—automatic deployment without breaking a sweat.

Prerequisites

Before we dive into the fun stuff, make sure you've got the following lined up:

  • A GitHub repository for the code you want to deploy.
  • Flask installed to handle incoming webhook requests.
  • A server or environment to host your Flask app (could be PythonAnywhere, AWS, Heroku, or even that old Raspberry Pi collecting dust).

Setting Up Shop

Cloning the Repository

First things first, let's clone the project and get things rolling.

git clone https://github.com/tyler-tee/push-deploy-repeat.git
cd push-deploy-repeat

Installing Dependencies

Time to set up a virtual environment. Because isolating dependencies is the cool thing to do.

python3 -m venv venv
source venv/bin/activate  # For macOS/Linux
# For Windows users:
# venv\Scripts\activate

pip install -r requirements.txt

The Secret Sauce

Open up config/update_config.json and add your own secret token.

{
    "SECRET_KEY": "your-super-secret-token"
}

You'll need this token later when setting up the GitHub webhook, so keep it handy.

Keeping Secrets

Crafting the Webhook Listener

Now, let's create a Flask app that listens for incoming webhooks from GitHub.

The Listener Nuts and Bolts

Here's the code for our webhook listener, reachable at /update_server:

import hmac
import hashlib
from ipaddress import ip_address, ip_network
import json
import os
import requests
from flask import Blueprint, jsonify, request
import git

update_server = Blueprint("update", __name__)

...

@update_server.route('/update_server', methods=['POST'])
def update():
    # Extract the client's IP address
    if 'X-Forwarded-For' in request.headers:
        # If behind a proxy, get the original IP
        forwarded_for = request.headers.get('X-Forwarded-For')
        # X-Forwarded-For can be a comma-separated list of IPs
        ip_list = [ip.strip() for ip in forwarded_for.split(',')]
        client_ip = ip_list[0]
    else:
        client_ip = request.remote_addr

    # Check if the IP is from GitHub
    if not is_github_ip(client_ip):
        return jsonify({'msg': 'Request IP does not match GitHub IP ranges'}), 403

    # Header validation
    header_reqs = ["X-GitHub-Event", "X-GitHub-Delivery", "X-Hub-Signature-256"]

    if not all(req in request.headers for req in header_reqs):
        return jsonify({'msg': 'Required headers missing'}), 400  # Bad Request

    # Event type validation
    event = request.headers.get('X-GitHub-Event')

    if event != "push":
        return jsonify({'msg': "Wrong event type"}), 400

    # Signature validation
    secret_key = load_secret()
    if not verify_signature(request.data, secret_key, request.headers.get('X-Hub-Signature-256')):
        return jsonify({'msg': 'Signature verification failed'}), 403

    # Payload validation
    payload = request.get_json()
    if not payload:
        return jsonify({'msg': 'Invalid or missing JSON payload'}), 400

    if payload['ref'] != 'refs/heads/main':
        return jsonify({'msg': 'Not main branch, ignoring'}), 200  # OK

    dir_path = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
    repo = git.Repo(dir_path)
    origin = repo.remotes.origin

    # Pull the latest changes from the main branch
    pull_info = origin.pull()
    if not pull_info or pull_info[0].flags > 128:
        return jsonify({'msg': "Didn't pull any information from remote."}), 200  # OK

    commit_hash = pull_info[0].commit.hexsha

    return jsonify({'msg': f'Updated server to commit {commit_hash}'}), 200  # OK

This script:

  • Validates the Request IP: Ensures the request comes from GitHub's IP ranges.
  • Verifies the Signature: Confirms the payload is from GitHub using your secret token.
  • Processes Push Events: Only acts on push events to the main branch.
  • Updates Your Code: Pulls the latest changes from GitHub.

Integrating the Blueprint

In your main Flask app (app.py), make sure to register the blueprint:

from flask import Flask
from update_server import update_server  # Import your blueprint

app = Flask(__name__)
app.register_blueprint(update_server)

if __name__ == '__main__':
    app.run(port=5000)

Running the Listener

Fire up the Flask app:

python app.py

Your app should now be running at http://localhost:5000. But unless your server is publicly accessible (and unless you're into that sort of risk, it shouldn't be), GitHub won't be able to reach it.

Server Running

Exposing Your Server to GitHub

To let GitHub communicate with your Flask app, you have a couple of options:

Option 1: Using ngrok

If you're running the app locally and want to test things out, ngrok is a handy tool that creates a secure tunnel to your localhost.

Installing ngrok

If you haven't already, download ngrok from ngrok.com and follow the installation instructions.

Starting ngrok

Run the following command to expose your Flask app:

ngrok http 5000

You'll get a forwarding URL like https://abcd1234.ngrok.io. This is the public URL GitHub will use to send webhook requests.

Option 2: Deploying on PythonAnywhere

If you'd prefer not to use ngrok or want a more permanent solution without managing your own server infrastructure, you can deploy your Flask app on PythonAnywhere. It's a cloud-based Python hosting environment that makes deployment a breeze.

Why PythonAnywhere?

  • Always On: Your Flask app will be running 24/7.
  • No Server Management: No need to set up or maintain servers.
  • Easy Setup: Ideal for hosting small to medium-sized apps.

Deploying on PythonAnywhere

  1. Sign Up: Create a free account on PythonAnywhere.
  2. Upload Your Code: Use their web interface or Git to upload your project.
  3. Set Up the Web App:
  4. Navigate to the Web tab and click Add a new web app.
  5. Choose Flask as the framework and specify the path to your app.py.
  6. Install Dependencies:
  7. Open a Bash console and activate your virtual environment if you have one.
  8. Run pip install -r requirements.txt to install your dependencies.
  9. Configure the Webhook URL:
  10. Your app will be accessible at https://yourusername.pythonanywhere.com/update_server.
  11. Use this URL when setting up your GitHub webhook.

PythonAnywhere

Setting Up the GitHub Webhook

Now, let's tell GitHub where to send its love letters (webhook payloads).

Adding the Webhook

  1. Navigate to your GitHub repository.
  2. Click on Settings > Webhooks.
  3. Click Add webhook.

Configuring the Webhook

Fill out the form:

  • Payload URL:
  • If using ngrok: https://abcd1234.ngrok.io/update_server (replace with your actual ngrok URL).
  • If using PythonAnywhere: https://yourusername.pythonanywhere.com/update_server.
  • Content type: application/json.
  • Secret: The same secret token from update_config.json.
  • Which events would you like to trigger this webhook? Choose Just the push event.

Click Add webhook and you're all set!

Testing the Setup

Time to see if all our hard work pays off.

Pushing a Test Commit

Make a small change to your code:

echo "# Testing automatic deployment" >> README.md
git add README.md
git commit -m "Test automatic deployment"
git push origin main

Watching the Magic Happen

  • If Using ngrok: Check the terminal where your Flask app is running. You should see logs indicating that it received a request and pulled the latest code.
  • If Using PythonAnywhere: Check the Server Log in the PythonAnywhere Web tab to see the incoming webhook and deployment actions.

Alternatively, check the Recent Deliveries section under Webhooks in your GitHub repository settings to see if the request was successful.

Webhook Setup

Security Matters

Before we get too excited, let's talk about security. We don't want just anyone triggering our deployment script.

IP Allowlisting

Our Flask app checks if the incoming request's IP address is within GitHub's official IP ranges. It fetches the ranges dynamically from the GitHub Meta API, so it's always up-to-date.

Validating the Signature

The app verifies the signature of incoming requests using HMAC SHA-256 and your secret token. This ensures the request actually came from GitHub and hasn't been tampered with.

Use HTTPS

Both ngrok and PythonAnywhere provide HTTPS URLs, which means the data transmitted between GitHub and your server is encrypted.

Security First

Deploying the Listener Permanently

Running the Flask app locally is fine for testing, but for a real deployment, services like PythonAnywhere are ideal.

Advantages of PythonAnywhere

  • Simplicity: No need to manage servers or worry about uptime.
  • Cost-Effective: Free tier available for small apps.
  • Scalability: Upgrade your plan as your needs grow.

Other Hosting Options

  • AWS EC2 or Lightsail: For more control and scalability.
  • Heroku: Simple deployments with a free tier.
  • DigitalOcean Droplets: Affordable VPS options.

Choose the hosting platform that suits your needs and deploy your Flask app there.

Enhancements and Next Steps

Now that you've got automatic deployments set up, why stop there?

Automate Additional Tasks

  • Restart Services: If your application requires a server restart after deployment, automate it within your webhook handler.
  • Run Tests: Integrate automated testing to ensure that new code doesn't break anything.
  • Notifications: Set up alerts (Slack, email, carrier pigeon) to notify you when deployments occur.

Use a CI/CD Pipeline

For a more robust solution, consider using continuous integration and continuous deployment tools like GitHub Actions, Jenkins, or Travis CI.

Wrapping Up

There you have it - Fully automated deployment using GitHub webhooks and Flask. Now you can push code to your heart's content, and your server will stay up-to-date without you lifting another finger.

Just think of all the time you'll save—not to mention the reduction in deployment-induced headaches.

Happy Coding

Closing Thoughts, Parting Bots

Congratulations! You've transformed your deployment process from a tedious chore into an automated breeze. Let's take a moment to highlight what we've accomplished:

  • Set up a Flask app that listens for GitHub webhooks.
  • Secured your application by validating requests with IP checks and signature verification.
  • Configured a GitHub webhook to automatically notify your server upon new commits.
  • Automated your deployment, eliminating the need for manual updates.
  • Deployed your listener using accessible hosting options like PythonAnywhere.

But this is just the beginning - There's a whole world of possibilities to explore:

  • Enhance your workflow by integrating testing, notifications, or continuous integration tools.
  • Scale your application by leveraging more advanced hosting solutions.
  • Dive deeper into automation to optimize other aspects of your development and deployment processes.

Automate All The Things

Further Reading