The purpose of this tutorial is to use the Fastly network to route traffic to HarperDB based on geo proximity.
Fastly and Compute@Edge
Fastly is a content delivery network (CDN) and an edge cloud platform. Compute@Edge is a Fastly offering that is designed to allow developers to run custom code at the edge of the Fastly network, closer to users.
This tutorial requires a few things to be set up.
- A HarperDB account
- A multiple node HarperDB cluster
- Your own domain with DNS records pointing to your nodes that have TLS certificates on them. *NOTE: If you do not have DNS or TLS certificates you can config Fastly to ignore certificate validation, however this is not recommended.
HarperDB Nodes / Instances
If you don’t already have the above requirements, start here to get an account with HarperDB. From there you’ll need to deploy at least two internet accessible HarperDB nodes (also referred to as instances) that are serving HarperDB via https.
HarperDB Multi Node Cluster
Then cluster your HarperDB nodes together in your configuration of choice. Here is a great video resource for using HarperDB Clustering, and this written tutorial provides a common use case scenario.
Fastly Getting Started
Create your Fastly account at https://www.fastly.com/signup, and follow the instructions to log in.
Next, navigate to the “Getting Started Guide.” The instructions will direct you to:
- Generate an API token
- Download/install the Fastly CLI
- Configure the Fastly CLI
- Install local dependencies (as needed)
- Create a new Compute@Edge project
- Choose a language
- Compile the project to a Wasm binary
- Deploy the project to a new Fastly service
Generate an API token
Navigate to “Account” by clicking your account name in the top right corner, then click “API Tokens” from the lower left menu. Click “Create Token”. Use the following options:
Submit the form. Next, save your token somewhere securely and click “Okay.” You’ll use this when you config the CLI.
Download/install the Fastly CLI
Install the fastly CLI https://developer.fastly.com/learning/compute.
Configure The Fastly CLI
Next, set up the Fastly CLI to use our API token. Run fastly profile create <yourproject> and answer the following prompts.
Install Local Dependencies
You will need a recent version of Node.js and npm if you don’t have them already.
Create a new Compute@Edge project
This will result in a shell project for your Fastly Compute@Edge code.
Compile the Project to a Wasm Binary
Next up, we’ll use fastly compute build to compile into a WebAssembly module.
Deploy the Project to a New Fastly Service
To deploy, run fastly compute deploy. At this point, Fastly starts a Compute@Edge free trial. At the prompts, you can accept an auto created domain name, or choose your own. To use Fastly’s free TLS your domain should be <name>.global.ssl.fastly.net, replacing <name> with a single word that claims the domain you're creating. You can also now access the Compute Console.
Fastly Compute Setup
Next we need to add Hosts (your HarperDB nodes).
From the Compute Console, select your domain (or version associated with that domain), and then select “Hosts” from the left side menu.
Add all of your hosts to this list. You may use up to five hosts for free.
These host references can be by DNS or by IP. If you use IP, you will need to click the pencil icon and edit the TLS settings. You will also need to do this if you decide not to use your own DNS and TLS certificates on your HarperDB nodes.
Next you need to edit each host to give them unique names. To do this click on the pencil icon next to the host.
Next you need to set up a lookup table using the Dictionary tab. Click on the “Dictionary” button on the left side of the screen. Click “Create your first dictionary”, name it “backends” and click “Add.”
Here you will create a mapping of Fastly points of presence (PoP) to your HarperDB nodes. You can find a complete list of Fastly PoPs by using the Fastly api. You should at least use the Fastly PoP that is closest to you so that you can test the routing.
The following screenshot shows an example of a mapping of a few PoPs to your HarperDB hosts by using the name of the host defined in the step above.
This lookup table can be modified to update your Fastly PoP to HarperDB node routing without having to modify any of your Compute code, which we will discuss in more detail later in the tutorial. You should also create a key called “default” pointing to one of your nodes. This will help in cases that a PoP is used that is not in your lookup table.
With Compute service set up in the console, it’s a good time to install dependencies then switch over to the IDE to add some code.
To install dependencies run npm install in your project.
Now, open src/index.js and replace the existing code with the following:
Now we’re ready to publish our service!
Run fastly compute publish.
Once this is done, your Compute service in the console will show up as “Active” and you will be able to access your HarperDB Cluster via the domain you set earlier, <name>.global.ssl.fastly.net. In the following screenshot I’m using a curl command via Postman to hit the “registration_info” operation using the username/password authentication for the node my traffic is routed to.
There are a couple more things that should be considered for a production environment.
- Health Checks to redirect traffic when a HarperDB node is offline
- A way to automatically update your lookup table when Fastly adds new PoPs
Deploying a Fastly Compute service is quick and easy while providing a way to route traffic to HarperDB nodes by using the client’s fastest connection to the Fastly network via Fastly PoPs. It also provides us a way to change our traffic routing on the fly with no code changes using Fastly’s dictionary.