Building Your Own Load Balancer In GO.
Implementation of round-robin load balancer algorithm in GO.
Table of contents
Overview
Modern applications grow to serve millions of users simultaneously. These applications look for fast and reliable ways to respond to every user with accurate texts, videos, and other data. Most applications handle such a high traffic volume by setting up a collection of multiple resource servers and duplicating data between them.
A load balancer is a device that sits between the user and this collection of resource servers. A load balancer acts as an invisible layer that ensures that the group of resource servers is used equally.
Load Balancers use different algorithms to control the distribution of the traffic to these servers. We will build one of these algorithms; the round-robin algorithm.
Pre-requisites:
Basic/Beginner knowledge of GO programming language.
GO is installed and running in your operating system.
A GitHub account
You can confirm if you have GO installed by simply running:
$ go version
You should get a response similar to this:
go version go1.20.2 darwin/amd64
We are going to keep things basic, no weird concurrency going on here. That said, let's whip up our code editor and start building the load balancer using the round-robin algorithm.
It's a Golang project, so we want to create a directory and our main.go file.
I'm using a Unix-based operating system, so to create a directory and navigate into it, I'll head into my side projects folder and run the following commands:
$ mkdir go-loadbalancer && cd go-loadbalancer && touch main.go
After creating the directories, we want to initialize our project as a GO module. In the directory you created, run:
$ go mod init github.com/{{YOUR_GITHUB_USERNAME}}
This should automatically generate a go.mod file in your project directory that should look like this:
That's all we need to do regarding folder structure and initializing the project. In this project, we will write all the code in our main.go file. To start with, we will have to declare our package main, just because GO tells us to and then we import important packages we will use in the project.
We will see the purpose of each imported package soon.
package main
import (
"fmt"
"net/http"
"net/http/httputil"
"net/url"
"os"
)
Next, we can create what a simple server should look like. A simple server struct that should have an address and a reverse proxy.
So, let's go ahead and write that:
type simpleServer struct {
address string
proxy *httputil.ReverseProxy
}
Next, let's write a function to create our collection of servers whenever we need them, this will be useful in the future to initialize the servers immediately after we run main.go.
The function will look something like this:
func newSimpleServer(address string) *simpleServer {
serveUrl, err := url.Parse(address)
if err != nil {
fmt.Printf("error: %v\n", err)
os.Exit(1)
}
return &simpleServer{
address: address,
proxy: httputil.NewSingleHostReverseProxy(serveUrl),
}
}
Nice! we will see that we have used two of our imported packages at this point.
Note that the httputil package has to be an explicit import, this is important as the httputil package contains many useful functions for working with HTTP components like a reverse proxy which we are using in this case. The url package is used to confirm that the address is a valid URL.
To make our code look a bit cleaner, let's write a generic error-handling function to avoid repetitive error handling across the file, we will handle all errors from this function and apply it where necessary.
func handleErr(err error) {
if err != nil {
fmt.Printf("error: %v\n", err)
os.Exit(1)
}
}
With this, we can change the newSimpleServer function to become:
func newSimpleServer(address string) *simpleServer {
serveUrl, err := url.Parse(address)
handleErr(err)
return &simpleServer{
address: address,
proxy: httputil.NewSingleHostReverseProxy(serveUrl),
}
}
Great! So far, we have our simple server struct, we have a function to create the servers and we are handling errors better. Next, we need to introduce an interface for our servers with certain methods to allow our servers to perform some operations.
type Server interface {
Address() string
IsAlive() bool
Serve(rw http.ResponseWriter, r *http.Request)
}
Next, we implement the methods of the simple server:
func (s *simpleServer) Address() string {
return s.address
}
func (s *simpleServer) IsAlive() bool {
return true
}
func (s *simpleServer) Serve(rw http.ResponseWriter, r *http.Request) {
s.proxy.ServeHTTP(rw, r)
}
In case you are wondering why we are defining methods just to return fields on the simple server struct; This approach makes our code better, easily maintainable, and extendable.
Now that we have created all of the methods of a simple server, we can create the struct for our load balancer. Before we go ahead with the implementation of the Load Balancer, let's first understand what the round-robin algorithm is and how it works.
Round-Robin Algorithm
The round-robin algorithm is the most straightforward algorithm for load balancing. In the round-robin algorithm, requests to the load balancer will route traffic to the group of servers sequentially. This means if there are 6 servers, the load balancer will send requests to each of the servers one after the other (sequentially), irrespective of the volume of requests the server is currently processing.
For the algorithm to work fine, we need a round-robin count to be able to calculate the next server to process a request after the next server is found, we must increment the round-robin count so that it will pick the next server in the sequence when processing the next request.
That means if you have 3 servers, s1, s2, s3 , and a round-robin count was initiated at 0. The load balancer determines the next server by evaluating the modular division of the round-robin count by the total amount of servers, in our case that would be:
0 % 3 = 0
That means upon the first request, it will use the very first server, after picking the first server the load balancer will update the round-robin count to 1. So, when a new request comes, the calculation for the next available server will be:
1 % 3 = 1
So it picks the second server, and the continuous increment of the round-robin count will continue to sequentially select each server in the collection of servers assigned to the load balancer.
Now, that we understand the concept, let's get back into our code editor and build the required functions.
To create the struct of our round-robin load balancer, we need to have three important things;
the port on which the load balancer will run
the round-robin count, and
the group of servers it will share requests amongst.
type LoadBalancer struct {
port string
roundRobinCount int
servers []Server
}
Our LoadBalancer struct will also have a function and some methods. Firstly, we will write one function that creates a new LoadBalancer:
func NewLoadBalancer(port string, servers []Server) *LoadBalancer {
return &LoadBalancer{
port: port,
roundRobinCount: 0,
servers: servers,
}
}
Then, we will implement two methods for the LoadBalancer; a method to get the next available server and then a method to serve the proxy of the targeted server.
Using the explanation of modular division above, our method for getting the target server will look like this:
func (lb *LoadBalancer) getNextAvailableServer() Server {
server := lb.servers[lb.roundRobinCount%len(lb.servers)]
for !server.IsAlive() {
lb.roundRobinCount++
server = lb.servers[lb.roundRobinCount%len(lb.servers)]
}
lb.roundRobinCount++
return server
}
Then we can write a method for serving the proxy of the given target server:
func (lb *LoadBalancer) serveProxy(rw http.ResponseWriter, r *http.Request) {
targetServer := lb.getNextAvailableServer()
fmt.Printf("Forwaring request to addresses: %v\n", targetServer.Address())
targetServer.Serve(rw, r)
}
Phew! We've come a long way, where do we GO next? (pun very much intended)
What's left now is to tie up all our Load Balancer methods and function in the main function, which is the entry point of our application.
Inside the main function, we will create the servers with the newSimpleServer function, we will be using LinkedIn, Bloomberg, and Uber
func main() {
servers := []Server{
newSimpleServer("https://linkedin.com"),
newSimpleServer("https://bloomberg.com"),
newSimpleServer("https://uber.com"),
}
}
Next, we need to create and attach these servers to the Load Balancer using the NewLoadBalancer function we created earlier, we will run the load balancer on port "8080"
func main(){
servers := []Server{
newSimpleServer("https://linkedin.com"),
newSimpleServer("https://bloomberg.com"),
newSimpleServer("https://uber.com"),
}
lb := NewLoadBalancer("8080", servers)
}
Next, we write the function to redirect the request to the proxy of the target server, like this:
func main(){
servers := []Server{
newSimpleServer("https://linkedin.com"),
newSimpleServer("https://bloomberg.com"),
newSimpleServer("https://uber.com"),
}
lb := NewLoadBalancer("8080", servers)
handleRedirect := func(rw http.ResponseWriter, r *http.Request) {
fmt.Println("Recieved request")
lb.serveProxy(rw, r)
}
}
Next, we add the routing system to handle only the index route, "/", and redirect it to the Load Balancer's target server proxy by coupling it with the handleRedirect function.
Then we listen and serve on the Load Balancers port, like so:
func main(){
servers := []Server{
newSimpleServer("https://linkedin.com"),
newSimpleServer("https://bloomberg.com"),
newSimpleServer("https://uber.com"),
}
lb := NewLoadBalancer("8080", servers)
handleRedirect := func(rw http.ResponseWriter, r *http.Request) {
fmt.Println("Recieved request")
lb.serveProxy(rw, r)
}
http.HandleFunc("/", handleRedirect)
fmt.Printf("Server is starting on the localhost at : %s\n", lb.port)
http.ListenAndServe(":"+lb.port, nil)
}
And Boom! We have a load balancer sending requests with a round-robin algorithm to a collection of 3 servers. Our entire main.go file should look like this now:
package main
import (
"fmt"
"net/http"
"net/http/httputil"
"net/url"
"os"
)
func handleErr(err error) {
if err != nil {
fmt.Printf("error: %v\n", err)
os.Exit(1)
}
}
type Server interface {
Address() string
IsAlive() bool
Serve(rw http.ResponseWriter, r *http.Request)
}
type simpleServer struct {
address string
proxy *httputil.ReverseProxy
}
func (s *simpleServer) Address() string {
return s.address
}
func (s *simpleServer) IsAlive() bool {
return true
}
func (s *simpleServer) Serve(rw http.ResponseWriter, r *http.Request) {
s.proxy.ServeHTTP(rw, r)
}
func newSimpleServer(address string) *simpleServer {
serveUrl, err := url.Parse(address)
handleErr(err)
return &simpleServer{
address: address,
proxy: httputil.NewSingleHostReverseProxy(serveUrl),
}
}
type LoadBalancer struct {
port string
roundRobinCount int
servers []Server
}
func NewLoadBalancer(port string, servers []Server) *LoadBalancer {
return &LoadBalancer{
port: port,
roundRobinCount: 0,
servers: servers,
}
}
func (lb *LoadBalancer) getNextAvailableServer() Server {
server := lb.servers[lb.roundRobinCount%len(lb.servers)]
for !server.IsAlive() {
lb.roundRobinCount++
server = lb.servers[lb.roundRobinCount%len(lb.servers)]
}
lb.roundRobinCount++
return server
}
func (lb *LoadBalancer) serveProxy(rw http.ResponseWriter, r *http.Request) {
targetServer := lb.getNextAvailableServer()
fmt.Printf("Forwaring request to addresses: %v\n", targetServer.Address())
targetServer.Serve(rw, r)
}
func main() {
servers := []Server{
newSimpleServer("https://linkedin.com"),
newSimpleServer("https://bloomberg.com"),
newSimpleServer("https://uber.com"),
}
lb := NewLoadBalancer("8080", servers)
handleRedirect := func(rw http.ResponseWriter, r *http.Request) {
fmt.Println("Recieved request")
lb.serveProxy(rw, r)
}
http.HandleFunc("/", handleRedirect)
fmt.Printf("Server is starting on the localhost at : %s\n", lb.port)
http.ListenAndServe(":"+lb.port, nil)
}
All we have to do to see it in action is to run the application and test our load balancer. Open your terminal and run the following:
$ go run main.go
Then, in another terminal tab, we will use curl to test the proxy as it rotates through the servers and pass the requests.
$ curl localhost:8080
I recommend highly that you use curl to test the requests to the load balancer because your browser might give some weird behaviours due to its caching features.
If you run the curl command multiple times you should see it rotating through the 3 servers.
There you have it, your very own load balancer built from scratch!
I hope you found this piece useful in helping you understand load balancers better.
To see the final codebase, check it out on my GitHub: https://github.com/Double-DOS/go-loadbalancer
To learn more about load balancing and its algorithms, read this great piece from AWS: https://aws.amazon.com/what-is/load-balancing/
If you enjoyed it and have any comments, questions, or feedback, I'll be happy to read them in the comment section!
Thank You!