Introduction to Load Balancing in BTP Cloud Foundry
Requests to applications running on the BTP CF can be routed using different load balancing strategies. The default strategy is round-robin, which evenly distributes requests across application instances. The least-connection load balancing directs requests to the application instances with the fewest active connections first. However, these algorithms are not ideal for scenarios that require consistent routing behavior.
A hash-based load balancing algorithm was recently introduced to tackle such scenarios. It allows you to define an HTTP request header containing a specific identifier – such as a tenant ID, a user ID, or a resource ID – to route requests to the same subset of application instances. Selection of a suitable load balancing strategy for your Cloud Foundry application depends on your use case.
This blog post provides an overview of the hash-based load balancing algorithm in Cloud Foundry, covering its use cases and a step-by-step walkthrough of configuring it for a sample application.
Use Cases for Hash-Based Load Balancing
Hash-based load balancing addresses a class of Cloud Foundry application constraints related to backing service limitations – such as finite database connection pools or in-memory caches. It consistently routes requests to the same subset of instances, enabling optimization in scenarios where those instances maintain stateful resources.
One use case is when users from different tenants send requests that are spread across all available instances, requiring each instance to establish connections to multiple tenant-specific databases. Instead, the hash-based load balancing algorithm routes each tenant's traffic to a consistent subset of instances, allowing those instances to minimize database connection overhead. This provides a flexible and scalable approach beyond traditional session affinity.
By pinning tenants, users, or resources to application instances based on an HTTP request header, this algorithm helps to optimize cache usage, reduce latency, and avoid redundant database connections.
Sample Application for a Hash-Based Load Balancing
To use the hash-based load balancing, you need to ensure that you are using BTP CF release 2606 or higher.
Hash-based load balancing is configured at the application route level. The configuration can be easily provided in the application manifest or directly via CF CLI commands “cf create-route“, “cf update-route“, and “cf map-route“.
If a hash header for a hash-based route is not present in the HTTP request, the platform's default load balancing algorithm, round-robin, applies.
Let’s create and deploy a sample app with a hash-based balanced route and see how requests are routed to specific application instances.
You can clone the Cloud Foundry routing samples repository. Our sample app is in the per-route/loadbalancing folder, where you can find manifest.yml structured as follows:
- name: per-route-loadbalancing-demo
routes:
- route: per-route-lb-hb.
options:
loadbalancing: hash
hash_header: tenant-id
hash_balance: 1.2 In the routes section for our test application, the “
The hash balance route option defines a balance factor. The value 1.2 means that no single application instance should handle more than 120% of the average number of in-flight requests across all instances managed by the Gorouter. In-flight requests exceeding this threshold will be redirected to other less-loaded application instances.
A hash balance factor of 0 (or leaving it undefined) means that all requests for a particular hash are routed to the same instance if it is healthy.
You can push the app via CF CLI command “cf push -f manifest.yml”.
Execute “cf app per-route-loadbalancing-demo” to verify that three instances of the application are running successfully. If so, the “cf routes” command should display a new route “per-route-lb-hb.
Hash-Based Load Balancing in Action
You can now curl the app, specifying a tenant ID in the HTTP request header named “tenant-id”. As you may recall, we configured this name in the app manifest. The following examples target the /requests endpoint of the test application with different hash header values.
curl -s https://per-route-lb-hb.
curl -s https://per-route-lb-hb.
To verify that the requests are routed to specific application instances based on the tenant-id, you can execute “cf logs per-route-loadbalancing-demo“ in a separate terminal and inspect the “app_index“ of the responses in real time. You should observe that all the requests with the same tenant–id are handled by the same application instance.
This simple test aims to give you a first impression of hash-based routing. To summarize, you have deployed an app with a route configured for the hash-based load balancing algorithm and have seen it in action for a simple scenario with multiple application instances and a single hash header.
References
All the technical details of hash-based load balancing can be found in the Cloud Foundry documentation.
A good deep dive into BTP load balancing can be found in the article “Support for Customizable Load Balancing Algorithms”.
