Client-side load balancing and service discovery

You can configure an Armeria client to distribute its requests to more than one server autonomously, unlike traditional server-side load balancing where the requests go through a dedicated load balancer such as L4 and L7 switches.

There are 3 elements involved in client-side load balancing in Armeria:

Creating an EndpointGroup

There are various EndpointGroup implementations provided out of the box, but let's start simple with EndpointGroup which always yields a pre-defined set of Endpoints specified at construction time:

import com.linecorp.armeria.client.Endpoint;
import com.linecorp.armeria.client.endpoint.EndpointGroup;

// Create a group of well-known search engine endpoints.
EndpointGroup searchEngineGroup = EndpointGroup.of(
        Endpoint.of("www.google.com", 443),
        Endpoint.of("www.bing.com", 443),
        Endpoint.of("duckduckgo.com", 443));

List<Endpoint> endpoints = searchEngineGroup.endpoints();
assert endpoints.contains(Endpoint.of("www.google.com", 443));
assert endpoints.contains(Endpoint.of("www.bing.com", 443));
assert endpoints.contains(Endpoint.of("duckduckgo.com", 443));

Choosing an EndpointSelectionStrategy

An EndpointGroup is created with EndpointSelectionStrategy.weightedRoundRobin() by default, unless specified otherwise. Armeria currently provides the following EndpointSelectionStrategy implementations out-of-the-box:

  • EndpointSelectionStrategy.weightedRoundRobin for weighted round robin.
  • EndpointSelectionStrategy.roundRobin for round robin.
  • StickyEndpointSelectionStrategy for pinning requests based on a criteria such as a request parameter value.
  • You can also implement your own EndpointSelectionStrategy.

An EndpointSelectionStrategy can usually be specified as an input parameter or via a builder method when you build an EndpointGroup:

import com.linecorp.armeria.client.endpoint.EndpointSelectionStrategy;
import com.linecorp.armeria.client.endpoint.dns.DnsAddressEndpointGroup;

EndpointSelectionStrategy strategy = EndpointSelectionStrategy.roundRobin();

EndpointGroup group1 = EndpointGroup.of(
        strategy,
        Endpoint.of("127.0.0.1", 8080),
        Endpoint.of("127.0.0.1", 8081));

EndpointGroup group2 =
        DnsAddressEndpointGroup.builder("example.com")
                               .selectionStrategy(strategy)
                               .build();

Connecting to an EndpointGroup

Once an EndpointGroup is created, you can specify it when creating a new client:

import static com.linecorp.armeria.common.SessionProtocol.HTTPS;

import com.linecorp.armeria.client.WebClient;
import com.linecorp.armeria.common.HttpResponse;
import com.linecorp.armeria.common.AggregatedHttpResponse;

// Create an HTTP client that sends requests to the searchEngineGroup.
WebClient client = WebClient.of(SessionProtocol.HTTPS, searchEngineGroup);

// Send a GET request to each search engine.
List<CompletableFuture<?>> futures = new ArrayList<>();
for (int i = 0; i < 3; i++) {
    final HttpResponse res = client.get("/");
    final CompletableFuture<AggregatedHttpResponse> f = res.aggregate();
    futures.add(f.thenRun(() -> {
        // And print the response.
        System.err.println(f.getNow(null));
    }));
}

// Wait until all GET requests are finished.
CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])).join();

Cleaning up an EndpointGroup

EndpointGroup extends java.lang.AutoCloseable, which means you need to call the close() method once you are done using it, usually when your application terminates:

// Release all resources claimed by the group.
searchEngines.close();

close() is a no-op for some EndpointGroup implementations, but not all implementations are so, especially those which updates the Endpoint list dynamically, such as refreshing the list periodically.

Removing unhealthy Endpoint with HealthCheckedEndpointGroup

HealthCheckedEndpointGroup decorates an existing EndpointGroup to filter out the unhealthy Endpoints from it so that a client has less chance of sending its requests to the unhealthy Endpoints. It determines the healthiness by sending so called 'health check request' to each Endpoint, which is by default a simple HEAD request to a certain path. If an Endpoint responds with non-200 status code or does not respond in time, it will be marked as unhealthy and thus be removed from the list.

import static com.linecorp.armeria.common.SessionProtocol.HTTP;

import com.linecorp.armeria.client.WebClient;
import com.linecorp.armeria.client.endpoint.healthcheck.HealthCheckedEndpointGroup

// Create an EndpointGroup with 2 Endpoints.
EndpointGroup originalGroup = EndpointGroup.of(
    Endpoint.of("192.168.0.1", 80),
    Endpoint.of("192.168.0.2", 80));

// Decorate the EndpointGroup with HealthCheckedEndpointGroup
// that sends HTTP health check requests to '/internal/l7check' every 10 seconds.
HealthCheckedEndpointGroup healthCheckedGroup =
        HealthCheckedEndpointGroup.builder(originalGroup, "/internal/l7check")
                                  .protocol(SessionProtocol.HTTP)
                                  .retryInterval(Duration.ofSeconds(10))
                                  .build();

// Wait until the initial health check is finished.
healthCheckedGroup.whenReady().get();

// Specify healthCheckedGroup, not the originalGroup.
WebClient client = WebClient.builder(SessionProtocol.HTTP, healthCheckedGroup)
                            .build();

DNS-based service discovery with DnsEndpointGroup

Armeria provides 3 DNS-based EndpointGroup implementations:

They refresh the Endpoint list automatically, respecting TTL values, and retry when DNS queries fail.

DnsAddressEndpointGroup is useful when accessing an external service with multiple public IP addresses:

DnsAddressEndpointGroup group =
        DnsAddressEndpointGroup.builder("www.google.com")
                               // Refresh more often than every 10 seconds and
                               // less often than every 60 seconds even if DNS server asks otherwise.
                               .ttl(/* minTtl */ 10, /* maxTtl */ 60)
                               .build();

// Wait until the initial DNS queries are finished.
group.whenReady().get();

DnsServiceEndpointGroup is useful when accessing an internal service with SRV records, which is often found in modern container environments that leverage DNS for service discovery such as Kubernetes:

import com.linecorp.armeria.client.endpoint.dns.DnsServiceEndpointGroup;
import com.linecorp.armeria.client.retry.Backoff;

DnsServiceEndpointGroup group =
        DnsServiceEndpointGroup.builder("_http._tcp.example.com")
                               // Custom backoff strategy.
                               .backoff(Backoff.exponential(1000, 16000).withJitter(0.3))
                               .build();

// Wait until the initial DNS queries are finished.
group.whenReady().get();

DnsTextEndpointGroup is useful if you need to represent your Endpoints in a non-standard form:

import com.linecorp.armeria.client.endpoint.dns.DnsTextEndpointGroup;

// A mapping function must be specified.
DnsTextEndpointGroup group = DnsTextEndpointGroup.of("example.com", (byte[] text) -> {
    Endpoint e = /* Convert 'text' into an Endpoint here. */;
    return e
});

// Wait until the initial DNS queries are finished.
group.whenReady().get();

ZooKeeper-based service discovery with ZooKeeperEndpointGroup

First, You need the armeria-zookeeper3 dependency:

build.gradle
dependencies {
    implementation platform('com.linecorp.armeria:armeria-bom:1.31.0')

    ...
    implementation 'com.linecorp.armeria:armeria-zookeeper3'
}

Then, use ZooKeeperEndpointGroup and ZooKeeperDiscoverySpec to retrieve the information of servers:

import com.linecorp.armeria.client.zookeeper.ZooKeeperDiscoverySpec;
import com.linecorp.armeria.client.zookeeper.ZooKeeperEndpointGroup;

String zkConnectionStr = "myZooKeeperHost:2181";
String znodePath = "/myProductionEndpoints";
ZooKeeperDiscoverySpec discoverySpec = ZooKeeperDiscoverySpec.curator(serviceName);
ZooKeeperEndpointGroup myEndpointGroup =
        ZooKeeperEndpointGroup.builder(zkConnectionStr, znodePath, discoverySpec)
                              .sessionTimeoutMillis(10000)
                              .build();

The ZooKeeperEndpointGroup is used to fetch the binary representation of server information. The ZooKeeperDiscoverySpec converts the binary representation to an Endpoint.

ZooKeeperDiscoverySpec.curator() uses the format of Curator Service Discovery which is compatible with Spring Cloud Zookeeper. If you registered your server with ZooKeeperRegistrationSpec.curator(), you must use ZooKeeperDiscoverySpec.curator().

You can use an existing CuratorFramework instance instead of the ZooKeeper connection string at this time as well.

import org.apache.curator.framework.CuratorFramework;

CuratorFramework client = ...
String znodePath = ...
ZooKeeperDiscoverySpec discoverySpec = ...
ZooKeeperEndpointGroup myEndpointGroup =
        ZooKeeperEndpointGroup.builder(client, znodePath, discoverySpec)
                              .build();

For more information, please refer to the API documentation of the com.linecorp.armeria.client.zookeeper package.

Eureka-based service discovery with EurekaEndpointGroup

First, You need the armeria-eureka dependency:

build.gradle
dependencies {
    implementation platform('com.linecorp.armeria:armeria-bom:1.31.0')

    ...
    implementation 'com.linecorp.armeria:armeria-eureka'
}

Then, use EurekaEndpointGroup to retrieve the information of servers:

import com.linecorp.armeria.client.eureka.EurekaEndpointGroup;

EurekaEndpointGroup eurekaEndpointGroup =
        EurekaEndpointGroup.of("https://eureka.com:8001/eureka/v2");

If you want to retrieve the information of certain regions or a service, use EurekaEndpointGroupBuilder:

import com.linecorp.armeria.client.eureka.EurekaEndpointGroupBuilder;

EurekaEndpointGroupBuilder builder =
        EurekaEndpointGroup.builder("https://eureka.com:8001/eureka/v2");
                           .regions("aws")
                           .appName("myApplication");
EurekaEndpointGroup eurekaEndpointGroup = builder.build();

For more information, please refer to the API documentation of the com.linecorp.armeria.client.eureka package.

Consul-based service discovery with ConsulEndpointGroup

First, You need the armeria-consul dependency:

build.gradle
dependencies {
    implementation platform('com.linecorp.armeria:armeria-bom:1.31.0')

    ...
    implementation 'com.linecorp.armeria:armeria-consul'
}

Then, use ConsulEndpointGroup to retrieve the information of servers:

import com.linecorp.armeria.client.consul.ConsulEndpointGroup;

ConsulEndpointGroup consulEndpointGroup =
        ConsulEndpointGroup.of("http://my-consul.com:8500", "my-service");

If you want to get the information of certain datacenter or filter endpoints, use ConsulEndpointGroupBuilder:

import com.linecorp.armeria.client.consul.ConsulEndpointGroupBuilder;

ConsulEndpointGroupBuilder builder =
        ConsulEndpointGroup.builder("http://my-consul.com:8500", "my-service");
                           .datacenter("ds1")
                           .filter("ServicePort == 8080")
                           .build();
ConsulEndpointGroup consulEndpointGroup = builder.build();

For more information, please refer to the API documentation of the com.linecorp.armeria.client.consul package.

Edit this page