Asynchronous REST endpoints

When developing a REST API, a major concern is how to scale the application to cope with large amounts of requests in an efficient way. This excellent article http://www.javaworld.com/article/2077995/java-concurrency/java-concurrency-asynchronous-processing-support-in-servlet-3-0.html explains the different models to deal with the requests: thread per connection, thread per request and the support for async Servlets introduced in Servlet 3.0.

The basic ideas are:

threads used to handle HTTP requests are expensive in terms of memory use; therefore it is not possible to have unlimited number of threads to deal with all the coming requests.
in a normal application, the threads handling a HTTP request block while waiting for resources such as files, databases, messages, etc.; in other words, most of the time HTTP threads sit idle.
to increase the efficiency of the threads, the specification Servlet 3.0 introduced async Servlets; this means that a HTTP thread can delegate the fulfilment of the response to some other thread while it becomes free to accept more requests.

This post will show some of the constructs to create a REST endpoint with asynchronous capabilities. The following examples are built using Spring Boot and can be found on https://github.com/fjab76/async-rest/tree/blog-entry

Synchronous version

To be able to compare the results given by the asynchronous version of the endpoint, we will set the baseline by creating a synchronous version.


@RequestMapping(value = "/sync", method = GET)
public String getSyncResource() throws InterruptedException {
    System.out.println("ThreadName-"+Thread.currentThread().getName());
    Thread.sleep(10000);
    return "hello sync";
}

In application.properties, we have added an entry to limit to 1 the maximum number of request processing threads to be created by Tomcat, which therefore determines the maximum number of simultaneous requests that can be handled. If not specified, this attribute is set to 200. See https://tomcat.apache.org/tomcat-7.0-doc/config/http.html for more details. By setting this value to 1, we can easily explore the effect of using synchronous vs asynchronous endpoints.

server.tomcat.max-threads=1

Now, if we start the application and open two browsers to hit http://localhost:8080/sync consecutively, the first browser displays ‘hello sync’ after 10 seconds and the second browser do the same after 20 seconds. In the console, this message is displayed, making clear that there is only one thread to handle all the requests, ‘http-nio-8080-exec-1’, and that it is done sequentially.

2016-05-30 19:15:50.381 INFO 2090 --- [nio-8080-exec-1] fjab.MyController : ThreadName-http-nio-8080-exec-1
2016-05-30 19:16:00.389 INFO 2090 --- [nio-8080-exec-1] fjab.MyController : ThreadName-http-nio-8080-exec-1

Asynchronous version with Callable

When repeating the same experiment to hit http://localhost:8080/callable, both browsers display ‘hello callable’ after 10 seconds and the message in the console shows how ‘http-nio-8080-exec-1’ dispatches the first request to the thread ‘MvcAsync1’, becoming free to accept the second request and dispatch it to the thread ‘MvcAsync2’ . ‘MvcAsync1’ and‘MvcAsync2’ execute the task defined by Callable and when are finished commit the response so that it is sent to the browser.

2016-05-30 19:20:46.533 INFO 2100 --- [nio-8080-exec-1] fjab.MyController : ThreadName-http-nio-8080-exec-1
2016-05-30 19:20:46.540 INFO 2100 --- [ MvcAsync1] fjab.MyController : ChildThread-MvcAsync1
2016-05-30 19:20:47.168 INFO 2100 --- [nio-8080-exec-1] fjab.MyController : ThreadName-http-nio-8080-exec-1
2016-05-30 19:20:47.168 INFO 2100 --- [ MvcAsync2] fjab.MyController : ChildThread-MvcAsync2

Asynchronous version with CompletableFuture

In this case, we hit http://localhost:8080/future in both browsers and, after 10 seconds, both browsers display the message ‘hello future’. The console shows this message

2016-05-30 19:34:28.248 INFO 2104 --- [nio-8080-exec-1] fjab.MyController : ThreadName-http-nio-8080-exec-1
2016-05-30 19:34:28.251 INFO 2104 --- [onPool-worker-1] fjab.MyController : ChildThread-ForkJoinPool.commonPool-worker-1
2016-05-30 19:34:29.062 INFO 2104 --- [nio-8080-exec-1] fjab.MyController : ThreadName-http-nio-8080-exec-1
2016-05-30 19:34:29.063 INFO 2104 --- [onPool-worker-2] fjab.MyController : ChildThread-ForkJoinPool.commonPool-worker-2

In this case, the tasks started by CompletableFuture run in threads from the ForkJoin pool.

Asynchronous version with DeferredResult

In this case, we hit http://localhost:8080/deferred in both browsers and, after 10 seconds, both browsers display the message ‘hello deferred’. The console shows this message

2016-05-30 19:42:36.769 INFO 2104 --- [nio-8080-exec-1] fjab.MyController : ThreadName-http-nio-8080-exec-1
2016-05-30 19:42:36.769 INFO 2104 --- [onPool-worker-5] fjab.MyController : ChildThread-ForkJoinPool.commonPool-worker-5
2016-05-30 19:42:37.874 INFO 2104 --- [nio-8080-exec-1] fjab.MyController : ThreadName-http-nio-8080-exec-1
2016-05-30 19:42:37.874 INFO 2104 --- [onPool-worker-6] fjab.MyController : ChildThread-ForkJoinPool.commonPool-worker-6

As stated by http://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/web/context/request/async/DeferredResult.html,

“DeferredResult provides an alternative to using a Callable for asynchronous request processing. While a Callable is executed concurrently on behalf of the application, with a DeferredResult the application can produce the result from a thread of its choice.”

In this case, the thread used to produce the result comes from the ForkJoin pool through CompletableFuture.

Asynchronous version with Spring @Async

In this case, we hit http://localhost:8080/async in both browsers and, after 10 seconds, both browsers display the message ‘hello async’. The console shows this message

2016-05-30 19:49:36.896 INFO 2104 --- [cTaskExecutor-2] fjab.MyController : ThreadName-SimpleAsyncTaskExecutor-2
2016-05-30 19:49:37.864 INFO 2104 --- [cTaskExecutor-3] fjab.MyController : ThreadName-SimpleAsyncTaskExecutor-3

The default task executor to run asynchronous tasks in Spring is SimpleAsyncTaskExecutor. By the way, in this example there is no trace corresponding to the main thread. This is so because the method annotated with @Async runs entirely in the new thread.

Byte by byte