PERFORMANCE.md

General performance guidelines

Throughput and latency

The on-prem client aims primarily to minimize latency, to be suitable in a RTB context. You should be able to get a response from the audience or proximity client within a few milliseconds, often less than one millisecond. Additionally, you can increase throughput by making multiple concurrent queries on various threads - the on-prem client is thread-safe. For throughput in the neighborhood of 10,000 queries per second with reasonable latency, try making use of the client from dozens of threads. But of course, no setup will be perfect for all execution events, so run your own benchmarks and decide on a concurrency profile that's right for your needs.

testMultiplePoints special configuration

Most methods on the client objects do not use any special threading tools on their own: requests are serviced from the thread in your application that initiated the reqeust. However, to ensure that the extra work required to test multiple points at once doesn't degrade latency, there is a special threadpool dedicated to servicing calls to ProximityClient.testMultiplePoints. This threadpool is initialized the first time you actually make a call to testMultiplePoints, so if you don't make use of this feature the threadpool will not impact your performance at all.

However, if you do make use of testMultiplePoints (eg, to handle adx parallelogram locations), make sure to consider the threadpool size. A larger threadpool will improve your latency for each call to testMultiplePoints, but having too many active threads in your JVM at once will degrade overall performance. The default threadpool size is the number of CPU cores you have, but you can change that with the threadPoolSize field in the ClientParams object.

You can still make multiple concurrent requests to testMultiplePoints, but to maintain good latency, you should limit the number of concurrent requests: rather than several dozen, just 8 or so concurrent requests performed best in our benchmarking. If you need to increase throughput further than you can sustain with one client machine, and you have already chosen a threadpool size large enough to maximize CPU usage on one machine, your best option is to start some additional servers to share the load.