API Science Blog Why Does Tail Latency Matter?

It's a complicated question, actually: latency versus throughput. Throughput is how many responses you can give to your varied customers in a certain amount of time. Latency is the amount of time it took for an individual customer to receive the response to their request. If your product is going to succeed, obviously you need both excellent throughput and low latency: you need to be able to simultaneously provide your product to a large customer base and ensure that your customers receive what they've requested as quickly as possible...