I'm not using uwsgi as a router, and the speed of the requests isnt the issue.
If for example, you randomly routed to a bunch of tornado servers that were overloaded/slow serving requests, you'd hit the same issue as that post.
Async/coroutines dont solve the problem, better routing does.
In this case, we use HAProxy, and you ideally would say "I know node X can handle 100 concurrent requests), and within HAProxy you could weight the node/set the maximum concurrent requests to said node, and it would distribute based on that.
The point is that the probability of the servers getting overloaded is greatly diminished, because they're not blocked for extended periods of time. An external API request can take several seconds. In that time, a framework that isn't async-capable is going to be blocked and unable to work on more requests in the dyno queue. Tornado (and node, etc.) can continue happily along.
On top of that, tornado has multi-processing support built-in, which adds intelligent routing at the dyno-level.
HAProxy is another solution, although it can't be setup on heroku so that's irrelevant. Also, it doesn't change the fact that your workers are blocked while executing long-running API requests. uwsgi won't save you either.
The linked doc literally says as much, toward the bottom: "So the only solution is for Heroku to return to routing requests intelligently. They claim that this is hard for them to scale, and that it complicates things for more “modern” concurrent apps like those built with Node.js and Tornado. But Rails is and always has been Heroku’s bread and butter, and Rails isn’t multi-threaded.
In fact a routing layer designed for non-blocking, evented, realtime app servers like Node and its ilk — a routing layer that assumes every dyno in a pool is as capable of serving a request as any other — is about as bad as it gets for Rails, where almost the opposite is true: the available dynos are perfectly snappy and the others, until they become available, are useless. The unfortunate conclusion being that Heroku is not appropriate for any Rails app that’s more than a toy."
If for example, you randomly routed to a bunch of tornado servers that were overloaded/slow serving requests, you'd hit the same issue as that post.
Async/coroutines dont solve the problem, better routing does.
In this case, we use HAProxy, and you ideally would say "I know node X can handle 100 concurrent requests), and within HAProxy you could weight the node/set the maximum concurrent requests to said node, and it would distribute based on that.