Async Ruby

Hacker News discussion

Ruby has an Async implementation!

It's available today, it's production-ready, and it's probably the most awesome thing that's happened to Ruby in the last decade, if not longer.

Async Ruby adds new concurrency features to the language; you can think of it as "threads with none of the downsides". It's been in the making for a couple of years, and with Ruby 3.0, it's finally ready for prime time.

In this post, I hope to show you all the power, scalability, and magic of Async Ruby. If you love Ruby, this should be exciting, really exciting!

Async gem

What is Async Ruby?

First and foremost, Async is just a gem and can be installed with gem install async. It's a pretty special gem because Matz invited it to Ruby's standard library, but the invite has not yet been accepted.

Async Ruby was created by Samuel Williams, who is also a Ruby core committer. Samuel also implemented Fiber Scheduler, a big Ruby 3.0 feature. It's "library agnostic" and may have other uses in the future, but currently, the main purpose of Fiber Scheduler is to enable seamless integration of async gem with Ruby.

Not a lot of gems get their custom-built Ruby integrations, but this one is worth it!

All of this tells you async is not "just another gem out there". The Ruby core team, including Matz himself, are backing this gem and want it to succeed.

Async ecosystem

Async is also an ecosystem of gems that work nicely together. Here's a couple of the most useful examples:

While each of the above-listed gems provides something useful, the truth is you only need the core async gem to access most of its benefits.

Asynchronous paradigm

Asynchronous programming (in any language, including Ruby) allows running many things at the same time. Most often, those are multiple network I/O operations (like HTTP requests) because async is the most efficient at that.

Chaos often ensues from multi-tasking: "callback hell", "promise hell", and even "async-await hell" are well-known downsides of async interfaces in other languages.

But Ruby is different. Due to its superb design, Async Ruby does not suffer from any of these *-hell pitfalls. It allows writing surprisingly clean, simple, and sequential code. It's an async implementation as elegant as Ruby.

A word of notice: Async does not get around Ruby's Global Interpreter Lock (GIL).

Synchronous example

Let's start with a simple example:

require "open-uri"

start = Time.now

URI.open("https://httpbin.org/delay/1.6")
URI.open("https://httpbin.org/delay/1.6")

puts "Duration: #{Time.now - start}"

The above code is making two HTTP requests. The total duration of a single HTTP request is 2 seconds, and this includes:

Let's run the example:

Duration: 4.010390391

As expected, the program takes 2 x 2 seconds = 4 seconds to finish.

This code is decent, but it's slow. For both requests, the execution goes something like this:

The problem is that the program is waiting for most of the time; 2 seconds are like an eternity.

Threads

A common approach to making multiple network requests faster is using threads. Here's an example:

require "open-uri"

@counter = 0

start = Time.now

1.upto(2).map {
  Thread.new do
    URI.open("https://httpbin.org/delay/1.6")

    @counter += 1
  end
}.each(&:join)

puts "Duration: #{Time.now - start}"

The code output is:

Duration: 2.055751087

We reduced the execution time to 2 seconds, and this indicates the requests ran at the same time. Good, problem solved then?

Well, not so fast: if you've done any real-world thread programming, you know threads are hard. Really, really hard.

If you intend to do any serious work with threads, you better get comfortable using mutexes, condition variables, handling language-level race conditions... Even our simple example has a race condition bug on the line @counter += 1!

Threads are hard, and it's no wonder the following statement continues making rounds in the Ruby community:

I regret adding threads.

—Matz

Async examples

With all the thread complexities, Ruby community is long overdue for a better concurrency paradigm. With Async Ruby, we finally have one.

async-http

Let's see the same example of making two HTTP requests, this time using Async Ruby:

require "async"
require "async/http/internet"

start = Time.now

Async do |task|
  http_client = Async::HTTP::Internet.new

  task.async do
    http_client.get("https://httpbin.org/delay/1.6")
  end

  task.async do
    http_client.get("https://httpbin.org/delay/1.6")
  end
end

puts "Duration: #{Time.now - start}"

And the example output is:

Duration: 1.996420725

Looking at the total run time, we see the requests ran at the same time.

This example shows the general structure of Async Ruby programs:

Once you get used to it, you see this structure is actually pretty neat.

URI.open

One thing that could be considered a disadvantage of the previous example is the usage of async-http, an async-native HTTP client. Most of us have our preferred Ruby HTTP client, and we don't want to spend time learning the ins and outs of yet another HTTP library.

Let's see the same example with URI.open:

require "async"
require "open-uri"

start = Time.now

Async do |task|
  task.async do
    URI.open("https://httpbin.org/delay/1.6")
  end

  task.async do
    URI.open("https://httpbin.org/delay/1.6")
  end
end

puts "Duration: #{Time.now - start}"

The only difference from the previous example is we swapped async-http with URI.open, a method from Ruby's standard library.

The example output is:

Duration: 2.030451785

This duration shows two requests ran in parallel, so we conclude URI.open ran asynchronously!

This is all really, really nice. Not only we don't have to put up with threads and their complexities, but we can also use Ruby's standard URI.open to run requests, both outside and inside an Async block. This can certainly make for some convenient code reuse.

Other HTTP clients

URI.open is vanilla Ruby, but it may not be your preferred way to make HTTP requests. Also, you don't see it often being used for "serious work".

You probably have your preferred HTTP gem, and you may be asking "will it work with Async"? To find out, here's an example using HTTParty, a well-known HTTP client.

require "async"
require "open-uri"
require "httparty"

start = Time.now

Async do |task|
  task.async do
    URI.open("https://httpbin.org/delay/1.6")
  end

  task.async do
    HTTParty.get("https://httpbin.org/delay/1.6")
  end
end

puts "Duration: #{Time.now - start}"

In this example, we're running URI.open and HTTParty together, which is perfectly fine.

The output is:

Duration: 2.010069566

It runs slightly longer than 2 seconds, which shows both requests ran concurrently (at the same time).

The takeaway here is: you can run any HTTP client inside an Async context, and it will run asynchronously. Async Ruby fully supports any existing HTTP gem!

Advanced example

So far, we only saw Async Ruby making requests with various HTTP clients. Let's unveil the full power of Async with Ruby 3.

require "async"
require "open-uri"
require "httparty"
require "redis"
require "net/ssh"
require "sequel"

DB = Sequel.postgres
Sequel.extension(:fiber_concurrency)
start = Time.now

Async do |task|
  task.async do
    URI.open("https://httpbin.org/delay/1.6")
  end

  task.async do
    HTTParty.get("https://httpbin.org/delay/1.6")
  end

  task.async do
    Redis.new.blpop("abc123", 2)
  end

  task.async do
    Net::SSH.start("164.90.237.21").exec!("sleep 1")
  end

  task.async do
    DB.run("SELECT pg_sleep(2)")
  end

  task.async do
    sleep 2
  end

  task.async do
    `sleep 2`
  end
end

puts "Duration: #{Time.now - start}"

We extended the previous example that contains URI.open and HTTParty with five additional operations:

All of the operations from this example also take exactly 2 seconds to run.

Here's the example output:

Duration: 2.083171146

We get the same output as before, which indicates all the operations ran concurrently. Wow, that's a lot of different gems that can run asynchronously!

Here's the point: any blocking operation (a method where Ruby interpreter waits) is compatible with Async and will work asynchronously within Async code block with Ruby 3.0 and later.

The performance looks good: 7 x 2 = 14 seconds, but the example completes in 2 seconds – an easy 7x gain.

Fiber Scheduler

Let's take a moment and reflect on something important. All the operations from this example (e.g., URI.open, Redis, sleep) behave differently based on the context:

Synchronously
Operations behave synchronously by default. The whole Ruby program (or more specifically, the current thread) waits until an operation completes before moving to the next one.
Asynchronously
Operations behave asynchronously when wrapped in an Async block. With this, multiple HTTP or network requests can run at the same time.

But how can, for example, HTTParty or sleep method be synchronous and asynchronous at the same time? Does async library monkey patch all these gems and internal Ruby methods?

This magic works because of the Fiber Scheduler. It's a Ruby 3.0 feature that enables async to integrate nicely with existing Ruby gems and methods – no hacks or monkey patching needed!

Fiber Scheduler can also be used standalone! With that approach, asynchronous programming can be enabled with just a couple built-in Ruby methods.

As you can imagine, the scope of the code that the Fiber Scheduler touches is huge: it's every blocking API Ruby currently has! It's no small feature by any means.

Scaling example

Let's crank things up and show another aspect that Async Ruby excels at: scaling.

require "async"
require "async/http/internet"
require "redis"
require "sequel"

DB = Sequel.postgres(max_connections: 1000)
Sequel.extension(:fiber_concurrency)
# Warming up redis clients
redis_clients = 1.upto(1000).map { Redis.new.tap(&:ping) }

start = Time.now

Async do |task|
  http_client = Async::HTTP::Internet.new

  1000.times do |i|
    task.async do
      http_client.get("https://httpbin.org/delay/1.6")
    end

    task.async do
      redis_clients[i].blpop("abc123", 2)
    end

    task.async do
      DB.run("SELECT pg_sleep(2)")
    end

    task.async do
      sleep 2
    end

    task.async do
      `sleep 2`
    end
  end
end

puts "Duration: #{Time.now - start}s"

This example is based on the previous one, with a couple of changes:

As before, every individual operation takes 2 seconds to execute. The output is:

Duration: 13.672289712

This shows that 5,000 operations, with a cumulative running time of 10,000 seconds, ran in just 13.6 seconds!

This duration is greater than the previous examples (2 seconds) because of the overhead of creating so many network connections.

We did almost no performance tuning (tweaking garbage collection, memory allocations etc.), and we still achieved the 730x "speedup", a pretty impressive result in my books!

Scaling limits

The best thing is: we're only scratching the surface of what's possible with Async Ruby.

While the maximum number of threads is 2048 (on my machine at least), the upper limit for the number of Async tasks is millions!

Can you really run millions of Async operations concurrently? Yes, you can – some users have already done that.

Async really opens new horizons for Ruby: think about handling tens of thousands of clients with a HTTP server or handling hundreds of thousands of websocket connections at the same time... it's all possible!

Conclusion

Async Ruby had a long, secretive development period, but it's stable and production-ready now. Some companies are already running it in production and reaping the benefits. To start using it, head over to the Async repository.

The only caveat is that it doesn't work with Ruby on Rails, because ActiveRecord doesn't support async gem. You can still use it with Rails if ActiveRecord is not involved.

Async's strongest point is scaling network I/O operations, like making or receiving HTTP requests. Threads are a better choice for CPU-intensive workloads, but at least we don't have to use them for everything anymore.

Async Ruby is super powerful and very scalable. It's a game-changer, and I hope this post demonstrates that. Async changes what's possible with Ruby, and will significantly impact the Ruby community as we all start thinking more asynchronously.

One of the best things is that it does not make any of the existing code obsolete. Just like Ruby itself, Async is beautifully designed and a joy to use.

Happy hacking with Async Ruby!