Ruby Fiber Scheduler

Hacker News discussion

Fiber Scheduler enables asynchronous programming in Ruby. The feature was one of the big additions to Ruby 3.0, and is one of the core components of the awesome async gem.

The best part is that you don't need a whole framework to get started! It's possible to achieve the benefits of asynchronous programming using a standalone Fiber Scheduler with just a couple of built-in Ruby methods.

The Fiber Scheduler consists of two parts:

Fiber Scheduler interface
A set of hooks for blocking operations built into the language. Hook implementations are delegated to the Fiber.scheduler object.
Fiber Scheduler implementation
Implements the asynchronous behavior. This is an object that needs to be explicitly set by the programmer, as Ruby does not provide a default Fiber Scheduler implementation.

Big thanks to Samuel Williams! He's a Ruby core developer who designed and implemented the Fiber Scheduler feature into the language.

Fiber Scheduler interface

Fiber Scheduler interface is a set of hooks for blocking operations. It allows for inserting asynchronous behavior when a blocking operation occurs. It's like callbacks with a twist: when the async callback is executed, the main blocking method does not run.

These hooks are documented with Fiber::SchedulerInterface class. Some of the main ideas behind this Ruby feature are:

Hook implementation

Let's look at the example showing how Kernel#sleep hook could be implemented. In practice all hooks are coded in C, but for clarity Ruby pseudocode is used here.

module Kernel
  def sleep(duration = nil)
    if Fiber.scheduler
      Fiber.scheduler.kernel_sleep(duration)
    else
      synchronous_sleep(duration)
    end
  end
end

The above code reads as following:

Other hooks work in a similar manner.

Blocking operations

The concept "blocking operation" was mentioned a couple times already, but what does it really mean? A blocking operation is any operation where a Ruby process (more specifically: current thread) ends up waiting. A more descriptive name for blocking operations would be "waiting operations".

Some examples are:

As a counterexample, the following snippet takes a while to finish, but does not contain blocking operations:

def fibonacci(n)
  return n if [0, 1].include? n

  fibonacci(n - 1) + fibonacci(n - 2)
end

fibonacci(100)

Getting the result of fibonacci(100) requires a lot of waiting, but it's only a programmer that's waiting! The whole time Ruby interpreter is working, crunching the numbers in the background. A naive fibonacci implementation does not contain blocking operations.

It pays off to develop an intuition on what a blocking operation is (and is not), as the whole point of asynchronous programming is to wait on multiple blocking operations at the same time.

Fiber Scheduler implementation

The implementation is the second big part of the Fiber Scheduler feature.

If you want to enable the asynchronous behavior in Ruby, you need to set a Fiber Scheduler object for the current thread. That's done with the Fiber.set_scheduler(scheduler) method. The implementation is commonly a class with all the Fiber::SchedulerInterface methods defined.

Ruby does not provide a default Fiber Scheduler class, nor an object that could be used for that purpose. It seems unusual, but not including the Fiber Scheduler implementation with the language is actually a good long-term decision. It's best to leave this relatively fast-evolving concern outside the core Ruby.

Writing a Fiber Scheduler class from scratch is a complex task, so it's best to use an existing solution. The list of implementations, their main differences, and recommendations can be found at Fiber Scheduler List project.

Examples

Let's see what's possible with just a Fiber Scheduler.

All examples use Ruby 3.1 and FiberScheduler class from the fiber_scheduler gem, which is maintained by yours truly. This gem is not a hard dependency for the examples, as every snippet below should still work if references to FiberScheduler are replaced with another Fiber Scheduler class.

Basic example

Here's a simple example:

require "fiber_scheduler"
require "open-uri"

Fiber.set_scheduler(FiberScheduler.new)

Fiber.schedule do
  URI.open("https://httpbin.org/delay/2")
end

Fiber.schedule do
  URI.open("https://httpbin.org/delay/2")
end

The above code is creating two fibers, each making an HTTP request. The requests run in parallel and the whole program finishes in 2 seconds.

Fiber.set_scheduler(FiberScheduler.new)
Sets a Fiber Scheduler in the current thread which enables Fiber.schedule method to work, and fibers to behave asynchronously.
Fiber.schedule { ... }
This is a built-in Ruby method that starts new async fibers.

The example uses only standard Ruby methods – both Fiber.set_scheduler and Fiber.schedule have been available since Ruby 3.0.

Advanced example

Let's see what running a multitude of different operations looks like:

require "fiber_scheduler"
require "httparty"
require "open-uri"
require "redis"
require "sequel"

DB = Sequel.postgres
Sequel.extension(:fiber_concurrency)

Fiber.set_scheduler(FiberScheduler.new)

Fiber.schedule do
  URI.open("https://httpbin.org/delay/2")
end

Fiber.schedule do
  # Use any HTTP library
  HTTParty.get("https://httpbin.org/delay/2")
end

Fiber.schedule do
  # Works with any TCP protocol library
  Redis.new.blpop("abc123", 2)
end

Fiber.schedule do
  # Make database queries
  DB.run("SELECT pg_sleep(2)")
end

Fiber.schedule do
  sleep 2
end

Fiber.schedule do
  # Run system commands
  `sleep 2`
end

If we ran this program sequentially it would take about 12 seconds to finish. But as the operations run in parallel, the total running time is just over 2 seconds.

You're not constrained to making just HTTP requests. Any blocking operation that's built into Ruby or implemented by an external gem works!

Scaling example

Here's a simple, although synthetic example running ten thousand operations at the same time.

require "fiber_scheduler"

Fiber.set_scheduler(FiberScheduler.new)

10_000.times do
  Fiber.schedule do
    sleep 2
  end
end

The code above completes in slightly more than 2 seconds.

The sleep method was chosen for the scaling example due to its low overhead. If we used network requests the execution time would be longer because of the overhead of setting up thousands of connections and performing SSL handshakes etc.

One of the main benefits of asynchronous programming is waiting on many blocking operations at the same time. The benefits increase as the number of blocking operations grows. Luckily, it's super easy to run large numbers of fibers.

Conclusion

Ruby can work asynchronously with just a Fiber Scheduler and a couple built-in methods – no frameworks are required!

It's easy to make it work. Choose a Fiber Scheduler implementation, and then use these methods:

Once you get it going, you can make any code asynchronous by wrapping it in a Fiber.schedule block.

Fiber.schedule do
  SynchronousCode.run
end

Whole libraries can easily be converted to async with this approach, and it rarely takes more effort than shown here.

The big benefit of asynchronous programming is parallelizing blocking/waiting operations to reduce the program running time. This often translates into running more operations on a single CPU, or even better, handling more requests with your web server.

Happy hacking with Fiber Scheduler!