Ruby Concurrency Progress Report

Last month I was able to spend some time discussing the concurrency model for Ruby 3 with Matz and Koichi. This progress report is a combination of both discussions.

Concurrency and Parallelism

Concurrency is the interleaving of tasks which are performing non-blocking operations. Parallelism is the simultaneous execution of tasks which are performing processor intensive work. With this in mind, we discussed having two separate abstractions. By doing this, we can keep both concurrency and parallelism simple by avoiding thread-safety issues which plague existing code.

# Channels for communication between isolates:
channel = Isolate::Channel.new

# @option bare [Boolean] minimal stdlib (e.g. exclude Thread, etc).
Isolate.new(bare: true) do |isolate|
	undef Thread # or maybe something like this?
	
	isolate.signal(:int) do
		
	end
	
	Async do |task|
		isolate.selector = task.reactor
		
		# fiber-based concurrency for all I/O operations:
		object = channel.read
	end
end

Isolate.new do
	# Default to blocking I/O:
	channel.write({foo: "bar"})
end

Isolate.wait_all

Non-determinism

Any time non-determinism is introduced, it is possible for some unexpected sequence of operations to occur, for example:

count = 0

5.times do
	Async do
		# Save the value of count:
		tmp = count
		
		# Perform some non-deterministic operation:
		internet.get(reference)
		
		# Increment the value of count:
		count = tmp + 1
	end
end

# What is the value of count? 5?
puts count

Pratically speaking, it's not possible to avoid non-determinsim in modern computers. That being said, we can minimise its impact on user code, and the ways in which non-determinism is invoked from user code.

Concurrency

We discussed how async and falcon improve existing Ruby web applications. However, we want to extend this with light weight hooks that can work across all Ruby implementations.

Life Cycle Management

One of the critical points of async, is managing the life cycle of resources (connections, sockets, files, etc). Life cycles are scoped to Async{} blocks, and async provides explicit support for stopping entire server hierarchies using Task#stop.

I/O Wrappers

async-io provides IO wrappers for Ruby, which in many cases are drop-in replacements:

Node.js

We discussed libuv which is the core of the Node.js concurrency model. It provides wrappers for many non-blocking operations and implements them using the most efficient method available.

Options

We considered a number of options:

Auto Fibers

This patch essentially adds green threads back to Ruby. It improves the concurrency of non-blocking I/O but doesn't necessarily address any other issues. It is a fairly large patch with no active maintainers at present.

Wrappers

As implemented in async-io. Wrap existing I/O constructs and other operations. It works but doesn't directly support legacy code.

New Methods

We already have a legacy of both blocking and nonblocking methods. It moves the problem of concurrency into the user code, and additionally, we don't support the full range of nonblocking operations (e.g. IO#gets_nonblock doesn't exist).

New Classes

New I/O classes to support non-blocking operations. We may yet explore this option for improvements to the user-facing implementation, but we'd rather avoid it for now.

Isolates

We discussed how the future of Ruby will affect existing code. In particular, the current light weight selector implementation is specific to the current Thread. When using Isolate, the selector would be specific to that container.

We also discussed the kind of methods that the Selector interface should have, including how to capture details of blocking operations. The main ones we have right now: wait_readable, wait_writable, wait_sleep, but we will expand this to capture blocking operations within the VM itself.

We considered how different designs would impact CRuby, JRuby and TruffleRuby, including the different process models required. CRuby supports fork which can allow child processes to share memory, while JVM implementations of Ruby require isolated threads to achieve the same levels of scalability/reliability.

Additional Keywords

With the async/await keywords, making a method asynchronous requires changing the entire lexical call-tree, which can be challenging. When using Fibers to implement the same non-blocking models, this is transparent to the user. We discussed the implications of this on Ruby code:

if production?
	def log(*arguments)
		# Non-blocking operation:
		post_to_remote_log_server(*arguments)
	end
else
	def log(*arguments)
		$stdout.puts(*arguments)
	end
end

# Is this non-blocking?
log("Hello World")

The fiber based model minimises the burden on user code when libraries make such changes - i.e. concurrency model is an implementation detail.

Blocking Behaviour

In Ruby, there are many blocking operations, e.g. system(...) and File.read. We discussed how this impacts the concurrency model, and looked at how we can mitigate some of these issues. In particular, one point that came up is wrapping the existing rb_nogvl function in CRuby to detect these blocking methods and report on them. In the case of RSpec, we could add assertions, e.g.:

expect do
	post_data
end.to_not perform_blocking_operations

Mutability

Another important issue is mutability. Because Ruby is a dynamic language, we can detect and report on mutability, e.g. using trace points. These could be used to implement detection of shared mutable state, and additionally provide code coverage tools to detect mutability issues.

cache = {}

expect do
	fetch_and_update(resource, cache)
end.to only_modify(cache)

Sponsorship

Many thanks to the 2019 Ruby Association Grant and all my sponsors who help enable this work.

If you are a company and you depend on Ruby, I invite you to support my work. I also provide commercial support agreements and contracting for businesses that want to improve their infrastructure using Falcon and Async. Feel free to get in touch.