The past two months have been busy. In New Zealand, we have just come out of lockdown, but the situation remains precarious.
Light Weight Per-Thread Fiber Scheduler
The proposal was recently accepted by Matz for experimentation and the CRuby implementation was merged. The interface provides a transparent bridge between native Ruby I/O and an underlying event loop, like that provided by Async. Between now and the release of Ruby 3, we need as much feedback on this interface as possible.
Async provides a scheduler implementation which conforms to the requirements of the proposal. It allows you to use native Ruby I/O in an event driven fashion. You can see some examples in the specs.
The current scheduler interface is not without limitations. We don't support all blocking operations including DNS resolution, thread synchronisation, process waiting, and libraries that use direct system calls for I/O. In addition, we are considering how to make Mutex aware of fibers and the event loop. That being said, this is a great first step and opens the door to the above improvements and more.
If you'd like to learn more about the scheduler proposal and it's implementation, you can checkout Improving Net::HTTP Concurrency, the Ruby Concurrency Progress Report and the Ruby Concurrency Final Report.
Socketry.io is an initiative to build a sustainable open source business for the Socketry organsation and all the related gems. Initially, we are offering a premium subscription model for the Falcon application server. Looking forward, we plan on adding long-term support releases including exclusive access to backported bug fixes, advance notice of security issues, expanded documentation, and other business-specific requirements. If you have a business that depends on Falcon, now is a great time to get involved and support its future!
The current implementation of the Socketry.io website is closed-source but it's general purpose and can work with any kind of subscription-based product. It is built on top of Stripe, Utopia and Relaxo. If you would be interested to help work on an open source release of the web application, feel free to let me know!
"How can I solve my problem with this library of code?" is the most important question that documentation needs to answer. But existing documentation tools often fail to provide the structure and organisation required for new users. Utopia Project provides task-centric documentation, cross-referenced with source code, and will be used as a starting point to rethink and improve the development experience.
I've already started rolling this out accross several projects:
In order to support the rich cross-referencing required by Utopia Project, I created Decode. It can quickly generate an index of Ruby source code using the parser gem, and can resolve symbols in source code comments and embedded in markdown easily.
As a general framework for source code analysis, its fairly abstract and in the future I hope to extend it to index C and C++ code, which is useful for handling native extensions.
Wrangling with Markdown documents is non-trivial, but the cmark implementation provides a full abstract syntax tree (AST) for reading, processing and generating documents. I made a fork of CommonMarker after wrangling with Kramdown. My primarily motivation was to improve the interface for document manipulations and rewriting. Utopia::Project uses this interface for processing source code comments, as well as rewriting the README, extracting the title of the document for the page title, etc.
Turbo Test (working title) is a new collaboration between myself and Sam Saffron to improve the parallelism of the RSpec tests within Discourse. I've already created a small proof of concept task runner using Async::Container and preliminary results are promising.
We aim to have a working prototype within the next month. If you are a business that would benefit from such a tool, please help to accelerate its development by sponsoring my work.
If you are a company and you depend on Ruby, I invite you to support my work. I also provide commercial support agreements and contracting for businesses that want to improve their infrastructure using Falcon and Async. Feel free to get in touch.