Blog @ Digital Bazaar

Fibers are the Future: Scaling Web Services Past 100K Concurrent Requests (Part 2/2)

How does the MODEST Web Services Processor scale to 100K and beyond?

Note that there were no optimizations made to the Linux kernel or the system on which the MODEST Web Services Processor ran. The application ran under a regular UNIX user account with default priorities. The hardware was a cheap, commercial off-the-shelf system. The secret lies in how the software is designed to operate and provide maximum throughput and concurrency.

The Secret of Fiber Efficiency

When you create a new POSIX thread under Linux, 8MB is allocated to the thread for data and stack space. This is far less than a process, which could take up tens if not hundreds of megabytes. When you create a new Fiber in MODEST, only the space that is needed by the Fiber is allocated – usually around ten to a couple of hundred bytes. Fibers perform co-operative multi-tasking, meaning that they voluntarily take themselves off of the work queue when they have completed a unit of work. This has two effects:

• Concurrency doesn’t come at a high memory cost (as it does with POSIX threads).
• Process/Thread scheduling overhead is greatly reduced.
• The unit of work is decided by the Fiber, not the Operating System.

Downsides of Fibers

There are some downsides to this model that one must be aware of before implementing such a system:

• If a Fiber is greedy or buggy, the entire system suffers. A fiber, by design, cooperatively yields processing time. If a fiber never yields processing time, it could consume 100% of a single CPU core.
• You must implement the web service in C++ if you are going to use the MODEST Web Services Processor. Most web service developers expect a web service to be written in Java, PHP, ASP or similar web technology.

Closing Thoughts

The computing industry is known for jumping from one technology to the next, often before the technology matures and often because a technology claims to solve a current aggravation. Our company had jumped on the Tomcat/Java/SOAP bandwagon when it first started rolling and while it helped us get to where we are today, we would have been much better off not adopting the technology in the first place and building our own. That is the lesson learned over the past four years – sometimes you’re better off rolling your own.

Instead of the typical Tomcat/Java/SOAP or the Apache/PHP/JSON stack, we launched Bitmunk 3.0 in July 2008 with MODEST/C++/JSON as our web services stack, and we’re very happy with how it has been performing. We will release the source to MODEST after we get the next stable release of Bitmunk 3 out to the public.

Acknowledgments

Thanks go to Dave Longley for his hard work on and excellent design of the MODEST Web Services Processor and the rest of the core technology that allows us to do what we do. Mike Johnson, for design and development of our websites, databases and UIs, as well as keeping all the operational stuff running at Digital Bazaar. David I. Lehn for his hand in making MODEST a reality as well as continuing to push us to do things like this research and blog post.

This research is the result of their efforts to continue improving Bitmunk such that we can all some day achieve a healthy ecosystem for the purchase and sale of digital goods using web browsers, open standards, and royalty schemes that fairly reward artists, fans and publishers.

1 Comment

1. Another common drawback of fiber-based systems is that you’re circumventing the OS and its understanding of your program’s parallelism. This means that if you happen to have a multicore or multiple CPUs, a traditional fiber system will not allow you to scale onto those other cpus.

   This is something a fiber-based system for modern architectures would have to take into consideration, such that there are multiple schedulers instead of a single one. (He says, feverishly trying to finish his own fiber system)

   Another thing to point out is that there’s definitely some pressure to move away from SQL and full-fledged databases towards something that can scale better. The proliferation of cloud datastores is one sign. Another is the idea that a p2p based datastore can actually be quite reliable and allow you to scale easily*.

   I say “move away”, but if it’s anything like other technology, they’ll pull to one side and meet in the middle somehow.

   * Chord – http://pdos.csail.mit.edu/chord/
   Similar idea, in video: http://skillsmatter.com/podcast/erlang/building-a-transactional-distributed-data-store-with-erlang

   Comment by Jonathan Turner — October 22, 2008 #

Sorry, the comment form is closed at this time.