Blog @ Digital Bazaar

Fibers are the Future: Scaling Web Services Past 100K Concurrent Requests (Part 2/2)

The Test Configuration

The test hardware and software configuration used the same one that was used in the previous blog post. We utilized a single computer containing a quad-core AMD Phenom (1.8Ghz per core) and 4GB of RAM (3.2GB available). The software was the Digital Bazaar MODEST Web Service Processor configured to run two different processing models:

• MODEST Operations (100,000 operations, executed from a thread pool size of 4 – one thread per CPU core)
• MODEST Fibers (100,000 fibers, running on 4 operations executed from a thread pool size of 4 – one operation/thread per CPU core)

The Operation-based processing model utilizes 100K Operations running on a thread pool size of 4 (one thread per CPU core). This model starts a new Operation for every web service request. Each operation will encode and decode a fairly complex and verbose JSON object 50 times in one shot.

The Fiber-based processing model utilizes 100K Fibers running on 4 Operations on a thread pool size of 4 (one thread per CPU core). There is one operation per CPU core, and one Fiber per web service request. Each fiber will concurrently encode and decode a fairly complex and verbose JSON object 50 times, performing one encode/decode cycle for every time it is scheduled to run.

The difference between the Operation-based processing model and the Fiber-based processing model is that 50 JSON encode/decodes are performed per Operation time slice. The Fiber-based processing model performs 1 JSON encode/decode per Operation time slice. The following table summarizes the differences between both processing models.

Test Results

Each processing model was run with a thread pool size ranging from 1 to 8 and a work unit number ranging from 1 to 100,000. All work units were entered into each system before processing began. For example, 100K fibers were entered into the system and then the run function was called in order to simulate 100K simultaneous web service requests. The time taken to complete given a thread pool size and workload is shown on the vertical axis of the graphs below.

The two graphs above demonstrate a number of interesting things:

• The MODEST Web Service Processor is able to process 100,000 work units concurrently where POSIX threads fails to achieve processing 400 concurrent work units.
• It is possible to achieve 96% efficiency when scaling to multiple cores using the C/C++ programming language and a simple fiber-based application framework. There is no need to shift programming languages (like shifting to ERLANG) to achieve efficient scalability.
• Linear scalability is demonstrated given additional processing cores.
• Fibers are faster when less cores are used, slower when load is high and more cores are used (due to fiber scheduling overhead).
• Even though Fibers have much more concurrency, the processing throughput isn’t greatly affected compared to Operations.

While Fibers at heavy workloads may be slower than Operations at heavy workloads – the amount of concurrency achieved by Fibers are much higher than that achieved by Operations. Whether you use Fibers or Operations is wholly dependent on your application or particular web service call. MODEST allows you to mix and match Fibers and Operations at run-time based on the web service application or based on a particular web service call. If your web service application is performing something that requires low latency, like video streaming, you will want to use Fibers. If your web service application is performing bulk data processing and requires the highest throughput, you will want to use Operations.

How does the MODEST Web Services Processor achieve this sort of throughput?

1 Comment

1. Another common drawback of fiber-based systems is that you’re circumventing the OS and its understanding of your program’s parallelism. This means that if you happen to have a multicore or multiple CPUs, a traditional fiber system will not allow you to scale onto those other cpus.

   This is something a fiber-based system for modern architectures would have to take into consideration, such that there are multiple schedulers instead of a single one. (He says, feverishly trying to finish his own fiber system)

   Another thing to point out is that there’s definitely some pressure to move away from SQL and full-fledged databases towards something that can scale better. The proliferation of cloud datastores is one sign. Another is the idea that a p2p based datastore can actually be quite reliable and allow you to scale easily*.

   I say “move away”, but if it’s anything like other technology, they’ll pull to one side and meet in the middle somehow.

   * Chord – http://pdos.csail.mit.edu/chord/
   Similar idea, in video: http://skillsmatter.com/podcast/erlang/building-a-transactional-distributed-data-store-with-erlang

   Comment by Jonathan Turner — October 22, 2008 #

Sorry, the comment form is closed at this time.