Blog @ Digital Bazaar

Bitmunk 3.1 Released - Browser-based P2P Commerce

Today marks a significant milestone in the evolution of the Bitmunk peer-to-peer commerce platform. The software release that went live earlier today is the culmination of over 26 months of development, hundreds of thousands of lines of code writes and re-writes and the dream of a small group of us that are trying to fundamentally change the way people buy and sell digital goods on the Internet.

On the surface, Bitmunk looks much like a web-based digital content store specializing in MP3 music sales. People can come to the site and purchase songs and albums for very competitive prices (cheaper than iTunes and Amazon.com).

There is, however, a deeper history and a grander goal for Bitmunk. This blog post outlines why today’s software release is such a significant step towards that goal. We are creating an open, standardized, Internet-scale peer-to-peer commerce infrastructure for the purchase and sale of digital goods. This mechanism, dubbed Collaborate Content Distribution, would allow anything digital to be found, bought and then re-sold via your web browser. This technology shifts the purchase of music, movies, television, books, and any other sort of digital good from being a purely corporation-to-consumer experience to a peer-to-peer experience.

If we’re successful, Bitmunk will help bloggers, artists, writers, tweeple, actors, novelists, and many other people that produce creative and knowledge-industry based content to make a living doing what they do best, without all of the barriers to distribution that have existed to date.

It all started with Bitmunk 1.0… (next page)

A Collaborative Distribution Model for Music

The music industry, via Choruss, is shopping a new music licensing model around to universities in the United States. Like some before it, this one attempts to address the still rampant music piracy occurring via peer-to-peer networks by enforcing a pseudo-mandatory collective licensing agreement on every student attending a participating university. There were a number of very interesting parts to the proposal that we would like to work on improving with Choruss and any partner universities. There were also a few propositions that we think are harmful to the industry, artists and fans as a whole.

It should be no surprise that we think that any sort of mandatory collective licensing is a very bad idea, as is the “covenant not to sue” approach that Choruss is currently pursuing. Voluntary collective licensing, as proposed by organizations like the Electronic Frontier Foundation, is not a good alternative either.

The basic approach proposed via collective licensing is to allow the general public unfettered access to all types of intellectual property, such music, movies and books. One would be allowed to download copyrighted works via BitTorrent, Limewire, and YouTube without worrying about the copyright-owner filing a lawsuit. ISPs would include a collective licensing charge on your monthly Internet connectivity bill, say $10 for movies, $10 for books, and $10 for music, that would be distributed to copyright owners based on what one downloads.

While this may seem like a good idea at first, the approach is fatally flawed…

Absorbing Costs Considered Harmful

Bitmunk was founded on a number of principles that we have, unfortunately, not codified on the website yet. One of those principles is the concept that we will always strive to give a detailed break-down of the costs associated with the purchase of any digital good on our network. While some of our customers may not care about where the money goes, others do want to know exactly how much is going to the artist. The fundamental principle at work here is transparency. We believe that transparency regarding how we run our network, manage our costs and reward artists, buyers and sellers is a fundamental operating principle for Digital Bazaar.

Displaying credit card fees have been a part of this transparency. We list all fees that credit card processors charge so that our customers know where their money is going. Typically, this has been about 4.17% per credit card transaction. That is money that goes directly to the credit card agency and we include it as a line item on our website so that our customers know that we aren’t profiting in any way from that charge.

Credit card processors vary widely in the services that they provide as well as their technical sophistication. We have been appalled at how backwards some of the transaction systems are in the banking industry. Credit card processing is no different. Typically, when you use your credit card, some online stores don’t check your address. In other words, they bypass address verification completely because many people enter their addresses incorrectly.

The seedy under-belly of credit card transaction processing…

W3C: RDFa 1.0 is Official

RDFa became an official World Wide Web Consortium Recommendation today. This means that it has undergone an intense amount of design, feedback, development and scrutiny to become a recognized world-wide standard for the expression of web semantics. Manu Sporny, Digital Bazaar’s Founder, has been directly involved with the RDFa Task Force and the standardization work involved with this new Web technology.

We would like to thank Ben Adida (Creative Commons), Chair of the RDFa Task Force, and Mark Birbeck (webBackplane), the primary designer of RDFa, for their vision and tenacity. In addition, we would also like to thank members of the task force - primarily composed of Ralph R. Swick (W3C), Shane McCarron (Applied Testing and Technology), Steven Pemberton (Centrum Wiskunde & Informatica), and Michael Hausenblas (JOANNEUM RESEARCH).

The Web is based on a core dedication to standards. It is individuals, such as those listed above, and their respective organizations that continue to lead the way in standards innovation on the Web. It is a largely thankless job, which is why we would like to extend our deepest appreciation and admiration for an excellent job on RDFa and all that those involved with the W3C continue to do for the World Wide Web and its citizens.

RDFa will have an effect on hundreds of millions of people around the globe. It is a great privilege and honor to be a part of that effort.

POSIX Threads Don’t Scale Past 100K Concurrent Web Service Requests (Part 1/2)

Hard times are upon our financial sector. The US financial markets are in turmoil. Many companies will be cutting spending as a squeeze is placed on operating budgets over the next couple of months, if not years. This is usually good news to the technology sector as most cost cutting measures depend on technology to keep productivity at the same levels as they were before the sky (and stocks) started to fall. These are exciting times as well in the IT sector. We are seeing a shift in the way we compute - from centralized IT to cloud computing, from one core per processor to many cores per processor, from closed data storage to open data portability, from a web of documents to a web of meaning.

At the heart of this transformation is the concept of a Web Service. Web services are used to perform operations on the cloud. They are used to read data from one place on the Web, process and transform that data in another location, and then send the data to yet another location on the Web. It is through this method that we get mash-ups like Google Maps, Facebook apps, Flickr albums and Twitter streams.

These web services are the workhorse of the current Web. They are highly available, highly concurrent, and usually have tens if not hundreds of thousands of people slamming them at a time. This can lead to heartache for software developers. The fine folks at Twitter have had scaling issues over the past two years that required painful changes to their service to avoid continued downtime.

This is a two-part blog post about how traditional software development does not prepare you for the realities of writing scalable web services. Our company focuses a significant portion of our R&D efforts on scalability. One of the lessons that we have learned over the past three years is that pure POSIX threads do not scale for web services.

Check out the graph below and note how the red line (the pure POSIX threads approach) does a very abrupt nose-dive while attempting to reach 400 concurrent web service requests.

You do not want to be in this position, EVER. Most software developers will inevitably choose to use pure POSIX threads for their application servers in order to scale their web services. They do this because most education institutions and websites drill it into our heads that to have concurrency, you must use threads. “I would never make that mistake!”, you exclaim. However, if you use a standard Apache 2 configuration (which uses MPM_prefork) and PHP for your web services (each PHP instance is run in a separate process), you have already made that mistake.

Read on to find out how to scale past 400 concurrent requests…

Bitmunk 3.0 Website Launches

Today, is a big milestone - the release of the Bitmunk 3.0 website. This is a release that has been in the making for 18 months. While much of the functionality facing our customers has not changed, everything behind the scenes has received a huge update. You can still search, browse, and purchase music and video through any web browser. We have kept everything that worked well the same, but have also made big improvements to the behind-the-scenes stuff that will help us start to tightly integrate Bitmunk into a variety of websites, web browsers, and mobile devices.

Read on to find out what has changed, why we’re excited about the changes, and what it means for the future of collaborative content distribution…

Blacksburg BarCamp 1.0

The very first BarCamp in Blacksburg is going to be happening on June 14th, 2008. Make sure to tell all your technologist friends and direct them towards the following website:

Blacksburg BarCamp 1.0

Here are the sessions so far for the day, but new ones could be added or these ones could be changed slightly on the day of the event based on camper feedback:

• Introduction to BarCamp (Manu Sporny)
• Multicore Environment Programming (Jonathan Turner)
• Blacksburg Technology Wiki (Cory Donovan)
• Exchange vs. Zimbra/Funambol (Joseph K. Goodman)
• Cloud Computing (Matt Pfeil)
• Ninja Audio Effects (Andy Fabian)
• Ruby on Rails (Josh Eckstein)
• Hacking Parallel Filesystems (Manu Sporny)

The event is starting at 10am this Saturday at Mailtrust:

755 University City Boulevard
Blacksburg, VA 24060

The room that we will be in is on the first floor and is about 1000 feet away from the main Mailtrust offices, towards the Kroger supermarket. If you are facing the Mailtrust logo, turn right and walk down about 1000 feet. There will be signs for Blacksburg BarCamp 1.0.

The event is free to attend and participate.

Dynamic Spectrum Auctions and Digital Marketplaces

Chances are that your cell-phone’s data connection is slow compared to most wired connection speeds. This is especially true in the United States. An emerging field, called Software Defined Radio and Cognitive Radio, is starting to make headway in changing the way we use the airwaves.

Two of the leading universities in the world on Software Defined Radio and Cognitive Radio research are Virginia Tech and Trinity College in Dublin. Digital Bazaar’s founder, Manu Sporny, was asked to join both research organizations along with researchers and officials from Supélec in Rennes, France, the University of Colorado at Boulder, the European Commission, and Vanu to take part in a workshop on the future of SDR/CR research.

Digital Bazaar’s part in all of this was to brainstorm business models and methods of auctioning wireless spectrum in real time. The workshop was hosted by the Centre for Telecommunications and Value-chain Research (CTVR) in Dublin, Ireland. It consisted of an intense 2-day workshop where ideas were exchanged as to the current state of SDR/CR research, what’s going right and what’s going wrong, as well as what the future should hold for this quickly maturing field.

Several ideas were generated at the workshop that dealt with using Digital Bazaar’s technology to buy and then re-sell wireless spectrum in real-time. Effectively, this means that you could buy and sell wireless spectrum from your cellphone or laptop on an as-needed basis.

Here’s how it would work: If you only need 100Kbps in bandwidth on average for sending/receiving e-mail then you can sign up for that wireless plan. If, however, you need 2000Kbps every now and then for the occasional large download to your laptop, you can purchase that wireless bandwidth a la carte from those around you. Not only would you buy the bandwidth, but you would lease the wireless spectrum from those around you as well - effectively increasing your overall download speed. One of the big things that was missing until now was a system that could handle the dynamic auctioning and billing issues. Luckily, Digital Bazaar has spent the last three years building such a system.

Using some of the same technology that Bitmunk employs, it is conceivable that a wireless router or laptop could buy and re-sell spectrum from another system without the need to have a plan setup with a wireless or mobile provider. This means that people could buy and sell bandwidth from each other when they need it and without having a permanent Internet service plan with any provider. Cheap, pay as you go wireless Internet.

We’re very excited about the opportunity to work with these leading research institutions in making this technological dream a reality.

The Next Generation of Bitmunk Technology

In the next several months, we will be releasing technology that has been in development in our R&D labs for over a year and a half. This will be version 3.0 of our technology and it is a massive leap in speed, size reduction, and interoperability. What follows is a quick run-down on what we’re working on for Bitmunk 3.0.

Historically, most of our underlying technology has been written in Java. While it helped us prove the concept and get something into our customer’s hands, let’s face it - nobody likes running Java applications. All of our systems have undergone a full re-write to native C++. We have focused on making the next generation technology completely cross-platform, so there will be native versions of the Bitmunk buying, selling and management software for Windows, Mac OS X, and Linux.

We have also focused on speed and size reductions to make sure that the Bitmunk software can be put anywhere that there is a computer connected to the Internet. This includes not only palm-tops and laptops, but mobile phones, web browsers, media players, home entertainment systems and a variety of other multi-media devices. Not only is the new Bitmunk software much smaller than the old Java stuff, but it’s fast. For example, the old software could perform about 50 transactions per second. Not bad, until you see what the new stuff can do. The new Bitmunk software can handle close to 4000 transactions per second. That means that at the most basic level, it’s 80 times faster than the old software. While we have made many optimizations, most of the slowness was attributed to the Java Virtual Machine.

Digital Bazaar’s R&D labs have also been working on a really cool new technology with the World Wide Web Consortium called RDFa. This technology will help us create a world-wide standard for expressing media such as songs and film in music blogs, movie sites, and other media-related pages.

We are also working hard to make integrating Bitmunk into next generation browsers such as Firefox, Songbird and Miro an easier task for those developers. Don’t be too surprised if you see our technology in some of those browsers before the end of the year.

While that’s all we can say for now, there will be a number of really cool initiatives that we’ll be announcing on this blog in the coming months. Get ready to see some really cool stuff.

Over One Million Songs Available on Bitmunk

When Bitmunk launched over three years ago, we had 3,280 songs available for sale. Bitmunk was the first peer-to-peer distribution platform out there that was DRM-free and rewarded fans for trading songs by giving them a cut of the sale. It brought the record companies, music fans and music distributors together. We knew we were on to something - creating a community that was fair to everybody just made sense. We also knew that it would be a very long, hard road ahead to convince the artists and record labels to trust their fans to distribute their content.

Three years later…