Blog @ Digital Bazaar

Bitmunk 3.1 Released - Browser-based P2P Commerce

Today marks a significant milestone in the evolution of the Bitmunk peer-to-peer commerce platform. The software release that went live earlier today is the culmination of over 26 months of development, hundreds of thousands of lines of code writes and re-writes and the dream of a small group of us that are trying to fundamentally change the way people buy and sell digital goods on the Internet.

On the surface, Bitmunk looks much like a web-based digital content store specializing in MP3 music sales. People can come to the site and purchase songs and albums for very competitive prices (cheaper than iTunes and Amazon.com).

There is, however, a deeper history and a grander goal for Bitmunk. This blog post outlines why today’s software release is such a significant step towards that goal. We are creating an open, standardized, Internet-scale peer-to-peer commerce infrastructure for the purchase and sale of digital goods. This mechanism, dubbed Collaborate Content Distribution, would allow anything digital to be found, bought and then re-sold via your web browser. This technology shifts the purchase of music, movies, television, books, and any other sort of digital good from being a purely corporation-to-consumer experience to a peer-to-peer experience.

If we’re successful, Bitmunk will help bloggers, artists, writers, tweeple, actors, novelists, and many other people that produce creative and knowledge-industry based content to make a living doing what they do best, without all of the barriers to distribution that have existed to date.

It all started with Bitmunk 1.0… (next page)

Admitting that Javascript was a Mistake

There was an interesting article that was written by Guillaume Marceau recently about visually expressing the usefulness of programming languages. The article uses star-line plots to show how different programming languages compare with one another in speed and expressiveness, as each is used to solve a number of common problems. It’s always nice to check your gut reaction to different programming languages against empirical evidence. Language choice can be as varied as our food preferences, often not based solely on fact. Like our palate, we may find that our preference for our favorite programming languages change over time. As we learn more and use our language of choice to solve real problems, the initial love affair may turn into a nightmare.

At Digital Bazaar, our initial fondness for Javascript is turning into a deep distaste for the way Javascript has evolved in the browsers…

The Looming Cloud Computing Bubble

The number of stories in the online media about Cloud Computing has increased sharply over the last six months. There is a great deal of excitement around this new buzz word, but what is it all about?

The following graph is from Google Trends and shows the average news volume between the term “Cloud Computing” (in blue), and “Web 2.0″ (in red):

As you can see, the Cloud Computing news reference volume overtook Web 2.0 this month, which means that we’re well on our way to another technology bubble. The speed at which this bubble will grow is not known, but one thing is for certain - the media and the IT darlings have latched onto something that they are intent on hyping.

If the IT industry is not careful, we may end up over-promising and under-delivering on the latest tech industry promise of always-on, data-in-the-cloud, always-available computing services. Or worse yet, forgetting about vendor lock-in and forgetting why data portability is such a good thing.

You know those people that can spend an hour saying almost nothing at all? Well, that’s Cloud Computing. It has a good chance of doing just as much harm as good, much like what we went through during the early 90s vendor lock-in and the late 90s dot-com bust.

Read on to find out why Cloud Computing is mostly hot air…

Bitmunk 3.1 Website Launch

The Bitmunk 3.1 website quietly launched on Wednesday 8pm EST. This release comes six months after the Bitmunk 3.0 release and went a great deal more smoothly than the 3.0 release. The only major hang-up was an issue with IPv6 and DNS AAAA records, both of which we have disabled for the time being. We will bring the IPv6 side of our service back online when we have the time to work on the issue. Apologies for the handful of people that were hitting our website via IPv6.

This release contains fairly minor updates and bug fixes to the Bitmunk front-end website. WebBuy still looks the same as it did, but the entire back-end has been swapped out. The majority of the updates concern back-end web service updates, database updates and new P2P web services in preparation for the Bitmunk 3.1 peer-to-peer Firefox 3 plug-in release in the coming months.

Fibers are the Future: Scaling Web Services Past 100K Concurrent Requests (Part 2/2)

In a blog post last month, we outlined why a traditional Apache+PHP setup will inevitably fail the growing needs of medium to large AJAX-based websites. The article is continued this month by analyzing different methods of scaling web services past the concurrency barrier inherent in a basic Apache+PHP setup.

While speaking with the technical minds of several companies in our industry, there were several very good questions raised about assumptions we had made when building our system. The feedback from the first blog post about this topic revolved around the following two questions:

• Why would anybody need to scale to 100K (one hundred thousand) concurrent web service requests?
• Isn’t database layout and server bandwidth the primary scalability issue for web services?

We tend to forget that not everybody spends their time thinking about scaling to this level. Scalability requirements that are quite natural to us sometimes come off as exceedingly steep web service requirements to others. The questions above are ones that we have grappled with in the past. Here are the answers that we’ve found over the past several years through vigorous internal debate.

Important questions to ask when scaling past 100K connections…

W3C: RDFa 1.0 is Official

RDFa became an official World Wide Web Consortium Recommendation today. This means that it has undergone an intense amount of design, feedback, development and scrutiny to become a recognized world-wide standard for the expression of web semantics. Manu Sporny, Digital Bazaar’s Founder, has been directly involved with the RDFa Task Force and the standardization work involved with this new Web technology.

We would like to thank Ben Adida (Creative Commons), Chair of the RDFa Task Force, and Mark Birbeck (webBackplane), the primary designer of RDFa, for their vision and tenacity. In addition, we would also like to thank members of the task force - primarily composed of Ralph R. Swick (W3C), Shane McCarron (Applied Testing and Technology), Steven Pemberton (Centrum Wiskunde & Informatica), and Michael Hausenblas (JOANNEUM RESEARCH).

The Web is based on a core dedication to standards. It is individuals, such as those listed above, and their respective organizations that continue to lead the way in standards innovation on the Web. It is a largely thankless job, which is why we would like to extend our deepest appreciation and admiration for an excellent job on RDFa and all that those involved with the W3C continue to do for the World Wide Web and its citizens.

RDFa will have an effect on hundreds of millions of people around the globe. It is a great privilege and honor to be a part of that effort.

POSIX Threads Don’t Scale Past 100K Concurrent Web Service Requests (Part 1/2)

Hard times are upon our financial sector. The US financial markets are in turmoil. Many companies will be cutting spending as a squeeze is placed on operating budgets over the next couple of months, if not years. This is usually good news to the technology sector as most cost cutting measures depend on technology to keep productivity at the same levels as they were before the sky (and stocks) started to fall. These are exciting times as well in the IT sector. We are seeing a shift in the way we compute - from centralized IT to cloud computing, from one core per processor to many cores per processor, from closed data storage to open data portability, from a web of documents to a web of meaning.

At the heart of this transformation is the concept of a Web Service. Web services are used to perform operations on the cloud. They are used to read data from one place on the Web, process and transform that data in another location, and then send the data to yet another location on the Web. It is through this method that we get mash-ups like Google Maps, Facebook apps, Flickr albums and Twitter streams.

These web services are the workhorse of the current Web. They are highly available, highly concurrent, and usually have tens if not hundreds of thousands of people slamming them at a time. This can lead to heartache for software developers. The fine folks at Twitter have had scaling issues over the past two years that required painful changes to their service to avoid continued downtime.

This is a two-part blog post about how traditional software development does not prepare you for the realities of writing scalable web services. Our company focuses a significant portion of our R&D efforts on scalability. One of the lessons that we have learned over the past three years is that pure POSIX threads do not scale for web services.

Check out the graph below and note how the red line (the pure POSIX threads approach) does a very abrupt nose-dive while attempting to reach 400 concurrent web service requests.

You do not want to be in this position, EVER. Most software developers will inevitably choose to use pure POSIX threads for their application servers in order to scale their web services. They do this because most education institutions and websites drill it into our heads that to have concurrency, you must use threads. “I would never make that mistake!”, you exclaim. However, if you use a standard Apache 2 configuration (which uses MPM_prefork) and PHP for your web services (each PHP instance is run in a separate process), you have already made that mistake.

Read on to find out how to scale past 400 concurrent requests…

Thoughts on HTML5, RDFa and Microformats

This article was authored by one of our founders, Manu Sporny. He is an Invited Expert for the RDF in XHTML Task Force at the World Wide Web Consortium and a very active participant in the Microformats community.

We are first and foremost a media services company serving the music, movie, television and electronic book industries. Our mission is to help artists from every walk of life, in every corner of the world, to make a living performing their craft.

Our company does this by creating tools, services and online environments that monetize digital creations for artists by using cutting-edge technology to perform digital distribution. Peer-to-peer networks, swarming distribution, micro-payment-based transaction networks, semantic web technologies, cryptography and steganography are just a few of the tools that we use to deliver on our promise to our artists, publishers and content fans.

Over the past several years, we have been working to help create a standard method of expressing information about music, movies and television on the web with the Microformats and RDFa communities. It has always been our understanding that Microformats are only going to take the web so far, and we will need a more robust, standardized way of expressing semantic information.

Recently, several discussion threads have been started on the WHATWG mailing list (the group of people working on HTML5), that were very discouraging. Not only because they discount the need for such a standardized semantic mark-up technology, but because they are making the decision on factually inaccurate “gut-reactions” about how Microformats work. The exchange expressed a failure to understand the core need of the semantic web by the HTML5 community, mostly because they have never had to create a Microformat through the Microformats Process, nor implement a Microformat parser.

The rest of this article is an attempt to educate the larger web community about the limitations of Microformats, the Microformats Process, and why the web needs both RDFa AND the Microformats community in order to make the semantic web a reality.

We started out where most in the HTML5 community are, we thought Microformats would be the solution to the semantic web…

Bitmunk 3.0 Website Launches

Today, is a big milestone - the release of the Bitmunk 3.0 website. This is a release that has been in the making for 18 months. While much of the functionality facing our customers has not changed, everything behind the scenes has received a huge update. You can still search, browse, and purchase music and video through any web browser. We have kept everything that worked well the same, but have also made big improvements to the behind-the-scenes stuff that will help us start to tightly integrate Bitmunk into a variety of websites, web browsers, and mobile devices.

Read on to find out what has changed, why we’re excited about the changes, and what it means for the future of collaborative content distribution…

Blacksburg BarCamp 1.0

The very first BarCamp in Blacksburg is going to be happening on June 14th, 2008. Make sure to tell all your technologist friends and direct them towards the following website:

Blacksburg BarCamp 1.0

Here are the sessions so far for the day, but new ones could be added or these ones could be changed slightly on the day of the event based on camper feedback:

• Introduction to BarCamp (Manu Sporny)
• Multicore Environment Programming (Jonathan Turner)
• Blacksburg Technology Wiki (Cory Donovan)
• Exchange vs. Zimbra/Funambol (Joseph K. Goodman)
• Cloud Computing (Matt Pfeil)
• Ninja Audio Effects (Andy Fabian)
• Ruby on Rails (Josh Eckstein)
• Hacking Parallel Filesystems (Manu Sporny)

The event is starting at 10am this Saturday at Mailtrust:

755 University City Boulevard
Blacksburg, VA 24060

The room that we will be in is on the first floor and is about 1000 feet away from the main Mailtrust offices, towards the Kroger supermarket. If you are facing the Mailtrust logo, turn right and walk down about 1000 feet. There will be signs for Blacksburg BarCamp 1.0.

The event is free to attend and participate.