Previous month:
September 2010
Next month:
November 2010

How Real is the Data Deluge?


It seems obvious that given the decreasing cost of storage and computation, there's going to be a significant increase in the volume of data that organizations accumulate over the next 10 years.  But the type of data being accumulated may be different from the areas where traditional DBMSs dominated.  It's not just about transactions; it's search patterns, on-line behavior, click-thru data, events fired off by smartphones, messages over Twitter & Facebook, log data of various kinds.

If an organization can figure out a better way identify prospects, or deliver more targeted ads, or optimize pricing decisions by analyzing terrabytes of data, they'd be crazy not to. Over the long term, companies that don't develop these capabilities will be at a competitive disadvantage.

As to what the implications are from a technological perspective, that's a whole different can of worms. I'm starting to see adoption of Big Data technologies like Hadoop, HDFS, Cassandra, MongoDB, XML databases, analysis with R, Pentaho, and loads of other technologies.  And MySQL continues to play a role here as do other traditional relational databases.  Over the next few months, I'm going to dig down deeper with people using these technologies to try and discern the emerging customer patterns.

If you're in this space or using some of these technologies, let me know your thoughts. What volume of data are you dealing with?  How many nodes or servers are you using?  Are you running on a public cloud, private cloud or hybrid? What technologies did you evaluate?  What about traditional DBMSs didn't work for this scenario? 

Help Bring Zork and the FyrevM to Android, Kindle et al

David Cornelson of TextFyre has embarked on an ambitious plan to create a new open source virtual machine, FyreVM.  This new VM will run Interactive Fiction games (e.g. Zork and newer works written in Inform) on a dozen different mobile platforms such as Android, WinPhone 7, Kindle, iPhone, iPad, Blackberry.  The goal of FireVM is to take advantage of specific user interface capabilities on each platform, whether it's the touch screen of Android tablets or the 5 way button on the Kindle.

To help with this project, TextFyre has started  a fundraising effort on Kickstarter with a goal of raising $5,000.  To make it interesting, Cornelson is offering several incentives for sponsors:

  •  $20  -- A copy of all TextFyre's current products
  •  $50  -- A copy of all TextFyre's current products plus two in the works
  • $100  -- Your IF game will be commercially published by TextFyre
  • $500  -- A Kindle loaded with TextFyre games and a t-shirt
  • $1000 --An iPad or Android tablet with TextFyre games and a t-shirt

The Kickstarter funding ends Saturday October 16. I hope you'll join me, other MySQLers, and IF fans  in making a donation. I think interactive fiction is an interesting area of retro computing and want to encourage the development of open source tools and platforms.  Note that Cornelson is publishing TextFyre under an open source license.