04
Mar
By Chris Neumann in Blogroll on March 4, 2009
   

MySpace decided to support one of its most important product launches of 2008 with an expansion of its Aster data warehouse. The data that would be collected would be used to provide information on trends in media and current interests on MySpace. The go-live date was October 2008.

MySpace Discusses Their Use of Aster nCluster

MySpace planned for the data warehouse right from the inception of the project to ensure that reporting was considered a first-class citizen in the overall launch process, rather than a post-launch activity. The result was that the data warehouse was up and running to receive the usage streams, even during a private beta release period, giving the warehousing team the necessary time to prepare for the onslaught of data that would result after the public release.

The launch was precisely on-time, and this video talks about the experience of MySpace in rolling out Aster nCluster on a broader scale after their initial deployment of Aster earlier in 2008. The combined Aster deployment now has 200+ commodity hardware servers working together to manage 200+ TB of data that is growing at 2-3TB per day by collecting 7-10B events that happen on one of the world’s largest social networks every day!

In fact, there is a very interesting incident that happened on the day of the new MySpace product launch. At about 7am, one of the servers in the Aster nCluster data warehouse failed. The failure was detected by our support team – and no scrambling ensued. Aster nCluster detected and isolated the failure, continuing to run the service with n-1 nodes without a blip and minimal performance change! Later, after the initial tense moments were behind us, the MySpace operations team walked over and replaced the failed hardware. The Aster database administrator then pressed a single button to re-include the node back to the nCluster data warehouse – the database continued to hum away with zero downtime.

The power of “Always-On”!

We will be co-hosting a case study by MySpace on their use of Aster at the Gartner BI Summit next week in National Harbor, MD on March 11.  If you’ll be at the event, please come by to hear what Hala has to say about their use of Aster to support their mission-critical operations at MySpace across multiple functions and departments.

Their Aster enterprise data warehouse supports frontline applications (e.g., MySpace TV, MySpace Video, etc.), as well as online marketing, sales, IT, finance, international, and legal.  MySpace is also planning  to incorporate data from Aster into a balanced scorecard for strategic alignment of the business around key performance indicators, as well as other future projects.

Some highlights from the video for folks who would rather read:

MySpace got up and running with Aster quickly

We were able to bring that up online and actually start processing the data into it within a matter of weeks, and I think very few technologies give you the ability to do something like that.”
- Hala Al-Adwin, VP of Data Services at MySpace

Aster is mission-critical to MySpace
With Aster, what we’ve been able to produce with commodity hardware has been a supercomputer-like infrastructure …the data that we collect and process is absolutely critical to the success of MySpace.
–Bita Mathews, Data Warehouse Manager, MySpace

Right now our key business performance metrics are all powered out of the Aster system.  If somebody went and shut it down, none of that would be available.  I think in a lot of ways, we were lacking that data before, and now that we’re used to having it, people are just hungry for more and more information.  So if all that went away, I think it’s kinda like going back to an age where there was no light.
–Hala Al-Adwan

MySpace’s data warehouse with Aster is extremely reliable
Aster is always on and available.  And this is very amazing thing about Aster, because it’s massive.  There’s a lot of hardware underneath the system.  When hardware fails, we can continue working, and although we know some engineers are fixing hardware, but that doesn’t stop us from continuing to run queries and producing our reports.
–Anna Dorofiyenko, Data Architect, MySpace

Aster is the blueprint for successful data warehouse deployments going forward
Integrating Aster and including them from the very beginning in the MySpace Music project from beginning to end is what allowed that to be the most successful data warehouse implementation we’ve had to date, and I think we should definitely use it as a blueprint for any future implementations we do.
–Christa Stelzmuller, Chief Data Architect, MySpace


Comments:

[...] Data has put up a blog post embedding and summarizing a video about its MySpace account. Basic metrics include: The combined [...]

Post a comment

Name: 
Email: 
URL: 
Comments: