I am very pleased to announce today that Aster nCluster now brings together the expressive power of a MapReduce framework with the strengths of a Relational Database!

Jeff Dean and Sanjay Ghemawat at Google had invented the MapReduce framework in 2004 for processing large volumes of unstructured data on clusters of commodity nodes. Jeff and Sanjay’s goal was to provide a trivially parallelizable framework so that even novice developers (a.k.a interns) could write programs in a variety of languages (Java/C/C++/Perl/Python) to analyze data independent of scale. And, they have certainly succeeded.

Once implemented, the same MapReduce framework has been used successfully within Google (and outside, via Yahoo! sponsored Apache’s Hadoop) to analyze structured data as well.

Read the rest of this entry »

Today we are pleased to welcome Pentaho as a partner to Aster Data Systems. What this means is that our customers can now use Pentaho open-source BI products for reporting and analysis on top of Aster nCluster.

We have been working with Pentaho for some time on testing the integration between their BI products and our analytic database. We’ve been impressed with Pentaho’s technical team and the capabilities of the product they’ve built together with the open source community. Pentaho recently announced a new iPhone application which is darn cool!

I guess, by induction, Aster results can be seen on the iPhone too.

We took a decision early on in building the company that we’d make our platform open in technology and have an inclusive philosophy on business.

I am glad to say that this year we have started delivering on our business philosophy.

We have good relationships with several smart consulting teams, and are actively working with them to bring innovative solutions to the market for our joint customers. We recently recommended a partner to a company where we were not a good fit because we felt that our partner could bring a lot of value to the prospect and that such introductions strengthen our extended network. We were genuinely surprised at the warmth it generated at both the company and the partner for us!

In the last few years, we’ve actively built our product to work on a variety of hardware platforms: we have customers running IBM, HP, Dell, and even white-box offerings! Earlier this week, we announced our partnership with Informatica. You will see a series of announcements appearing in the next few months.

We are actively looking for a person who can lead our efforts in establishing meaningful partnerships in the data warehousing space. If you know one, or are one, who shares an inclusive philosophy, drop us a note!

I had the opportunity to work closely with Anand Rajaraman while at Stanford University and now at our company. Anand teaches the Data Mining class at Stanford as well, and recently he did a very instructive post on the observation that efficient algorithms on more data usually beat complex algorithms on small data. He followed it up with an elaboration post. Google also seems to believe in a similar philosophy.

I want to build upon that observation here. If you haven’t read the posts, do read them first. It is well-worth the time!

I propose that there are 2 forces in action that help simple algorithms on big data beat complex algorithms on small data:

1. The freedom of big data allows us to bring in related datasets that provide contextual richness.
2. Simple algorithms allow us to identify small nuances by leveraging contextual richness in the data.

Let me expand my proposal using Internet Advertising Networks as an example.

Read the rest of this entry »

I’ve remarked in an earlier post that the usage of data is changing and new applications are on the horizon. Over the past few years, we’ve observed or invented quite a few interesting design patterns for business processes that use data.

There are no books or tutorials for these new applications, and they are certainly not being taught in the classrooms of today. So I figured I’d share some of these design patterns on our blog.

Let me start with a design pattern that we internally call “The Automated Feedback Loop”. I didn’t invent it but I’ve seen it being applied successfully at search engines during my research days at Stanford University. I certainly think there is a lot of power that remains to be leveraged from this design principle in other verticals and applications.

Read the rest of this entry »

I am glad to share the news that one of our first customers, MySpace, has scaled their Aster nCluster enterprise data warehouse to more than 100 Terabytes of actual data.

It is not easy to cross the 100TB barrier, especially when loads happen continuously and queries are relentless, as they are at MySpace.com.

Hala, Richard, Dan, Jim, Allen, and Aber, you have been awesome partners for us! It has been a great experience for Aster to work with you and we can see the reasons behind MySpace’s continued success. Your team is amazingly strong and capable and there is a clear sense of purpose. Tasso and I often remark that we need to replicate that culture in our company as we grow. At the end of the day, it is the culture and the strength of a team that makes a company successful.

And to everyone at Aster, you have been great from Day 1. It is impressive how a fresh perspective and a clean architecture can solve a tough technical challenge!

Thank you. And I wish everyone as much fun in the coming days!

Have you ever discovered a wonderful little restaurant off the beaten path? You know the kind of place. It’s not part of some corporate conglomerate. They don’t advertise. The food is fresh and the service is perfect – it feels like your own private oasis. Keeping it to yourself would just be wrong (even if you selfishly don’t want the place to get too crowded).

We’re happy to see a similar anticipation and word-of-mouth about some new ideas Aster is bringing to the data analytics market. Seems that good news is just too hard to keep to yourself.

We’re serving up something unique that we’ve been preparing for several years now. We’re just as excited to be bringing you this fresh approach.

My name is Mayank, and I co-founded Aster Data Systems with George and Tasso in 2005.

Shortly after incorporation, the three of us were eating lunch at a Chinese restaurant and out popped a fortune slip from a cookie reading:

Indeed. The Internet is changing the speed at which we communicate, processes are being automated to react and execute in the blink of an eye, and data is playing a key role in guiding execution. Analysis of data is moving front-and-center, breaking out of the passive world of warehousing and reporting, as applications create intelligent processes, and companies live-and-die by their ability to monetize their data.

A new set of applications are being written – or waiting in-the-wings to be written – that will leverage data to act smarter. Consider the rapid evolution of online advertising networks: in the past 5 years, we have seen a spate of successful companies carving out a niche for themselves in the market. Their differentiation? The unique ability to match advertising inventory with consumer segments. Their basis of differentiation? Data!

Read the rest of this entry »