If you read this blog, you’ve probably seen the news about the partnership between Aster Data and Dell on their new PowerEdge C-Series servers (link to their page). Together we have enabled some really successful customers such as MySpace and Mint.com and proven that Dell hardware with Aster Data software easily scales to support large-scale data warehousing and advanced analytics.

Aster Data CEO Mayank Bawa explains this combination in more detail, as well as Aster Data’s history and the distinct advantages offered by the partnership with Dell, including Online Precision Scaling™, out-of-the-box advanced in-database analytics, and always-on availability.

This Monday we announced a new web destination for MapReduce, MapReduce.org. At a high level, this site is the first consolidated source of information & education around MapReduce, the groundbreaking programming model which is rapidly revolutionizing the way people deal with big data. Our vision is to make this site the one-stop-shop for anyone looking to learn how MapReduce can help analyze large amounts of data.

There were a couple reasons why we thought the world of big data analytics needed a resource like this. First, MapReduce is a relatively new technology and we are constantly getting questions from people in the industry wanting to learn more about it, from basic facts to using MapReduce for complex data analytics at Petabyte scale. By placing our knowledge and references in one public destination, we hope to build a valuable self-serve resource to educate many more people than what we could ever reach directly. In addition, we were motivated by the fact that most MapReduce resources out there focus more on specific implementations of MapReduce, which fragments the available knowledge and reduces its value. In this new effort we hope to create a multi-vendor & multi-tool resource which will benefit anyone interested in MapReduce.

We’re already working with analysts such as Curt Monash, Merv Adrian, Colin White and James Kobielus to syndicate their MapReduce-related posts. Going forward, we expect even more analysts, bloggers, practitioners, vendors, and academics to contribute. If traffic grows like we expect, we may eventually add a community forum to aid in interaction and sharing of knowledge and best practices.

I hope you enjoy surfing this new site! Free to email me for any suggestions as we work to make MapReduce.org more useful for you.

It looks like Aster Data isn’t the only company hoping to demystify advanced analytics as a growing number of organizations try to decide whether to dip their toe in the water or dive in completely. There was an interesting article from Doug Henschen of Intelligent Enterprise last week on their launch of the “Advantage in Analytics” TechCenter.

The main impediments to growth for the analytics market have been “lack of awareness, skills and a demand for fact-based decision-making within organizations,” Doug writes. This preceded our announcement of the Aster Analytics Center and MapReduce.org earlier this week to support those very goals.

The Aster Analytics Center provides best practices, and ready-to-use analytics solutions to jump-start development of advanced in-database analytics. Further, www.mapreduce.org provides valuable information to companies interested in understanding MapReduce-based analytics and learning how other companies have built rich data-driven applications using MapReduce and SQL-MapReduce. We expect these resources, along with Intelligent Enterprise’s “Advantage in Analytics” TechCenter, to be integral in increasing awareness and skills. And as more organizations learn the benefits, demand will become nearly universal. Hats off to Intelligent Enterprise for supporting these goals.

Along with the release of Aster Data nCluster 4.5, we added a new Solution Partner level for systems integrators and developers.

Why is this relevant?

Recession or no-recession, IT executives are constantly challenged. They are asked to execute strategies based on better analytics and information to improve effectiveness of business processes (customer loyalty, inventory management, revenue optimization, ..), while staying on top of technology-based disruptions and managing (shrinking or flat) IT budgets.

IT organizations have taken on the challenge by building analytics-based offerings  leveraging existing data management skills and increasingly taking advantage of MapReduce, a disruptive technology introduced by Google and now being rapidly adopted by mainstream enterprise IT shops in Finance, Telco, LifeSciences, Govt. and other verticals.

As MapReduce and big data analytics goes mainstream, our customers and ecosystem partners have asked us to make it easier for their teams to leverage MapReduce across enterprise application lifecycles, while harvesting existing IT skills in SQL, Java and other programming languages.  The Aster development team that brought us the SQL/MapReduce innovation, has now delivered the market’s first integrated visual development environment for developing, deploying and managing MapReduce and SQL-based analytic applications.

Enterprise MapReduce developers and system integrators can now leverage the integrated Aster platform and deliver compelling business results in record time (read how ComScore delivers 360 degree view of digital world to enterprise customers, Full Tilt Poker gains the upper hand tackling online fraud using Aster).

We are also teaming up with leaders in our ecosystem like MicroStrategy to deliver an end-to-end analytics solution to our customers that includes SQL/MapReduce enabled reporting and rich visualization. Aster is proud to be driving innovation in the Analytics and BI market and was recently honored at  MicroStrategy’s annual customer conference.

I am delighted with the rapid adoption of Aster Data’s platform by our partners and the strong continued interest from enterprise developers and system integrators in building big data applications using Aster. New partners are endorsing our vision and technical innovation as the future of advanced analytics for large data volumes.

Sign up today to be an Aster solution partner and join the revolution to deliver compelling information and analytics-driven solutions.

We’re a couple of days away from our second Big Data Summit event in as many calendar quarters, and it’s shaping up to be jam-packed with good presentations and conversation around innovations in data management and advanced analytics.  For people not aware, we’re conducting Big Data Summits regionally in North America and Europe with an eye on helping educate organizations who are looking for ways to tackle the enormous amount of data growing in (and outside) of their four walls. More importantly, we’re helping people answer the question, “what do I do with all this data?”.  If you can’t make it to the Bay Area on Feb 18th, look for one coming soon to a venue near you.  Here are some previews of what’s coming on Thursday:

1) Intuit (formerly mint.com) will be talking about how they anonymize consumer data and provide financial benchmarks and insights to individuals.  e.g., Do you spend more on dinner than the average Joe or Jane in California?

2)  Mobclix, a mobile advertising network, will discuss how they enable application developers for the iPhone, Android, BlackBerry, etc. to make more money by providing targeting advertising based on better analytics. (And how they’ve deployed it on Amazon Web Services to scale up & down on demand)

3) Merv Adrian from IT Market Strategy will keynote to talk about trends in data management and the questions you need to ask yourself as you consider ways to tackle big data problems. He’s also moderating a panel that I’m hearing will have some cameo appearances from other industry analysts with big brains.

4) Dell is our platinum sponsor and they’ll be talking about some new hardware they’re rolling out for big data computing and putting on display at the show.

5) Our CEO and co-founder will be talking about our approach to harnessing the power of big data and giving a glimpse of the future from Aster Data.

6) Other sponsors include Amazon Web Services, Informatica, and a new partner of ours, Impetus.

Hope to see you at the show.

Last week I attended Bank of America’s Technology Innovation Summit in Silicon Valley. In attendance were leading technology executives from Bank of America who outlined needs and challenges for the global banking giant. BofA’s annual IT spend is greater than $5 Billion, serving  almost 59 million, or one out of every two U.S. households and distribution strength of about six thousand branches, 18,000 ATMs and 24 million online banking customers, and more than 3,000 customer touches every second. Key themes discussed involved Cloud computing, Information Management, Security, Mobility and Green IT. And as I sat through the panel discussions and spoke to some of the IT leaders, it became evident that underpinning all the major business and IT initiatives for the global bank was a central theme – Lots of data, need for better and faster insights.

A senior BofA IT executive stated “Broad BI and data mining remain objectives, not realized goals”. There was a high level of interest in analytics and a big drive to be information-driven across business units.

Clearly, for a large bank like BofA, the business drivers exist. For example, the consumer channels executive was interested in understanding consumer behavior across different channels. In a saturated marketplace for retail customers and facing stiff competition from Chase (now owns WaMu), Wells Fargo (now owns Wachovia), BofA is keenly interested in strengthening its bond with its existing customer base.  With thousands of interactions per second, every interaction with the customer is an opportunity to learn more about customer behavior and customer preferences.

 In the credit card division, early detection of fraud patterns can translate into big savings for a market that is undergoing dramatic transformation due to reforms mandated by Congress. 

On the IT front, BofA has lots of existing investments in BI tools and data management software.

So where is the gap? Why are BI/data mining unrealized goals?

The answer lies in re-thinking and challenging the status quo in data management and analytic application development in today’s big data IT environments. Google, Amazon, and other innovators are leading this and it is only a matter of time before leaders in the financial services industry follow suit. A new mandate and architecture for big data applications  is emerging.

This new class of analytic applications will require a strategic investment in infrastructure that embraces assimilating advanced analytics processing right next to the terabytes to petabytes of enterprise data for key business initiatives including

• Customer service effectiveness to predict customer requirements as well as fully understand customer relationships across branch office, ATM, online, and mobile channels
• Ability to respond faster to regulators or to management and driving decisions based on insights driven from accurate, timely data

Broader, more pervasive BI and richer Analytics is on the threshold of becoming a reality!

Aster Data 4.0 is here and for those of you who subscribe to Aster Data’s blog, “Winning with Data”, you may have noticed that we’ve changed things up a bit.  This blog is now called the “Big Data Blog” and will continue to be a mash-up of opinions and news from the team at Aster Data. Topics will continue to be a mix of technical deep-dives as well as company announcements and content.

At the same time, our CEO and co-founder Mayank will be sharing his thoughts on a separate blog called “Winning with Data” where he will talk about his perspectives of the market trends, customer use-cases, technology evolution and company growth. You can find all of his previous posts there, as well as fresh content starting with the announcement of Aster Data’s massively parallel data-application server.

We just wrapped up our first of a two-part series on Mastering MapReduce together with Curt Monash. We’ve spent a lot of time discussing MapReduce with Curt and wanted to help educate the community on exactly what it is and how it applies to data management and analysis.  We’ve published the recorded webcast and below are the slides we presented from an Aster Data perspective which outline:

- What is Aster Data’s SQL-MapReduce?
- Example industry applications of SQL-MapReduce
- Walking through the SQL-MapReduce syntax

Curt has also posted his slides on DBMS2 with a great overview on dispelling the myths around MapReduce, and how MapReduce and SQL play nicely with each other.

We had great turn-out and questions from the sessions.  If you have any questions after reviewing the material, please drop a comment.

On Friday we had series of events and announcements around our new Aster-Hadoop Data Connector which utilizes key new SQL-MapReduce functions to provide ultra-fast, two-way data loading between HDFS (Hadoop Distributed File System) and Aster Data’s MPP data warehouse.

In addition to the Big Data Summit we held in New York City (which we’ll detail in a separate post), Colin White presented on a Webcast with Aster on the various use-cases for Hadoop within data warehouse environments. Colin does a great job summarizing what Hadoop is, how it’s different from an RDBMS, the different types of users for each, and how they co-exist nicely in customer environments.

Below are the slides to view if you weren’t able to attend the event, which will also be available for on-demand viewing soon in our resource library.

I’m very excited about the upcoming Big Data Summit in New York City on Thursday evening (October 1st). Sponsored by Aster Data, Microstrategy, and Informatica, we have an incredible speaker lineup including LinkedIn, comScore, and Colin White from BI Research. Check out the Facebook page for the event here.

To kick off the festivities, we’re holding a live webinar earlier in the day at 9 a.m. US Pacific time. Colin White and myself will be discussing Hadoop and data warehousing - how they’re similar, how they’re different, and what they can be used for (both separately and together). In fact, we’ll be making an important announcement of a new product offering that you won’t want to miss. If you haven’t already, I urge you to register by clicking here.

Mark it on your calendar - something “big” is coming on October 1st.