17
Oct
   

“Big data” has always been a favorite subject of discussion among the Aster Data team. We’ve been talking about big data at least since 2009, long before the term became burning-hot. The big data hype has confused many organization (and vendors) in the market about the best technology or method to solve their analytical business problems.

However, our vision hasn’t changed: from the time we founded the company in 2005 to today where we are part of the Teradata family. Teradata Aster continues to lead the market with technology innovations and reference architectures which provide clear guidance and deliver significant business value to our customers

Today, we are pushing the limits of analytical technology once more, by launching the Teradata Aster Big Analytics Appliance. The Big Analytics Appliance is a unique machine that can help enterprises see their business in high-definition. By harnessing all existing and new data types in the enterprise, we enable organizations to leverage our powerful SQL-MapReduce framework and business-ready analytics & apps which solve specifics business problems in marketing attribution, fraud detection, graph analysis, pattern analysis, and much more. It unleashes the creativity of bright analysts to go discover new insights to help their organizations grow revenue and create sustainable competitive advantage.

So what is the Big Analytics Appliance? It’s five things in one box:

  1. Aster + Apache Hadoop (100% open source via the Hortonworks HDP distribution), fully integrated in one box
  2. ANSI-standard SQL and next-generation MapReduce, fully integrated
  3. More than 50 ready-to-use MapReduce  apps, to deliver immediate business value
  4. Full ecosystem connectivity for both Aster and Hadoop; with BI, ETL and other existing IT systems
  5. The latest-generation, most efficient hardware platform, specifically optimized for Aster, Hadoop, and Big Analytics

Loyal to our Stanford roots, the appliance comes in Cardinal-red color!

Teradata Aster Big Analytics Appliance

The Big Analytics Appliance packs a long list of essential and unique technologies, including:

  • SQL-MapReduce®,  industry’s only true SQL/MapReduce integration
  • SQL-H™, industry’s only ANSI-standard SQL and Hadoop integration
  • Teradata Viewpoint, the most advanced database monitoring platform now extended to Aster and Hadoop
  • Teradata TVI a very sophisticated hardware support and failure prevention software, now ported to Hadoop as well as to Aster
  • Infiniband network interconnect - makes ultra-high-performance connectivity between Aster and Hadoop, as well as scalability, a non-issue
  • Small factor disk drives and dense enclosures - make this appliance one of the most dense and space-efficient big data platforms in the market

And, of course, everything in this appliance is packaged, integrated, pre-tested and supported by Teradata - the most trusted brand in data management and analytics.

I also want to take a moment to talk about our Unified Data Architecture vision for the enterprise. When most vendors out there talk about big data at a very high level without explaining where it fits and how it relates with traditional technologies like data warehousing, we decided to do the hard work of figuring out how different technologies complement each other and for what purpose. The result of that was the diagram below that showcases how Teradata, Aster & Hadoop can work together in tandem to provide a complete data solution for enterprise environments:

Teradata Unified Data Architecture

We also went one step further and now have a matrix that explains what technology (or technologies) are more appropriate for what use case - given a workload/use case and a specific type of data. The result of that exercise is below:

Processing as a Function of Schema Requirements by Data Type

When To Use Which Technology? The best approach by workload and data type

If you want to know more about our Unified Data Architecture vision, read the whitepaper we co-authored with Hortonworks, or feel free to contact us and we’ll be happy to discuss with you this concept and how it’d fit into your environment.

Through tightly integrating Aster and Hadoop, the new Big Analytics Appliance addresses a large part of the Unified Data Architecture; and via the Teradata-Aster and Teradata-Hadoop connectors, Teradata now has all the necessary pieces to help enterprises extract the maximum business value from all their data and execute on their Big Data vision. At Aster, just like at Teradata, we are committed to continuously provide the best innovations to help our customers have the power to make the best decision possible.

P.S. If you want to try out Aster without ordering a full Aster box, we now allow you to download an Aster virtual appliance! Go give it a try: http://www.asterdata.com/AsterExpress



12
Jun
   

Back in 2005, when we first founded Aster Data, our vision was to take some of the latest technology innovations – including MPP shared-nothing architectures; Linux-based commodity hardware; and novel analytical interfaces like Google’s MapReduce – and bring them to mainstream enterprises. This vision translated into a strategy focused not only on big data innovations, but also on delivering technologies that make big data viable for enterprise environments. SQL-MapReduce®, our industry-leading patented technology that combines standard SQL processing with a native MapReduce execution environment, is one example of how we make big data enterprise ready.

Today we have completed another major milestone on providing value to our customers by announcing a major innovation: Aster SQL-H™, a seamless way to execute SQL & SQL-MapReduce on Apache™ Hadoop™ data.

This is a significant step forward from what was state-of-the-art until yesterday. What was missing? A common DBMS-Hadoop connector operating at the physical layer. This means that getting data from Hadoop to a database required a Hadoop expert in the middle to do the data cleansing and the data type translation. If the data was not 100% clean (which is the case in most circumstances) a developer was needed to get it to a consistent, proper form. Besides wasting the valuable time of that expert, this process meant that business analysts couldn’t directly access and analyze data in Hadoop clusters. Other database connectors require duplicating the data into HDFS by using proprietary formats; a cumbersome and expensive approach by any measure.

SQL-H, an industry-first, solves all those problems.

First, we have integrated Aster’s metadata engine with Hadoop’s emerging metadata standard, HCatalog. This means that data stored in Hadoop using Pig, Hive & HBase can be “seen” in an Aster system as if they are just another Aster view. The business implication is that a business analyst using standard SQL or a BI tool can have full and seamless access to Hadoop data through the Aster’s standard ODBC/JDBC connector and Aster’s SQL engine. There is no need to have a human in the middle to translate the data or ensure its consistency; and no need to file tickets or call up experts to get the data the business needs. Everything happens transparently, seamlessly, and instantly. This is an industry first, since today all available Hadoop tools either do not provide standard SQL interfaces that are well optimized, do not provide native BI compatibility, or require manual data translation and movement from Hadoop to a third party system. None of these approaches are viable options for SQL & BI execution on Hadoop data, thus making it hard for enterprises to get value from Hadoop.

Secondly, SQL-H provides a high-performance, type-safe data connector, that can take a SQL or SQL-MapReduce query that involves Hadoop data, automatically select the minimum subset of data in Hadoop that is required for execution of the query, and run the query on the Aster system. The performance of running SQL and SQL-MapReduce analytics in Aster is significantly higher than Hadoop because (a) Aster can optimize data partitioning and distribution, thus reducing network transfers and overhead; (b) Aster’s engine can keep statistics about the data and use that to optimize execution of both SQL & MapReduce; (c) Aster’s SQL queries are cost-based-optimized which means that it can handle very complex SQL, including SQL produced by BI tools, very efficiently.

In addition, one can take advantage of SQL-H to apply the 50+ pre-build SQL-MapReduce apps that Teradata Aster provides on Hadoop data, thus doing big data analytics that are impossible to do in every other database without having to write a single line of Java MapReduce code! These apps include functions for path & pattern analysis, statistics, graph, text analysis, and more.

Teradata Aster is committed to groundbreaking product innovation as the key strategy in maintaining our #1 position in the big analytics market. SQL-H is another important step that we expect will make Hadoop and big data analytics much more palatable for enterprise environments, allowing business analysts, SQL power-users & BI tool users to analyze Hadoop data without having to learn about Hadoop interfaces and code.

If you want to find out more we’ll be talking about SQL-H at Hadoop Summit, on webcast taking place June 21st, at the upcoming Big Analytics 2012 events in Chicago & New York, and at the annual Teradata Partners event.



26
Apr
By Paul Barrett in Digital Marketing, Social Media, Teradata Aster on April 26, 2012
   

What questions are important to you about Social Media?

John Lovett from Web Analytics Demystified just published a new white paper on Social Analytics. Lovett, who has written the book on Social Analytics (literally), lays out a compelling vision for Deeper Social Analytics for companies.  He clearly presents the value of companies to go beyond surface level analytics of likes, followers and friends and challenges the CMO to ask deeper and more important questions.

I love the three key questions presented in the paper that really hit the c-suite.  These are

  • What is the Audience?
  • What is the Activity?
  • What is the Action?

These 3 questions provide a framework to share social media initiatives with business leaders and strip away all the non-business related questions that become so distracting in understanding the impact of social media on the enterprise.

Although we are still in the early, Black and White TV stage of social analytics, Teradata Aster has been heavily influenced by our customers’ needs in the social space.  Customers like LinkedIn,  Gilt Groupe, MySpace, and Myzinga have redefined how consumers interact with each other through music, shopping, and content.  Attensity running on Aster promises to bring together big data and social analytics that starts to deliver on Mr. Lovett’s proposition of deeper social analytics.

Teradata’s strategy to marry big data analytics and marketing applications with its industry-leading database solutions is steeped with the concept of deeper analytics.  In social analytics, we have identified 10 key business questions that should be asked about every social post.   In a market where posts can go viral, impact brand, customer perception, and revenue, being able to quickly and effectively navigate deeper social analytics becomes a mission critical capability.

Beyond John’s questions, the 10 key questions are:

  • What was said?
  • What is it about? (ie,  product, service, brand, experience)
  • Is it a common sentiment?
  • What are the trends on this topic?
  • Who said it?
  • What is their value?
  • How engaged are they?
  • What is their influence?
  • How do I respond?
  • Was my response effective?

To effectively answer these questions CMO’s need a set of marketing technologies.  These include:

  • A social listening platform that analyzes social timelines for owned, embassy and public feeds.  These tools identify what was said, what it is about and if it is a common sentiment.
  • The second level demands a customer hub to understand who posting and what the customer relationship is with them and to measure the customer value of that relationship.
  • The third level requires social network analytics and the ability to find implicit and explicit social connections.  This helps illuminate how engaged customers are and their level of influence – or who influences them
  • The fourth level is where integrated marketing management and customer facing marketing applications come it.  Once you understand what was said, who said it and the potential impact – how do you respond?  Is it a one-on-one conversation, a social discussion or was a bigger issue identified that may result in marketing campaigns?

Are these the right business questions to ask?  What else do you want to know about social media posts in your business?



13
Apr
By Mayank Bawa in Analytics, Business analytics, Teradata Aster on April 13, 2012
   

We live in interesting times!

In the past 30 years, data was used to record business events and report on business events. Over the last 5 years, data has gotten closer to business. Now data is being used to record business events, report on business events as well as influence business events. We now realize that the more data we record, the more comprehensively data can influence business events.

Hence the excitement of “big data” - it is a business opportunity for each line of business - to influence business events to have favorable outcomes.

The responsibility for technologists is to provide the right platforms and tools to make influencing business easy and simple.

There are TWO relentless forces that are playing out in the big data space to which technology has to respond.

The first force is the diversity of data. As we record more data, we end up having different formats of data to manage. About 20% is relational, but we also have text, emails, PDF, Twitter feeds, Facebook profiles, social graphs, CDRs, Apache logs, JSON formats, …

The second force is the richness of analytics. As we influence more business, we end up having richer analytics to perform. About 20% is SQL, but we also have time series analysis, statistical analysis, geo-spatial analysis, graph analysis, sentiment analysis, entity extraction, …

Note that I am not saying MapReduce doesn’t have a diverse set of analytics to do: MapReduce is a way of programming to do analysis - time series, statistical, geo-spatial - each require different MapReduce programs to be written.

Today, the platforms and tools for big data are very complex. They expect lines of business owners to write programs to manage different forms of big data, to write sophisticated programs to analyze big data, to master the management and administration of big clusters and be self-sustaining in managing data quality. This last point is very important - data values change over time. We have to keep values consistent, otherwise our analysis will be wrong and our influence on business will be negative - garbage in, garbage out rule of computing.

As a result, big data is in danger of entering the DIY (do it yourself) space. A line of business is now expected to support big clusters = big administration = big programs = big friction = low influence.

We have to acknowledge these challenges as technologists. If we let big data solutions be a DIY solution, only pockets of enterprise will embrace big data - the rest of the non-technology savvy business leaders will be left out of the opportunity.

We have to simplify this equation. We need to enable line of business owners to benefit from big data a lot more easily. We have to make it simpler for business leaders to get from big data to big analytics.

Our goal, big data = small clusters = easy administration = big analytics = big influence.

This entails solving the following problems:

[1] Make platform and tools to be easier to use to manage and curate data. Otherwise, garbage in = garbage out, and you will get garbage analytics.

[2] Provide rich analytics functions out of the box. Each line of programming cuts your reachable audience by 50%.

[3] Provide tools to update or delete data. Otherwise, data consistency will drift away from truth as history accumulates.

[4] Provide applications to leverage data and find answers relevant to business. Otherwise the cost of DIY applications is too high to influence business - and won’t be done.

At Teradata Aster, we are continuing to lead the big data revolution. We have led the revolution for the past 5 years, and helped shape the market and technologies. We are convinced that the path to big data success is to connect it with Big Analytics in the coming 5 years.



21
Mar
   

The conversation around “big data” has been evolving beyond a technology discussion to focus on analytics and applications to the business.  As such, we’ve worked with our partners and customers to expand the scope of the Big Data Summit events we initiated back in 2009 and have created Big Analytics 2012 - a new series of roadshow events kicking off in San Francisco on April 19, 2012 .

According to previous attendees and market surveys, the greatest big data application opportunities in businesses are:

- Digital marketing applications such as multi-channel analytics and testing to better understand and engage your customers

- Using data science and analytics to explore and develop new markets or data-driven services

Companies like LinkedIn, Edmodo, eBay,  and others have effectively applied data science and analytics to take advantage of the new economics of data. And they are ready to share details of what they have learned along the way.

Big Analytics 2012 is a half-day event, is absolutely free to attend, and will include insight from industry insiders in two different tracks: Digital Marketing Optimization, and Data Science and Analytics. Big Analytics 2012 is a great way to meet and hear from your peers such as: executives who want to learn more about leveraging advanced analytics to a competitive advantage, interactive marketing innovators who want access to “game changing” insights for digital marketing optimization, enterprise architects and business intelligence professionals looking to provide big data infrastructure and data scientists and business analysts who are responsible for developing new data-driven products or business insights.

Come to learn from the panel of experts and stay for an evening networking reception that will put you in touch with big data and analytics professionals from throughout the industry. Big Analytics 2012 will be coming soon to a city near you. Click here to learn more about the event and to register now.

 



19
Mar
By Tasso Argyros in Analytics, Business analytics, Interactive marketing, Teradata Aster on March 19, 2012
   

Tomorrow, I will have the pleasure of presenting “Radical Loyalty - Data Science Applied to Marketing” at the GigaOm Structure:Data event with Marc Parrish, the VP of Membership and Customer Retention Marketing at Barnes & Noble. In contrast with most talks at this event, Marc and I will be focusing on the business opportunities of Big Data and specifically on marketing loyalty programs and how they relate to Big Data analytics.

The concept of a loyalty program is certainly nothing new. Brick and mortar companies have been leveraging customer loyalty in a variety of unique ways for decades. What’s different is the ability of businesses to use new types of data to take their customer loyalty insights and strategies to a completely new level. At tomorrow’s conference, we will explore ways in which modern retailers like Barnes & Noble with a strong digital marketing strategy leverage their customers’ loyalty using Big Data and how to make loyalty programs worthwhile for customers and their needs.

Barnes & Noble has proven an ability to innovate their business model by leveraging data. I look forward to sharing some insight with Marc on retail and other real world applications of Big Data.



15
Mar
By Steve Wooledge in Analytics, MapReduce, Teradata Aster on March 15, 2012
   

Yesterday I presented at the Los Angeles Teradata User Group on the topic of “Data Science: Finding Patterns in Your Data More Quickly & Easily with MapReduce”. One point discussed was the common misnomer that big data is about volume, which is certainly part of the issue organizations are facing. However, the big story in big data is the complexity and additional processing required to make “unstructured” data actionable through analytics. This is where procedural frameworks like MapReduce can help. Here is a great post by Teradata’s own Bill Franks about unstructured data which helps describe the requirements unstructured data demands in the context of analytics.

As Franks notes, “the thought of using unstructured data really shouldn’t intimidate people as much as it often does.” Read more to learn why.

 



28
Feb
By Stephanie in Interactive marketing on February 28, 2012
   

On a recent webinar, Rob Bronson from Forrester Research pointed out that 45% of Big Data implementations are in marketing.  One of the use cases we most hear about for customers is the need to move from single-touch attribution methods like last-click and first-click to multi-channel, multi-touch attribution.  Today we announced an extension of our Digital Marketing Solutions to deliver multi-touch attribution. 

When I speak with customers about moving to multi-touch attribution it feels like hearing about HDTV for the first time.   More clarity, more detail, and a richer experience that is more like the real-life experience of consumers.  So, multi-touch attribution is basically the HD equivalent of single-touch attribution.

What’s different?  First of all, consumers interact across many touch-points, social, mobile, search, websites as well as offline channels.  Most existing attribution solutions look at multiple touch-points within a single channel, like an ad network or web visitors.  With a Big Data Analytics approach it is easier to blend more channels into the mix and find customer connections.

This is critical today, because it better reflects the customer journey.   To be customer-centric, it is critical to be able to look at your brand through the eyes of the consumer.  A few years ago, this was impossible or at least difficult and expensive.  Now Big Data marketing analytics makes it possible to see the multi-channel journeys with incredible clarity.

As consumers dynamically adopt new technologies, keeping up with them is one of today’s marketers biggest challenges.  To do that, you can’t be stuck in legacy single-touch or annual reviews of attribution.  Big Data Analytics makes it possible to discover new patterns, test new programs and iterate to optimize in the time scales that the market demands.

An additional value is that Big Data Analytics can deliver a 3D-type enhancement to attribution.  Teradata Aster gives you the ability to use different measures for each touch point so you can use uniform, variable or exponential weightings in your model in order to test and iterate to get the right approach for your business.

Another big difference using Teradata Aster to analyze attribution is to be able to link to additional data in a Teradata Data Warehouse to include Revenue, Profit and Lifetime Value which extends attribution beyond conversion to real bottom-line performance.

Lastly, the ability to integrate into the Aprimo marketing platform makes this insight actionable.   With Aster and Aprimo being part of Teradata, it becomes possible to operationalize your Big Data Analytics more effectively.

The infographic above highlights why some marketers might feel like they have an attribution problem.  You can download a PDF of it here. On the same page, you will also find a white paper we created with Aprimo to go into more detail around what attribution looks like today, and an On-Demand webinar with Forrester and Razorfish that looks at attribution in some depth.  For those who want to read more, check out an addition to this Delicious stack.

So my question for this post is – Do you have an attribution problem?  And if so, how can having multi-touch, multi-channel attribution model make it better?



21
Feb
By Tasso Argyros in Analytic platform, Analytics, Analytics tech, Database, MapReduce on February 21, 2012
   

It has been about seven years since Aster Data was founded, four years since our industry-first Enteprise SQL-MapReduce implementation (first commercial MapReduce offering) and three years since our first Big Data Summit event (the first “Big Data” event in the industry as far as I know). During this whole time, we have witnessed our technology investments take off together with the Big Data market - just think how many people had never even heard the word MapReduce three years ago, and how many swear by it today!

As someone who was caught in the Big Data wave since 2005, I can tell you that the stage of the market has changed significantly during this time - and with it, the challenges that Enterprise customers face. A few years ago, customers were realizing the challenges that piles of new types of data were bringing - big volumes (terabytes to petabytes) and new, complex types (multi-structured data such as weblogs, text, customer interaction data); but at the same time, the opportunities that the new analytical interfaces, like MapReduce, were enabling. Fast forward to today and most enterprises are trying to put together their Big Data strategies and make sense of what the market has to offer - and as a result there is a lot of market noise and confusion: it is usually not clear what use cases apply to traditional technologies versus new; how to reconcile existing technologies with new investments; and what type of projects will they give them highest ROI versus a long and painful failure.

Teradata and Teradata Aster have a high interest in customers being successful with Big Data challenges and technologies, because we believe that the growth of the market will translate into growth for us. Given Teradata’s history in being the #1 strategic advisor to customers around data management and analytics, we only want to offer the best solutions to our customers. This includes our products -which are recognized by Gartner as leading technologies in Data Warehousing and Big Data analytics- but also our expertise helping customers how to use complementary solutions, like Hadoop, and making sure that the total solution works reliably and succeeds in tackling big business problems.

With this partnership, we are taking one more step towards this direction. So we are announcing three things:

1. Teradata and Hortonworks will work together to jointly solve big challenges for our customers. This is a win/win for customers and the industry.

2. Our intent to do joint R&D to make it easier for customers that use products from Teradata and Hadoop to utilize these products together. This is important because every enterprise will look to combine new technologies with existing investments, and there is plenty of opportunity to do better.

3. A set of reference architectures that combine Teradata and Hadoop products to accelerate the implementation of Big Data Big Data projects. We hope that this will be a starting point that will save enterprises time and money when they embark on Big Data projects.

We believe that all the above three points will translate into eliminating risks and unnecessary trial and error. We have enough collective experience to guide customers to avoid failed projects and traps. And by helping clear up some of the confusion in the big data market, we hope to accelerate its growth and the benefit to Enterprises that are looking to utilizing their data to become more competitive and efficient.



29
Sep
By Tasso Argyros in Analytic platform, Analytics, MapReduce on September 29, 2011
   

One of the great things about starting your own company (if you’re lucky and your company does well) is that you take part in the evolution of a whole new market, from its nascent days to its heyday. This was the case with Aster and the “Big Data” market. Back when we started Aster, in 2005, MPP systems that could store and analyze data using off-the-self servers was still a pretty new concept. I also recall in 2008, when we first came out with our native in-database MapReduce support — and our SQL-MapReduce® technology — we had to explain to most people what MapReduce even was. In 2009, we came out with the first Big Data event series — “Big Data Summit” — because we knew we were doing something new and wanted a term to describe it. “Big Data” caught on more than we had imagined back then, and the rest is history. Product innovation was at the core of Aster’s existence, and we kept pushing ourselves and our product to become the best platform for enterprise-class data analytics using both SQL and MapReduce as first class citizens on one analytic platform.

Today there is a lot of innovation in the big data market. However, we see a “chasm” between the SQL technologies—which are very enterprise-friendly—and the new wave of open source big data or “NoSQL” software which is used extensively by engineering organizations. In the middle is a very large number of enterprises trying to understand how they can use these new technologies to push their analytical capabilities beyond purely SQL, while at the same time utilizing their existing investments in technologies and people. This is the problem that Aster solves.

With last week’s announcement, the launch of our Teradata Aster MapReduce solutions which include Aster Database 5.0 software (formerly Aster nCluster) and our new Aster MapReduce Appliance, we bring to market the best answer for the organizations that are “caught in the middle.” Unlike SQL-only systems focused primarily on analyzing structured data, our database and appliance provide support for native MapReduce which enables a new generation of analytics, such as digital marketing optimization, social graph analysis, fraud detection based on customer behavior, etc. Our newly extended libraries of pre-built MapReduce analytical functions allows such applications to be developed with significantly less time and cost versus other MapReduce technologies. And, unlike other MapReduce-based systems, we offer full SQL support, integration with all major BI and ETL vendors and a data adaptor to EDW systems that allows enterprises to utilize existing tools and skills to bring big data analytics to their businesses. Finally, with our new appliance, we leverage Teradata’s strength and engineering to provide a proven and performance-optimized system for businesses to start analyzing untapped diverse data while cutting down on time, cost and frustration!

As we move forward, Aster is committed to being the leader in SQL and MapReduce analytics for multi-structured data. Having spent 6 years in this market, we believe that it’s not just the coolest technologies that will win, but the ones that make it easier for business analysts and data scientists within organizations to solve their business problems and innovate with analytics. With the launch of our new Teradata Aster solutions — including the revamped SQL-MapReduce interfaces and the new Aster MapReduce appliance—we are pushing the state of the art towards this direction (or as my marketing team likes to say – “bringing the science of data to the art of business”). :)