Clearly there exists the potential to take what I will call “operational” optimization and bring it to the next level. I mean relevancy is the name of the game in regards to that bag baby.

You seem to be indicating and almost professing the value of behavioral networks. The challenge will come in determining not only how to regress behavior in order to deliver more relevant content but more so in one’s ability to track sequential behavior to try to guess what’s going to be of interest next. I believe one of your existing clients could speak on this topic far more than i could though

-jd

Well there are the latency and cost factors which Aster seems to have a new solution for. However we may optimize techonology, there is still a tipping point of sorts. It is fair to say the above information is used today for internal decision making processes to expand share or increase profit. This is done inside the firewall. Question is, when does it become more profitable for companies to externalize these analysis to drive the customer experience?

Thanks – dm

(1) Control: The key here is to separate strategy from operational tactics. For example, etailers can have a strategy to give $5 off to customers who have discarded their shopping cart, either via email or after inactivity. The operational piece is to iterate upon different messaging, mediums of contact (on-site, email), threshold when this offer is extended. So Control lives with the Intelligent Expert, Operations lives in nimble processes.

(2) Cost: It doesn’t cost much to set up a feedback loop. It is more of a way of realizing its power, and taking steps towards it. I’d not be surprised to learn that Google was getting feedback into its ranking when it was a small company. Yep, it was a small company once without the billions it makes now.

Yes, ideally companies could automate this process. But I think there are a few significant barriers to this type of approach:

(1) Institutional – Managers and firms generally desire control (which is not per se negative). Take the example of product price testing. While we could rely on analytics to drive the pricing levels we show to users (based on demand, other factors) we instead take the analysis offline. In this case, there are qualitative factors that influence our decisions, i.e. overall product strategy, future bundling opportunities, etc. Oh, and distrust in automated technologies that directly effect the top-line.

(2) Cost – These types of systems are (a) not easy to architecture and (b) not easy to implement. Sure, companies like Google, et al w/ vast resources can see a positive ROI. But most startups (and even established companies) that focus on a vertical or do not have significant scale cannot allocate resources to a project with nebulous returns.

You make the point that the manual/human process is expensive but do you have data to support that? e.g. The cost of implementing an automated system vs. hiring someone with domain knowledge?

I work in the data group at a startup so I deal w/ these issues on a day-to-day process. I’d be interested to hear more of your thoughts.

I’m currently in the ETL field, so I’m familiar with the technology you’re discussing… I’ll definitely be paying attention. Congrats on the MySpace implementation – 100TB is a great accomplishment.