Scaling

Typical big data analytics scaling issues:

• Too expensive and complex to scale analytic processing
• Too much planning and administration required to scale

Causes:

• Inflexible systems
  • Advanced planning required to move data correctly
  • Complexity: repartition, re-indexing, constraints, etc.
• “Super-size” upgrades
  • Existing systems limited by hardware constraints
  • Proprietary and expensive hardware pre-configured in cabinets and “rolled in” (at huge cost to you)
  • Huge costs in people resources/time to migrate data over, set up the new configurations, and hope everything works
  • Repeat as data volumes grow
• Downtime required
  • Upgrades are complex and require hours, if not days to complete
  • Extra data loading time required to compensate for downtime
• Error prone upgrades
  • Complexity leads to user error or system glitches
  • Improper load balancing and poor performance typical as kinks are worked out

Remedies

• Always-parallel performance and scalability
  • Online incremental scalability
  • Heterogeneous commodity scale-out
  • Scalable online back-up and restoration
• Always-on availability and resiliency
  • Live system administration
  • One-click scaling
  • Online fault-tolerance and recovery