How to make historical analysis work
on real-time data

‘As of’ root cause analysis at any point in the pastHistorical data analysis is typically enabled using data duplication technologies. But is this method still valid today when users need to analyze historical data that’s moving fast and changing rapidly throughout the day? All we know is that in ActivePivot, we practically had to re-invent our core database to support the requirements of our customers who wanted to travel back in time and analyze large volumes of dynamic data.
Continue reading

Posted in Big Data Analytics, Technology | Tagged , , | Comments Off

From Multi-Core to Many-Core: Do not delay in Making Your Java Application “NUMA-Aware”

NUMA Aware ApplicationIn the last post, I explained the difference between SMP and NUMA architectures as we enter the “many-core” era. I also asked the following question: “Is it reasonable to expect massive performance improvements when you run an existing application on new NUMA-enabled hardware?” The answer is yes. However, improved performance is not guaranteed and you must be prepared to rewrite the code of your application to get the best out of many-core hardware. Continue reading

Posted in Big Data Analytics, Technology | Tagged , , | Leave a comment

To NUMA or not to NUMA

Symmetric Multi-Processing (SMP)When your business is analyzing big data with the goal of providing answers in split seconds, you find yourself trying to squeeze every bit of speed into your solution. Among other things, this also includes finding the optimal processor architecture. This is why we’ve spent quite a lot of efforts studying the memory architecture alternatives – NUMA (non-uniform memory access) and SMP (symmetric multiprocessing) – to see which one could provide us with the best results.

Continue reading

Posted in Big Data Analytics, Technology | Tagged , , , | Leave a comment

In-memory analytics: doing things radically different

Think differentIn our previous post, How In-Memory Computing is Accelerating Business Performance, we explained the disruptive potential of in-memory computing. Performance gains resulting from faster execution of queries were one of the top benefits mentioned. However, in-memory computing goes way beyond performance gains, allowing organizations to do things differently and achieve new levels of competitiveness. This post illustrates this with a few examples.

Continue reading

Posted in Big Data Analytics, e-Commmerce, Supply Chain, Technology | Tagged | Leave a comment

From 24 hours to 24 seconds – How In-Memory Computing is Accelerating Business Performance

simulationsCountless articles are written about Big Data every day. Beyond the hype, the Big Data phenomenon is a real change agent delivering capabilities that were never thought of before. Financial institutes and banks, for example, can calculate and asses their risk in near-real time, throughout the day.

To a large degree, the phenomenal performance and interactive analysis capabilities of Big Data projects are enabled by in memory computing. In-memory databases have become the foundation of a new generation of business applications that bring the power of analytics to the hands of decision makers. Continue reading

Posted in Big Data Analytics, Finance, Technology | Tagged , , | Leave a comment

Dynamic Data Bucketing: A Use Case for Complex Aggregation

Fast & Dynamic Data BucketingComplex aggregation has become a common requirement for business users looking to analyze sophisticated metrics across multiple dimensions. There are numerous use cases for complex aggregation such as cross-currency aggregation, which was explored in our last post. Dynamic bucketing is another use case example.

This blog takes a deep look at the technical considerations for a successful implementation of dynamic bucketing.

Continue reading

Posted in Big Data Analytics, Technology | Leave a comment

Real-Time Cross-Currency Analytics: A Use Case for Complex Aggregation

SQL vs Post Processors: Currency Aggregation ExampleIn previous posts, we’ve delved into the principles of multidimensional databases. Among all the benefits that a multidimensional database delivers is complex aggregation, a process by which KPIs are written once and are immediately available across any dimensions, through any filtering, letting the user follow his train of thought.

But how does complex aggregation actually work? This post explores a concrete use case, articulates the technology challenges behind complex aggregation and demonstrates why ETL and SQL relational databases are not a fit.

Continue reading

Posted in Big Data Analytics, Technology | Tagged , , , | Leave a comment

Why Complex Event Processing (CEP) is not enough for real-time decision making

Real-time decision making is commonly linked with Complex Event Processing (CEP). Indeed, CEP systems can extract and alert about meaningful events from streams of data. However, for decision makers to have context and turn a notification into a meaningful, actionable event, CEP must be supplemented with mixed workload and multidimensional capabilities.

Let’s take a look at what it means.

Continue reading

Posted in Technology | Tagged , , , , , | Leave a comment

Mixed Workload databases: A Primer

In a previous post comparing multidimensional and relational databases we mentioned that the decision making imperatives in the Big Data era were disrupting the clear-cut border between OLTP and OLAP, enabling a new type of mixed workload database that addresses both needs.

This post takes a closer look at mixed workload systems – what they are, how they work, and what are they useful for. Continue reading

Posted in Technology | Leave a comment

Multidimensional or relational? What’s the right system for you?

Relational and multidimensional databases differ on almost any possible dimension:  tables, columns and rows vs. cubes, measures and dimensions; queries across joint tables vs. pre-calculated aggregations across dimensions; Structured Query Language (SQL) vs. Multidimensional Expressions MDX.  And the list of goes on and on.

Nevertheless, talk to a vendor from any of these two camps, and he will argue that the system can successfully perform any task. Cross-dimensional analytics, for instance, can be performed using both types of systems. Continue reading

Posted in Technology | Leave a comment