From Big Data to Real-Time Information
By Steve Shine, Co-President, Ingres
The rise of Big Data and its implications to database technology
Emerging business analytic needs are exposing limitations in traditional database technologies and are fuelling the growth of highly specialised analytics platforms. This article presents an overview of the database market and the latest technology breakthroughs.
The Data Explosion
Over the past few years, an increasing number of business processes such as sales, customer services and logistics, have become available online, along with the data they accumulate. As these individual databases are becoming connected, we have seen that every part of the company is now linked to the data centre and every action, be it a sales transaction or a mouse click, is stored in the database. The result is that organisations are being flooded by data which pose complex management issues.
A recent study by analyst house IDC¹ has shown that the “Digital Universe” grew by 62% in 2009 to nearly 800,000 petabytes (a petabyte is a million gigabytes) of data. And in 2010, the Digital Universe would grow to 1.2 million petabytes, or 1.2 zettabytes. This explosive growth means that by 2020, our Digital Universe will be 44 times as large as it was in 2009. Obviously, such vast data quantities pose real challenges to companies and their CIOs who want to provision for easy and quick access to information, seamless data management as well as accommodate future data growth. In addition, compliance with regulations adds to the complexity of the task by rules related to data security and information privacy.
From Big Data to Specialised Business Analytics
Through data integration, analytics and reporting, an organisation can fully utilise all of the information coming into the business to uncover unique operational insights and gain competitive advantage. Emerging analytic needs are fuelling the growth of highly specialised platforms. A variety of niche business analytics tools are available in the market which enable smarter and more strategic business decisions to optimise operational performance and increase profitability.
Database capacity, processor speeds and software enhancements will continue to drive even more sophisticated analytical applications.. However, the explosion of data is exposing limitations in traditional database management technologies. We have seen analytical and database environments converging and this trend is likely to accelerate in the future across all industry segments, whereas today it is most prominent in health care, government and retail. For example, in-database predictive analytics helps marketers predict consumer behaviour.
In-memory vs. Vectorised database
Industries are beginning to rely on real-time analysis to gain competitive advantage. For example, financial services organisations require information to be accessible at the speed of thought. In this instance, disk storage may be too slow and many are hailing in-memory database technologies as the way forward.
The main value proposition of an in-memory database is that it enables great improvements in performance and greater speed of retrieving information because it runs directly in the computer’s main memory, which is suitable for high volume transaction systems with vast amounts of data, like e-commerce. In-memory technology has a downside; it is much more expensive than disk storage. Is there an alternative to in-memory databases which would support real-time analytics in the age of Big Data? I believe there is, in the form of Vectorised database.
VectorWise enables businesses to keep pace with the explosive growth of business data and to gain competitive advantage by embedding analytic capabilities in their real-time operations and improving management of information systems. VectorWise business analytics help prevent businesses falling back on inadequate or summarised data sets due to existing database performance limitations. Instead, as they do not have to summarise the original data and being on-line they can drill down into the data to the detail they need to make the best decisions possible. VectorWise unlocks the full processing power of today’s sophisticated chip technology, meaning businesses can gain a significant competitive edge through the performance gains. For example, VectorWise can correlate customer data from multiple sources including the vast volumes of data from web and mobile interactions. It also enables compliance with the complex new regulatory demands for deep analysis of financial and product data and it does so at a reasonable cost.
Ingres customers have already experienced performance increases of up to 70 times compared to other databases. For example, The Rohatyn Group, a New York-based asset management company focused on global emerging market investments, had been searching for a database that would fulfill its most intense data processing needs. With VectorWise, the company can now process millions and millions of rows of data with seemingly infinite width and depth and without the need for new expensive hardware or complicated schemas. The VectorWise Database removes former data analysis obstacles and makes a rapid impact on business decisions. Ultimately, this reduces cost and risk as well as accelerates the success of business intelligence projects.
The unprecedented growth in data volumes is having a significant effect on many organisations. As databases increase in size, so too do the costs and complexities associated with data management, business analytics and business intelligence. It is of utmost importance for organisations to be able to extract information critical to their business effectively and, increasingly important, in real-time, to be able to retain competitive advantage.
Specialised business analytics and in-memory databases are responding well to the growing demand for fast data analytics however it is software developments which leverage advancements in modern chip technology that will ultimately lead the way forward in database technology. The VectorWise database is the biggest technology leap with proven immense performance increases² over the competition which shortens the time of business intelligence projects. The reduced costs and increase in overall efficiency enable organisations to make quick and informed business decisions and as a result maintain their competitive advantage.
About the Author
Steve comes to Ingres from Sybase where he was senior vice president and general manager for EMEA, overseeing all operational, sales, financial and human resources in the region for the past three years. While at Sybase, he achieved more than 200 million in revenue and managed 500 employees, charting over 50 percent growth in the Business Intelligence market for Sybase. Prior to Sybase, Steve was at Canadian-based Geac Computer Corporation for ten successful years, helping to successfully turn around two major global divisions for the ERP firm.