Data warehousing technology began as a framework to better manage, understand, and capitalize on data generated by the business, and it worked extremely well for many years. This is a space that Microsoft knows well with warehousing capabilities since 1995 with SQL Server.
However, there are several forces working to stretch the traditional data warehouse. Data volume is expanding tenfold every five years. Even the most robust SMP warehouse will require costly forklift operations to a larger and more expensive hardware footprint to keep up with the growth. Companies are using real-time data to optimize their businesses as well as to engage in dynamic, event-driven processes. The variety of new data types is proliferating with over 85 percent of new data coming from non-relational data such as logs, mobile, social, RFID, and devices.
Modernizing your data warehouse with new technologies can help you meet the needs of today’s enterprise to connect all volumes of any data with agile and familiar BI to business decision makers. This was validated by The Data Warehousing Institute (TDWI) who published a checklist to enable the modern data warehouse. At a high level, your new warehouse must be able to handle:
- Data of All Volumes: The modern data warehouse can scale up to any volume of data starting from terabytes up to multi-petabyte scale on all data types – relational and non-relational. As an example, Virginia Tech was able to crunch data from DNA sequencers (growing over 15 petabytes a year) to do cancer research.
- Real-Time Performance: The ability to work with data in real time to keep up the pace of increasing demands without losing performance. As an example, MEC was able to bring queries of customer online visitation metrics from four hours down to minutes.
- Any Data Types: The ability to seamlessly integrate over any data types from traditional relational sources to new non-relational sources. As an example, Direct Edge was able to join non-relational stock exchange messaging with their relational stock ticker data.
Microsoft has a comprehensive solution to modernize your data warehouse across software, appliance, and cloud for this new world of data. We invite you to learn more of our offerings:
SQL Server 2014
SQL Server Fast Track
|Parallel Data Warehouse with PolyBase||SQL Server for DW in WA Virtual Machine|
|Non-relational Data (Hadoop)||Hadoop on Windows||Parallel Data Warehouse with PolyBase||Windows Azure HDInsight|
- SQL Server 2014: Microsoft SQL Server 2014 will be generally available on April 1 and includes technologies like an updateable in-memory columnstore that can increase data warehouse queries 10-100x.
- SQL Server Parallel Data Warehouse: SQL Server Parallel Data Warehouse (PDW) is a Massively Parallel Processing data warehouse appliance that can scale out queries to Petabytes of data. It includes PolyBase, a feature that can allow you to seamlessly query & integrate both relational data and Hadoop data with the same T-SQL query.
- SQL Server hosted in a Windows Azure Virtual Machine: SQL Server Enterprise for data warehousing can be installed and hosted in the cloud on Windows Azure Virtual Machines. This image takes advantage of best practices from the Fast Track reference architecture to tune SQL Server for data warehousing in Windows Azure.
- Windows Azure HDInsight: Windows Azure HDInsight is a Hadoop-based service from Microsoft that brings a 100 percent Apache Hadoop solution to the cloud. You can seamlessly process data of all types with simplicity, ease of management, and an open Enterprise-ready Hadoop service all running in the cloud. We recently announced the general availability of HDInsight running Hadoop 2.2 clusters.
- Hortonworks Data Platform For Windows: Through a strategic partnership with Hortonworks, Microsoft co-developed the Hortonworks Data Platform for Windows to customers who want to deploy Hadoop on their own servers. Learn more about Microsoft’s contributions to Hadoop.
You can learn more about Microsoft solutions for the modern data warehouse by tuning in for a live stream of our April 15th Accelerate your insights event and by visiting these resources: