Analytics – the final touch to your Data Warehouse or the secret behind it?

December 19, 2023

Data is a precious thing and will last longer than the systems themselves. Data-driven companies, which get analytics to their teams faster, are three times more likely to find their financial performance superior compared to their competitors, reported the Economist Intelligence Unit.

The term Enterprise Data Warehouse has fundamentally shaped our way of thinking when it comes to information data management and analytics.

Before data warehousing, reporting, and provisioning data from disparate systems was a slow process that often failed to meet the organization’s need for actionable information. However, such a traditional approach to information management has become more and more problematic for being expensive, time-consuming, and inflexible. As a result, companies are shifting to more agile methods to stay competitive in today’s dynamic business environment.

According to Gartner, around 85% of all big data projects are deemed to be a failure, resulting in about $159B of wasted investment. The common reason, behind this negative outcome, is that their overall data strategy is often outdated. For example, many organizations try to ensure that their data, as well as the systems around it, are perfect before opening them up to users. At the end of the day, if your data isn’t in good shape, why continue with analytics? Well, think again.

“Upon implementation, business analytics has an average of 112% 5-year ROI.”

If you wait to build your perfect data architecture before starting with reporting, your employees will not be able to make use of data analytics to optimize their decision-making process and improve their performance. Another issue is that the needs of the end-users are continually changing. Therefore, it is highly possible that by the time you have built the perfect system with the perfect data, it will already be obsolete.

It is important to note that building sound data architecture and ensuring high data quality is still very important. Yet, it is a mistake to think that you need to cleanse your whole data and build the perfect Data Warehouse system before you can start with analytics. Nevertheless, according to a study from IBM “upon implementation, business analytics has an average 112% 5-year ROI”, so why should you wait to make use of it?

There are many examples of successful companies that have shifted from a more traditional data architecture to a more agile one, which can serve their needs better.

We were recently approached by one of our clients, a large, international enterprise with over 14 000 employees, to help them with a Data Warehouse migration project. They wanted to migrate their data from their old Data Warehouse to a new and better one. However, our client’s global IT team failed to complete the “transition” stage, due to the seemingly impossible task to compare the data business rules in the two systems in a comprehensive and flexible way. Thus, they could not ensure that the new Data Warehouse system is superior to the old one. When they contacted us, they were already struggling with the project for a few years. They had spent many resources and due to the prolonged delay were risking a reporting crisis. In order to solve their issues, we developed a QlikView dashboard, where they could compare the data in the two Data Warehouses and easily prototype new business rules, thus minimizing the friction during the migration process. This kind of architecture can also be extremely useful in cases of mergers and acquisitions, where a transition between different systems often occurs.

“Using the power of the Qlik platform, the client saved 75% to 90% in time and money over building a Data Warehouse.”

In other cases, the Data Warehouse can be omitted altogether. This is possible due to the ability of BI platforms such as Qlik to combine multiple data sources without relying on a Data Warehouse. To illustrate this point, let’s look at the following data integration example. A Qlik team was approached by a client with the request to integrate 50+ sources into a single data model. To meet the client’s needs, the team had to build approximately 100 scripts in the ETL.

Additionally, they had to find a way to replicate an ERP “matching” functionality, as the client wanted to be able to match receipts against order quantities to calculate the remaining open order amounts by line item. To solve this issue, the consultants decided to run the data through multiple scripts to break down single-receipt records to correspond to the quantities in the order table. In the end, using the power of the Qlik platform, the consultants were able to successfully design and build a very complex ETL using only Qlik. This saved their client 75% to 90% in time and money over building a Data Warehouse.

“… global investments in FinTech have more than tripled since 2014 to cross $12 billion.”

Another interesting architecture is the one employed by most FinTech companies. Due to the unique characteristics of their business, FinTech companies must be extremely agile, and this, of course, includes their data architecture. Instead of having just a Data Warehouse, they are using a middleware reporting app to deploy fast reports before the data even gets to the Data Warehouse. This kind of architecture allows them to be more agile, efficient and to gain a competitive edge. Therefore, the statement of PricewaterhouseCoopers that “…global investments in FinTech have more than tripled since 2014 to cross $12 billion” should not be surprising. FinTech solutions play a significant role in today’s financial services market. For instance, Revolut’s customer base has increased from about 4 million to over 7 million during the first 8 months of 2019 and continues to expand rapidly.

As one can see, there is a wide variety of different data architectures that companies can choose from, depending on their needs and goals. The most important thing to remember, however, is that regardless of the architecture type you pick, most decisions are made by people striving to keep up with the pace of today’s hypercompetitive business environment. Therefore, the companies that get data to their teams faster and solve specific problems will outperform and eventually annihilate those who are preoccupied with data perfection and keep contemplating on how to best architect the systems around it.

How B EYE Can Help You

At B EYE, we empower business users by transforming the data deluge into meaningful insights. We have a proven track record in helping Fortune 500 companies discover and implement new opportunities and data-driven strategies. Relying on our long-standing experience and next-generation BI technologies, we can help you optimize your data architecture to serve your needs in the best possible way. Our team of professionals can help you structure your whole ETL process, create highly polished pixel-perfect reports and efficiently bring data to your people.