An Interview with Bill Busch
Companies rely on information and data as the differentiator and key to success and business growth. For years, organizations have turned to business intelligence systems to make sense of the vast amounts of information and data available in order to answer critical business questions. According to a number of Forrester Research surveys, business intelligence (BI) has been the number one software priority of organizations’ project plans since 20101 and one of the top three IT investment areas for more than a decade.
Despite the focused investment in BI, the traditional systems that are prevalent in today’s organizations have historically been slow to change and have forced users to operate within a set of rigid capabilities. However, in the last few years a new set of technologies and methods have been developed that have resulted in BI systems that are flexible and easily changed. Business users can prepare data for analysis and create their own reports, all without IT involvement. BI organizations realize that enabling business users to be self-sufficient greatly increases the BI value proposition. New data can be added to a BI ecosystem within days versus months with traditional systems. In short, BI systems can now support the speed of business.
What are some of the trends impacting BI solutions today? How can organizations prepare for navigating a rapidly changing business environment?
Although there are many trends impacting BI, the two most prevalent are the speed of business and Big Data. Having ready access to enterprise data is now essential for businesses to recognize and act upon market trends. BI systems traditionally have required months for new data to be added and this has hindered an organization’s ability to perform timely analysis. However, using agile BI tools, methods and modern data architectures, new data can easily be made available to business analysts. With data discovery tools like QlikView, Endeca and Spotfire, business users can, without IT involvement, leverage quality data warehouse data and combine it with data that is external to the data warehouse. The time-to-value is reduced greatly, thus supporting faster decision cycles.
Although there are many trends impacting BI, the two most prevalent are the speed of business and Big Data. Having ready access to enterprise data is now essential for businesses to recognize and act upon market trends. BI systems traditionally have required months for new data to be added and this has hindered an organization’s ability to perform timely analysis. However, using agile BI tools, methods and modern data architectures, new data can easily be made available to business analysts. With data discovery tools like QlikView, Endeca and Spotfire, business users can, without IT involvement, leverage quality data warehouse data and combine it with data that is external to the data warehouse. The time-to-value is reduced greatly, thus supporting faster decision cycles.
How does an agile BI solution differ from a traditional BI approach?
The first and probably most important change is that BI systems are not limited to information stored in data warehouses or data marts. Merv Adrian from Gartner research coined the term “Logical Data Warehouse,” which essentially enables users to leverage data from many different data sources, thus, creating one logical or virtual data warehouse. Whether it is a Big Data lake, data refinery, staging area, data mart or a data warehouse, end users should be able query the data where it is stored. In an agile BI environment, users easily can join data from different sources without worrying whether joint paths have been established or roll ups have been defined. The business analyst is in control and can prepare and analyze data to meet his/her individual requirements.
The first and probably most important change is that BI systems are not limited to information stored in data warehouses or data marts. Merv Adrian from Gartner research coined the term “Logical Data Warehouse,” which essentially enables users to leverage data from many different data sources, thus, creating one logical or virtual data warehouse. Whether it is a Big Data lake, data refinery, staging area, data mart or a data warehouse, end users should be able query the data where it is stored. In an agile BI environment, users easily can join data from different sources without worrying whether joint paths have been established or roll ups have been defined. The business analyst is in control and can prepare and analyze data to meet his/her individual requirements.
Another major difference is the autonomy of the business user to create and publish BI content. IT organizations should not be in the business of creating BI content. The tools that they deploy should enable power users to create BI dashboards, reports and applications, and then publish this content for consumption. This takes 80 percent of the content creation off of IT’s plate, enabling IT to focus on data quality and increasing access to enterprise data resources.
What steps should an organization take to implement agile BI solutions?
Enabling business agility must be a core objective for the BI program or else other competing objectives may overshadow the agile BI objective. The architecture needs to allow for pervasive data access. BI reporting, dashboard and analytical tools must enable users to quickly perform analysis and then develop and publish BI content based on that analysis. The architecture should minimize the need for modeling or analysis to publish new data for business use. Processes should be honed to reduce unnecessary work. Finally, the BI organization needs to be focused less on traditional IT development and more on increasing usage of the BI systems. Agile BI is a significant paradigm shift.
Enabling business agility must be a core objective for the BI program or else other competing objectives may overshadow the agile BI objective. The architecture needs to allow for pervasive data access. BI reporting, dashboard and analytical tools must enable users to quickly perform analysis and then develop and publish BI content based on that analysis. The architecture should minimize the need for modeling or analysis to publish new data for business use. Processes should be honed to reduce unnecessary work. Finally, the BI organization needs to be focused less on traditional IT development and more on increasing usage of the BI systems. Agile BI is a significant paradigm shift.
What are the biggest challenges that businesses have with enabling agile BI solutions?
There are several major challenges. First, most IT organizations believe agile BI means to deliver BI through an agile SDLC process like SCRUM. Although delivering value incrementally is desirable, agile BI actually is focused on enabling the end-user through improved BI tools and greater data access.
There are several major challenges. First, most IT organizations believe agile BI means to deliver BI through an agile SDLC process like SCRUM. Although delivering value incrementally is desirable, agile BI actually is focused on enabling the end-user through improved BI tools and greater data access.
I also see IT organizations still focusing on one-size-fits-all data management where all published data needs to be modeled and fully integrated into the data warehouse. Corporations are dealing with tens of thousands of tables and data elements reaching into the millions. It simply is not cost effective to integrate all this data into a traditional data warehouse or into a single BI semantic layer. IT organizations need to change their focus from traditional data modeling to inventorying and classifying data so that business users can find and access the data they need. This does not mean that data modeling goes away, but IT organizations must right-size their data management activities (including analysis and modeling) based on the value of the data to the organization. In the world of Big Data, this results in about 20 percent of the data being modeled.
The other area where IT organizations have challenges is around data quality. With data discovery technologies in particular, IT tends to take a hands-off approach versus governing and managing these capabilities. I have seen data discovery environments that resemble the Wild West, and the data discovery tool gets blamed for the mess. However, if the tool had been governed with similar effort as other critical BI capabilities, then the data quality challenges would have been mitigated.
What are the benefits in evolving a current BI strategy into a flexible and agile BI environment?
The initial benefit companies see is the reporting backlog will disappear because content creation is moved from a small number of IT professionals to a larger number of BI power users. The enhanced data access will enable quicker, better decision making, which eventually results in higher revenue.
The initial benefit companies see is the reporting backlog will disappear because content creation is moved from a small number of IT professionals to a larger number of BI power users. The enhanced data access will enable quicker, better decision making, which eventually results in higher revenue.
As a result of this overall approach, we see business stakeholders finally owning BI content and data quality, which is a good thing. This results in better alignment between BI initiatives and business initiatives. If IT truly embraces the consultative role, a strong working partnership between IT and business stakeholders will develop.
Do BI development and support organizations have to change to support an agile BI environment?Yes, they do. Most BI teams within IT organizations are development and IT-project focused. This changes in organizations supporting agile BI. The development role now focuses on only a small portion of the data. The majority of the effort should enable data access and make the data easier to query. IT resources need to become consultants that coach end-users on best practices and how to best deliver BI content. Yes, IT will still move data from one system to another and build a data warehouse, but this will be focused only on a limited number of high-valued data elements.
How does Perficient help companies implement agile BI strategies and technologies?Perficient has a number of offerings that help companies transition to a more agile approach to BI. In our standard BI or data strategy assessment and roadmap offerings, we have built the capability to benchmark and improve the agility of a BI program. For companies not seeking a full BI/data strategy, we have a targeted agility assessment, which helps companies develop a plan to implement agile BI capabilities through technology, process and organization improvements. When companies are looking to purchase new software for the BI/data warehouse environment, we include specific agile and business-enablement requirements in Perficient’s product selection engagements. Finally, we offer a complete set of information architecture and implementation services that incorporate Perficient’s agile BI thought leadership.
originally shared by
Forrester Research. “TechRadarTM: BI Analytics, Q3 2013.” July 11, 2013