BI in Financial Services

Businesses are suffering from the inability to effectively exploit their data assets. The cost of this corporate disability is literally immeasurable, and touches every aspect of business operation - from increased customer churn to a waste of IT resources. Many Financial Services organisations have realised the need to develop an enterprise-wide platform to manage information and turn it into 'business intelligence', and have initiated a data warehousing project, only to have it become a partially integrated point solution acting as little more than a data dumping ground. The general Business Intelligence (BI) market is maturing, as are the underlying technologies and architectures, so why has BI failed to deliver so much of its promise?

Our view is that the way that the tools are being used within organisations is far from mature. All too frequently the purchase of BI products is sanctioned on a case-by-case basis in an attempt to plug short-term information gaps within a department or functional area. The classic example is to use BI to address the shortcomings in the native reporting and analysis capabilities of an operational application such as mortgage processing or insurance claims administration. Whilst the deployed BI tool may address this particular issue, it does nothing to enhance wider organisational decision-making and performance management, which by their nature require the integration of data from multiple sources. Nor are such "point" solutions likely to improve the many, less well-defined procedures that cut across departmental or system boundaries. This raises a variety of difficult issues, both technical and cultural, most of which have hitherto been swept under the carpet. That looks set to change.

The Financial Services Industry is being forced to undergo sweeping changes in the wake of corporate scandals such as those at Enron, WorldCom, and Parmalat. This, combined with the threat of terrorist attacks post 9/11, has driven governments and regulators to respond with a whole raft of legislation intended to restore public confidence. Meanwhile, industry bodies have already been developing voluntary codes such as Basel II, which will require banks to implement much more sophisticated systems to manage risk and capital. As a result, compliance is becoming a major driver for investment in BI throughout the Financial Services sector.

Whilst some of the detailed provisions are still under discussion, it is already clear that regulators like the UK Financial Services Authority (FSA) will require more detailed and frequent reporting of transaction-level data, in some cases in 'near real time'. Financial institutions are now expected to have and use the right data - to know their customers - and to demonstrate that management is in control of every process. That means providing evidence that 'closed loop' systems are in place to monitor, detect, alert, and respond appropriately to events or exceptions - and simply to correct mistakes. This has obvious implications for data quality and staff training, but it also requires that BI becomes integrated with business processes.

These new regulatory reporting requirements tend to be seen as "operational", and the typical response is to initiate multiple projects to develop or enhance existing point solutions within existing functional boundaries. Providing such closed loop, operational BI may be more urgent and is undoubtedly easier than full enterprise integration, but is unlikely to yield the maximum possible return on investment.

Butler Group advocates taking a more strategic approach to BI, and treating the investment necessary to ensure compliance as an opportunity to develop value-added, integrated BI services that leverage information flows to improve operational efficiency, financial performance, and customer service.

Integrated Business Intelligence (IBI) embeds the collation, analysis, and dissemination of relevant information into the organisation's operational and decision-making processes at all levels, so that its use becomes part of everyday activities. It enhances the automatic detection of significant events and changes, facilitates the selection of timely, intelligent responses, and enables proactive management of enterprise performance.

To realise this vision, the organisation needs to understand its information flows: in terms of the lifecycle of data as it moves through each process, and the value it adds at each step. It is all about providing the right information, to the right person, at the right time - but also in the right context.

Business Issues

Regulatory changes are forcing banks, insurers, and investment firms to improve their risk management, fraud prevention, and anti-money laundering systems. At the same time they face unrelenting competitive pressures, which make it increasingly important for them to understand individual product and customer profitability, and to leverage the insights gleaned from every customer interaction. Many Financial Services organisations have recognised the contribution that BI technologies can make in these areas, but few have deployed them to maximum effect.

A business' culture can make or break a BI deployment. Getting this right starts well before any product or vendor selection is made, and hinges on unravelling the complex and often political issues surrounding information ownership. The problem stems from the fact that business processes span departments and functional areas of the business. No one individual 'owns' the process, and so when problems arise, as they inevitably do, the result tends to be more of a finger pointing exercise than a serious commitment to rapidly addressing the issue. Only by defining who needs to know what, why, where, and when, can firm foundations for information sharing, collaboration, and individual empowerment be laid.

Introducing BI also raises issues about the roles of business and IT. Producing reports and analysing data have historically been the task of technical specialists: most people regard this work as either too difficult or beneath them, yet are frustrated at their dependence and the consequent delays in getting access to the information they need. The new generation of BI tools is starting to change these attitudes, and with appropriate training many Financial Services organisations are succeeding in introducing self-service reporting delivered through personalised interfaces via an Intranet.

The ability to personalise information delivery is key, and not every user will work the same way. An equities trader, for example, will need real-time market data feeds and graphical analyses whilst an underwriter might only need to review claims history once each quarter before adjusting premium rates. The overall BI architecture will have to support many different types of decision, from strategic to operational.

The value of information decays over time, and there may be a limited window in which it remains actionable. For the trader in our example above, that window may literally be a few seconds, but in most cases it will be measured in minutes, hours, or days, and a 'just in time' solution will suffice. Competition and regulation are putting increasing pressure on Financial Services firms to be able to react faster, so plans for architecting an IBI environment should consider carefully what and where 'near real time' capabilities are really needed.

Let us not forget that the foundation of successful BI is the accuracy, consistency, completeness, and timeliness of the underlying data. Data quality needs to be far greater to support operational decision-making, at the level of an individual customer, than it does for more strategic decisions, based on summary-level data, which are traditionally the focus of BI systems.

Data integration is also critical: unless the data from different source systems have been matched and checked for consistency, there will not be a single customer view to underpin lifetime value analysis or cross-selling initiatives. This is the classic rationale for building an enterprise data warehouse, an undertaking that few Financial Services institutions have completed.

We would suggest that implementing IBI technology without tackling data quality and integration issues is like sticking a fresh dressing on a festering wound without cleaning it up first. The problems will only get worse!

Data quality also has important cultural implications, which are linked to the issues surrounding information ownership. No-one wants the job of cleaning up bad data, and it is cheaper and easier to trap and address any problems at source. Employees must therefore learn the importance of getting it right first time, with data quality control embedded into every process.

IBI also opens up possibilities to increase efficiency via automation - not just by using alerts to enable management by exception, but also by automating some of the routine decision-making itself. It is perfectly feasible to set up programmatic trading systems, or to process credit checks based on pre-defined rules - but people will not always be comfortable to leave such things to a machine.

Whilst harnessing the potential of BI requires education and training, integrated BI necessitates a radical change in mindset: decisions will be based on facts, with all the relevant information expected to be available on tap - no waiting, guessing, or rummaging around.

As with other strategic initiatives, IBI needs strong, business-led project and programme management, focusing on benefit realisation, not just on delivering the enabling technology. That means having a top-level champion to overcome the political barriers to process and data integration and drive through the necessary data quality improvements. It also means involving the senior management team in thinking through how the use of BI can be aligned to strategic goals. Corporate Performance Management (CPM) is therefore a key component of IBI.

Technical Issues

Companies operating within the Financial Services Industry, especially the major banks, are used to handling very large volumes of transactional data on a daily basis. The advent of e-business, and increasing demands for the retention and analysis of detailed electronic records, are fuelling a rapid expansion in the scale of data warehouse and BI solutions in this sector.

At the same time, banks, insurers, and investment management firms are looking to expand the user base for BI, internally at first, but increasingly to suppliers, intermediaries, and customers. Web technologies are enabling them to do this across multiple delivery channels, with applications ranging from providing on-line fund information to outsourcing the entire back office. This Web-based self-service reporting has a two-fold application: widening BI use by employees on the intranet, allowing ad hoc reports to be created using intuitive interfaces without calling upon power users, and via the broader Internet, enabling on-line bank customers, for example, to customise statements and to drill down into expanded detail.

There is an industry-wide drive to accelerate and automate transaction processing, particularly within trading, settlement, and card payment systems. Integrating additional and relevant information into these processes will contribute significantly to achieving Straight Through Processing (STP). This requires a more complex, real-time architecture, which uses Message Oriented Middleware (MOM) to integrate processing between applications. In addition to STP, this kind of infrastructure will also support real-time, closed-loop BI applications; for example, to detect and intercept potentially fraudulent transactions.

Most BI tools on the market today are capable of generating alerts, and delivering them to the appropriate people automatically. But without some filtering or prioritisation mechanism, there is a real risk of information overload. Focusing on what is important puts the 'intelligence' into BI, and gives the information its value. If you had 30 e-mails warning that your mailbox is full, you might not spot the one telling you to check up on Mr Leeson.

These alerting mechanisms are driven by rules, and are becoming increasingly sophisticated. With most Business Activity Monitoring (BAM) solutions, the rules are entered manually, but there are several BI vendors who offer predictive modelling, statistical analysis, and data-mining tools, which are capable of generating and maintaining rules automatically. This process is usually undertaken 'off-line' by analytical specialists, but the resulting models can be embedded into a BI application and executed in real time. Searchspace and Mantas, for example, have used their behaviour detection software to develop anti-money laundering, fraud prevention, and customer profiling applications for several major banks.

It is generally accepted that the value contained in unstructured information is considerable, yet it is rarely included in BI applications because of the cost and difficulty of integration. Compliance increasingly requires access to all communications between employees and customers, especially documents and e-mails. Some of the largest Financial Services organisations are already using scanning and text mining technologies to generate structured data from paper-based documents. Most BI vendors are still weak in this area, but some, such as Autonomy, specialise in managing unstructured data, while other vendors with a content management pedigree, such as Hummingbird, have recognised the need and begun to address it. At the same time, operational systems vendors such as Mobius, which specialises in automated cheque processing, are moving into this space by adding generic content management and report management functionality to their vertical applications, and are also moving into horizontal markets.

Metadata is another key issue. Even now, most vendors' tools can barely import and export one another's database schemas, never mind metric definitions and business rules, making it extremely difficult to create a transparent, end-to-end, information supply chain. The aim should be a unified metadata repository, accessible to every component within the overall enterprise architecture, and beyond. The Object Management Group's Common Warehouse Metamodel (CWM) standard can help here in ensuring interoperability between tools. Many BI vendors are now using XML to create adaptors to make the necessary links. The arrival of XML also makes the definition of standards for information exchange much easier, and there are many examples within the Financial Services industry, including those used by SWIFT and VISA. In particular we mention eXtensible Business Reporting Language (XBRL) as the emerging standard for financial reporting.

IBI is clearly not an off-the-shelf proposition. Although many of the leading BI vendors now offer end-to-end BI solution platforms, none yet offers integration across performance, content, change, and process management facilities within a single, enterprise-scale environment - a level of integration that would be ideal for meeting compliance needs. Enterprise application integration tools are also needed to glue these components together and provide real-time links with operational systems. We are likely to see a period of acquisitions and alliances over the next two years as these underlying technologies mature and converge. In the meantime, Butler Group's advice is to build a strategic-level IBI infrastructure based on open standards. Compliance solutions in particular need to be open and flexible, since regulatory requirements are still evolving.

Market Issues

For the mainstream BI vendors featured in this Report, the Financial Services sector currently represents between 15% and 25% of revenues from their BI market, and this sector is widely expected to grow rapidly in absolute terms over the next two years. The drive to achieve regulatory compliance has led to a surge of interest from potential buyers, and many more projects are starting up, in most cases with substantial budgets. It is no surprise, therefore, to see the competition 'hotting up' as well.

In terms of BI maturity and ambition, banks are generally ahead of insurers, with the largest financial institutions taking the lead. That said, BI technologies are still generally being used to implement 'point' solutions for departmental reporting, not, as we would advocate, to support enterprise-wide data integration, end-to-end information management, collaborative decision-making, and performance improvement. Educating potential buyers thus remains a significant challenge.

BI vendors need to get better at selling solutions - to business rather than IT - but many do not have sufficient domain understanding. Some are addressing this by recruiting banking, insurance, and investment management experts; others rely on consulting and systems integration partners. This in-depth industry and business process knowledge becomes even more important for successfully implementing IBI to support operational decision-making, and those who have it will be in demand.

Microsoft is now beginning to offer functionally-rich, scalable, end-to-end BI solutions at significantly lower cost, and is gaining increasing credibility (and market share) within the Financial Services industry. This is good news for second- and third-tier organisations, which often cannot afford to buy and integrate best-of-breed software components.

Faced with this powerful new competitor, a maturing market with few 'virgin' customers, sceptical buyers, and the tight IT budgets of the last two years, the established BI vendors have been forced to look for strategies to differentiate themselves. Some have tried to sell industry-specific and/or function-oriented solutions as packaged applications, e.g. for planning and budgeting, sales analysis, compliance, or risk management. Others, including CorVu, Hyperion, Cognos, and Business Objects, have either caught the wave of interest or are enjoying an existing presence in CPM. Meanwhile, some have focused on their core strengths - Informatica and Group 1 on data integration, for example, and SAS and SPSS on complex analytics - whilst embracing open standards so that they can be readily integrated with other best-of-breed products.

In the fledgling IBI market, these horizontal BI vendors also face competition from generic business process management, content management, and change management tool vendors such as TIBCO, FileNet, Metastorm, Mobius, and Serena, who are targeting the same business needs, particularly within the Financial Services industry. We anticipate convergence between these technologies, and some consolidation, driven by the demand for compliance solutions that integrate all these components.

At the same time, many specialist banking, insurance, and investment management application vendors, such as FiServe, Eurobase, and FirstApex, have added management reporting or even OLAP functionality to their operational systems solutions, enabling users to monitor process efficiency and investigate unexpected variances and trends. Others have gone a step further, embedding reporting and analytics into the business process. Examples include facilities for real-time, pre-deal limit checking, payment card fraud detection, and real-time customer authentication from the likes of Misys, Murex, Thomson Finance, Algorithmics, Carreker, and ID Analytics. Although these are generally 'point' solutions with limited scope for cross-vendor integration, they may allow smaller financial services organisations to achieve closed-loop BI vertically rather than horizontally. We have already seen some interesting merger activity in this space - for example, the acquisition of Whitelight by Sungard. In the enterprise application market we have again seen the major vendors adding enhanced reporting modules like SAP Business Warehouse.

There are no clear leaders in this rapidly evolving market place, and buyers should consider a range of options to find the best strategic fit. User organisations looking for IBI capabilities should evaluate niche vendors specialising in the Financial Services sector as well as established BI tools.

Key Findings

Key Findings

• Business Intelligence (BI) technologies can help financial institutions to manage risk, detect fraud, leverage customer insights, and gain visibility into their profitability, but few have deployed them to maximum effect.
• Established BI tools face competition in the Financial Services sector, both from vertically-focused operational and analytic application vendors, and also from in-house custom development.
• No vendor yet offers all the required components to deliver true 'closed loop' BI across the enterprise, so solutions must be built around open standards with a unified metadata repository that complies with the Common Warehouse Metamodel (CWM).
• Compliance necessitates integrated business intelligence, but this should be seen as a strategic investment opportunity and used to create competitive advantage. Vendors offering overlapping technologies can give a head start.
• Data accuracy, consistency, and completeness, always important, are given greater urgency by compliance.
• Routine integration and analysis of both structured and unstructured data, in near real time, will require a more sophisticated IT infrastructure with Message-Oriented Middleware (MOM) to link applications.
• The use of XML for information exchange and fast messaging is growing rapidly, with emerging standards such as eXtensible Business Reporting Language (XBRL) for financial reporting.
• Microsoft is turning traditional enterprise reporting into a commodity, forcing established players to find strategies to differentiate themselves.
• Ad hoc, self-service reporting and analysis are increasingly demanded by customers as a vehicle for widening the use of BI within the organisation.