Contents

FOCUS

Acting on the Digital Imperative

VIEWPOINT

The End of Two-Speed IT

FOCUS

Simplifying IT to Accelerate Digital Transformation

VIEWPOINT

Agile Development’s Biggest Failure Point—and How to Fix It

FOCUS

Bridging the Trust Gap: The Hidden Landmine in Big Data

Q&A

Changing the Game with a Data Lake: An Interview with Centrica’s David Cooper and Daljit Rehal

INDUSTRY SPOTLIGHT

Ensuring Digital Readiness in Financial Services

INDUSTRY SPOTLIGHT

Building a Digital Technology Foundation in Insurance

Preface

Companies on digital transformation journeys face three key questions: How do I speed up my digital delivery? How do I scale the new digital world across my entire organization? How do I create value from my digital transformation?

When executives ask these questions, they usually discover that the answer, in part, is that their organizations need a new set of capabilities. But building these capabilities at scale while driving a transformation and maintaining “business as usual” is challenging.

To meet those (seemingly) multiple challenges, it’s critical to understand that, essentially, they are all pieces of one overarching challenge: to embed digital at the core of the business. The digital transformation is not a passing trend and cannot be relevant to only part of the business. All capabilities throughout the organization must be digital ready, because “business as usual” is rapidly becoming synonymous with “digital as usual.”

In this edition of BCG Technology Advantage, we look at the digital journey from a variety of perspectives. Our feature article shows how companies whose digital transformations succeed are acting on the digital imperative in order to outperform the competition. Clearly, as several other articles herein detail, lessons about digital are being learned:

- How two-speed IT must give way to an all-agile approach

- How IT can become dangerously complex and how to simplify it

- How to fix the most common reason for failure in agile development

- How companies are realizing the importance—and the means—of properly handling consumer data to maintain consumer trust

In an interview with Centrica’s David Cooper and Daljit Rehal, we get a detailed look at a particular aspect of digital transformation: the move from traditional data warehouses to data lakes. We also look at the journey from industry-specific points of view, with articles devoted to what going digital means for financial institutions and insurers.

I hope you find these publications interesting. Please send any feedback to TechnologyAdvantage@bcg.com.

Ralf Dreischmeier
Global Leader, Technology Advantage practice

focus

Acting on the Digital Imperative

by Ralf Dreischmeier, Karalee Close, Thomas Gumsheimer, Peter Hildebrandt, and Adal Zamudio

Since publication of “The Digital Imperative” (BCG article, March 2015), we have heard from CEOs around the world who are eager to take the next step toward digital transformation. Most executives tell us they understand why their companies need to transform, and most have a basic understanding of what must be done. But a fundamental question remains: how do they make it happen?

The transformation will vary somewhat depending on the company’s current location on the digital journey. (See the exhibit, “Digital Transformation Is a Journey.”) At the outset, companies need to frankly assess their digital maturity and identify gaps relative to competitors. It’s essential to make the right moves in the early days of a digital transformation. To do so, companies need to understand where digital initiatives can have the greatest impact on their business.

After analyzing more than 1,000 companies that were undergoing digital transformation, across multiple industries and geographic locations, we concluded that the most successful companies outperform their competition by excelling in three fundamental areas:

- Speed. How can we move quickly—at digital speed?

- Scale. How can we change the way we work across the entire enterprise?

- Value. How can we create tangible value for the business—and for customers?

To compete in the fast-moving digital world, companies need to execute along these three dimensions, in parallel.

Speed

Large organizations are accustomed to managing large-scale, multiyear projects with fixed timelines and set budgets. It’s not uncommon for an enterprise resource planning (ERP) migration project to take five to ten years from start to finish. But in today’s fast-paced digital economy, such a model is not just outdated—it’s potentially fatal to the organization. To survive, companies need to work and make decisions more rapidly than ever before.

The digital transformation, when done right, is not a monolithic endeavor. Rather, multiple projects run simultaneously on parallel tracks. When selecting initial pilots, teams rally around a business objective (such as increasing conversion rates by a certain percentage) and then brainstorm product offerings that will best achieve that objective. Each team has a captain, or product owner, who drives the process forward and tracks outcomes. (See “Agile Development’s Biggest Failure Point—and How to Fix It,” BCG article, August 2016.) The product owner designs each project to meet an accelerated delivery cycle, allowing teams to rapidly test the project, quickly learn its strengths and weaknesses, and continually adjust course as needed. Each project contributes to a compound effect that benefits the entire enterprise and catalyzes additional pilots. The beauty of this approach is that companies generate results quickly, which helps fund the journey toward ever greater opportunities for innovation. At the same time, the parallel projects gradually build up capabilities for speed within the organization.

A leading North American bank took 18 months, on average, to release new digital products into the market. With startup attackers moving quickly on the digital front, the bank needed to shorten its development period without compromising quality or regulatory compliance. The bank changed three things: the way it organized teams, the way it made decisions, and the way its teams worked. First, the company created cross-functional teams, bringing together people from the business side and the technology side, to follow a product from inception to release. These teams rallied around a business objective, such as increasing incoming applications for mortgages by 10% or increasing conversion rates on credit card applications by 5%.

Unlike in the typical development model, in which IT teams deliver specific functionality (creating a button here or a search box there), the bank’s cross-functional teams had the freedom to design a product as they saw fit—and supervisors evaluated them on their ability to meet their stated business objective. Teams also embraced a new way of working, through agile principles. Rather than engaging in extended philosophical debates on the merits of various offerings, they created prototypes and put these in front of users. By analyzing user feedback, they learned what customers wanted—thus clarifying what they should build. By implementing these new processes, the bank released products in approximately half the time that the process used to take (roughly 8 months instead of 18 months), while maintaining compliance and improving quality.

Scale

To succeed at scale, companies need to create a culture in which initiatives thrive across varying locations and markets. Any discrete business unit can create an app or launch a pilot—and many do—but scaling up requires a shift in mindset across the entire enterprise. Senior leaders must provide strong, steady support from the top. Cross-functional teams must extend their digital capabilities and ensure that agile principles and the associated culture become embedded across the organization. Meanwhile, internal business units—such as HR, budgeting, and IT—must receive training and development to encourage the right culture and behaviors.

It is also important for teams to think through the end game for each initiative. While brainstorming potential pilots, they need to ask tough questions: Why does this project matter? Can it create significant value? Does it yield a sustainable advantage? Rigorous analysis is an essential part of identifying initiatives that create a true competitive advantage and can scale up rapidly. Digital products can deliver strong ROI when designed thoughtfully (by targeting a key customer segment, for example) and developed with a keen focus on rapid, comprehensive scalability.

Starbucks has become an industry leader in mobile payments, and the company’s journey illustrates the value of scalability. Its digital strategy began in 2008 when the company built an online community and became active on Twitter and Facebook to nurture customer loyalty. In 2009, the company launched its first app, which enabled customers to find Starbucks stores and get nutritional information about its products. This simple app scaled easily and laid the foundation for the company to advance its loyalty program via mobile. Next, Starbucks launched an app to make digital payments possible, using existing technology (rather than developing a new leading-edge technology). Customers could scan a barcode at the point of sale, eliminating the need for money to change hands. Again, the app’s simplicity—from both a technology and a business model perspective—made it seamlessly scalable across store locations. The app also helped Starbucks strengthen its relationship with participants in its loyalty program (who, on average, spend three times as much money on Starbucks items as typical customers do).

The company’s most recent app, which builds on the previous versions, has become a model for the industry. Its Mobile Order & Pay app enables customers to place orders online, pay ahead of time, and pick up their orders without waiting in line. While Starbucks executives may have had this vision in mind from the outset, they started with smaller initiatives that gave them the opportunity to experiment, learn what customers like, build their internal digital capabilities, and improve their offerings. The results have been stunning: app usage continues to grow, revenue is at an all-time high, and today more than 20% of all orders at Starbucks in the US arrive through Mobile Order & Pay.

Value

Above all, a digital transformation must deliver value—not five or ten years down the road, but now. Thus far, digital hasn’t delivered its full potential in many large organizations. In our view, this is because those companies haven’t successfully tackled change management, either from a technology perspective (assessing which initiatives will deliver the most value) or from an organizational perspective (understanding how to manage work differently across the organization). Companies need to address these two aspects of change management in an integrated way—and in parallel.

Once digital projects are underway, project leaders must relentlessly focus on measuring their value in terms of real-world outcomes: customer response, increased revenue, reduced churn, cost savings, time savings, and so forth. Companies must kill initiatives that don’t deliver value. Output is irrelevant—and counterproductive in the long run—if it doesn’t create value. To succeed, organizations must think creatively about how best to capitalize on their assets.

Transport for London (TfL), an organization responsible for all public transit in London, wanted to provide its customers with a mobile app to help navigate public transportation options. Instead of spending public funds to develop its own app, TfL invested in a framework that allowed third-party developers to access TfL’s transportation data and use it to create innovative travel apps, maps, and services on their own. Since the program launched in 2009, more than 8,000 developers have signed up for it. One local startup, Citymapper, used TfL’s data to create an urban navigation app that has become the go-to resource for Londoners—and the company has expanded to cover more than 30 cities. Overall, every £1 invested by TfL in its open data framework has yielded £58 in benefits for Londoners. By embracing open data, TfL created much greater value for its customers, and did so much more quickly than it could have on its own, while also creating extraordinary opportunities for startups like Citymapper.

Although transformation tends to be viewed as an audacious undertaking, the most successful digital transformations start with focused initiatives that deliver on all three dimensions: speed, scale, and value. Once the earliest digital initiatives prove their value, they catalyze the next round of more ambitious follow-on projects. At the same time, a culture shift occurs within the organization, as the company adjusts to digital as a new way of doing business. A well-executed digital transformation doesn’t just even the playing field for large companies—it tips the odds in their favor. With speed, scale, and value on their side, incumbents can fend off attackers and win in the digital economy.

Ralf Dreischmeier is a senior partner and managing director in the London office of The Boston Consulting Group and the global leader for the firm’s Technology Advantage practice. He has developed technology and digital strategies for clients in financial services, telecommunications, media, and consumer goods, and has supported multiple large-scale business and technology transformations. You may contact him by e-mail at dreischmeier.ralf@bcg.com.

Karalee Close is a partner and managing director in BCG’s London office. She is the leader for the Technology Advantage practice in Western Europe and South America and the global topic leader for digital, big data, and advanced analytics in the Health Care practice. Over the past 20 years, she has worked at the intersection of technology and health care across many sectors, including biopharma, medical technology, payers and providers, software, and telecommunications. She spent eight years in hospital management before joining BCG. You may contact her by e-mail at close.karalee@bcg.com.

Thomas Gumsheimer is a senior partner and managing director in the firm’s Berlin office and the global leader for digital transformation and big data in Central Europe, the Middle East, and Africa in the Technology Advantage practice. He has extensive experience in developing and implementing strategies and large programs, especially corporate and organization transformation programs, as well as digital transformation and cost-saving programs. You may contact him by e-mail at gumsheimer.thomas@bcg.com.

Peter Hildebrandt is a principal in BCG’s Los Angeles office, specializing in technology and operational issues in the financial services industry. You may contact him by e-mail at hildebrandt.peter@bcg.com.

Adal Zamudio is a principal in the firm’s London office and a core member of the Technology Advantage practice. He specializes in digital and IT simplification and transformation. You may contact him by e-mail at zamudio.adal@bcg.com.

viewpoint

The End of Two-Speed IT

by Hanno Ketterer, Benjamin Rehberg, Christian N. Schmid, and Djon Kleine

Back in 2012, as established companies began to make a serious push into digital, BCG advocated a concept known as “two speed IT.” It was something of a compromise—a very necessary one. If IT organizations were going to support digital initiatives, they needed to work in faster, more flexible, more collaborative ways. Yet management often viewed these methods—based on principles set out in 2001 in the Agile Manifesto—as untested and maybe even a bit wonky. Two-speed IT was a way of saying, Don’t worry: you can use the new techniques for new areas like digital, and the traditional approach for mission-critical core functions.

It was a good idea at the time, but times have changed. Today, two-speed IT is a compromise that companies can no longer afford to make. The future of IT is one speed: all-agile. That’s not just because agile has proved itself at countless startups and major technology companies—and for all types of software development, digital and nondigital alike. It’s not just because agile’s footprint is expanding to industries like banking and insurance. (See “Ensuring Digital Readiness in Financial Services,” BCG article, April 2016.) And it’s not just because today’s companies can draw on fleshed-out playbooks when implementing agile. (See “Five Secrets to Scaling Up Agile,” BCG article, February 2016.) More than anything, it’s because two-speed IT creates—or will create—significant challenges for companies that continue to employ it.

Two-speed IT was a great intermediate stage, but it is not a long-term solution. And its term is up.

The Problems with Two-Speed IT

With its iterative development cycles, multidisciplinary teams, and continuous testing, agile represents a sea change from the traditional “waterfall” approach, where development flows sequentially from conception to testing and where separate teams take over at each phase. The differences between the models—and the processes, culture, and even mindset they require—make the appeal of two-speed IT easy to understand. But operating at two speeds, we have observed, creates three problems.

It’s harder to attract and retain talent. Recruiting and developing top-tier talent are perhaps the most important challenges that CIOs face today. You can’t do great things without great people. But two-speed IT puts companies at a significant disadvantage in the war for talent. The organization is effectively split into two parts—each with its distinct, and inevitable, culture. There is the “fast” group, which is seen as doing all the exciting, cutting-edge work. And there is the “slow” group, which is viewed as doing the staid and traditional work. The dinosaur projects. The dull stuff.

It’s not hard to guess which group everyone wants to join. This causes a problem because having top talent in the slower group is particularly important. Here is where the hard challenges of transforming legacy systems are tackled—and where the larger part of IT spending still goes. But when people see themselves as stuck in the slow group with no chance to switch sides, they’ll look for opportunities elsewhere.

Two-speed IT, we are seeing, leads to talent drain. It also makes it harder to hire talent. Today’s digital generation looks for—and expects—a workplace that emphasizes the flexibility, cooperation, and adaptability that are hallmarks of agile.

It leads to “hurry up and wait.” In today’s IT environment, fast-moving agile initiatives increasingly rely on core and legacy systems. Consider, for example, a digital front end that links to a back-end platform. In such a case, two-speed IT means slamming on the brakes. Fast-moving projects will often run up against—and be delayed by—slow traditional test-and-release cycles. What could have been running tomorrow is now set to run after the summer—maybe. This “slowest common denominator” issue is becoming increasingly problematic as digital applications become more central to business and must interact closely with core systems.

It keeps the larger organization from realizing the benefits of agile. Within many two-speed companies, there is a well-entrenched notion that, changed world or not, the more methodical waterfall approach is still better suited for legacy and very large projects. But it’s not. Large projects are particularly susceptible to delays and rising costs, and tend to have very low success rates. Part of the problem is that testing comes only at the end of the process, so errors are found late in the game, when fixes become time-consuming, difficult, and expensive. Agile, with its iterative cycles and continuous testing, finds and corrects errors as development progresses. There is no last-minute—and nightmarish—back-to-the-drawing-board scenario.

The waterfall approach works well when the goal is fixed—if you know, for instance, that you need to build a bridge across a river. But in today’s IT realm, fixed goals are the exception. Whether it is a digital front end or a core business system, requirements change frequently because of customer feedback, competitors’ moves, evolving regulatory environments, and alterations made to associated systems. Agile-related processes incorporate change better than waterfall methods do because they were designed to incorporate change. This adaptability is something the entire IT organization—not just part of it—needs to benefit from.

In a world where customers have more choices than ever before, the ability to develop core systems faster and more flexibly is crucial. To quote Peter Jacobs, the CIO of ING Bank Netherlands: “I would rather work agile at my core bank system than at the channels.”

Making All-Agile Work

While a single speed can “spread the wealth” of agile throughout the IT organization—and beat back the challenges that two speeds create—the model won’t work without the support and commitment of senior leaders. They can mobilize the troops and help steer—and, when necessary, push—the initiatives and changes that will ease the move to all-agile. A number of steps, we’ve found, are particularly crucial.

Identify and empower agile champions. Two-speed IT has helped companies get agile up and running in part of their organization. The experience and talent already developed can be harnessed to spread agile concepts—and knowledge—throughout IT. The most enthusiastic and communicative agile team members can serve as mentors to those just getting started—providing insights on what works, what doesn’t work, and how to do things better.

Create the right technical environment. Legacy systems are not a deal breaker for agile. Indeed, agile’s main principles can be translated to work on any project, and industries that still rely heavily on legacy applications and infrastructure—such as banking, insurance, and aerospace—have already started to embrace, and benefit from, agile.

But there are modern technologies and practices that can make the agile approach more effective. A decoupled architecture—in which applications, infrastructure, and data interact with one another through standardized interfaces like APIs and microservices—allows teams to work more independently of one another. Now they’re in control of their own development speed (and if one service breaks, just that service is down—not the whole system). Companies can also increase speed and efficiency—often dramatically—by combining agile with techniques like continuous delivery and continuous deployment of applications. This reduces the manual tasks—and the resources—required. Companies should be taking these steps anyway to improve their responsiveness and accelerate their digital transformation.

Implement agile in an agile way. A large established company is likely to implement agile very differently than a startup will. After all, bigger, older organizations must account for the layers of processes and hierarchy developed over the years. Similarly, agile will take different forms even within a single organization. Whereas one team may find two-week sprints optimal, another may determine that four or six weeks work better. Agile on a legacy mainframe, meanwhile, won’t look the same as agile on a mobile shopping app. And because some projects, like a major enterprise-resource-planning transformation, won’t lend themselves to going live in little pieces, agile may mean releasing code to the testing environment—but not the production environment—every day. Agile is a flexible set of principles, not a rigid doctrine. It should be implemented in that spirit.

Offer incentives to middle management. Agile changes the role of middle managers. Eventually, many of the coordinating tasks that have historically fallen to them will disappear. In agile, managers are much closer to the content and the technologies. While they still have some traditional managerial responsibilities, like recruiting and evaluations, they now work in the teams themselves. And on these teams, they are equal to every other member—serving, for example, as a fellow developer. Instead of instructing others, they work as coaches and advisors.

Given these shifts, it’s easy to understand why middle managers would resist the migration to agile: they can see themselves losing control and power. How to avoid this perception? One way is to start getting these managers closer to the front—in both body and mindset—through education, training, and participation in agile conferences and the agile community. KPIs used in measuring a manager’s performance should be tweaked as well. They should encourage the quick development and deployment of features but also tolerate some failures as long as the overall system stays stable. This is much more in line with how agile works.

Develop a digital culture. Migrations from two-speed to all-agile IT won’t happen overnight. And with the war for talent continuing, it’s important to send a message—to current and prospective employees—that agile and the workplace it creates are the company’s future. Hackathons—marathon sessions where teams compete to develop software and even hardware—have been used to foster a fast-moving “think outside the box” culture. (In fact, Facebook’s ubiquitous “like” button traces back to a company hackathon.) The idea is to take steps that let technology experts know that they can stay—and succeed—as technology experts; that, contrary to the old days and the old ways, they don’t need to take a managerial position to make a career at the company.

Establish joint business and IT teams. One of the hallmarks of agile is the cross-functional team, in which members representing the business and IT work together. Migrating to agile means breaking down organizational barriers and fostering communication and collaboration across once-isolated domains. (See The Power of People in Digital Banking Transformation, BCG Focus, November 2015.) Flexibility is crucial here, too. A key tenet of agile is that someone from the business side serve as the “product owner.” But for IT4IT products and tools, such as telepresence, it will make more sense for this owner to come from IT. Once again, the experience and practices already developed on the agile side of two-speed IT can prove invaluable.

Taking Agile Even Further

Unlike two-speed IT, the all-agile model is a long-term solution—and not only for the IT organization. Think about the main principles of agile: short iterations that enable teams to quickly spot errors and react to changes; collaboration in multidisciplinary teams; and progress that remains visible—and tested—as work continues. These are principles that can be utilized to great effect throughout a company, increasing its responsiveness to customers and competitors alike.

Already, we are seeing agile move beyond IT into areas such as product management and marketing, and functions that include human resources and risk management. (See The Agile Marketing Organization, BCG Focus, October 2015.) Spotify and ING are notable examples of companies that are bringing an agile style of working to IT and the business alike. (See “Building a Cutting-Edge Banking IT Function: An Interview with Ron van Kemenade, the CIO of ING Bank,” BCG article, December 2015.)

Today’s businesses are under mounting pressure to get products to market and systems deployed while minimizing risk and delay. Two-speed IT was an important step in gaining experience in new and better ways to do this. Now it’s time to take the next step. A return to a single speed—one based on agile principles—will improve efficiency and outcomes across all technology delivery and, ultimately, across the company. The result: better experiences for customers—and a competitive edge for the business.

Hanno Ketterer is a senior partner and managing director in the Amsterdam office of The Boston Consulting Group and the global leader of the insurance sector in the firm’s Technology Advantage practice. For more than 20 years, he has led large-scale technology transformations and postmerger integrations in the banking and insurance industries. You may contact him by e-mail at ketterer.hanno@bcg.com.

Benjamin Rehberg is a partner and managing director in BCG’s New York office and the global topic leader for software and agile transformation in the firm’s Technology Advantage practice. He has more than 15 years of experience advising financial services clients in IT-related areas. You may contact him by e-mail at rehberg.benjamin@bcg.com.

Christian N. Schmid is a principal in BCG’s Munich office. He is active in the firm’s Technology Advantage practice and is the global operational leader of the Simplify IT topic in the Financial Institutions practice. For the past ten years, he has driven large-scale transformations—including digital and agile transformations, simplification and restructuring programs, and postmerger integrations—for numerous companies. You may contact him by e-mail at schmid.c@bcg.com.

Djon Kleine is a consultant in the firm’s Amsterdam office. A member of BCG’s Technology Advantage practice, he has experience in technology-related transformations involving digital, agile, software-development, and cloud-computing topics. You may contact him by e-mail at kleine.djon@bcg.com.

focus

Simplifying IT to Accelerate Digital Transformation

by Michael Grebe, Sabine Döschl, Christian N. Schmid, and Jonathan Koopmans

Corporate IT functions are being hit with a one-two punch. Digital technologies increase the need to adapt ever more quickly to disruptive new applications and services. Those technologies also create rising customer expectations. For B2C companies, disruptive innovators—such as Airbnb, Simple, and Uber Technologies—continuously roll out new models and services, and each one raises the competitive bar. They also teach consumers to expect more from all the other companies that they do business with. In the B2B sector, innovative entrants such as Alibaba.com and Mercateo enable companies to create entirely new value chains and propositions.

To fight back, traditional companies need to become much more agile, by shortening time to market and delivering IT services more quickly; flexible, by scaling IT volume up and down quickly, for example; and efficient, by reducing IT costs. Simplifying IT is an essential task for any IT organization that wants to stay competitive. It is also a critical foundation for digital transformation.

This article analyzes the causes and patterns of IT complexity, examining two routes that companies can take to eliminate complexity and prepare for a digital transformation that is based on The Boston Consulting Group’s Simplify IT approach. Part of the Smart Simplicity framework, Simplify IT employs six levers that can be applied individually or in combination, depending on any given company’s needs.[1] (See Simplify IT: Six Ways to Reduce Complexity, BCG Focus, March 2013.)

Complexity and Digital Transformation

In a perfect world, IT simplification and digital transformation go hand in glove. Simplifying IT helps a digital transformation succeed: it is far easier, for example, to develop new digital services in a “clean” IT landscape—with a reduced number of applications, interfaces, and technologies—than in a complex one. At the same time, a digital transformation offers opportunities to simplify IT and allows CIOs to take advantage of a significant investment budget along with strong buy-in from the business function and commitment from top management. (See “Digitization and Simplification: Getting the Best of Both,” BCG article, December 2014.)

The CIO of ING Bank put it this way: “We decided that since we were investing a lot in the simplification of our landscape and the consolidation of data centers, platforms, operating systems, and middleware—and since we were looking at a three- or four-year journey anyway—we should do all of the above: build a new digital platform that was cloud-based and API-based, and Web scale at the same time.” (See “Building a Cutting-Edge Banking IT Function: An Interview with Ron van Kemenade, the CIO of ING Bank,” BCG article, December 2015.)

That said, we continually encounter companies whose failure to address IT complexity hampers or substantially slows their digital-transformation programs. This is both unfortunate and unnecessary. We have developed a clear methodology for simplifying IT that is suitable for any kind of company; even companies that are not undertaking a full-scale transformation will reap substantial savings and facilitate the application of digital technologies from a simplified—and more agile, flexible, and efficient—IT operation.

Simplify IT’s Six Levers

The following are the six levers for simplifying IT:

1. Intelligent Demand Management. This helps business units develop a clear understanding of the IT resources—and cost drivers—that are needed to keep day-to-day IT operations running and to take on new projects, so managers can maximize value with informed decisions, effective resource allocation, and efficient use.

2. Application and Data Simplification. Companies look for ways to consolidate and decommission applications and clearly define interfaces—or replace them with less complex alternatives—and to simplify the data landscape.

3. Infrastructure-Technology-Pattern Reduction. Companies embrace opportunities to minimize the number of unique configurations of hardware and software, including middleware and databases, and to automate deployment, such as through standardization.

4. Simplified IT Organization and an Enabled IT Workforce. Companies can remove bottlenecks by trimming management layers, optimizing spans of control, and ensuring an effective interface between the business function and the IT department. They can also ensure that the IT staff is an appropriate size and that staff members have the requisite expertise and digital skills, such as mobile development and user-experience design.

5. Effective Governance and Simplified Processes. Good governance models position IT as a strategic partner of the business unit while streamlining IT processes. IT functions can also leverage agile methods, such as scrum, to speed up application development.

6. A Shared-Services Model and Optimized Sourcing. Companies pool demand for both external and internal resources as needed and establish a sourcing model tailored to the strategy and requirements of the organization.

We recently analyzed some 500 Simplify IT projects and about 50 Simplify IT quick assessments performed by BCG over the past three years. (See Exhibit 1.) The quick assessments were based on a questionnaire that includes quantitative and qualitative questions, from both an IT and a business perspective, on each of the six levers. Each response was given a complexity score ranging from 0, for low complexity, to 100, for high complexity. The analysis helps companies that are evaluating their IT simplification needs to focus on three key issues:

- The typical root causes of the complexity, both within the IT department and at its intersection with the business function.

- The complexity patterns that emerge and need to be addressed.

- The extent of the effort needed to eliminate non-value-creating complexity and become more agile, flexible, and efficient—as well as speed up digital transformation.

Common Causes of IT Complexity

Our analysis pinpoints four major root causes of IT complexity, each growing out of companies’ particular situations.

Fast Growth. To facilitate rapid business expansion at fast-growing companies, IT often employs hastily constructed, short-term, or siloed solutions that do not scale easily. New functionalities are implemented quickly, but they are not always efficient because they increase the number of point-to-point connections or satellite applications. This is especially true for companies that rapidly develop decentralized digital services and fail to align them with existing architecture. (See Recasting IT for the Digital Age, BCG Focus, March 2016.)

Incomplete Postmerger Integration (PMI). As companies grow through acquisition, IT functions are not always fully integrated during PMI, which can lead to significant redundancies. These redundancies, and the frequent absence of automatic interfaces between systems, also make it difficult to implement digital end-to-end services.

Low Business-IT Collaboration. Many companies—however unintentionally—grow IT complexity by employing an old-fashioned business-IT collaboration model. This is especially true of companies that historically have considered IT as a support function rather than as a partner to the business. Such companies tend to wait until late stages of a project before involving the IT department. They also follow the classic waterfall development process. What’s more, the business function, which pays for IT services, considers itself the sole decision maker and believes that it can sidestep or ignore process and architectural guidelines. As a result, new requirements disrupt existing application landscapes and processes, leading to a wide variety of different technologies employed for similar functionality. We also find that in such organizations the IT department does not have up-to-date skills and has not adopted new ways of working, such as agile—two shortcomings that undermine its ability to add value and be perceived as a partner.

Cost Pressures. As growth slows or markets plateau, companies come under strong cost pressures that are reflected in both the business and the IT functions. This can lead to underinvestment, meaning that business requirements tend to be met at the lowest cost, with no consideration given to related complexity increase and project follow-up costs. At the same time, digital transformations can lead to even higher IT costs, at least in the near term.

Common Patterns of IT Complexity

These causes lead to three common complexity patterns: data landscape complexity; poor business-IT governance and IT process complexity; and application and IT-infrastructure landscape complexity. (See Exhibit 2.) Not surprisingly, the extent and nature of each varies by both company and industry. Financial-services companies have greater experience with managing these types of complexity and thus handle them better. For example, most banks and insurance companies, which started to renovate their IT landscapes several years ago, score 13% better overall than other companies.

Data Landscape Complexity. Companies struggle more with data landscape complexity and data landscape management than any other issues. Approximately half of the companies in our sample scored lowest in this area. Data simplification requires uniform definitions and common standards for enterprise-wide data objects (pieces of information that are relevant to the entire company), but we found that such definitions and standards are often only partly specified; this was the case for 45% of companies. In addition, only about 10% of companies have established a “single source of truth” for their data, that is, unique storage spaces, or databases, for enterprise-wide data objects. The lack of standards and multiple databases can lead to serious problems: inconsistent data, such as data from multichannel customer engagements, hobbles the ability of data analytics programs to analyze customer behaviors and deliver digital services.

Many companies also lag in establishing clear data-management and governance policies and practices, such as defining a clear target-data landscape and establishing a central and overarching data-management role within the organization. These are important foundations for data-simplification programs. Good practices include implementing state-of-the-art data quality and security management processes and prioritizing the performance of such processes on both the business and IT agendas. Clear data-management and governance policies are not IT-only tasks; the entire business organization needs to own them.

Business-IT Governance and IT Process Com-plexity. The complexity of business-IT governance and IT processes was rated high or medium (at least 50 on our 100-point scale) by 60% of the companies that participated in the assessment. This pattern is common in companies with low business-IT collaboration and appears mainly in organizations where IT is seen as a service provider—a “butler of the business”—rather than as a strategic partner. The business-IT interface usually does not follow an established process (much less best practice); instead it takes the form of a complex set of ad hoc and decentralized interactions. This typically results in redundancies and workaround solutions. New business functionality is not weighed against the incremental complexity it inevitably adds to the application landscape and IT processes, such as numerous interfaces and increased testing efforts, which reduce speed and agility for the company as a whole.

Complicating matters further, if IT development processes are perceived as slow and bureaucratic when it comes to implementing new business requirements, impatient businesses try to either shortcut those processes by reaching out directly to developers to get things done more quickly or start building their own solutions, a phenomenon often referred to as “shadow IT.” This multiplies the complexity increase and compounds the problem, especially when solutions that do not comply with the overall IT architecture guidelines are employed, thus undermining digital speed or creating a security hazard—or both.

In addition, many companies do not make adequate use of agile working methods and processes. Traditional software-development methods, such as the waterfall model, tend to dominate. Although, in our experience, three-quarters of companies use some agile methods, the vast majority have only just started agile pilots: only 5% or fewer of the development teams that we encounter work in an agile mode. Introducing agile development methods and ensuring that a dedicated business owner works with the development team can significantly speed up IT delivery. (See The Power of People in Digital Banking Transformation, BCG Focus, November 2015.) Some companies might also consider applying digital to simplify IT processes—for example, by deploying code to cloud environments, fully automating delivery processes, or using robotic-process automation to monitor data center operations.

We have seen companies achieve high productivity gains through joint business-IT efforts to improve IT processes, speeding up incident management and application development, for instance, by 25% and 30%, respectively. New methods of collaboration between development and operations (such as DevOps) can speed time to market by 20%. Improving business-IT governance and thereby strengthening the role of IT and its relationship with the business—for example, by ensuring that IT has a strong role in new product-development processes and in prioritizing the IT project portfolio—is yet another way to eliminate this cause of complexity.

Application and IT-Infrastructure Landscape Complexity. The third major problem area is the application and IT infrastructure landscape, where complexity constrains companies’ ability to implement digital innovations quickly and efficiently. Nearly two-thirds of the companies in our sample evidenced weaknesses in such capabilities as developing application program interfaces and microservices for connecting new services or fostering methods such as continuous delivery (the automation of the entire application development and deployment process) or deployment.

Specific IT infrastructure capabilities required for digital services and innovations include the adoption of continuous delivery, the availability of real-time-capable infrastructure components, and the use of adequate cashing and buffering systems, such as buffer databases, to reduce latency time and processed-data volume. These are the capabilities that enable digital-native companies to use continuous deployment and other methods to release new services or software more than 100 times a day. Older, more established companies can also develop and use such resources to reduce the time for provisioning new software from weeks to days or even minutes.

One European bank furthered its digitization efforts by overhauling its application infrastructure. It designed an IT application architecture around three main domains that corresponded to three key functions: client relationships, client services, and transaction processing. The bank was able to develop applications for each domain much faster by using the most appropriate methodology in each case. It reduced the cost of developing applications by one-third while also shortening its development schedule.

Eliminating Complexity

Companies can undertake two types of IT simplification projects. The first and most far-reaching consists of comprehensive IT-simplification programs, which we define as those applying three or more Simplify IT levers. These have the potential to cut up to 30% of the total IT cost base and to substantially facilitate digital transformations by establishing a clean and decoupled IT-application landscape and by automating IT processes.

This approach, which usually involves a mul-tiyear journey, entails a revolutionary change of the entire corporate IT function: technology, processes, and ways of working. Removing legacy systems makes room for state-of-the-art and more efficient technologies, software, programming languages, and infrastructure. Broader usage of cloud platforms, systems, and software is common. Such large-scale simplification programs generally aim to facilitate and enhance digital transformation. They are often coupled with agile transformations, at least for nonlegacy areas. To ensure maximum impact, companies undertaking full-scale simplification programs still need to address the complexity issues at the business-IT interface by forming cross-functional, feature-oriented teams to cut coordination overhead, reduce idle time, and increase agility and flexibility for application delivery.

The second approach is to focus on a particular problem or need that requires one or two IT simplification levers. Despite its relatively narrow scope, this undertaking can still have a big impact. Some application and data-landscape simplification projects have achieved savings of 15% to 20% of the full IT cost base; others have cut up to 50% of the applications-related cost base as companies were able to reduce their number of applications by 40% to 70%. In addition to the direct cost benefits, a simplified application landscape also makes it easier to connect new digital services or to implement end-to-end process automation.

Similarly, projects that focus on simplifying the IT infrastructure typically achieve cost savings of 5% to 15% of the IT budget, and companies have been able to reduce the cost of physical servers by up to 80% by increasing server virtualization and decreasing the number of different configurations. These efforts also foster the application of approaches such as DevOps and continuous delivery, which facilitate the rapid implementation of digital innovations and promote quick reactions to changing customer requirements. IT organization projects, which are frequently undertaken in combination with changes to the business-IT governance and processes, also typically achieve cost savings of 15% to 20% of the IT budget. Companies benefit from fewer skills-related bottlenecks, increased decision-making speed, and higher user satisfaction as well.

An essential success factor for all types of simplification projects is strong buy-in from business management, which must commit to actually eliminating complexity in application and data landscape, often through decommissioning redundant or rarely used functionalities.

A proven IT-simplification methodology will increase agility, flexibility, and efficiency—all key success factors in digital transformation. The challenge can appear daunting, but the tools to overcome it are available. Our process starts with the Simplify IT quick assessment—a sound analysis of the sources of complexity and an evaluation of which levers can be brought to bear. The result can be both significant near-term savings and a simplified IT landscape, which not only allows for faster and more efficient implementation of new digital services but also accelerates digital transformation.

Note

1. For more information about Smart Simplicity, visit our website at http://on.bcg.com/SmartSimplicity.

Michael Grebe is a senior partner and managing director in the Munich office of The Boston Consulting Group. He is active in the firm’s Financial Institutions and Technology Advantage practices and is the global leader of the Simplify IT topic. For nearly 20 years, he has helped design and deliver large-scale transformations—including postmerger integrations, core banking-system migrations, restructuring programs, and IT simplification programs—for numerous financial institutions. You may contact him by e-mail at grebe.michael@bcg.com.

Sabine Döschl is a principal in the firm’s Munich office and the operational leader of the Simplify IT topic in the Financial Institutions practice. As a core member of BCG’s Technology Advantage practice, she has driven large-scale IT transformations—including postmerger integrations, IT strategic alignments, IT simplification programs, and IT outsourcing—for clients in various industries. You may contact her by e-mail at doeschl.sabine@bcg.com.

Christian N. Schmid is a principal in BCG’s Munich office. He is active in the firm’s Technology Advantage practice and is the leader of the Simplify IT topic in the Financial Institutions practice. For almost ten years, he has driven several large-scale transformations—including digital and agile transformations, simplification and restructuring programs, and postmerger integrations—for numerous clients. You may contact him by e-mail at schmid.c@bcg.com.

Jonathan Koopmans is a principal in the firm’s Amsterdam office. He has more than 15 years of experience in designing and managing complex IT and digital transformations, focusing on the financial services and insurance industries. He is the global leader of the Simplify IT topic in the Insurance practice. You may contact him by e-mail at koopmans.jonathan@bcg.com.

viewpoint

Agile Development’s biggest failure point—and How to fix it

by Michael Sherman, Stephen Edison, Peter Hildebrandt, David Ritter, and Benjamin Rehberg

Agile development has gone mainstream, but the software development methodology that helped hundreds of Silicon Valley startups explode onto the scene has delivered decidedly mixed results for more traditional organizations. In most cases, there is a critical point of failure: the product owner.

A vital part of agile development, product owners are responsible for aligning the business, IT, and customer perspectives, and they have extraordinary power to create value. They are the people who understand the customer’s needs; know how to increase revenue, cost-efficiency, and customer satisfaction; and see enough of the big picture to deliver a highly competitive product. Too often, however, this critical role is filled by a safe pair of hands when what’s needed is someone who can drive a process forward, maintain momentum and speed, and create meaningful value. If the product owner lacks these critical capabilities, the business pays a heavy price in wasted capital, customer dissatisfaction, and, ultimately, stagnation.

So how do business and IT leaders identify and cultivate product owners? And how do they create an atmosphere that fosters their effectiveness? Product owners should be the entrepreneurs in the organization: passionate about their mission, motivated to create measurable value, and supported by a culture of empowerment, transparency, and trust. Companies that put the right people in this position and make their success a priority will see immediate results.

What to Look for in a Product Owner

The success of agile largely depends on the strength of product owners. (For a detailed discussion of agile principles, see “Five Secrets to Scaling Up Agile,” BCG article, February 2016.) By and large, developers will build whatever they’re told to build. If they make a mediocre, overly complex, or exorbitantly costly product, the fault most often lies with the product owner. The product owner is the central node where creative ideas come to thrive and get to market quickly—or to wither and die.

Companies commonly make one of two mistakes when selecting a product owner. Often they tap a junior employee with limited experience and therefore a limited understanding of how the project fits into the larger mission. Product owners need enough seniority to inspire and motivate peers across multiple business units. By earning the respect of teams in customer experience, enterprise architecture, and risk and compliance, for example, the product owner can help ensure that projects move smoothly without costly bottlenecks. Other companies err in the opposite direction, selecting a senior executive who is too harried to devote adequate time and may not adapt well to the highly responsive, iterative nature of agile development.

So what should companies look for when appointing product owners? In our view, the key is to find people who think and behave like entrepreneurs. Consider how successful entrepreneurs behave in pitch meetings with venture capitalists. They have passion and enthusiasm for their products and ready answers to big-picture questions: Who are your customers? What problem are you solving for them? How will you make money? How quickly will you become profitable? These questions separate the forward-thinking product owners from the pencil-pushing project managers.

When selecting a product owner, company leaders need to think like a venture capitalist and identify an individual who can be trusted to steward a major investment. The ideal candidate will exhibit four critical qualities:

- Collaborative Leadership. Product owners must have the maturity and confidence to convey a vision for the product, solicit continuous feedback, negotiate tradeoffs, and drive the process forward. They must ensure that decisions are fact-based and that key stakeholders are appropriately consulted and informed.

- Business Sense. Effective product owners pride themselves on having a comprehensive understanding of both the product and the customer. It’s not enough to create something cool. They stay focused on prioritizing the projects that will deliver tangible and measurable value for users.

- Creativity. The best product owners work lean and mean, inspiring their teams to tackle challenges, setbacks, and budget constraints with smart workarounds and creative solutions.

- Passion. Product owners need to be passionate about the project and fully dedicated to its success. They should have a bias for action, although their decision making must always be guided by empirical data and input from key stakeholders.

The most successful product owners will be visionaries, though not necessarily in the mold of a Steve Jobs. They do need a strong vision for how the product will create value, the flexibility to adapt on the fly, and a willingness to learn and grow along with the product. However, much of their success comes from simply managing a sound process: conducting market research, understanding the customer’s needs, identifying where the product will create the most value, prioritizing the most important features, testing ideas, capturing customer feedback, and continuously refining their vision over time. The best product owners temper grand visions with practical decisions that create measurable value. (See the sidebar, “Improving the Customer Experience for Airline Travelers.”)

The Supreme Importance of Accountability

If product owners are treated as entrepreneurs, it’s only natural that they be evaluated on the basis of outcomes. Executives have every right to expect a tangible return on investment. But beyond the bare minimum of delivering features on time and within budget, it’s surprisingly rare for product owners to be held accountable for their products’ performance. Just as entrepreneurs must develop a pitch that persuades investors to give them money, product owners should be required to create a business case that explicitly demonstrates the value of new digital initiatives.

- Establish a business case. Prior to receiving funding, product owners need to establish metrics that can be used to evaluate a project’s success. Questions the business case should address include: What value can be captured? Which customer segments will be targeted? How can we secure quick wins? How can we reduce risk? The business case should serve as a living, breathing document—sketching out an initial roadmap that will be continuously revised and improved. A mammoth tome that requires exhaustive effort upfront and then is forgotten in a drawer is worthless.

- Track performance. Because this is a new way of working for many companies, product owners need to provide 360-degree transparency into all aspects of a product’s journey from ideation to maturation. As the product evolves, the product owner must provide detailed updates on progress, outcomes, and obstacles (through metrics and open sharing of the team’s work). A product owner’s performance should be measured on real-world outcomes, including adoption rate, customer satisfaction, revenue gains, and cost savings. Vanity metrics such as page views or downloads are merely a distraction. The critical metrics will focus on where the product has performed well—and how quickly it produces results.

When product owners are held accountable for value creation and employees’ incentives are aligned around this point, agile teams are well positioned to produce tangible results—for customers and the business.

Creating a Culture Where Product Owners Thrive

Even the strongest product owners will struggle in a company that doesn’t accommodate an agile mindset. Traditional businesses need to embrace a new way of working based on speed, experimentation, and adaptation.

- Give product owners a long leash. To be successful, product owners need the authority to make pivotal decisions independently and in the moment. Senior executives should have total confidence in their product owners. In a fast-moving agile environment, product owners may be asked to make multiple decisions every day on product releases, prioritization of workflow, feature tradeoffs, and much more. If required to consult with a committee or a senior executive before handing down a decision, the product owner will serve as a mere functionary. Great product owners do not make all the decisions; rather, they are collaborative leaders who get the best from their teams and other stakeholders and drive an inclusive, data-driven decision-making process. But they are also unwilling to let a project stagnate. Product owners should be fully empowered to experiment, set priorities, and guide the agile development process on the ground.

- Celebrate fast failures. For product owners working in an agile culture, the goal is to experiment boldly, iterate quickly, and use feedback from each release to improve the product. In a typical company, failure is often perceived as shameful or catastrophic. But it is a natural byproduct of risk taking and should be celebrated. The trick is for product owners to fail quickly. Fast failures can be highly instructive. With rapid customer feedback, the cause and effect are clear, and product owners can make adjustments accordingly. They can shift course and reallocate resources in response to real-world data. In addition, when product owners drive quick iteration and incremental adaptation, a fast failure has limited consequences for the business. Although it takes sustained effort to create a culture that celebrates fast failures, companies that embrace it will be rewarded with higher levels of innovation from their product owners.

As entire industries are disrupted by fast-moving tech startups and innovative business models, agile methodologies can help companies stay ahead of the competition. But agile methodology on its own is not enough. Agile development needs to be driven by a passionate and entrepreneurial product owner backed by a culture of transparency, empowerment, and trust.

Michael Sherman is a partner and managing director in the Dallas office of The Boston Consulting Group. He is a leader of the Technology, Media & Telecommunications practice, and he leads the firm’s software sector in North America. You may contact him by e-mail at sherman.michael@bcg.com.

Stephen Edison is a principal in the firm’s Dallas office. He serves technology, media, and telecom clients. You may contact him by e-mail at edison.stephen@bcg.com.

Peter Hildebrandt is a principal in BCG’s Los Angeles office. He specializes in technology and operational issues in the financial services industry. You may contact him by e-mail at hildebrandt.peter@bcg.com.

David Ritter is a member of the firm’s Technology Advantage; Technology, Media & Telecommunications; and Marketing, Sales & Pricing practices. He advises clients on using technology for competitive advantage, developing effective IT and data strategy, and optimizing IT infrastructure. You may contact him by e-mail at ritter.david@bcg.com.

Benjamin Rehberg is a partner and managing director in BCG’s New York office. He is a core member of the firm’s Financial Institutions and Technology Advantage practices, and the global leader of the software & agile topic within the Technology Advantage practice. He specializes in technology and operational issues in the financial services industry. You may contact him by e-mail at rehberg.benjamin@bcg.com.

Focus

Bridging the Trust Gap

The Hidden Landmine in Big Data

by John Rose, Frederik Lang, and Alexander Lawrence

This is the first in a series of articles exploring what really matters for organizations that collect and use consumer data.

Big data has the potential to be both friend and foe. The Boston Consulting Group conservatively estimates that trusted uses of big data and advanced analytics could unlock more than $1 trillion in value annually by 2020. (See “The Value of Our Digital Identity,” BCG article, November 2012.) However, recent BCG consumer research has uncovered a previously hidden obstacle to successfully unleashing this enormous opportunity: data misuse.

Data misuse does not refer to a use of data disclosed in an agreement that no one reads when signing up for a credit card, mobile phone, or social media service; it is not even about whether a use actually causes harm to consumers. Data misuse occurs when consumers are unpleasantly surprised upon learning that data about them has been collected or that it has been used in new ways—that is, outside of the original purpose for which it was gathered—and when they perceive such practices to be potentially harmful and feel that the company should not engage in them. (An example would be when a company originally collected data in order to complete a transaction or ensure that potential customers are good prospects but is now using it for marketing purposes.) Our research suggests that consumers’ reaction to data misuse—defined in this way—can cause them to reduce their spending with a company by about one-third.

Executives will not mitigate data misuse by writing even longer and more complex legal documents for consumers to ignore, or by working even harder to ensure that companies don’t run afoul of regulations and legal agreements. Instead, company leaders at the highest levels must develop new ways to manage and use data, rather than confining the discussion to legal or IT, as it is at most companies. Even organizations that use data for completely legal and fully disclosed reasons are on a collision course with their customers. The steps companies take now to assess and address this risk will confer significant, long-term, and sustainable competitive advantage and head off the looming threat to their earnings performance.

The Weakened State of Consumer Trust

Issues of privacy and trust continue to be at the top of consumers’ minds. In fact, feelings about these issues have intensified over time. When we surveyed consumers across 20 countries and multiple generations in 2013, it was clear that they all cared deeply about the expanding use of “their” data. (See The Trust Advantage: How to Win with Big Data, BCG Focus, November 2013.) For instance, in every generation and in most countries, consumers were five to ten times more likely to share personal data with an organization if they trusted that the data would not be used to harm them. Moreover, 83% of US consumers agreed that they needed to be cautious about sharing personal data online—again with only small differences across generations and across most of the countries surveyed.

A new BCG survey of 8,000 consumers in the US and five European countries shows that these concerns remain at high levels in most product and service areas. Consumers who say they are concerned about the sharing of personal data online increased slightly from 83% to 86% in the US. Four out of five US millennials are similarly concerned. Consumer willingness to allow companies to use data in new ways remains roughly five to ten times higher among those who trust a company to prevent harmful uses than among those who do not.

Of greater concern, nearly half the consumers we surveyed believe that companies are neither being honest about their use of data nor taking adequate steps to protect it. In fact, only about 20% of consumers across all the countries surveyed trust companies to “do the right thing” with their data, and approximately 30% across all the countries surveyed believe that companies will not do the right thing. This is particularly troubling given that 71% to 79% of the surveyed consumers said they would be unlikely to share or let data about them be used by a company they did not trust.

Because consumers are already skeptical that companies will be honest about, protect, or otherwise do the right thing with existing uses of data about them, they are primed to view most new uses of data with significant distrust—and, even worse, as probable misuses.

The Cost of Crossing the Line

Companies face a hefty penalty for doing the wrong thing with consumer data. They lose access to five to ten times the data they could have used had they excelled at creating trust. What’s new from our recent research is the real revenue impact of that loss of trust.

Consumers are now demonstrating that they will “vote with their feet”—stopping or significantly reducing spending—if they believe that a company has misused data about them or other consumers. As noted, this can have dramatic results: in the US, customers who are aware of and concerned about a data misuse reduce their spending by about a third in the first year. Overall, that means a 5% to 8% loss of total company revenues in the first year after these customers have stopped or reduced their spending, dropping to a 3% to 5% loss in year two. (The overall revenue loss is a function of the much larger size of the total user population.) However, as consumer awareness and concerns increase, we believe that data misuse has the potential to cut overall revenues by 10% to 25% in year one, dropping to 5% to 15% in year two. While differences exist between the US and the European countries surveyed, the potential revenue losses are comparable.

This consumer reaction to data misuse is significantly greater than the reaction to data breaches or other cybersecurity events. In fact, 25% more US consumers have reacted to a data misuse than to a data breach by stopping or reducing their spending.

The impact of data misuse on a company is a function of several factors:

- The Size of the Population Affected. Our survey suggests that 20% of US consumers believe they have been affected by data misuse. This group comprises 14% who are aware of a misuse involving data about them (those “directly affected”) and 6% who are aware of a misuse involving data about other consumers (those “indirectly affected”). (See Exhibit 1.) We expect this combined population to significantly increase over the coming years.

- The Behavior of the Affected Population. Of those surveyed in the US who believe they were directly affected, 76% took some sort of action: half cut their spending by 56% on average; the other half stopped using the company’s services entirely. Combined, the actions of these two groups of directly affected consumers led to a 33% drop in the company’s revenues in the first year. Of those indirectly affected, 71% took some sort of action: three-fifths reduced their spending by an average of 48%, while two-fifths stopped using the company’s services entirely. The actions of these two groups of indirectly affected consumers led to a year-one reduction in spending of 30%.

- The Passage of Time. By the end of the first year, many of the consumers who had abandoned the company returned, reducing the impact in year two. In the US, only 8% of the reduction in revenues resulting from the actions of the directly affected population, and 7% of the reduction in revenues resulting from the actions of the indirectly affected population, endured past the end of year two. (See Exhibit 2.) By year three, we would expect the balance to be restored—assuming the company does not again misuse customers’ data.

- The Geographic Area Affected. This is not a geographically isolated phenomenon. As noted above, companies experience similar levels of economic damage from data misuse in the US and Europe. More than one-third of revenues from directly and indirectly affected consumers will be lost during the first year, with the reaction slightly harsher in Europe than in the US. About one-tenth of revenues from these consumers will be lost during the second year, with differences between Europe and the US beginning to level out. (See Exhibit 3.)

Why the Cost of Crossing the Line Will Increase

A worrisome trend—and one that will increase the potential impact of data misuse—is the increasing attention that the phenomenon is receiving in traditional and social media. One example is the uproar that greeted Uber’s disclosure of the uses of its “God view” capabilities. This software functionality allows the company to track the location of drivers in real time and tie that data to their passengers. While the feature can be seen as relatively benign in the context of fleet management, many considered Uber’s internal data-sharing practices to be an invasion of privacy (there were even allegations that Uber had broadcast the data on giant screens at parties in cities where it was launching a new service). It is worth noting that much of Uber’s collection and use of data has been within the bounds of both the law and the company’s privacy policy, as is the case with many perceived incidents of data misuse.

As coverage of data misuse becomes the new normal in traditional and social media, the number of consumers who become aware that data about them is being collected and used in new ways, who consider those uses to be potentially harmful, and who feel that companies should not engage in such practices is likely to increase—and with it the economic impact of any spending reductions that consumers may make in response.

Few companies see the landmines that are looming under the surface of their attempts to use data in new ways. They overlook these risks because their focus is on data privacy—a necessary requirement in an intense and rapidly changing regulatory environment. In the process, however, they are overlooking the needs of their customers, who define data misuse on the basis of their perception of right and wrong, not on the basis of regulatory rules or legal agreements. Make no mistake, this strategic blindness will lead to a painful loss of revenues and customers.

As we will discuss in the next article in this series, most companies are poised to fail in their pursuit of new data uses. In fact, they have set themselves on a recklessly conservative path, which is leading them to unnecessarily limit their own opportunities while at the same time ensuring that they act in ways that engender the negative consumer reactions they hope to avoid. By focusing privacy and data stewardship practices on the regulations and guidelines that have arisen around big data and advanced analytics—many of which were designed to protect consumers—companies are creating a gap between themselves and their customers. The economic harm that is likely to result is something that few if any C-suite executives can afford to disregard.

Instead, companies need to fundamentally change their approach to data stewardship. They can build consumer trust by making significant improvements in the four main dimensions of robust data stewardship: internal policies and practices; current and new data usage; transparency about current practices; and usage-specific notification and permissions. Adopting best practices in these areas will not only help companies avoid the pitfalls of perceived data misuse but also enable them to expand the range of opportunities they can pursue. Ultimately, companies will foster a broader and deeper level of consumer trust.

Next up in this series: a detailed examination of current business practices and the disconnect between consumers and companies over data privacy.

John Rose is a senior partner, managing director, and fellow in the New York office of The Boston Consulting Group. His focus is the risks companies face from the use of consumer data and the steps they can take to create sustainable competitive advantage through effective data stewardship. You may contact him by e-mail at rose.john@bcg.com.

Frederik Lang is a consultant in the firm’s Copenhagen office. He has expertise in sales, cost optimization, and strategy across a range of industries. You may contact him by e-mail at lang.frederik@bcg.com.

Alexander Lawrence is a consultant in BCG’s New York office. Before joining BCG, he worked as an attorney specializing in mergers and acquisitions. You may contact him by e-mail at lawrence.alexander@bcg.com.

Q&A

Changing the Game with a Data Lake

An Interview with Centrica’s David Cooper and Daljit Rehal

UK-based Centrica is a leading multinational energy and services company. Its brands include British Gas and Hive. BCG’s Jon Brock and Sesh Iyer recently spoke with Centrica’s David Cooper, group CIO, and Daljit Rehal, strategic systems director, about how the company established a data lake at British Gas as part of its efforts to transform Centrica’s data analytics capabilities. Edited excerpts from the discussion follow.

David and Daljit, can you briefly describe the circumstances that led to Centrica’s decision to establish a data lake at British Gas?

Cooper: When we began this project, about three years ago, British Gas’s information architecture and supporting infrastructure were largely patchwork in design. Essentially, British Gas had assembled a collection of technologies over time, with limited thought given along the way to how they might be forged into a coherent information architecture. Simultaneously, we were facing increasing demands internally. British Gas wanted to do more with the data it had, but our systems were struggling to provide the necessary access and functionality.

Rehal: Also, we were at a point where we needed to make decisions about British Gas’s data warehouses. They all needed hardware upgrades and new appliances. And we hadn’t yet begun to dip a toe in the world of big data; we were just trying to do the basic business reporting and analytics. We knew that we would have to find a way to accommodate the substantial growth in our data volume that would result from smart metering, connected homes, feeds from our website, sensors in various devices, and so forth. In addition, it occurred to David and me that neither of us had ever met anyone in an IT organization who said, “I’m really happy with my data warehouse.” Bottom line, we realized that we needed different technologies and a different approach.

What are some of the key advantages of the technology you’re using now over traditional data warehouse technologies?

Rehal: Some of the most visible advantages are on the cost front. A lot of the new technology can run on cheaper hardware. It can also be scaled in very small increments. So there can be material cost savings.

Cooper: Another big advantage, cost-wise, is that a lot of the new technology is open source. So we can avoid such things as expensive vendor lock ins.

What are some of the critical functional advantages of the technology?

Cooper: The data lake holds raw, not summarized, data. It also accommodates more frequent updates than the traditional data warehouse. In the past, with a traditional data warehouse, it was routine to have delays of 48 hours, and potentially much longer, between the time something occurred and when the data was actually stored. With the data lake and its more than 200 servers, we’ve removed the traditional bottlenecks. We can take and perform analytics on data from our source systems up to four times a day.

Rehal: The data lake is also inherently unstructured, so it accommodates a wider mix of data and data types—it’s essentially a dumping ground for data. You can populate it with whatever you want to, and there’s no need to spend months and months creating engineering structures to accommodate it. We’ve got everything in our data lake, from GPS data for the company’s vans to smart-meter readings for customers to data from our billing platforms. The data from most of British Gas’s systems is in there already, and we aim to add the rest as soon as possible. We’ve also designed a high level of data security into the lake; we’ve had that in place from the inception.

You’ve engaged in large-scale, complex transformation initiatives within British Gas, focused on such things as billing and customer relationship management systems before. Did you approach this effort differently?

Cooper: Our approach toward this differed in several critical respects. First, we didn’t have the benefit of being able to study the examples of other major companies that had tried this—no FTSE companies had attempted it. So we started in much more of a proof-of-concept mode. We needed to determine if this was something that we could actually do, given the huge amount of complexity. Removing all of a major company’s historical information systems and replacing them with a data lake isn’t simple. If it were, everyone would be doing it. We ran our first big proof-of-concept test on half a dozen Raspberry Pi computers clustered together. From there, we graduated to PCs and then to servers.

Rehal: There were other significant differences from our normal approach. Most project management methodologies, whether agile or waterfall, start with a requirement-scheduling exercise and then proceed to developing a solution. Our approach here was radically different because we didn’t need to determine requirements—we were going to take all of the data, since doing so wouldn’t cost us that much more than just taking part of it. We’d take all of it and worry about whether people found it useful later. Hence, we didn’t need to schedule requirement-capture workshops and so forth. This was a radical change to our way of thinking and approach.

How do you manage data governance? That seems like it could be a sizable challenge, given the volume of data in the lake.

Cooper: The program itself acted like an ignition point for the business to start thinking, or rethinking, about data governance. Through acquisitions and considerable organic growth, the company had forgotten much of what it probably knew at one point. We had to start again, with a new set of data owners and the identification of dedicated data stewards. In terms of metadata management and processes, we developed our own solutions and are in the process of rolling them out.

What are some of the major business and IT cost benefits of the data lake project so far?

Cooper: There have already been significant business benefits. We now have the foundation in place to enable the business to do essentially all of the things it wants and needs to do with the tremendous amounts of data, including smart data, that it’s accumulating daily. These are capabilities that didn’t exist before that can make a meaningful difference to the business. An example is the enhanced forecasting ability that the company now has. Even a modest improvement in, say, the ability to estimate likely power and gas usage across the customer base over a given period of time can translate into material economic benefits for British Gas.

A second example of a business benefit enabled by the data lake is the new functionality we’ve been able to deliver to our people in the field. In the past, when one of British Gas’s engineers was making a service call to a customer’s home, he or she would have access only to the information necessary to perform the specific task required. Now, through the apps that we have created for them, the engineers have access, via their handheld and mobile devices, to information such as which other British Gas products or services this particular customer has and the history of previous customer contacts. This greatly enhances the engineer’s ability to make real-time decisions that can enhance the company’s level of service.

There are numerous other benefits to our business users as well, including such things as better ability to gauge and attribute performance and identify revenue leakage. And we’re at the tip of the iceberg, as the prospect of real-time pricing is right around the corner. So we expect that the business benefits will continue to unfold.

Rehal: In terms of IT cost benefits, many of those are realized early owing to the cost avoidance of having to buy bigger data warehouse appliances, as mentioned. The data lake has also helped us save money by enabling us to reduce our storage footprint and decommission a lot of redundant storage.

We’ve also benefited on the cost front by bringing processing to the lake—that is, migrating certain analytical platforms, such as SAS, to it—rather than extracting the data and taking it to processes outside the lake. This has had the added benefits of curbing data leakage and giving us greater opportunity to control the data and make it more secure.

Ultimately, because of the complexity of our estate, we were able to justify the entire data lake investment on the basis of IT cost savings. That was much easier than trying to do so on the basis of various business units’ individual business cases.

Have you encountered any significant cultural challenges as the initiative has proceeded?

Cooper: The resistance from both the business team and a lot of our IT people was enormous initially. People were comfortable with what they were familiar with. This was particularly true for some of our IT staff, especially those with the longest histories in data. These individuals had amassed certain qualifications and experiences over their careers and viewed the pending changes as a threat to their personal worth. Some also thought that the initiative would never work and would end in tears. But once the project started to succeed, they began to align.

Rehal: There was also strong resistance initially from the business units, especially some of the leadership, who were very concerned with the particular choice of technology solutions. To win them over, we had to get them thinking about outcomes. One of those outcomes, we pointed out, was potential independence from the IT unit. We asked them: Do you want the world of self-service? Do you want to be free of the huge waterfall-type governance process you’ve been dealing with?

This worked in many cases, though ultimately we encountered two different types of communities. There were people who said, “I just want the data. I don’t want anything else from you. I’m clever, I’ve got clever people on my team, I can do the analysis.” For those people, the notion of autonomy from IT was a very easy sell. There was another group within the business units, though, that was so hooked on the existing IT that it would not accept the idea until the finished product was delivered, with all the requirements captured and the designs finished.

Cooper: Make no mistake, the leap from the old environment to the new one is, in fact, substantial. When you look at some of our traditional reports for simple things like the number of customers who received products and services from British Gas today versus yesterday, the query, written in SQL, is huge. Today, that can be replaced with a small Java program that is a fraction of the length and is instantly reusable. This is a substantially different universe, and the transition was hard on some individuals. And again, there were two camps. There were some who said, “OK, this is an opportunity for me to learn something new.” There were others who said, “I don’t want to be part of this; I’m going to go somewhere else.”

So yes, there have certainly been cultural challenges. We’ve worked through them with single-mindedness and by building on our successes.

Are there any lessons you’ve taken from the journey so far that you think could be particularly valuable for other companies contemplating such a move?

Rehal: You will probably face considerable skepticism and opposition, so have a clear vision and be brave!

Where do you expect Centrica to be in several years as the journey continues?

Rehal: I’d expect that in, say, two or three years’ time, as we continue to expand what we’re doing with British Gas across the rest of Centrica, our information architecture will be relatively simple—simple, clean, and with one copy of the data rather than several. Our costs will be lower as a result. We’ll have modern, cutting-edge technology and people who love working on it. Our more effective use of our data will continue to unlock opportunities for the business.

Thanks, Daljit and David.

Jon Brock is an associate director in the London office of The Boston Consulting Group. He is a member of the Technology Advantage practice and a specialist in IT strategy. You may contact him by e-mail at brock.jon@bcg.com.

Sesh Iyer is a partner and managing director in the firm’s Washington, DC, office. He is a member of the Technology Advantage; Technology, Media & Telecommunications; and Energy practices. You may contact him by e-mail at iyer.sesh@bcg.com.

Industry spotlight

Ensuring Digital Readiness in Financial Services

by Hanno Ketterer, Heiner Himmelreich, and Christian N. Schmid

Across industries, digital innovators are pushing customer service standards higher every day, offering ever-expanding arrays of cutting-edge, digitally enabled services and features. Prominent examples include Netflix’s low-latency, high-availability offering; Amazon’s real-time personalized offers based on customer profiles; and Uber’s convenient, intuitive user interface.[1]

These new offerings and enhancements are driving customers’ expectations higher for all businesses, including banks and insurers—and these institutions are being forced to respond. The need for banks and insurers to improve their digital capabilities is becoming even more urgent with the emergence of new market entrants and services (such as Apple Pay) from the “fintech” and technology sectors, which are attacking banks’ and insurers’ core businesses and reshaping those industries.

To meet these challenges, banks and insurers are embarking on a comprehensive digitization journey, one that is different from the path they were on just three years ago, when their efforts were confined largely to isolated initiatives such as division-focused big data projects, the launch of individual apps, and improvements to their online and mobile channels. Today, banks and insurers are changing the ways that they interact with customers, giving customers a wider range of choices and greater control over the interaction itself. Banks and insurers are also providing customers with new, value-added, digitally enabled functionalities, such as rapid validation of loan approvals and the ability to open accounts quickly.[2] Ferratum, an international provider of mobile consumer loans, for example, can complete an application within two minutes. ING-DiBa, Germany’s third-largest retail bank, allows customers to identify themselves by holding an ID card in front of a webcam, rather than having to display postal or branch-based identification.

But banks and insurers are not focusing solely on improving the front end. Several institutions have also started to upgrade their back-end operations, which is often a much more complex challenge. A number of banks, for example, have begun to establish multidimensional master-data-management capabilities, strengthening their ability to leverage big data, meet regulatory requirements, and ensure consistent and timely reporting. Others are adopting agile ways of working, deploying continuous-delivery software engineering and DevOps to improve performance.[3]

Some financial institutions are enhancing these efforts by increasing the standardization of their infrastructure as well as their use of automated deployment, measures that can support a more rapid software-release cycle and better leverage front-end development. GE Capital, for example, developed its innovative Fleet Optimizer application, which is aimed at optimizing spending and utilization rates for clients of the company’s Fleet Services business, within 70 days—from whiteboard to production—aided by continuous delivery and greater standardization.

Some banks and insurers are also experimenting with new and evolving digital technologies, such as robotics process automation and self-learning machines, which together have the potential to deliver step changes in speed and efficiency. Arago’s AutoPilot, for example, can complete 80% of all IT service management tasks autonomously. In addition, these institutions are experimenting with new, customer-facing technologies. Deutsche Bank, for example, has added to its online investment platform a “robo advisor,” which uses algorithms to create portfolios for investors.

Overhauling their digital capabilities promises to have a transformative effect on banks’ and insurers’ business. But getting there will put a significant strain on IT departments and, especially, chief information officers, who can play a determining role in ensuring that their companies are ready and able to become truly digital enterprises. (See Recasting IT for the Digital Age, BCG Focus, March 2016, for a deeper look at opportunities and challenges related to corporate IT’s role in digital transformation.)

Enabling Transformation

To successfully transform themselves into truly digital enterprises, banks and insurers must adopt or develop four critical elements or capabilities (see Exhibit 1):

- A simplified application landscape and infrastructure

- Multidimensional master data management

- Fully automated delivery processes based on the principles of continuous delivery and DevOps

- An agile workforce, and a new way of working, across IT and the business

A Simplified Application Landscape and Infrastructure. The application landscapes and infrastructures of most banks and insurers are complex, fragmented, and full of legacy assets. They were built application by application over time and now include various platforms, infrastructure stacks, technologies, and customized or home-grown applications, often with several versions or variations of each. As a result, they are typically incapable of real-time responsiveness. They are also difficult and expensive to maintain, which ultimately means less value delivered per dollar spent on IT.

In addition, these environments are susceptible to broad reductions in service availability if one part of the chain breaks down. This is a huge liability in a world where customers expect digital services to be available 24/7. Further, the mix of technologies and methodologies employed—for example, batch and dialog components—makes it very difficult to implement real-time, end-to-end processing (a critical component of digital services such as real-time loan verification and payments).

To remedy this and ready themselves for digital transformation, banks and insurers are striving to simplify their application and infrastructure architectures and decouple their application landscapes. Following the lead of Web-born companies such as Spotify and Netflix, they are replacing complex interfaces and centralized service buses with more flexible, decentralized interfaces, such as application programming interfaces (APIs).[4] In such an environment, development teams can freely change the services they own, as long as they adhere to defined interface guidelines. The resulting fast release cycles allow these institutions to quickly bring new functionality to customers, helping to distinguish them from competitors and increase customer loyalty.

Financial institutions are also replacing their legacy applications with new technologies to enable straight-through processing. Our experience suggests that banks and insurers can gain outsized benefits by decommissioning not just some applications but entire platforms—for example, replacing mainframe hardware and software with a fully automated (and hence considerably cheaper) x86-based cloud server.

A success story here is Australia’s ING Direct, the country’s first fully cloud-enabled bank, which transformed itself with its Bank in a Box platform. By replacing its legacy infrastructure with “zero touch,” cloud-based hardware, ING Direct has gained several critical advantages, including greater automation and scalability of processes. This encapsulation of infrastructure is allowing the bank to focus on the development of digital services—a core competitive differentiator—rather than on infrastructure, which is essentially a commodity. The effort has been such a success in Australia that ING Direct is deploying it globally. The availability of new virtualization technologies, such as those produced by Docker, will make it easier for other financial services companies to go down this path.

Leading IT organizations at financial services companies are supporting these measures with significant efforts toward standardization and complexity reduction. We have observed that standardization works best when it is driven centrally and when it is mandatory for all parties. This approach translates into accelerated automation, leading to faster, more efficient development of digital services (as well as greater value per dollar of IT spending).

Collectively, actions such as these can create a simple, decoupled architecture that allows quick changes and reduces the need for testing. Infrastructure costs fall significantly because cheap, automated commodity hardware replaces costly legacy hardware, freeing up budget to develop digital services. And the use of APIs and microservices reduces dependencies among applications. This, in turn, improves service availability—a core differentiator in a digital business model. Such an architectural foundation is also a good starting point for unleashing the full power of agile methodologies and continuous delivery, which can enable rapid releases to production and accelerate the company’s digital transformation. (See Exhibit 2.)

Multidimensional Master Data Management. A consistent view of the company’s data—meaning that all data related to a particular service, customer, or other business-related object or process, as well as relevant data from third-party providers, is stored together in a consistent way—is a foundation for most digital applications. For example, integrated customer journeys (with the customer initiating a transaction or process in one channel and continuing or finishing it in another) require a high degree of data consistency to permit seamless switching among channels.

Most banks and insurers, however, still struggle to get consistent, high-quality sets of data that are free of redundancies. They employ several different database management systems and data models. Legacy applications are often connected to their own databases, translating into a high level of redundancy in physical and logical data storage across the organization. Typically, such systems can store, process, and analyze only structured data—meaning that these institutions can leverage only approximately 20% to 30% of the data potentially available to them. (This is because their legacy systems typically cannot store unstructured data, which includes such things as pictures, movies, tweets, and Facebook posts.) And the high cost of proprietary storage systems drives up data-related costs materially.

Some banks and insurers, however, have substantially elevated their capabilities on this front. They are working to revamp their data management practices and provide a digital-ready data infrastructure. They aim to replace inconsistent data pools with shared-data clusters to ensure data integrity and data access by all systems, enabling seamless customer journeys and end-to-end processing. Note that these efforts do not entail replacing all existing data stores; rather, they require the use of technical and conceptual layers to eliminate or manage redundancies and ensure consistency.

Several leading banks and insurers are also considering the use of data lakes to facilitate data storage and the analysis of unstructured data.[5] Often this is accompanied by the use of distributed architectures, such as Hadoop, which can increase data availability and reduce outages. (Netflix, to cite an example from outside the financial services industry, has a highly distributed data architecture that ensures availability of the whole system at all times, even if individual sites or regions break down.) In addition, data lakes and distributed architectures facilitate advanced analytics capabilities, which can help companies derive maximum value from the data they collect. GE Aviation employs a data lake that houses more than 1,500 terabytes of flight data and facilitates analysis that is 2,000 times faster, and ten times cheaper, than what is possible in a traditional data-warehouse-based environment.

Fully Automated Delivery Processes Based on the Principles of Continuous Delivery and DevOps. Banks and insurance companies often release software according to fixed, unyielding schedules. Testing is often still performed manually or in semiautomated fashion and primarily occurs at the end of projects or “sprints,” after hundreds to tens of thousands of lines of code have been written. There are long cycle times for deployment and lengthy lead times for infrastructure provisioning.

These institutions can take their development practices to the next level by using continuous-delivery software engineering and DevOps to fully automate their delivery processes. But this requires a change in the demarcation line between development and operations and infrastructure: developers and operations staff must work together in joint teams, and teams access standardized infrastructure based on an infrastructure-as-a-service (IaaS) model.

To be sure, most banks and insurers will not be releasing software to production on a daily basis. But a daily error-free build in the development environment can lead to a zero-defect mentality among staff and eliminate the need for huge testing cycles at the end of projects or sprints. It also increases the quality of the software produced and therefore the stability of digital-service provisioning. This can reduce errors by as much as 50%, freeing up funds that can be used for digital transformation. Another advantage of such an environment is that developers can self-deploy using self-service portals (rather than having to engage the infrastructure department), which can increase speed to market for new services by up to 20%. The resulting rapid-release cycles also enable fast deployment of new functionalities, which can further differentiate the institution from competitors.

Multiple companies can attest to the benefits of this approach. Union Bank, for example, has realized an 80% reduction in testing costs and slashed the time necessary for setting up development environments from 42 to 3 days. Nationwide Insurance has reduced critical defects by 80% and increased system availability by 70%.

An Agile Workforce, and a New Way of Working, Across IT and the Business. Our observations indicate that the software development process in many banks and insurance companies is far from optimal. Development teams remain intact only for the duration of their respective projects and work in isolation from the business. Developers focus on completing projects and take no responsibility for the full life cycle of the applications they create; instead, they pass their work along to operations, which takes care of maintenance and fixing bugs. Team members are often geographically dispersed, making it difficult to react to changing market requirements.

Agile methodologies can help overcome these problems. Success, however, depends on how they are deployed. Agile teams perform best when they are multifunctional—meaning they include business staff, developers, product management personnel, data analytics staff, and user-interface and user-experience design personnel, among others—and when their members are colocated. They also tend to do well when they focus on features rather than components. This model gives each team a long-term mandate, fostering end-to-end accountability as well as the accumulation of deep, specific knowledge by developers, which can increase their productivity significantly. It also allows teams to operate with a relatively high degree of autonomy when it comes to handling incidents and structuring the development process. (See Exhibit 3.) This close interaction between the business and IT, combined with each team’s ownership of the digital service it is working on, boosts efficiency and ensures that only the features that the business and client really need are developed.

Most banks and insurers today are at least experimenting with agile. Many have launched pilots that deploy it or have development teams (focused on mobile apps, for example) that are using it. And some of these efforts have been quite successful. But many institutions are struggling to scale up agile across their entire organization in a beneficial way. Here, banks and insurers can learn a lot from a number of “digital disruptors,” companies that have found a way to deploy agile at scale. ING Bank, for example, has successfully adopted Spotify’s agile-based approach, which uses “squads”—long-lived, cross-functional, cross-component teams that are dedicated to specific customer features and that stay together for years, not just for a project’s duration. (For more on how banks are scaling agile, see The Power of People in Digital Banking Transformation: The Digital Financial Institution, BCG Focus, November 2015.)

Implementing agile not only improves the efficiency of the development team, it also helps attract high-performing IT talent. BCG research shows that an agile environment fosters qualities that are highly valued by today’s top-tier talent—in particular, an appreciation for one’s work and good relationships with colleagues. Attracting the right talent could be a critical differentiator for banks and insurers in today’s rapidly evolving digital environment.

Making the Right Moves

For banks and insurers, digital transformation can be an exciting journey as well as an extremely rewarding one commercially. (See “Building a Cutting-Edge Banking IT Function: An Interview with Ron van Kemenade, the CIO of ING Bank,” BCG article, December 2015.) We believe that the following steps are critical for a strong start and ultimate success:

- Design a rough target architecture—spanning applications, infrastructure, and data—rapidly from the top down. The design effort should be a joint initiative that involves the entire IT organization: architects, application domains, operations, and infrastructure. The target architecture should be successively detailed in the months ahead.

- Be agile in planning but apply rigorous program management to ensure progress. Move toward the target architecture, even if the path is not fully visible. A wave-based approach (allotting, for example, six months per wave) can help the company deal successfully with uncertainty. It can also help it to quickly embark on the transformation and adjust to requirements that change along the way.

- Make automation of the delivery process a top priority. Strive for fast automation of core systems and the application and infrastructure delivery process. As a rule when migrating to a new target landscape, focus first on migration and second on automation.[6]

- Scale agile. Start with pilots and quickly roll agile out to the whole organization. We have observed that at leading companies, most pilots and agile initiatives start in IT before they are expanded throughout the business. Pilots are the starting point for building an at-scale agile organization that is aligned with the vision and aims of the company.

- Build DevOps teams. Best-practice companies break down the silos that separate development and operations by forming joint DevOps teams and applying the principles of continuous delivery, which encourage teams to be fast and flexible.

- Train your team and infuse it with digital talent. Promote an engineering culture and the development of deep technical expertise—for example, by staging events such as hackathons and discussion panels comprising the CIO and developers. Hire agile coaches and developers from fintechs, technology firms, and software vendors or offer secondments to your people. Create an appropriate physical environment (such as by colocating staff) and an IT culture in line with that of “true” technology companies—one that will attract and help retain top talent.

Notes

1. To ensure maximum availability, Netflix continuously simulates outages with its Simian Army series of tools. Chaos Kong, for example, simulates the infrastructure breakdown of an entire region.

2. BCG’s Retail Banking Operational Excellence benchmarking indicates that leading digital banks can validate a loan within 15 minutes, compared with about an hour for most traditional banks, allowing them to file the loan contract before the client even leaves the branch.

3. Continuous delivery is the ongoing, rapid updating and release of software in response to changing business needs. In DevOps, development and operations engineers work together throughout an application’s life cycle—not just during the development and deployment stages—in order to achieve greater speed and higher quality.

4. These can be particularly valuable. Once an API is in place, it can easily be reused, facilitating the addition of more functionality to a service (for example, a branch or agency locator added to an iWatch app). APIs are also expandable, allowing a company to launch a service with limited functionality and develop it as necessary. Finally, APIs facilitate integration of third-party offerings and connection to social media platforms, such as comparison portals.

5. A data lake can also be positioned as a consolida-tion layer between physical data stores and applications, acting as a “single source of truth.”

6. An exception might be when legacy components will remain in place for several years. In such cases, it could be beneficial to also automate those parts of the application landscape.

Hanno Ketterer is a senior partner and managing director in the Amsterdam office of The Boston Consulting Group and the global leader of the insurance sector in the Technology Advantage practice. For more than 20 years, he has led large-scale technology transformations and postmerger integrations in the banking and insurance industries. You may contact him by e-mail at ketterer.hanno@bcg.com.

Heiner Himmelreich is a partner and managing director in the firm’s Amsterdam office, a member of the global operating committee of the firm’s Technology Advantage practice, and BCG’s global leader for IT sourcing. For more than 20 years, he has worked on a wide range of IT sourcing projects in several industries, including development and implementation of an IT sourcing strategy, sourcing contract renegotiations, IT infrastructure and application management sourcing, cloud migrations, redesign of the retained organization, and improvement of vendor relations. You may contact him by e-mail at himmelreich.heiner@bcg.com.

Christian N. Schmid is a principal in BCG’s Munich office. He is active in the firm’s Technology Advantage practice and is the global operational leader of the Simplify IT topic in the Financial Institutions practice. For the past ten years, he has driven large-scale transformations—including digital and agile transformations, simplification and restructuring programs, and postmerger integrations—for numerous companies. You may contact him by e-mail at schmid.c@bcg.com.

Industry Spotlight

Building a Digital Technology Foundation in Insurance

by Hanno Ketterer, Jonathan Koopmans, and Rolf Mäurers

Insurers today face a host of digital to-dos if they want to stay competitive—much less gain an advantage on their peers. These undertakings include digitizing the customer experience, building digital offerings and business models, and constructing in-house digital capabilities. Underpinning them all is the question of how to adapt legacy IT systems and architectures to the needs of digital business models. This challenge can require new front-end architectures to mimic the mobile-first customer experience of digital natives. It can also necessitate a fundamental overhaul of core insurance systems to digitize end-to-end customer journeys and automate decision making in basic functions such as underwriting and claims handling. The prospect is daunting.

Most insurers need to overcome significant constraints in their current IT landscape. For example, BCG research shows that about 35% of all applications in the industry run on legacy technology stacks that are not “cloud ready” and that a similar percentage of incumbents still rely on static HTML-based digital channels that do not work well on mobile devices—the consumer’s digital device of choice.

We recently researched the readiness of insurers to go digital. We interviewed CIOs and IT architects at leading insurance companies worldwide. We also interviewed executives at prominent solution providers.[1] We conducted a “follow the money” assessment of some $17 billion in venture capital investment in more than 900 technology startups with relevance to the insurance sector. And we analyzed the main IT trends on the basis of four years of architecture benchmarking with top insurers in the German market.

Insurance CIOs and other IT executives will not be surprised to learn that we found multiple pain points at all levels of IT architecture. For example, only 36% of insurers use a central customer-data repository or CRM application to engage with clients, and only 64% have mobile apps. Such shortcomings limit insurers’ ability to gain a full view of client needs and to provide omnichannel interactions. The average age of core insurance systems in the companies we interviewed and benchmarked was 13 years.

Insurance companies have their IT work cut out for them. This article provides a framework for their efforts that is based on three questions:

- What are the main technological building blocks of a digital insurer?

- What emerging architectural strategies can help insurers accommodate future developments in technology?

- How do incumbent insurers jump-start digital implementation and stay ahead of the competition?

Key Trends, Developments, and Building Blocks

The complexity of the digital IT challenge is due in large part to the sheer number of technology trends and developments that have an impact on IT architectures today. There are no fewer than nine—including social media and mobile technology, the Internet of Things (IoT), open ecosystems, big data and advanced analytics, and cloud computing. (See Exhibit 1.)

The combination of these digital developments affects the entire IT landscape, which leads many insurers to go beyond building digital channel functionality and undertake an integrated front-to-back overhaul of the IT landscape across six architecture layers. (See Exhibit 2.) Others decide to focus on one or more layers as initial priorities. Substantial amounts of venture capital have been pouring into these six layers, indicating that tech players and their financial backers see big opportunities for improvement in current systems, platforms, applications, and approaches.

Front End, or Customer Engagement. This layer provides device-, location-, and context-aware customer interfaces and enables digital companies to deliver tailored advice and recommendations, as well as a rich multichannel, multidevice digital customer experience. More than $4 billion of venture capital is backing innovation in omnichannel user-experience platforms, social-network listening tools, and IoT and telematics platforms.

Smart Process and Decision Management. By using automated decision engines and artificial intelligence, this layer offers tailored customer-centric services based on microsegments and personalized risk profiles. Some $3.5 billion in venture funding is at work here. In contrast, legacy systems, in which core business processes (such as pricing and underwriting) are “hard coded,” allow for only static decision making based on broad customer segments and statistical patterns.

Back End, or Core Insurance Systems. This layer contains all systems of record for the core insurance business (the policy-administration, claims, and billing functions) and its support (risk management and finance, for example). Digital platforms integrate modular product architectures and “zero touch” processes. The former enable insurers to package multiple product and service components into a broad customer proposition, while the latter are completely automated processes that can be changed with minimal involvement from IT. This layer has attracted some $500 million in venture capital.

Central Data. This layer captures all data (both structured and unstructured) for real-time processing and analytics. The recipient of more than $4 billion of venture funding, it provides a “single source of truth” that gives insurers a 360-degree view of the customer and can reduce customer churn or detect fraudulent claims. In contrast, in today’s legacy IT landscape, data is typically scattered across multiple systems and not available for real-time analysis.

Cloud-Based Infrastructure. This layer allows for scalable high-performance digital services and rapid time to market for new digital solutions. Some $500 million of venture capital has been committed here. Cloud solutions replace on-premises legacy systems that depend on (expensive) self-owned data centers and a central team to manage the IT infrastructure and provide IT services.

Integration and Security. This layer manages these two functions by decoupling front-end from back-end platforms, integrating applications with external parties on the basis of open application programming interfaces (APIs), and managing security and privacy across the IT landscape. It has attracted more venture funding than any other layer—more than $5 billion—for good reason: data security and customer privacy are huge issues. Integration and security for most insurers today involve proprietary interfaces with partners, aggregators, brokers, and clients, with perimeter security and data privacy confined to enterprise IT systems.

Emerging Architectural Strategies

Given the complexity of the challenge, it’s no surprise that across the industry, insurance companies are taking fundamentally different approaches to building their digital platforms. Digital architecture is moving beyond mainstream software, with most digital functionality available now through ready-to-go software as a service (SaaS) platforms or open-source software. Four architectural patterns are emerging: mainstream software platforms, integrated core insurance suites, cloud platforms, and open-source platforms.

Many insurers continue to rely on mainstream software platforms from vendors such as Oracle and IBM. (The old adage “Nobody got fired for hiring IBM” still holds.) These platforms offer broad functionality ranging from mobile-first capability to API managers that span all the layers of the reference architecture, with the exception of core insurance. This architectural strategy appears most suitable for companies with complex legacy-integration challenges that need industrial-strength solutions. For example, one European insurer built a business-process-management layer with IBM software on top of its legacy system and accelerated the time to market of process changes by a factor of ten.

Other companies, most notably in the property and casualty segment, are betting on integrated core insurance suites. Core insurance software vendors such as Guidewire Software and SAP are rapidly expanding their offerings into digital portals and analytics so that they too can offer end-to-end solutions across all layers of the reference architecture. This approach is best suited for insurers that want an out-of-the-box or zero-touch solution to managing customers throughout their life cycle. For its MyDirect auto insurance offering in the US, for example, MetLife implemented a “wall to wall” Guidewire solution. The platform now handles 80% of all customer transactions through fully self-service channels.

A new generation of technology vendors is making rapid inroads into the insurance industry by challenging established software vendors and bypassing the central IT department with cloud platforms. A prime example is Salesforce.com, which has established a dominant position in the customer engagement layer with a client base of more than 2,000 insurers. Amazon Web Services (AWS) and Microsoft Azure are also entering the mainstream. Some insurers are moving completely away from internal data centers to these cloud platforms. The main advantage of this approach is its low-cost usage-based pricing model. One insurance startup runs its entire core insurance solution at AWS and manages 50,000 quotes a day at a cost of only €2,000 per month.

We are also seeing the rapid adoption of open-source platforms such as Liferay for portals and Hadoop for big data. About 40% of the insurers we interviewed use these types of solutions to tap into the innovation speed and talent pool of the open-source community. A good example is The Wall, which gives MetLife a 360-degree view of its 118 million customers and was built on MongoDB in just three months.

Jump-Starting Implementation

As with their architectural strategies, insurers are also taking fundamentally different pathways toward their desired digital technology end state. (See Exhibit 3.) The two principal decision axes are whether to buy or to build and whether to focus expressly on the digital front end or to start with the core insurance back end. The tradeoffs create four pathways with radically different investment and risk profiles. Each also has a different business impact.

Four factors substantially determine the suitability of each approach for a particular company: speed of change, investment required, implementation flexibility, and appetite for risk.

A customer-centric-package approach is the best option for insurers that are looking for mature mainstream digital functionality and don’t see a need to establish a digital competitive edge. These packages, which employ standard software, work well with stable legacy systems that allow for easy integration with the front end (such as through an enterprise service bus). The packages also require little upfront investment owing to their pay-per-use models, but they constrain rapid experimentation or radical innovation. Their biggest implementation risk is in data consistency, particularly maintaining a full customer view across legacy systems. Speed of change is typically less than six months.

Digital native front-end platforms are the best option for insurers facing competitive pressures and needing fast-differentiating digital solutions. They are custom built to drive digital innovation in customer engagement. Like the customer-centric-package approach, they require stable legacy systems and the ability to integrate them with new platforms. Speed of change depends primarily on how fast the insurer can build an internal engineering capability. These platforms can be significantly less expensive than customer-centric packages. They offer full control of the front end, but implementation of end-to-end digital customer journeys is constrained by the legacy IT back-end systems.

AIG moved to a digital native approach for its new front-end applications. It established a mobile innovation-and-delivery center in the heart of the California tech industry and assembled a dedicated data sciences team to develop custom-built analytics engines. The company gradually phased out legacy back-end systems, taking a natural end-of-life approach and replacing them with core insurance solutions from a leading software provider. AIG is also constructing a “data lake” to create a “single version of truth” across channels and products.

An integrated core-insurance suite is typically the best option for insurers facing a scattered legacy landscape at the end of its useful life. This approach, however, which involves the modernization of back-end systems to enable digitization of end-to-end processes, requires top-down commitment to endure the disruption of a large-scale transformation that encompasses significant reengineering of business processes as well as major data migration. It also involves a big upfront investment (which can be 100% to 150% of the annual IT budget). Transformations take time—typically two or more years (often even longer in the life insurance segment), although this can be shortened to about six months in the case of a greenfield project that starts with a clean slate.

Achmea, a large multiline insurer in the Netherlands, chose to modernize its application and data landscape through an SAP solution rolled out in a phased manner. On the basis of the SAP core insurance suite, Achmea is implementing digital customer-service and process chains with integrated front-to-back functionality, from quotation and underwriting to claims handling. The insurer expects efficiency gains, improvements in flexibility, and access to innovations. For example, the solution enables Achmea to configure new products within weeks, including front-to-back functionality.

A digital native end-to-end platform is the best option for insurers that put a strategic priority on technology-led innovation. This approach requires a world-class engineering capability that helps the insurer compete with actual digital natives. The level of investments depends heavily on the complexity of the business model, but it is not necessarily prohibitive or even large. One company built an auto insurance startup from scratch with a development team composed of a handful of people. Greenfield implementation can also be fast (as little as 12 months); regulatory approvals are often the bigger constraint. The main risk factor can be the difficulty of maintaining custom-built software owing to the competition for, and attrition of, key engineers.

One Asian insurer that pursues an agile early-adopter strategy for customer-facing technology manages all of its software development in-house. It applies a regular multiyear renewal cycle to its fully integrated core insurance platform in order to avoid large-scale legacy issues. It also taps into the open-source community, rather than traditional vendors, to keep pace with innovation.

Three Major Considerations

The extent of any insurer’s digital IT task is a factor of its digital business strategy and ambitions. Insurers should focus on three overall considerations.

First, digital affects the entire IT landscape. Many successful companies take an integrated front-to-back approach that goes beyond mere digital-channel functionality. For those that choose to focus more narrowly, at least initially, the smart-process and decision-management layer is key to offering customer-centric tailored services and maximizing lifetime value.

Second, digital architectural strategy should extend beyond the solutions offered by mainstream software. Most digital functionality today is available through ready-to-go SaaS platforms or open-source software.

Third, implementation pathways (such as build or buy) should be carefully designed because they pose radically different investment and risk profiles. Insurers that aspire to radical innovation typically invest in building an internal engineering capability, while those with less extensive goals can rely on mainstream commercial software. Insurers with major legacy IT constraints should take an end-to-end transformation approach. Others have the option of a front-end focus.

The complexity can be confounding, but companies should not be put off. The range of solutions available today, both tailored and off the shelf, vary widely, but they make it possible for every company to determine how best to address its particular circumstances.

Note

1. The authors are grateful to executives from the following technology companies for their input: Accenture, Earnix, eBaoTech, Guidewire Software, HCL Technologies, IBM, Innovation Group, Microsoft, Novidea, Pegasystems, Salesforce.com, SAP, SAS Institute, and Wipro.

Hanno Ketterer is a senior partner and managing director in the Amsterdam office of The Boston Consulting Group. He leads BCG’s Technology Advantage practice in insurance globally and manages large technology transformations for insurance companies and banks. You may contact him by e-mail at ketterer.hanno@bcg.com.

Jonathan Koopmans is an associate director in the firm’s Amsterdam office. He has more than 15 years of experience in designing and managing complex IT and digital transformations, focusing on the financial institutions and insurance sectors. You may contact him by e-mail at koopmans.jonathan@bcg.com.

Rolf Mäurers is a managing director of Platinion, a wholly owned BCG subsidiary. You may contact him by e-mail at maeurers.rolf@platinion.com.

note to the reader

Acknowledgments

The authors would like to thank their BCG colleagues Astrid Blumstengel, Biljana Bajic-Bizumic, Thomas Gießen, and Stuart Scantlebury for their contributions to this publication. They also thank Katherine Andrews,Mickey Butts, Gary Callahan, Alan Cohen, Catherine Cuddihee, Angela DiBattista, David Duffy, Kim Friedman, Abby Garland, Gerry Hill, Sara Strassenreiter, and Amy Strong for their help in the publication’s writing, editing, design, and production.

For Further Contact

Ralf Dreischmeier

Senior Partner and Managing Director

Global Leader, Technology Advantage practice

BCG London

+44 20 7753 5353

dreischmeier.ralf@bcg.com

Jon Brock

Associate Director

BCG London

+44 20 7753 5353

brock.jon@bcg.com

Karalee Close

Partner and Managing Director

BCG London

+44 20 7753 5353

close.karalee@bcg.com

Sabine Döschl

Principal

BCG Munich

+49 89 231 740

doeschl.sabine@bcg.com

Stephen Edison

Principal

BCG Dallas

+1 214 849 1500

edison.stephen@bcg.com

Michael Grebe

Senior Partner and Managing Director

BCG Munich

+49 89 231 740

grebe.michael@bcg.com

Thomas Gumsheimer

Senior Partner and Managing Director

BCG Berlin

+49 30 28 87 10

gumsheimer.thomas@bcg.com

Peter Hildebrandt

Principal

BCG Los Angeles

+1 213 621 2772

hildebrandt.peter@bcg.com

Heiner Himmelreich

Partner and Managing Director

BCG Amsterdam

+31 20 548 4000

himmelreich.heiner@bcg.com

Sesh Iyer

Partner and Managing Director

BCG Washington

+1 301 664 7400

iyer.sesh@bcg.com

Hanno Ketterer

Senior Partner and Managing Director

BCG Amsterdam

+31 20 548 4000

ketterer.hanno@bcg.com

Djon Kleine

Consultant

BCG Amsterdam

+31 20 548 4000

kleine.djon@bcg.com

Jonathan Koopmans

Associate Director

BCG Amsterdam

+31 20 548 4000

koopmans.jonathan@bcg.com

Frederik Lang

Consultant

BCG Copenhagen

+45 77323400

lang.frederik@bcg.com

Alexander Lawrence

Consultant

BCG New York

+1 212 446 2800

lawrence.alexander@bcg.com

Rolf Mäurers

Managing Director

Platinion Cologne

+49 221 5895 8300

maeurers.rolf@platinion.com

Benjamin Rehberg

Partner and Managing Director

BCG New York

+1 212 446 2800

rehberg.benjamin@bcg.com

David Ritter

Director

BCG Boston

+1 617 973 1200

ritter.david@bcg.com

John Rose

Senior Partner, Managing Director, and Fellow

BCG New York

+1 212 446 2800

rose.john@bcg.com

Christian N. Schmid

Principal

BCG Munich

+49 89 231 740

schmid.c@bcg.com

Michael Sherman

Partner and Managing Director

BCG Dallas

+ 1 214 849 1500

sherman.michael@bcg.com

Adal Zamudio

Principal

BCG London

+44 20 7753 5353

zamudio.adal@bcg.com

The Boston Consulting Group (BCG) is a global management consulting firm and the world’s leading advisor on business strategy. We partner with clients from the private, public, and not-for-profit sectors in all regions to identify their highest-value opportunities, address their most critical challenges, and transform their enterprises. Our customized approach combines deep insight into the dynamics of companies and markets with close collaboration at all levels of the client organization. This ensures that our clients achieve sustainable competitive advantage, build more capable organizations, and secure lasting results. Founded in 1963, BCG is a private company with 85 offices in 48 countries. For more information, please visit bcg.com.

© The Boston Consulting Group, Inc. 2016. All rights reserved.

For information or permission to reprint, please contact BCG at:

E-mail: bcg-info@bcg.com

Fax: +1 617 850 3901, attention BCG/Permissions

Mail: BCG/Permissions

The Boston Consulting Group, Inc.

One Beacon Street

Boston, MA 02108

USA

To find the latest BCG content and register to receive e-alerts on this topic or others, please visit bcgperspectives.com.

Follow bcg.perspectives on Facebook and Twitter.

10/16