Why Is Knowledge Management So Difficult

Published Date: 02 Nov 2017

Knowledge management promises much, but often delivers very little. There are no simple solutions to this challenge. This article starts by trying to define what knowledge management is. It then identifies where the problems lie and suggests five steps to resolve those problems. The article is based on research in a dozen leading companies, including HP, Ericsson, ABB, Skandia and Xerox. If you keep even half an eye on the management press, chances are you have come across the concept of knowledge management. Emerging at the beginning of the 1990s, knowledge management is now a well-established school of thought with its own dedicated consulting companies, journals and management gurus.

The "big idea" with knowledge management is that in a fast-moving and increasingly competitive world, a firm’s only enduring source of advantage is its knowledge – the knowledge of its individual employees, and the knowledge that gets built into its structures and systems. Consulting companies and R&D organizations have known this for years, but increasingly even "old economy" firms in oil, steel and consumer products are recognizing the importance of knowledge assets as the source of their success. The problem with knowledge management is that most companies struggle to make it work. A Bain and Co. study found that while some companies were "extremely satisfied" with their progress in knowledge management, the majority expressed a below-average level of satisfaction. And in a survey of European and US companies (Ruggles 1998), the results were even more worrying. Only 13% of respondents rated their ability "to transfer existing knowledge within the organization" as good or excellent, and "measuring the value of knowledge assets and/or the impact of knowledge management" was rated good or excellent by only 4% of respondents. Drawing on research in a dozen companies, this article tries to make sense of the practical implications of knowledge management. Through interviews with more than 50 executives in companies like HP, Ericsson, ABB, Skandia and Xerox, I have looked at the approaches taken to managing knowledge and the effect on individual and company performance. While these companies have all undertaken plenty of initiatives in knowledge management, the results of their efforts typically lie somewhere between disappointing and acceptable. There were no outright failures, but no great success stories either (see the box overleaf for a discussion of HP’s experiences).

Why is knowledge management so difficult? The key problem is that knowledge management is so central to the make-up of the firm that it cannot be separated out and acted upon in the way that a single business process or management system can. In all companies, knowledge is already being "managed" through informal networks. To do it better involves not only developing new tools, but also eliminating old ways of working. As a result, changing a firm’s knowledge management system is not unlike changing its culture – it involves fundamental changes to people’s behavior, and it typically takes many years to bring about.

In sum, knowledge management promises much, but often delivers very little. There are no simple solutions to this challenge, but it is possible to make some progress through a more complete understanding of what exactly knowledge management is, where the problems lie, and the steps you can take to resolve those problems. That is the purpose of this article.

Knowledge Management at Hewlett-Packard Laboratories

HP has developed a sophisticated approach to knowledge management. Of all the companies in the study, HP has come farthest in making knowledge management work. Yet managers acknowledge that they are still experiencing some teething pains and frustrations. A plethora of knowledge management tools and techniques has been established over a number of years.

IT applications for coordinating activities: HP has a market research database to make research reports available to all staff, and an external standards database to help with lobbying. In addition there are databases for managing research projects and HP’s patent portfolio.

Organizational tools for coordinating activities: HP’s laboratories are organized into centre that focus on specific technologies, but there is a high level of fluidity across centre, and typically strong links between centers and development groups. HP has an active rotation program. As one person said, "There are 12 different ways to rotate here". There are also experiments underway with creating a "virtual laboratory" in which people co-ordinate their efforts remotely.

IT applications for problem-solving: HP has an elaborate competence database to search for employees with specific skills. There are also communication facilitators such as intranet forums and video-conferencing.

Organizational tools for problem-solving: There is a well-established best practice transfer process, including a unit that takes responsibility for best practices across the laboratories. Individuals are encouraged to seek out information from other labs and from outside HP, and to share information with others. There are also informal networks of individuals with similar technological interests. More broadly, HP has managed to develop a strong "one company culture" that prevents individuals from hoarding their knowledge.

How well do these tools and systems work? Certainly as well as in any other company that participated in the study, but not as well as HP would like. Three issues in particular stand out. First, with so many tools, people were either not aware of all of them, or were too busy to use them all. As a result, a lot of knowledge management still took place through the informal networks. Second, while knowledge flows within the labs was good, the links between labs and development groups were of very variable quality, and this constrained researchers’ ability to develop technologies that would be usable on a corporate wide basis. Third, some of the IT applications such as the patent database were developed primarily for management rather than for researchers, and were not readily accessible without explicit permission.

What is Knowledge Management?

At the heart of the knowledge management movement is the simple concept of the firm as a "social institution". The firm draws value from the individuals within it, and from its ability to harness their knowledge. But individuals also draw value from the firm they work for, to a far greater extent than a simple contract-based view of the world would suggest. They learn from their colleagues and are able to accomplish tasks they could not do on their own. Equally important, people are innately social animals – they like to share experiences, they like to gossip, and they like the feeling of "belonging" to a department, or a firm.

This may not be an organizational philosophy that resonates with everyone, but the fact is that many firms both small and large – do work this way, so there is potentially a great deal of value in understanding how to make it work. And to do so is essentially about creating structures and systems that enable, rather than constrain, social activity and knowledge sharing.

Knowledge management, by this logic, can be seen as a set of techniques and practices that facilitate the flow of knowledge into and within the firm. Two related bodies of thinking, Organization Learning and Intellectual Capital can also be identified . In practical terms, there are three elements to knowledge management (Figure 1). First, the firm should encourage individuals to interact – to work together on projects, or to share their ideas on an informal basis. Second, systems are needed to codify the knowledge of individuals so that it can be used by others. A key insight from the knowledge management movement is that most valuable knowledge is tacit – it is held so deeply by the individual that it is hard to express or write down. If ways can be found for transferring that knowledge to others in the firm, either through personal interaction or by recording it explicitly, then that knowledge becomes an asset of the firm, and a key source of advantage. Third, the firm needs to get access to new knowledge from outside its boundaries, as a means of updating and renewing its knowledge base.

The Related Concepts of Organization Learning and Intellectual Capital

Knowledge management is one, but by no means the only, set of concepts concerned with how companies make the most of their knowledge assets. Two others are particularly influential – Organization Learning and Intellectual Capital. While their key ideas overlap to a great extent with knowledge management, they also have their own particular nuances.

Organization learning was the forerunner to knowledge management, and gained its support largely on the back of Peter Senge’s (1990) The Fifth Discipline. The underlying premise is similar – that knowledge is a scarce and valuable resource, so for firms to succeed in a fast-changing world they have to become better at learning (i.e. sharing knowledge among individuals). The difference is that organization learning is about managing the processes of learning while knowledge management is more concerned with techniques for building up and applying stocks of knowledge. The Intellectual Capital movement took shape around the same time as knowledge management. It emerged out of the world of accounting as an attempt to identify and measure intangible assets. By breaking a firm’s intellectual capital down into such elements as human capital (the capabilities of employees), customer capital (existing relationships), and structural capital (patents, operating systems, practices), its advocates were able to come up with useful measures that could then be monitored and evaluated over time (Ambler 1999). The best-known example of this was the Navigator model developed by Skandia, a Swedish insurance company, in which the elements of Intellectual Capital were identified, measured, and reported on as a supplement to the Annual Report.

Three Elements of Knowledge Management

Improving the informal flows of knowledge between individuals

Building systems for codifying and sharing knowledge within the firm

Tapping into new knowledge from sources outside the firm

Applying these concepts involves three sets of tools: information technology systems, formal and informal structures and specific KM tools. Information Technology systems: In many firms, knowledge management evolved out of information management, for the obvious reason that knowledge and information are closely related concepts. McKinsey and Accenture, for example, both have highly sophisticated databases that provide libraries of information about their proprietary methodologies, clients, previous engagements and so on. These are essentially repositories of codified knowledge. They do not capture the tacit knowledge or expertise of leading partners, but they provide a form of collective memory that consultants can tap into quickly and efficiently. Formal and informal structures: A large part of knowledge management is simply about facilitating the natural interactions between people. One approach is to design the physical layout of offices so that social interaction is encouraged. Ericsson and GlaxoWellcome, for example, designed their R&D labs using lots of glass, open-plan layouts and hub-and-spoke structures to facilitate informal discussions. A second approach is to design the formal structure around the key knowledge flows. Cross-functional project teams are a case in point a way of formalizing meetings to ensure that all the individuals involved bring their relevant knowledge to bear on the project. A third approach is to facilitate informal interactions through what are usually called communities of practice – groups of individuals with common interests and problems who are dispersed throughout the firm. These individuals, it is argued, will naturally seek one another out to share their experiences and learn from one another, so the firm can play a subtle role in facilitating their interaction, for example by creating discussion forums on the intranet. Specific knowledge management tools: Finally, there are a number of specific knowledge management tools that firms use. One is the transfer of best practice – HP for example, has a structured process for taking a technique or practice in one laboratory and transferring it to another laboratory in a different part of the world. Another is the designation of centre of excellence for example 3M Europe identified individuals or groups with specific expertise (e.g. business intelligence in the UK, key account management in Sweden), in order that their knowledge could be picked up and used in other parts of the firm (see Moore and Birkinshaw 1998).

Why Knowledge Management Often Fails

Despite the range of tools and techniques, the success rate of knowledge management is mixed, and in my own research I have seen plenty of cases of program and initiatives that did not deliver the results they were looking for. The following four observations summarize why this is the case:

Firms do not sufficiently recognize that they are already doing it.

Information technology is often regarded as a substitute for social interaction

Knowledge management typically focuses too much on recycling existing knowledge, rather than generating new knowledge.

Most knowledge management techniques look like traditional techniques.

Knowledge management is never zero-based: to make it work, you need to recognize that you are already doing it. While the proponents of knowledge management might like to argue otherwise, the reality is that firms have been managing their knowledge since the dawn of time. For example, formal organization structures are typically designed to ensure that knowledge exchange takes place between those who most need it. Informal social networks the groups of people who have lunch together or go for a drink after work are mechanisms for knowledge transfer. And industry associations are vehicles for sharing ideas and discussing common problems. A recent study by Niklas Arvidsson (1999) looked at the sales and marketing affiliates of such Swedish firms as Alfa Laval, Ericsson, and Volvo. The study examined the flows of knowledge between these affiliates, and sure enough it found that there were plenty of knowledge transfers going on. However, when Arvidsson compared the flows of knowledge with the ratings of which affiliates were the highest performers, he discovered that most transfers did not flow from "best" to "worst" – they simply flowed along existing lines of communication. Those affiliates that were part of the "in crowd" exchanged knowledge freely, while those that were new to the group or geographically isolated were essentially shut out of the loop (Figure 2). The result was a lot of cases of "mediocre" or "worst" practice transfers. The only exception to this rule was Alfa Laval which had gone to enormous effort to build linkages between all its affiliate companies and to hold regular management meetings at which knowledge sharing could take place. Another example is a new media consulting company I work with. In looking at how knowledge was shared between software programmers, it became apparent that these people would often use external discussion groups with people they had never met, rather than the company’s own intranet forum. This was a real eye-opener for the managers, because they had assumed that the intranet would be a useful tool for sharing ideas and solving problems. Armed with this insight, they were then in a position to rethink the support they gave to their software programmers, and the function of the corporate intranet. The point, in other words, is that knowledge sharing is already taking place, but it is happening in an adhoc way along the lines of people’s informal networks. Knowledge management can help to make this a more systematic process, for example by monitoring and comparing the performance of different subsidiary companies or R&D units. This at least helps the firm to "know what it knows" but it still does not guarantee that the flows of knowledge will be diverted away from their traditional and well-trodden paths.

Information Technology is never a substitute for social interaction

The internet revolution has caused some writers to make absurd claims about how the world of work will change. One popular article by Tom Malone and Robert Laubacher (1998) foresaw the emergence of the "e-lance economy" in which individuals would work as freelancers rather than as members of firms. Another line of thinking talked about the "paperless office". And many have argued that telecommuting will take over from traditional forms of commuting. These arguments are just plain wrong, for the simple reason noted earlier that individuals need social interaction – both for its own sake, and because it provides a powerful vehicle for learning. The Social Life of Information (Brown and Duguid 2000) provides an excellent counterpoint to the argument that technology is going to change the way we work. It explains the importance of the social interaction between people that lies at the heart of knowledge management. This simple insight has important implications for the management of knowledge. First, it helps to explain why most knowledge databases are so poorly used. Most people would much prefer to talk to a colleague about their latest ideas rather than try to find something he or she wrote. And second, it cautions us to recognize that IT tools and "social" tools such as communities of practice are complementary. Again, this flies somewhat against the received wisdom. A recent article by Morten Hansen and colleagues (2000) suggested that firms should focus on either a "codification strategy" which involved putting the firm’s knowledge onto IT databases, or on a "personalization strategy" which involved building strong social networks. But experience suggests that the complementary nature of the two strategies is such that you need to do both. Knowledge management typically ends up focusing too much on the recycling of existing knowledge and not enough on generating new knowledge. Most of the knowledge management literature is inherently inward-looking. It is concerned with tools such as transfer of best practice and knowledge databases. These are extremely important but ultimately focused on what I would call "operational efficiency" – making current activities work in a more streamlined way. Much more important, over the long term, is the ability to bring new knowledge into the organization, and turn it into new products and business models. There are many ways of doing this. 3M, for example, has strategic accounts with many leading customers to pursue joint innovation projects. Ericsson and Nokia both have outposts in Silicon Valley whose job is to tap into the latest thinking there and transfer it back to head office. Cisco aggressively acquires small companies as a way of building its knowledge base. All of these approaches to knowledge generation have their own challenges, but they are critically important because they represent a way of renewing the firm’s knowledge, rather than just recycling it.

Most knowledge management techniques end up looking just like the traditional techniques you have been using for years. The deeper that firms get into these sorts of tools, the more they find that managing knowledge is integral to the working of the entire organization. For example, Skandia, the Swedish insurance company, developed its "Navigator" model on Intellectual Capital foundations, but it ended up functioning exactly like the Balanced Scorecard. The concept of communities of practice is alluring, but essentially it is just about encouraging people to communicate with one other and share their ideas, which is an idea that is as old as the hills. Knowledge databases have an important role to play in sharing knowledge, but the challenge of aligning such tools to user needs is the same one that IT managers have always faced.

These observations are not meant to be disparaging. Good management practice always boils down to a few basic challenges, so the fact that knowledge management leads to a certain amount of déjà vu among seasoned managers is actually a reassurance that it is addressing important issues. The acid test is whether these techniques end up being better understood, and better implemented, as a result of the knowledge management "framing". In my assessment, they are certainly being better understood, but the jury is still out on whether implementation is also improving.

Making Knowledge Management Work

These arguments paint a complex and perhaps over gloomy picture of knowledge management. There are no easy quick fixes for a company that is pursuing, or considering, a knowledge management program.

But the following five basic guidelines may make it easier to think through how to structure knowledge management efforts:

Map the knowledge flows in the organization.

Map the stocks of knowledge and use them to encourage sharing of best practice.

Focus efforts on mission-critical rather than nice-to-have knowledge.

Raise the visibility of knowledge management activities.

Use incentives to institutionalize new knowledge sharing activities.

Map the knowledge flows in the organization

Before starting any specific change initiatives, you need to get a handle on the current state of the knowledge flows in your company. They can best be understood through "social network" analysis, which involves mapping the web of connections between individuals in an organization. The analysis shows very clearly which parts of the company suffer from a lack of knowledge flows, and these can then be highlighted for attention. For example, in one division of Skandia it became clear that the German subsidiary was being left out of the loop on important discussions. On closer investigation it became clear that the problem was language – the German subsidiary head did not speak good English, so he had chosen to opt out rather than embarrass himself. The problem was readily addressed by identifying one of his team with good English and bringing him into the discussions.

Map the stocks of knowledge and use them to push the sharing of best practice

A similar procedure can be used to identify the key sources of expertise in the company. One established approach is to undertake a "best practice audit" in which units or businesses are rated on a series of key metrics and then ranked in "league tables" of performance. In Ericsson, for example, R&D units are rated in this way, and the information is then used to start a discussion with each unit about how their performance can be improved, and which other units they can learn from. In Intel and GE such league tables are taken a stage further, and are used as a major input into performance evaluations. For the laggards, the incentive to learn from the units at the top of such league tables is extremely strong.

Focus efforts on mission-critical rather than nice-to-have knowledge

Good change programs always start with the "low hanging fruit" – the stuff that can be grasped with minimal effort. In terms of knowledge management, this means groups who need access to new knowledge to do their job. In one pharmaceutical company, for example, this meant focusing on the project teams doing in-licensing of drugs. In a consulting company, the first knowledge initiative focused on making sure that client teams had the technological and interpersonal support they need to do deliver leading edge solutions to their clients.

Raise the visibility of knowledge management activities

Making knowledge management effective in the long term requires a fundamental shift in the behavior of employees. The chief executive and other senior figures have an important role to play in this process, by raising the visibility of knowledge management and by demonstrating its importance. One chief executive made a point of putting best practice sharing on the table at every management meeting – by getting a subsidiary manager to present the latest thinking from his or her market.

Gradually the team began to take some of these ideas home with them to share with their people. Another approach is to appoint a Chief Knowledge Officer whose role is to champion all knowledge sharing and creation activities (Earl 2000). Ultimately, however, knowledge management has to become a way of doing business, and just as with Total Quality Management this means moving it from a staff function and a couple of specific projects to an organization-wide responsibility

Use incentives to institutionalize new knowledge sharing behaviors

Another useful way of instilling a shift in behavior is by making knowledge management part of the incentive system. By this I don’t mean a financial reward every time an employee shares their ideas with a colleague – this would be expensive, impossible to monitor, and would encourage exactly the wrong sort of narrow-minded behavior. Much better – as most consulting companies have realized – is a "soft" form of incentive, typically a section of the annual performance review in which an individual’s contribution to the company’s knowledge is recorded and evaluated. This ensures that individuals are reminded of the importance of knowledge management on a regular basis, while avoiding the problem of specifying too narrowly what the appropriate knowledge-sharing behaviors are. Knowledge management has been around for more than ten years, but it has got a lot of life left in it. My purpose in this article was to identify some of the core ideas, but also to show why they are so difficult to put into practice. The tone is deliberately somewhat skeptical, but I think that is appropriate if knowledge management is to move from being yet another interesting "fad" to an enduring management technique. Many of the ideas in knowledge management are, indeed, old ones with new labels. But at the same time there are some important and lasting principles that help us better understand how knowledge-based firms work.