I was reading a blog post by Tom Davenport, author of numerous books and articles on analytics, and his recent post caught my attention in relation to the book Sea of Thunder I’ve been reading. “It’s clear that one of the major problems in our current economy is that senior executives in financial firms have relied on faith to ensure that their analytics are correct and their risks are tolerable” (¶ 1). I would even argue that a problem in a number of companies is that management don’t fully understand how the data they use for managing their business is built. Accuracy of data is one thing, but accurate data that doesn’t answer the management problem at hand isn’t worth the bits that are used to store it.

I have a friend who works for a large organization who has relayed the discussions he has had with his fellow employees in his department. They are responsible for developing and disseminating data related to turnover of personnel and other HR related metrics. The problem is that the data being disseminated is inaccurate, because when the first numbers were developed, management didn’t like the numbers, so they changed standard metrics formulas to make the numbers look better. One manager, afraid of standing behind the numbers, has misrepresented their HR metrics to the entire senior management team. How can this happen in the age of Sarbanes-Oxley? And if it is happening with the HR metrics, could it be happening in other departments as well? If management can’t trust the numbers they are being given, they can’t effectively manage the company.

Accuracy of data isn’t just restricted to the business world of today. Back in World War II, the problem existed in another form. In the book Sea of Thunder I’m reading right now, the author quotes The Illusive War Results: The Truth of the Imperial Headquarters Reports and how the Japanese questioned returning pilots in order to get reports that they wanted rather than an accurate assessment of their raids: “One pilot described seeing a ‘pillar of fire,’ possibly an oil tanker or a carrier burning ‘at a far away place.’ A senior officer demanded, ‘Wasn’t that an aircraft carrier?’ The pilot responded, ‘It could have been.’ The chief officer pressed: ‘It was an aircraft carrier, right?'” (p. 165). The Japanese Navy was driven by a desire to please their superiors, which is why inaccurate data was passed up the chain of command.

The problem wasn’t just in the Japanese Navy, as Evan Thomas describes in this passage: “Generally speaking, the American commanders were more skeptical of their pilots’ reports. They benefited from code-breaking as a reality check, and, as the war progressed, American pilots probably became less hyperbolic than their Japanese counterparts” (p. 167). Although, one can’t blame soldiers in war from being over zealous in their damage assessments. I, thankfully, have never experienced combat, but I couldn’t fault them for making errors when they only had fractions of a second to do an assessment while attempting to avoid flak and other planes (both on our side and theirs).

Evans goes on to quote John Lawrence, one of Halsey’s air combat intelligence (ACI) officers, “‘The pilots were under enormous strain,’ risking their lives on every mission. ‘We wanted to encourage the pilots to go back and attack.’ The ACI staffers were faced with a dilemma. ‘The great question,’ said Lawrence, ‘was to what extent you could water down the pilots’ reports without killing morale'” (pp. 167-168).

When we provide data to upper management, we have to make sure we are not coloring, or distorting, the true picture. In previous jobs I have been responsible for developing and disseminating data to upper management, and I’m still responsible for the accuracy of reports that are developed as a result of requests from management.

Truth and reality are dependent upon the quality of the data being supplied. If the data isn’t accurate, then management won’t have a true picture of the situation and won’t be able to effectively manage the company. We as leaders, at all levels of management, have to ensure the data we are using to make decisions, or the data we are providing to others, is accurate and pertinent. Even though, like the Japanese, we want to please our superiors, we have to realize that producing inaccurate data is worse than being honest about the situation of the company. Too much is riding on the data we provide.

— Davenport, T. (2008, February 7). Faith-based Analytics and the State of the Financial Industry. http://discussionleader.hbsp.com/davenport/2008/02/faithbased_analytics.html.

— Thomas, E. (2006). Sea of Thunder. New York, NY: Simon & Schuster.

Advertisements