STANFORD GRADUATE SCHOOL OF BUSINESS – If you’ve spent any time at all in business school, the question, “Does management matter?” seems almost absurd. After all, graduate students spend two years or more reading case studies and delving into the details of management strategies, and studies of “managerial technology” were published as early as 1887. It turns out, though, that the answer to that question is far more complex than it appears.
While there’s no doubt that there are “astonishing differences in productivity across both firms and countries,” economists aren’t convinced that management actually matters to economies, Stanford economist Nick Bloom says. Why the skepticism? “One reason is the belief that competition will drive badly managed firms out of the market. Another is the complexity of management, making it hard to measure,” he says.
Until relatively recently, researchers had limited access and quality data, especially in the area of human resource management. The case studies so popular in management education “are a great way to internalize a deep example, but the drawback is they are built on just one data point. They are not quantitative,” says Kathryn Shaw, the Ernest C. Arbuckle Professor of Economics. “It isn’t easy to find cases where there is an actual hypothesis on why management matters.”
Now, however, the spread of information technology has had a huge impact not only on how workers are managed but also on the ability of researchers to compare the effectiveness of different strategies. More data is produced by firms and government agencies. But data alone can be misleading. Talking to insiders is also essential to interpret it, says Shaw, who is widely credited with developing a research framework called “insider econometrics” when she began studying steel mills in the ’90s.
For more than a decade, researchers at Stanford and other leading universities have been examining management strategies related to human resources practices in an effort to establish clear causal connections between the actions of management and increased productivity or competitive advantage. They have made those connections in studies of industries and occupations that include steel and software manufacturing, auto-glass installation, fruit picking, nursing, banking, and textiles. They’ve analyzed management practices such as team-based pay, incentive pay, social networks and information sharing, call monitoring, and more.
In a 2009 paper coauthored with Columbia’s Casey Ichniowski, a frequent research partner, Shaw explained that three fundamental questions are addressed by insider econometric studies:
- Why do firms in the same industry adopt different
- Does the adoption of a new management practice
- If so, why does the new management practice
“This research approach addresses these questions by combining insights from industry insiders with rigorous econometric tests about the adoption and productivity effects of new management practices using rich industry-specific data,” they wrote.
Who are those “insiders”? In a typical business school case study, it’s the top managers; in an insider econometrics study, an insider could be a worker on the assembly line. By talking to people intimately involved with production, researchers can see relationships that otherwise would remain hidden.
The value of that practice was demonstrated by a well-known insider econometrics paper published in 2005 that looked at different types of incentive pay for fruit pickers on one farm. When the company switched from a compensation plan in which worker pay was based on how much they produced relative to other workers to a simple piecework plan, productivity soared by 58%.
Seeing the increase in productivity was very simple, of course. But insider econometrics asks why the increase occurred. After conducting extensive interviews with the field workers, the researchers realized that workers who picked alongside their friends under the original plan didn’t want to excel because extra pay would come at the expense of fellow workers. Once that impediment was removed, they felt free to do their best, and productivity increased.
Interesting as it was, the fruit picker study involved one firm with fewer than 200 employees. But armed with far richer data sets than were available even a few years ago, researchers can look across entire industries.
In 2009, Shaw and four colleagues examined the relationship of compensation to innovation across the software industry, a huge sector employing thousands of workers. They hypothesized that firms operating in the more volatile segments of the industry would hire more top-flight talent and pay more than firms competing in more stable segments of the software market.
Testing that hypothesis would have been difficult in the past. Detailed data on a broad sampling of employees was not available, which is why most compensation studies, until recently, have focused on CEO pay. Empirical studies, the researchers say, had yet to establish a link between product market strategy and human resource practices using data covering more than a small number of firms or a select group of employees.
Now, however, much broader and deeper data is available through the U.S. Census Bureau and state agencies that track unemployment insurance contributions. The researchers analyzed salary, including exercised stock options and bonuses, and revenue data from software firms in 10 states. Using that information, they were able to calculate the different potential payoffs for various types of software products and how those payoffs related to compensation. And their assumption was correct. [Click here for more details]
“In addition to data [from the Census Bureau and other agencies] we also have far more data from within firms. This may have begun with manufacturing and operations, where the machines today
measure everything about performance. But now it has extended to measuring employees’ traits and behavior and performance,”
For instance, oDesk, a large online labor market based in Redwood City, Calif., gave Christopher Stanton, a 2011 PhD candidate who
is supervised by Shaw, access to anonymized records of more than 300,000 contractors who have used the service since its founding
in 2005. He used that data to build a model of how agencies —
third-party groups working within a labor market such as oDesk — improve the hiring process and lead to higher initial salaries for
workers, particularly in developing countries.
Another Shaw student — Sara Champion, a 2011 PhD candidate — used an even larger data set to study the effect of teacher accountability standards on the work habits of teachers. She looked at pay records of 700,000 teachers in three states, eventually compiling 3.7 million data points.
She found that those standards and the threat of actions against educators whose students don’t progress have led teachers to significantly reduce the amount of time they spend “moonlighting,” that is, working at second jobs outside the school. Champion is collecting and analyzing additional data, hoping to determine whether the decline of moonlighting corresponds to higher student achievement.
While these four studies are quite dissimilar, all rely on a careful regression analysis of the collected data to pinpoint a cause for changing behavior. The ability to perform that analysis is so important that Shaw tells students in her data-driven management class that if they can’t do a regression, they’re in the wrong class. (A regression analysis is a technique for modeling and analyzing several variables with the goal of understanding the relationship between a dependent variable and one or more independent variables.)
Being qualified to analyze data doesn’t mean it’s easy to get it; indeed, diplomacy and the use of informal connections can be almost as important as mathematics. Companies may be concerned about privacy issues or be reluctant to divulge proprietary information, says John Roberts, the business school’s John H. Scully Professor of Economics, Strategic Management, and International Business. “Normally companies don’t hand out data, so we have to get in and earn their trust.”
When Edward Lazear, the school’s Jack Steele Parker Professor
of Human Resources Management and Economics, was studying incentive pay for auto-glass installers in the late 1990s, he had the “ultimate ally” inside the firm, CEO Garen Staglin, MBA ’68, a business school alum, Roberts recalls. “Had Garen not made the data available, Ed could never have done this tremendously influential work,” Roberts said. In that case, piece-rate pay was better than hourly pay.
Another example is Yanhui Wu, a former journalist now at the London School of Economics, who leveraged his knowledge of the publishing industry to get inside a major Chinese daily newspaper to study productivity between 2004 and 2006. The issue is sensitive enough that Wu is not free to mention the name of the newspaper, but he found that when the paper centralized more authority in upper management, reporter productivity increased. Product quality, as defined by the managers, also increased.
Insider researchers get as close as possible to the firms and people they’re studying. Some are taking that a step further by conducting controlled experiments with the cooperation of management and comparing productivity not just across industries but also across countries. Stanford’s Roberts, Bloom, Aprajit Mahajan, and two colleagues conducted a two-year field experiment examining management practices in the Indian textile industry.
Rather than simply observe, they hired consultants to work with one group of companies and teach their management “best practices” to improve productivity. A second group of firms that were not steered toward change acted as the experiment’s control.
The researchers saw productivity in the first group increase by about 10%, along with gains in profitability, while productivity of firms in the control group increased by just 1%. [See details of study]
It showed, the researchers said, that management really does matter.
— Bill Snyder
Also on Stanford Knowledgebase: