What follows is a draft of a chapter I prepared for a book entitled “Stuff That Actually Matters”. The book’s theme was to provide background knowledge and theory on social issues, to help people make sense of current events. The hope is to avoid some of the ideological traps that can and do ensare those who understand machines and mathematics better than human beings.
My chapter is based on “Seeing Like a State”, by James C. Scott, and addresses some of the ways big projects, run by well-intentioned people, can nonetheless fail and cause massive harm.
Paved With Good Intentions
In the 1940’s, Chicago’s poor were crowded into tenement houses, enduring awful conditions. In an effort to provide better living conditions, the city built high-rise housing, such as the iconic Robert Taylor Homes. The intent was to provide stable, safe, affordable housing. The result was a cycle of poverty, violence, and incarceration for the residents. In the 1990’s, the city undertook a new initiative to tear down the “projects”, moving tens of thousands of residents into “mixed income” communities and subsidized private rentals (“section 8” housing). This plan, too, faltered and left many families living in even worse conditions.
It’s a common enough story: a project begun with the intent to improve the human condition ends having hurt those it was meant to help. Urban highway projects intended to alleviate traffic congestion lead to divided cities and still worse traffic. Programs to lift subsistence farmers out of poverty with improved agricultural techniques have often ended in agricultural collapse and famine. History shows technologies failing to achieve their promises, too: film, television, computers, and the Internet were all supposed to revolutionize education, but as Larry Cuban describes in “Oversold and Underused”, that has never come to pass.
The simple diagnoses for these failures are incompetence, greed, or hatred. The cycle repeats when a new generation decides they are competent, charitable, and altruistic, and thus need not learn any lessons from previous failures. But a closer look reveals some instructive deeper themes.
A Recipe For Failure
James C. Scott’s comparative analysis, “Seeing Like a State”, is one such closer look. While it is focused on state-sponsored projects, his findings are applicable elsewhere. He identifies four components of large-scale projects that ultimately end in failure.
The first component, administrative ordering, refers to the sort of simplifying assumptions any large-scale project must make. For example, a project as basic as a census assumes that each person has a single name and lives in exactly one place. Similarly, a model for agricultural prediction will define an expected yield per acre planted, glossing over details like microclimate, drainage, and so on.
The second component, high-modernist ideology, sees science and technology as providing solutions to every problem, and by contrast portrays indigenous approaches as primitive and obsolete.
Many of Scott’s examples focus on agriculture, where mid-century western agronomists were convinced that their techniques and technology – largely developed in the plains of the US – could increase crop yields globally. He describes finely tuned indigenous farming practices, adapted through generations to difficult terrain. For example, many practiced intercropping, planting more than one crop in the same field to secure loose soil against erosion. This practice was seen by the newcomers as primitive and wasteful, not least because the resulting fields did not display the straight rows of produce so iconic to the US Midwest.
It is worth noting that professional scientists do not necessarily adhere to high-modernist ideology: by training, scientists are careful to describe the caveats and limitations of their findings. Technologists take those findings, discard the caveats and limitations as minor details, and proceed to make plans based on the conclusions. The high-modernist ideology is evident in their promotion of these plans.
The third and fourth components constitute an imbalance of social power: a small group has the power to see their project implemented, and the people affected by that implementation are unable to mount an effective resistance.
Often, the power is political power, as in the Ujamaa villagization project in Tanzania. In the 1970’s, the government of Tanzania attempted to move its residents, formerly mostly subsistence farmers scattered throughout the country, into regular villages, as a part of the broader Ujamaa program. Having recently escaped the yoke of colonial rule, the Tanzanian government was popular and had a “mandate” to improve the new nation’s circumstances. Ultimately, the project involved forced movement of thousands of rural farmers who did not have the political clout to resist the project’s increasingly coercive implementation.
Just as often, powerful businesses or organizations can attempt high-modernist projects. For example, the Bill & Melinda Gates Foundation heavily supported the “common core” project in US education. Without the foundation’s vast financial backing and political connections, the project would not have existed. The people most affected by the implementation – students, teachers, and parents – were fragmented by disagreements over other contentious issues like privatization, funding, and pay, and unable to speak with a consistent voice on the program.
How do these components come together to cause ambitious, progressive projects to fail?
The administrative ordering required to carry out a large-scale project seldom matches the messiness of the real world, and especially of real people. Abstraction is a critical part of any engineering discipline, and critical to managing any large-scale endeavor. But when applied to represent a person with a row in a database, or to capture the particularities of a patch of soil in a surveyor’s map, something vital is lost. The scientifically-determined instructions for efficient cropping in an “average” patch run afoul of the particularities of this patch bisected by a brook or that patch blasted by dry wind from the nearby hills, and the yield suffers. Some users don’t fit into the database – perhaps they do not have a first and last name, don’t fit the binary gender options, or they live rough and do not have a fixed address that passes validation.
From the top down, these seem like trivial details, but for the individuals affected, it is life and death: a family’s barren plot does not “average out” with the bountiful acreage next door when that family is starving. This difference in perspective corrodes from both ends: project managers struggle to keep the “big picture” in the face of an avalanche of details, while on the ground the project appears ever more nefarious. People resist or find workarounds, abandoning any sense of shared purpose.
As the project is implemented, a few additional pathologies manifest. The abstractions made in the design phase lead to quantifiable metrics for progress, and the adage “if you can’t measure it, you can’t manage it” quickly leads to its contrapositive, “only manage what you can measure.” Measurements naturally have errors, as any scientist will attest. And when measurements are aggregated in a hierarchical management structure, detail is lost and each level tends to “round up” the numbers to present a favorable picture to upper management. The result is executive-level reports that bear little resemblance, even on the attributes being measured, to reality. These reports omit the hardships occurring on the ground and lead to decisions based on faulty information.
The Ujamaa project began with an appealing notion of a communal village, with shared, local governance and shared fields. But it is much easier to measure the number of people trucked to a village location than to assess the quality of communal decision-making. Officials competed for political favor by relocating more people and establishing more villages, regardless of the actual conditions of those people in those villages. Based on the resulting aggregate numbers, Nyerere made boastful claims of success that must have seemed farcical to any peasants who heard them.
The misalignment between the perspective of those managing a project and the perspective of those impacted by it quickly erodes any sense of common purpose that may have existed during the early stages. The edicts from on high are nonsensical or impossible. From the management perspective, the people are resistant to what is best for them, and if they would just follow instructions, they would see how wonderful the plan is. Even if coercion wasn’t present at the start, it soon enters the picture, increasing the divide and quickly becoming an end in itself. Officials often cut corners by omitting parts of the plan that might curtail their own power, such as devolving control to village governments – a double standard villagers were sure to resent.
Common core illustrates these effects well: the project began from what appears to be a consensus: that the US education system is underperforming and that improving it would be beneficial to the nation, society, and the economy. It is founded on evidence-based research in education (high-modernist ideology) and establishes nationwide standards for what students should understand (administrative ordering). Teachers see this as limiting their autonomy and their skill at adapting instruction to the students in their classroom. The large-scale measurement that comes with common core – high-stakes tests – are seen as a threat to teachers’ employment and to students’ success. And parents protest that the curriculum is not appropriate to what they believe their children should know. Opponents brand the project’s supporters as greedy self-dealers looking to profit from their children’s schools. Over the decade of implementation, the parties have lost any mutual trust in common motivations and are now in pitched political battle. Common core is unlikely to succeed in raising education levels in such an environment.
High Modernist Ideology Today
While many of the historical examples discussed so far focus on government-led projects, the problem is not exclusive to the public sector. In fact, with the rise of global corporations and a western penchant for public-private partnerships, the private sector is often heavily involved in broad, ambitious projects.
All four components make appearances throughout the modern economy. To build systems at scale, engineers make simplifying assumptions in the form of abstractions, statistical models, and inferences from limited data. The fervent promotion of big data, social networking, and the sharing economy as modern miracles is certainly a kind of modernist ideology. Free-market proponents, too, display an ardor for market solution that borders on the religious.
We see, too, a power imbalance in the private sector that allows huge projects to proceed without substantial input from those who are affected. For example, Google and Facebook are at once indispensable and unaccountable to vast numbers of people.
Private prisons, covered in detail in Kim Moir’s chapter, can be viewed through Scott’s lens. Such prisons are sold to governments as cost-saving measures, on the faith that private business is ipso facto less expensive. In fact, the cost savings come at the expense of the people incarcerated in the system, as the contractual relationship reduces the visibility and monitoring of the conditions in the jails. The operators may have laudable intent, but their profit margin depends on economies of scale, so they cannot tailor services to specific communities, and have a strong incentive to increase populations to fill beds and provide more profit to shareholders. The ensuing alienation of the communities they serve is inevitable.
A more technical lesson comes from Facebook’s “Free Basics” program, intended to give rural India free access to Facebook and its affiliates – but not to the rest of the Internet. Mark Zuckerberg unveiled this plan with lofty, ambitious language about Internet connectivity as a human right and reducing access costs to allow anyone to connect. Yet the program was met with fierce resistance from those it meant to serve and was soon banned by the Indian telecom regulator.
The lesson here was a simple one: millions of people cannot be dazzled into submission by lofty rhetoric and fancy technology. What Facebook saw as an untapped pool of valuable consumers for paying advertisers was, in fact, millions of real people with different needs. Individually, they saw little benefit to the service, but understood that it meant being in thrall to Facebook. Facebook’s response, a massive public-relations campaign, is reminiscent of government frustrations at people “irrationally” resisting what is good for them.
The hubris of high modernist ideology has, in various forms, led many projects to failure. While grand state-planned projects are easy to spot and criticize, the same pattern repeats on a smaller scale in market-driven projects. How can we avoid these problems?
What Scott calls high-modernist ideology is a kind of hubris: modern science and technology – or just common sense – have given us the answer, and any doubts or contradictory evidence must be based on old beliefs and narrow-mindedness. The simplicity is alluring, but the real world is complex, especially where people are involved. The problem of people bullying autonomous vehicles illustrates this clearly: it is not enough to build a vehicle that can operate on an empty road or even a road shared with other responsible users. The technology must be capable of handling dangerous situations where other users are making illogical, unsafe decisions.
An over-reliance on administrative ordering signals problems to come. People show enormous diversity on all scales, and a proposal that assumes sameness ignores that diversity at its peril. Ride-sharing companies have embraced this diversity, tailoring their service to local preferences, needs, and regulations, while still providing a consistent experience for their customers. And the most successful charities make a deep commitment to local decision-making and rely on local staff to implement their projects.
Scott’s advice focuses on what he calls “metis”: the local knowledge and individuality of the people involved in a project. In the Ujamaa program, this was the intimate familiarity that farmers had with the land they had grown up on: its microclimate, intercropping techniques to avoid erosion, and so on. By removing peasants from their traditional land and forcing them to farm unfamiliar territory, the Tanzanian government discarded an incredibly rich knowledge-base, replacing it with crude approximations that delivered lower agricultural yields.
Incorporating metis is “messy”, expensive, and difficult to automate or scale. It often involves people talking to people, whether in focus groups, a customer service relationship, or with local specialists on the ground. And it involves incorporating those conversations materially into the project.
Existing systems and practices, however broken they might be, embody substantial knowledge. This is true even in software engineering: a deployed application has already seen patches for countless bugs and exhibits lots of small behaviors users rely on. Tempting though it might seem, replacing the application wholesale just returns to the beginning of that process, re-introducing bugs and breaking important features. There is wisdom in preserving existing knowledge by minimizing disruption, preferring transformation to outright replacement. The same analysis applies to government programs or to whole markets, where “disruptive change” can amount to a temporary blindness to the hard constraints that have shaped the existing market.
Projects that “drain the swamp” and start from scratch are attractive in their simplicity, but beware: that simplicity will be their undoing. A solution that explicitly recognizes complexity, incorporates local knowledge, and is flexible enough to adapt to changing conditions is far more likely to succeed.