The Blackout History Project: A Virtual Machine that Would Go of Itself

James T. Sparrow

The Blackout History Project, GMU

Presented at
Using the World Wide Web for Historical
Research in Science and Technology

Stanford University
21 August 1999

[1: start with splash page up]

The Blackout History Project is a little different from the other four original Sloan-funded grants within Science and Technology in the Making (STIM). It leans more toward social history than toward the history of technology, and its exploration of the latter centers around two dramatic technological failures: the Great Northeast Blackout of 1965 and the equally infamous New York Blackout of 1977. You might suspect that studying technological failure is not the best way to start a web-based research pilot. Quite the contrary.Ý The web is teeming with people obsessed with technological failure--witness the mounting Y2K mania. And the history of this particular pair of failures has a special place in the public memory of New York. From my few years living in New York City after college, I had noticed that New Yorkers loved telling stories about themselves. (Who doesn't?) And this was a story nearly everyone had told, in one form or another, since the dark days of the 1970s. (Spike Lee's recent movie, "Summer of Sam," is a well-publicized example of this phenomenon.) So, the propitious combination of topic and medium gave the project an appeal that promised to mitigate the many obstacles that face the social historian of the recent past.

To attract attention, and thus respondents, it helps to have a little drama.Ý In the case of the blackouts, the drama was suggested by the very opposition between the two events. [2: Great Northeast Blackout of 1965] The first power failure, in 1965, was caused by a single faulty relay that triggered a cascading system collapse of interconnected utilities, darkening seven northeastern states and a large part of Ontario in less than 13 minutes. The second failure was more localized in its causes--a series of unfortunate lightning strikes along a narrow transmission corridor in Westchester County, just north of New York City, which separated Con Edison from the upstate generation on which it relied to meet nearly half its daily load.

Yet if the technological causes of these two events were distinct, both highlight our society's profound dependence on electricity and other complex, interdependent systems. They reveal, by way of contrast, the social arrangements which such ubiquitous technologies sustain. New Yorkers who lived through these events sense this telling fact almost viscerally, remembering the blackouts as moments of truth that mark the City's harrowing descent into criminal chaos. Blackout "survivors" recall in almost chorus-like fashion how the 1965 blackout had unveiled the City's underlying civility and basic goodness, prompting men to direct traffic, office workers to rescue trapped co-workers from dank elevator shafts, and allowing neighbors (that endangered species of New York) to get to know each other over candlelight. [3: New York Blackout of 1977] Almost twelve years later the moral of the story was reversed: within the first hour of darkness in that sweltering night during the "Summer of Sam," looters acted unhesitatingly to tear down store grates, commencing a two-day spree of mayhem and rioting in the poorest sections of the five boroughs. In 1965 New Yorkers had worried momentarily about Russian sabotage, or whispered ominous speculations of UFO activity, but in their homes and with their neighbors they came together. A few short years later their neighbors had become the menace, and all it took was "the flick of a light switch" to set off a citywide riot.

The significance of this opposition is more than topical: it's the "hook" on which I hung my plans for recruitment and popular engagement of the site. It had an added appeal. These two power failures also represented milestones in the recent history of the electric utility industry, introducing after 1965 a strong government oversight of interconnection and reliability in the form of power pools and coordinating councils, and encouraging the practice of load shedding after the events of 1977 demonstrated the perils of placing legal liabilities before social responsibilities. For their historic significance, and because of their relevance to the reliability questions raised by deregulation and the catastrophic visions inspired by Y2K, the blackouts also drew the attention of my second study population, managers and employees in the electric utility industry.

Now I had two large, interested audiences to draw to my site and hopefully catch in the historical spider's web I was building. Two potential sources of historical perspective seemed at least twice as good as one; and their intersection, made possible by the web's capacity to combine broadcast publication with nano-niche targeting, evoked the intriguing prospect of "multivocality" and "non-linear narrative." Inspired by this vision of historical subjects rapturously enfolded in strands of intersecting discourse, I decided upon a model of database-driven, object-oriented content accumulation that would allow my respondents to "make" their own history recursively, by precisely linking their memories and donated materials to other parts of the site. This, I thought, was basically how public memory worked, and now we could use the web to both model and facilitate it. I was about to get a post-modern lesson in "power/knowledge" (pun intended), and it wouldn't be what I expected.

I devised parallel structures for the site, assuming that despite their differences in substantive interest, the two groups--New York "survivors" of the blackouts and electric utility industry employees--could interact with the site using the same mechanisms. [4: forum] The discussion forum might feature very different topics (blackout experiences, electric utility perspectives, New York in the 1960s and 1970s) but each would work the same, and I hoped there might even be some crossover (e.g. Con Ed employees who were off-duty). Two on-line survey instruments, each aimed at the respective audiences, would feature a number of identical attitudinal and demographic questions to allow for cross-group comparisons. [5: sabotage survey] Interviews were likewise peppered with overlapping questions. This blurring of feedback boundaries would be most pronounced (and, I hoped, productive) in the comments which all visitors could make on any page by clicking on a ubiquitous lightbulb. [6: highlights 77, never again, click lightbulb] I wasn't quite so naÔve as to assume that my two study populations would behave in the same fashion. (Power engineers are tough, but not New York tough.) I expected New Yorkers would probably lean toward highlights like the timeline [7: timeline], whileÝ electric utility industry people would gravitate to certain sections, such as the forum. [8: electric utilities forum] But the idea was to create a multifaceted on-line space that would be all things to all respondents: a forum in which to vent special-interest or just picayune observations for some, a big tent of the past for others. First-time visitors would be drawn into the site according to their curiosity, directed accordingly by the highlights, [9: transition rollovers] which would act as a sort of portal to the site, and the nuggets they found there would channel them to sections allowing them to satisfy their particularistic interests. [10: blackout baby]

This seemed feasible because of the metadata system I devised with the Stanford team. It would do everything: keep contributions distinct, categorize them for search, yet record all interactions to track overlap and intersections of visitors' feedback. Most importantly it would allow newly contributed objects, such as a comment on an interview, to inherit the metadata properties of the object to which it referred. I envisioned retired Con Ed system operators leafing through the FPC report on the 1965 power failure, or the DOE report on 1977, and being inspired to contradict the official findings with a direct observation from their own experience. Likewise I hoped New Yorkers would have their memories jogged by something someone said in an interview, or by a photograph in Life, and consequently insert a comment that opened up the meaning and perspective of otherwise static materials. But what I learned is that neither blackout survivors nor retired electric utility employees are at home on the web, much less within an object-oriented, metadata-driven time machine. They may be comfortable, even sophisticated users of the internet, but they are not accustomed to reliving their lives in it the way other media, such as film, television or radio, might allow them to. The web lacks the authority of a documentary or a published book. At the same time, it confronts the potential respondent with the omnipresent anonymous gaze of a potentially global audience, and that produces a form of stagefright which discourages frank or thorough disclosure.

The blurred/dual audience proved to be a blessing and a curse. I did have retired Con Ed people talking in broader terms about he meaning of the blackout [11: interview transcript], and at least one New Yorker was obsessed with technological failure to the point where the archival materials on the electric utility industry interested him. I think the site is more interesting because of this dual emphasis on social history and technology. But addressing two audiences at the same time, within the same web site, was exhausting and distracting. Generally speaking, power engineers and other members of the electrical utility industry are not provoked by the same sorts of materials that push the buttons of New Yorkers.

I'm not sure it paid off. Perhaps it would have been better to create two totally separate web pages, or begin with one group--the power engineers in the electric utility industry--and then invite the general public to the site after it had matured with contributions from sysops and linemen.

Whatever strategy might have worked best, I quickly learned that I could not bootstrap participation in this project from sheer content or topical appeal: my archival documents, far from being the "content magnets" I have envisioned, were a source of frustration or indifference for web visitors. For reasons of time, funding and project emphasis we found it was best to make scanned documents available as Adobe Acrobat files. [12: PDF from archive] But this only angered those respondents interested in these items, since (a) a technical limitation of many older Acrobat readers is they require a full download before viewing what are very large files (hundreds of pages), and (b) not all items could be scanned, and seeing citations to undigitized materials only served to tantalize people. A similar problem besets the interviews, many of which are (or will be) available in RealAudio format. [13: real audio of interview] When you can actually hear this, it lends an authenticity and texture to the project that can't be matched by transcript. But even I have trouble getting my laptop to load Real Audio without some fatal glitch. As with the PDFs, RealAudio was a more cost-effective way to present these primary sources--even more "authentic" in their trueness to the original--but they didn't meet the iron rule of web rubustness. You simply have to go with the lowest common denominator if you want to ensure that your materials will be seen by the widest audience, and to ensure that they persist (e.g. HTML 3.2). The positive feedback loop here is a killer: innovators can be trapped in a spiral of their own originality and technological prowess if the crowd they seek to attract decides to go the other way.

In this respect my early and fundamental emphasis on using metadata to "grow" the site was a problem. I think the metadata system we devised turned out to be a success on the technical merits, and has tremendous value as a tool for future research projects targeted at tightly-knit communities of respondents whose social networks mesh well with the internet. However, my specifications made it so complex that I'm still trying to iron out features of the on-line survey, and thus have had to rely on a quickly-built replacement in Filemaker (using Tango) to plug the gap. This was a costly investment that delayed my ability to reach out to my target population, and one which held absolutely no appeal to them. [14: metadata form] Within a longer time frame I think the metadata system would be more effective, and I think its development was merited by the wide use to which it may be applied. But if I were to do this over again, I would start with technologically more primitive devices, such as simple CGI-scripts for web forms and newsgroups that you can download and run in an afternoon, and stir up initial participation, and then port that material over to a well-tested database of metadata-encoded objects.

In fact, it might have been helpful to break apart the various aspects of our projects and fund them separately: some money for digitization or access to pre-digitized objects, some other money for cataloguing,Ý some money for recruitment and interviews, some money for database programming, and then some money to let the researchers do what they do best; research. To some extent this was done by having the Stanford Team handle digital library issues like metadata and programming--and their involvement in this project has been a critical factor in what accomplishments have been possible. But due to the big differences in the five projects, we all ended up spending lots of time on production-related tasks that might better have been outsourced. This project has turned me into something of a jack of all trades, shifting hats and jumping somersaults in a desperate effort to get the attention that would draw feedback.

It turns out that I didn't need the exciting new technology of the web to get that attention. In fact, some tried and true technologies worked very well without the aid of a single electron mediating their influence. Personal, face-to-face contacts, accompanied by networking, yielded an impressive number of respondents. True, working the email and the phones amplified that impact, and continues to draw more people into the study--but not without my constant involvement to keep the top spinning. The same could be said of another "low-tech" approach: I placed a print ad in New York Newsday and the New York Post, and that garnered a momentum of contributions to the site that is still adding surveys to the database. [15: Newsday ad] Again, the success of this approach was amplified by other technologies: a toll-free number connected to a capacious voice mail system makes it easy for me in some cases to record reminiscences without even doing an interview, and listing the URL in the ad clearly attracted some people to the site. Print and other established media have also treated us well when it comes to editorial content: my assistant, John Summers, recently placed a very effective piece with the History News Service that linked the ire of the Giuliani Administration at Con Ed's recent reliability problems to the 1977 and 1965 blackouts. An ad and a possible article in the IEEE Power Engineering Review should also do much to drum up visits to the site by members of the electric utility industry. (The jury's still out on this point, as we are still in the midst of recruitment, and about to put up the second survey for power engineers.) The lesson for the Blackout project is clear: not only is print not dead, it has trumped the web (so far) in its ability to get people where they live. The problem with print is that it is labor-intensive when done effectively, and its persistence (outside of libraries) is evanescent for most laypeople.

If conventional research techniques--archival work, interviewing, fact-checking--are not dramatically and unambiguously improved by the use of the web, the reverse is clearly true. Web-centered research does require an education of study populations, as we have heard repeatedly throughout this conference. More than that, it requires a habituation and a conceptual adjustment that transcends the instrumental reasons most people browse the web today. This means building outward from the contexts in which people already make sense of their lives, and understanding that technologies work because of the social relationships they facilitate. More than one caller to the toll-free number hadn't noticed the URL in the ad that sent them to me, but when notified they chose to do the on-line survey rather than the telephone interview. (This was not always the caseósome preferred the personal connection of the interview.) Even when I was able to locate an ideal donor of historical materialsóan electrical engineering professor at Cornell (Robert J. Thomas) with an electronic case study of the blackoutsóthe file he sent by email attachment as we spoke over the phone turned out to be written in a version of Hypercard that we never could untangle.

When we think about this for a minute, it becomes clear that not only is the web a limited medium in which to conduct research on the vast majority of living subjects today, but it is barely considered to be a genuine medium by its best practitioners. If even the technological wizards of Silicon Valley need to meet face to face for a deal to be real, this profound reality has to ring home as a first consideration. According to a report by the Commerce Department in July, less than half of white households even own a computer; for white families earning between $15,000 and $35,000 a year, that proportion drops to 32%. For black and Mexican families the proportion is even lower: roughly 20%. Overall internet use falls along similar lines. When you compound the class-based realities of computer ownership with the educational factors that combine to determine computer literacy, you can see how far we have to go before the web is a reliable universe from which to draw study populations. This should not prevent us from building projects that will enrich and improve the fast-accumulating information cornucopia of tomorrow, but it should shape the paths along which we seek to evolve this medium.

So, was this project worth the investment? Or, to frame the question in a different, equally important light: does it rate a journal article? Yes and No. Yes, it was worth the investment, but the Blackout Project has not yet produced primary historical materials that would change the way we think about these events. This email message I got yesterday demonstrates the nature of many contributions [16: where were you when the lights went out?]: what at first appears flippant, on second look turns out to be an interesting leadóbut the historical value of the contribution (even the identity of the contributor) remains ambiguous. Although we are still in the process of recruiting people to the study, I suspect that even when, in the late fall, we direct a second survey and fresh sections of the site toward the more focused electric utility industry people, the contributions will continue to elude rigorous inspection. It remains to be seen whether the social context of web-based contribution produces more than anecdotal responses. And even recursively linked anecdotes don't rate a journal article.

So why was it worth the investment? I would make two arguments. First, the metadata system developed with the Stanford Team is a tool that can greatly enrich on-line data collection and scholarly exchange, even if it has yet to yield big results in this project (aside from comments in context). The real test of this tool will come later in the fall, when two new audiences are invited to become participants in the site: scholars and web designers. Then the recursive qualities of the system, its ability to capture the context of submission and later appropriation, will enjoy a fuller application. But perhaps as important as the tangible by-product (software code that works) is the process by which I have begun to disentangle the social contexts which make historical memory valuable and operative to my audiences. If we pay close attention to the dissonances and resonances between the spaces we build on-line and the actual people we beckon to visit them, we can slowly inculcate a more widespread habit among out target audiences of "making" history before it evaporates into the ether.