The Library of Congress Unlocks The Ultimate Archive System

Ken Weissman Culpeper, Virginia USA ©2010 CreativeCOW.net. All rights reserved.

Our division within the Library of Congress is responsible for the Library's 6.3 million piece collection of audio-visual and film materials. I direct the film preservation program, and prior to the advent of the Packard Campus for Audio-Visual Conservation in 2007, we were primarily focused on the nitrate film collection: 140 million feet, dating back to the 1890s.

Cellulose nitrate is a very flammable material, but it also happens to be a very high quality clear plastic -- 35mm motion picture film was almost exclusively nitrate, from the beginning of filmmaking through 1950 or so in the US, possibly as late as the early 60s in Russia and eastern Europe. But after a number of notable fires, it was banned for projection in a theater except under very special circumstances.

Photo by Ken Weissman, Library of Congress, as are all photos in this article, except where noted. Article title graphic: scene from "The Last Days of Pompeii" (1913), courtesy of the Library of Congress.

It was replaced by cellulose triacetate, but because of nitrate's nature to deteriorate, the content on all of the thousands of previously collected films was at risk. They needed to be copied to safety prints, which is primarily what we do.

The bulk of the nitrate-era films are black and white, so that has been our specialty. Chemical formulas have been tweaked from time to time, but the development process hasn't changed very much since it was invented.

At the same time, not many people work with black and white motion picture film anymore. In some ways, we are redefining the art of black and white film development.

But copying old films to new stocks is not as simple as it sounds. There are many factors that can impact your ability to be able to do that very well, especially with nitrate archival films. Shrinkage is the most notable. Original era nitrate negatives were showing significant shrinkage shortly after they were shot and processed. After it was developed, the film had to be dried again, so that it didn't stick to itself, and one of the plasticizers used to keep the film flexible was quite volatile, and could shrink considerably. In some cases, you can actually see the perforation from the original negative printed onto the original positive.

Other challenges come from running a fairly soft material -- the nitrate film -- through pieces of steel, the projectors, and the sprockets. If things weren't running quite perfectly, bases and emulsions got scratched. Static electricity attracted dirt to the surface, where it could be embedded in the emulsion.

RTI-Treise film processors

All of these problems can show up on new prints, so we have various pieces of equipment designed to help us overcome those.

The introduction of wet gate printers, or immersion printers, was the sea change in the world of preservation. They allowed laboratories to make fairly pristine copies from films that were otherwise horribly scratched, by immersing the film in a liquid -- perchloroethylene, the same fluid used in dry cleaning -- to fill in the scratches.

Even when we look at films that were preserved prior to the wet gate era, and those done after the wet gate era from the same negative, it's a remarkable difference.

Now for many people, "restoration" refers to just taking a version of the film that they have available to them, and making a new copy. I object to that usage, because it overlooks the first question: which is the right version to restore? There can be several different negatives and, especially as you get back to the 40s and earlier, there are often no notes to describe what changes were made, or why.

"Mr. Smith Goes to Washington," courtesy of Sony Pictures.

For example, we did a restoration of "Mr. Smith Goes to Washington" in 1989-90, and we were able to identify six different versions of the film, from roughly 119 minutes to 132 minutes. So, what are you restoring TO? In our case, we chose to restore to the longest version. Based on research, including newspaper reports from the time, we became convinced that this was the version screened at the original premiere - but that is hardly always the case with every film. This is why we work so diligently to determine what it is we're restoring, and also where the best materials are. We are not afraid to ask colleagues at other archives if we could borrow material on a title, to find if it's as good as ours, or perhaps better.

In fact, there have been times where we determine that the best material is that in other archive. If they significant material to contribute that might be better than ours, perhaps they should take the lead. Not that we want to set anybody else's preservation or restoration agendas, but we have a responsibility to maintain goodwill with our fellow archives, and to work together to determine where a project might best take place.

Once we know that we are working with the best material, on the right version of the film, there are other questions where the answer is not so easy. Let's say, for example, that a film appears underdeveloped. Clearly, there's art involved, so we make a qualitative judgment as to whether it was intentional, or poorly done lab work. If necessary, we can essentially force-process the master positive and to a certain degree adjust the gamma to make it more "normal." Now there are ethical decisions that have to made: if this is the way it has always been seen, even by the original audiences, what right do we have to change it?

Whenever we make decisions like that, we document our reasoning. I have no problem with people being critical of our approach. If someone were to provide evidence that we took the wrong approach, we can go back to the original and redo it with the new information -- although, quite frankly, we don't get criticized very often, because I think our reasoning is pretty sound.

PAPER PRINTS

We started as a photochemical laboratory, and are primarily a photochemical laboratory to this day. It has only really been in the past half a dozen years, or less, that you can even begin a conversation that might convince people in the know that preserving motion pictures might be done digitally. So here in our lab, we began a pilot digital project for a very special collection that we have in the Library of Congress: the paper print collection.

These paper prints exist because of a vaguery in the copyright law at the time that motion pictures were invented. The Copyright Office at the Library interpreted the law to say that a motion picture film is simply a series of still photographs, and therefore the still photographic copyright law applied. If you wanted to copyright a motion picture, you had to provide the Library of Congress two copies of the film, and they had to be on paper. Not film.

A process was invented to literally create long strips of photographic paper, exactly the size of 35mm film stock, and then create contact prints from the original 35mm negatives, onto those long strips of paper. These were then deposited with the Library.

A reel from one of the Library's paper prints. Click image for larger.

There are over 3000 titles within that collection, some of the earliest films ever made -- from 1894-1915, with the vast majority from before 1912. Most of them are unique. In other words, these paper prints are the only copies of these films. They represent the single largest collection of early motion pictures in the world, by far. The Library is rightfully very proud of this collection.

The paper prints had been locked in a vault in the bowels of one of the library buildings, and rediscovered by librarian Howard Walls in the late 30s. The paper itself is still stable, but for the most part, you can't see the images very easily except by looking directly at the paper -- where of course there is no motion.

This is why there have been several attempts over the course of history since their rediscovery to put them back on film. One of the first was by Kemp Niver, and his company called Renovare. He took these 35mm prints (and there are some that are actually a larger gauge than that), and re-photographed them using a clever device that he built, printing to 16mm film. We have used various models of these Niver printers, including one where we replaced the 16mm camera with a 35mm camera, in order to print back to 35.

All of the processes have been interesting, and they've all been successful to some degree. However, they've also been unsuccessful to a great degree, in that the images are alternately soft, or fuzzy, or very shaky. There was also no way to accurately register the images. In fact, we've concluded that in many cases, the images aren't very well registered on the paper.

The obvious solution is to scan the images, then take advantage of digital processing to stabilize them, correct positioning and so on. Our first scans of the paper prints were 2K x 2K, which theoretically should have been good enough, but in our analysis of the imagery, we think it might be better to go to 4K x 4K. But that's one part of the pilot program, to figure out exactly how to do it. It's more of a theoretical workflow because we haven't practically implemented it yet, but we're getting close.

PRESERVATION FOR "BEYOND FOREVER"

One of the Library's main missions is to preserve America's memory for future generations of Americans, with no end point. Our goal is to have these images, and the sounds associated with the images, available for your grandchildren's grandchildren's grandchildren and beyond -- literally for hundreds of years.
Several years ago, the Image Permanence Institute developed the concept of Preservation Index (PI). This is a measure of how ambient temperature and relative humidity affect decay, and is expressed in years.

We have differing conditions depending upon the materials stored in them and their use. Our nitrate film is stored at 39 degrees Fahrenheit, with 30% relative humidity. This equates to a Preservation Index of 655 years.

For non-nitrate film preservation masters (also known as "safety film"), we store at 25 degrees Fahrenheit, 30% relative humidity, for a PI of 2125 years. All of our new film preservation masters go into this storage environment.

The remainder of our collection includes magnetic tapes of all types and flavors, both audio and video -- Edison cylinders, wire recordings, metal stampers, etc. You name it, we probably have it. These are stored at 45 degrees Fahrenheit, 30% relative humidity, for a PI of 429 years, and in some cases, at 50 degrees, 30% relative humidity, for a PI of 244 years.

It is important to note that, yes, the Preservation Index is measured in number of years, but as defined by the Image Permanence Institute, it is not a fixed number. The PI is relative, since you seldom know how the previous storage of a collection item has impacted its overall life expectancy. Take a roll of film for example. Even new film stock is subject to a wide variety of conditions on its way to permanent storage. Kodak makes it and stores it under pretty good conditions, but then it is shipped by a truck through who knows what kind of weather. Maybe it sits in an uncontrolled storage warehouse for several days before it is shipped to a customer. Even under cool office conditions, film would only last about 50 years before serious degradation could occur, but stored at 25F and 30% relative humidity, you can expect it to last 40 times longer than that - 2000 years.

These numbers are relatively non-controversial, so we can take them as a starting point.

That's why, as we move further into digital technologies, the plan for now is still to scan the images, restore or preserve them as needed, then run them back to film, and put the film away at 25 degrees, 30% relative humidity, for practically forever. For most people, in practice, somewhere between 600 and 2000 years is beyond forever. Because frankly, once you get to that point, what are you really worrying about?

The Packard Campus includes 35 climate controlled vaults for sound recording, safety film, and videotape, and 124 individual vaults for more flammable nitrate film, one of which is shown below.

On the other hand, you have hard drives, or any kind of digital storage. You can't look at the files in and of themselves. You need a lot of technology to bring them back, and always have to wonder if, when the time comes, if that technology will still be available. The beauty of film is that the machine it takes to look at the images is amazingly simple. You need a light source, you need a lens, you need something to shine it on, and then you can see it. I don't think I'm going to get radically criticized for saying that that technology is never going to go away.

Digital certainly has its contributions to make. Scanning film, working with it digitally, and printing back to film avoids the degradation you get from even the most controlled processing for multiple generations of film to film. However, as we look at staying in the digital realm, we're asking ourselves, "Okay, let's say we don't do film to film transfer as the main preservation workflow any longer. What's the impact of an all-digital workflow on data, and our data infrastructure?" And the numbers are really, really scary.

Speaking very broadly, with 4K scans you wind up in the neighborhood of 128 MB per frame. Figure that a typical motion picture has about 160,000 frames, and you wind up around 24TB per film. And that's just the raw data. Now you process it to do things like removing dust, do digital restoration work. Each of those steps develops additional data streams and data files to be saved.

I asked one of our IT people today, what if were to digitally preserve a typical feature film? A typical film scanner might have 5 to 10 terabytes of direct storage that's associated with it. Once you fill that, then you have to offload the media to another SAN for post processing, and then once you're done with post processing, you then have to put in a deep archive. If you want to do anything with it later on, you have to pull it back out of that deep storage archive.

The process of pulling out a single terabyte of data from the deep archive obviously depends on the speeds of the digital infrastructure. Right now, I'm told that moving one terabyte from our robotic tape-based system to a SAN where we could do some more processing on it takes between 3 to 5 hours to complete -- and that's with 10 gigabit interfaces. The numbers are staggering.

Everyone is pretty much agreed that you had better migrate that data after five years to the next latest greatest thing, or you risk losing it. And of course, you want to have a backup copy. I've even been at several conferences and meetings in the last couple of years where people are saying, "No, no, no, no, you want to have at least TWO backup copies." On separate servers, separate geographic locations, the whole bit, because a single backup that you make might not be able to be restored. You want that second backup, just in case.

And you've committed to migrate it all, every five years. That's not going to be cheap, and it's going to take a lot of time.

There have been conversations among the Association of Moving Image Archivists over the past few days, though, weighing what it would actually cost to store a film for 500 years or more, perhaps MUCH more, versus migrating data over the same span. There are legitimate arguments on both sides, but suffice it to say that cold storage at 25 degrees and 30% relative humidity for CENTURIES costs money too, and it's not necessarily simple to do it over that length of time.

There may be no way that you can actually calculate it, but I can't help feeling in my heart of hearts that the simple solution is usually the best. And film is a pretty simple solution.

We have had a very robust preservation program over the years, and we still have a long way to go. We have only been able to restore a small percentage of the films that we are preserving, and we're collecting new films all the time. For example, we recently acquired a privately owned collection of films about three years ago that added almost 15 million feet to our nitrate film collection, and we know that there are many more large collections out there.

In the meantime, we have our prototype project for digital archiving in place, and we are looking to build on that in the future.

Ken Weissman Culpeper, Virginia USA Ken has worked for the Library of Congress since 1981, and is now Supervisor of the Film Preservation Laboratory at the Packard Campus for Audio Visual Conservation. Ken has directed the Library's restoration of such films as MR. SMITH GOES TO WASHINGTON, THE MALTESE FALCON, WHERE ARE MY CHILDREN?, THE BLUE EAGLE, BIG FELLA, and, most recently, a restoration of Paul Robeson's THE EMPEROR JONES under a grant from the National Film Preservation Foundation.