National Archives Announces Major Milestone of Electronic Records Archives System

Archivist of the United States Allen Weinstein announced today the launch of a major initiative
which lays the foundation for preserving electronic and all other records generated by the government and providing public access to them. The Initial Operating Capability of the National Archives Electronic Records Archives (ERA) is the beginning of far-reaching changes in the management of U.S. government records.

In its initial stage, the new system will support the basic process of determining how long federal agencies need to keep records and whether the records should be preserved in the National Archives afterwards. ERA will support this process for all federal records, whether they are paper, film, electronic, or other media. In July, the National Archives will start moving approximately three and a half million computer files into ERA.

These historically valuable electronic records range from databases about World War II soldiers to the State Department's central files on foreign affairs. The records eventually will be accessible online in ERA.

ERA is a multi-year project spearheaded by the National Archives and Lockheed Martin, the development contractor, to create a "permanent" solution for the ever-changing challenge of preserving electronic records.

Because new formats of electronic records are constantly being created and older formats become obsolete quickly, the "permanent' solution" cannot be a one-time fix. It has to be a dynamic system which can grow to accommodate ever-increasing volumes, be extended to deal with new formats, and evolve to enable records on obsolete formats to be accessed on new computers. The goal is to enable researchers 50 or 100 years from now to find and retrieve
electronic records using the best technology available to them, regardless of what hardware or software was used to create them. ERA will also move record keeping out of filing cabinets and into cyberspace. It provides a foundation for the National Archives and all other federal agencies to perform business transactions online to improve the way government records
are organized, stored, and retrieved. Besides the direct benefit to government, these capabilities will make it easier for citizens to discover what records the government has and to access them.

The National Archives has received ongoing counsel from the Advisory Committee on ERA which is chartered under the Federal Advisory Committee Act to advise the Archivist of the United States on technical, mission, and service issues related to ERA. Its members include experts in archives, records management, libraries, computer science, history, and the
law, including representatives of other federal agencies and of state governments.

Chairman of the Advisory Committee on ERA and co-inventor of the Internet Robert Kahn says, "The Internet has made it possible for people using a computer to instantly communicate across continents and around the world. When the full system is deployed, ERA will make it
possible for people to access U.S. government information across generations."

Watch more breaking news now on our video feed:



Bookmark http://universeeverything.blogspot.com/ and drop back in sometime.

Computer Scientists Develop Solutions For Long-Term Storage of Digital Data

Although the digital age is well under way, one crucial detail remains to be worked out--how to store vast amounts of digital information in a way that allows future generations to recover it.

"The problem is how to build a large-scale data storage system to last 50 to 100 years," says Ethan Miller, associate professor of computer science in the Baskin School of Engineering at the University of California, Santa Cruz.

Tape libraries are widely used for data storage, but digital tape has many shortcomings as an archival medium. Miller's group has come up with a new approach, called Pergamum, which uses hard disk drives to provide energy-efficient, cost-effective storage. The declining cost of hard drives has made them more competitive with tape, and they offer numerous advantages for searching and retreiving data. "It's like the difference between a VCR and TiVo," Miller says.

Pergamum, named after the ancient Greek library that made the transition from fragile papyrus to more durable parchment, is a distributed network of intelligent, disk-based storage devices. The team that developed it includes UCSC graduate students Mark Storer and Kevin Greenan, along with researcher Kaladhar Voruganti of NetApp (formerly Network Appliance), a company that focuses on storage and data management solutions.

Archival storage is a big issue for businesses, partly due to legal requirements for the preservation of financial and business records, and also because data mining strategies can turn stored data into a valuable resource. Long-term storage is also a growing issue for individuals who are filling their personal computers with digital photos, movies, and documents.

"There is a risk that an entire generation's cultural history could be lost if people aren't able to retrieve that data," Storer says. "Everyone is switching to digital cameras, but we've never demonstrated that digital data can be reliably preserved for a long time."

Pergamum has attracted a lot of attention from industry since Storer presented it at a leading conference in the field, the USENIX Conference on File and Storage Technologies (FAST '08), held in San Jose in February. Robin Harris, an industry consultant who writes an influential blog called StorageMojo, called the Pergamum paper his "favorite FAST '08 paper)."

The researchers designed the system to provide reliable, energy-efficient data storage using off-the-shelf components. It also has the ability to evolve over time as storage technologies change. "You want to avoid 'forklift upgrades,' where you have to get rid of the old system and transfer all your data to a whole new system," Miller says.

According to Storer, businesses are beginning to recognize that archival storage is very different from simply backing up their data. "A backup is a safety net--you hope you won't need it. Archival data you do want to use--it's a valuable resource and you want to be able to mine it for information," he says.

Tapes work well for backups, in which data are written once, rarely read, and not kept indefinitely. But archival data should be easy to read, query, browse, and search, and tape has inherent weaknesses in these areas. Existing disk-based systems offer excellent performance, but rely on power-hungry central controllers.

"Energy usage is a big issue, so a lot of our effort in designing Pergamum focused on dramatically reducing power use," Miller says.

Pergamum uses individual building blocks consisting of a hard drive; a small, low-power processor (like the chip in an iPhone); a flash memory card; and an ethernet port. These units, called "tomes," are connected using relatively inexpensive ethernet switches.

"Each tome is like a minicomputer, but with very low power demands," Miller says. "When not in use, it can shut down almost completely."

Even when active, the devices use very little power (less than 13 watts), which can be delivered over the network using Power over Ethernet technology. As a result, each unit is essentially a self-contained box with a network connection. The flash memory provides low-power, persistent storage so that many operations can be performed without activating the hard drive.

For reliability, Pergamum uses two levels of redundancy--within and between disks--to protect from both disk failures and errors in writing data to a disk (so-called "latent sector errors"). Tomes can be easily added to expand the system or to replace failed disks. And if hard disk drives become obsolete in 10 years, Pergamum won't suffer the same fate. The system doesn't care what the actual storage medium is, as long as the device can implement the simple protocol that will allow it to function as part of the network.

"In 50 years, the devices might use holographic storage," Storer says. "As long as you can wrap the new storage medium in this intelligent layer that speaks the protocol, it can participate in the network."

Watch more breaking news now on our video feed:



Bookmark http://universeeverything.blogspot.com/ and drop back in sometime.