Archiving Projects Among National Broadcasters

　　This article identifies best practices and archive policies among broadcasters throughout the world.

　　Kane has studied and worked with more than 60 major TV and radio operations in Europe, North America and Asia, and has completed a benchmark report sharing other broadcasters?experiences in the specification and implementation of digital archive systems and associated new working practices.

　　The major elements of any broadcast archiving policy include the following:

　　‧Tapeless architecture, along with Digital Storage and Media Asset Management

　　‧Implementation strategy

　　‧Metadata

　　‧Workflows

　　‧Format issues

　　‧Deep Archive Recovery

　　Metadata is "data about data" and is the information (such as title, duration, and format) which identifies and describes "essence" or raw media data.

　　If "operational process" describes the overall set of tasks which are required to achieve a given job, then "workflow" is the word used to describe the individual tasks and steps required.

　　So what does the report reveal in each of these categories? What is the archiving "best practice"?

Tapeless Architecture
　　The first requirement of a tapeless architecture is that it must link together all parts of the enterprise, and include acquisition, production, delivery and archiving in a single integrated infrastructure. This philosophy is not new, of course: it has been like this in radio for over a decade, and in TV is becoming more and more common, driven by the progressive replacement of videotapes by IT files.

　　100% of the Kane's panel of 61 broadcasters who produce news are using an IT-based production architecture, and this is rapidly spreading to other genres. For playout, 87% of the panel use only video servers or IT disks, with no VCR involved.

　　There are two crucial elements to a tapeless architecture which need special mention.

　　Most broadcasters no longer archive current production on media containers such as videotape, and Digital Storage has become the norm. 80% of Kane's panel now use digital storage, including disks and robotic data libraries using data cassettes. Disks are used for on-line material which must be instantly available, and data libraries, which are still cheaper than online disks, are used for near-line storage, where retrieval takes slightly longer. Data libraries are often used to share media between post-production, playout and archive, with specific partitions.

　　Equally important is Media Asset Management (MAM). Since there is no longer a physical container, with its contents written on the box, new methods are needed to index and locate media objects. MAM systems integrate metadata and essence within the same application, and are rapidly replacing non-integrated "text" documentary IT applications or paper index cards. MAM Systems rely on a central database for metadata and include ingest, categorization and keywording, a search engine, low resolution browsing, quality control and usually a broad range of interface capabilities.

　　MAM systems give users comprehensive searching tools, which work over multiple media and data types. They can directly browse found material, and select any of it for further processing or even for immediate delivery or transmission.

Implementation strategy
　　After a first generation of digital storage and MAM applications which were largely dedicated to a single function (such as production, delivery or distribution) broadcasters are now clearly looking for a more cross-functional approach.

　　However there is a division between those who implement a commercially available software package (such as the Flemish-language Belgian state broadcaster VRT who chose Ardendo) and those who build "home-grown" MAM applications (such as the BBC or NHK.) Some broadcasters acted as test-beds for applications which later became standard products (SVT with Ardendo, SBS Seoul with Konan).

　　Among those who adopt standard packages some broadcasters combine different products and vendors in a "best of breed" philosophy, such as Turner (who combine components from TMD, Pharos, and Pro-Bel), while others have a more traditional "one vendor" approach such as PBS with Artesia.

Metadata
　　Metadata is a much-discussed and often contentious subject. The choice of category is crucial, and must serve both current and future needs for description, interchange, and perennity. Achieving consensus on an industry-wide scale without the category list (or "schema" becoming unmanageably large is a real challenge. Metadata is often divided into 5 categories: identification and documentary, editorial, technical or related to physical storage, rights, and administrative.

　　Many broadcasters also try to use a standard thesaurus, or at least a controlled vocabulary, in order to improve search efficiency, reducing variation in indexation choices made by individual archivists.

　　Broadcasters also try to keep track of all rights contracts regarding elements of programmes such as footage, additional music or photos.

　　MAM applications manage, in the same repository, all media types such as video, audio, still images such as photos and graphics, and documents such as scripts and subtitle. A "container" concept is used to link objects together and enable all elements of a package to be replayed together.

　　Some broadcasters are pioneering methods to assure interoperability between systems, and persistence over time. Some, such as Disney, implement a standard taxonomy and adopt the same data definition through the entire organization. Others (such as VRT Belgium) use an external data dictionary like P-Meta, developed by the BBC and adopted by the EBU (ie. VRT), or Dublin Core. Some norms such as OAIS, also provide guidelines for persistence.

Workflow
　　Understanding and optimizing workflow is a key to efficient operation; for example eliminating a situation where synopses, credits and storyboarding are applied to a given package several times, by different teams, in various different systems, and are still not available openly in the archive.

　　One very general workflow change in this area is the involvement of archivists much sooner in the production process, in order to ensure the effective collection of the proper metadata at all stages.

　　Metadata collection is facilitated by Production Management Systems that help production collect seamlessly all useful items for future archive purposes.

　　Alternatively some metadata are automatically collected and generated by systems. Shooting information is generated by cameras and transmission metadata is generated by automation.

　　Best practice is to input information only once. This requires interfacing different IT and broadcast systems such as traffic, production management, MAM (logging) and automation.

Formats
　　The growth of what is sometimes called "360 degree delivery" means that broadcasters are now being expected to deliver content in a whole range of different formats: Standard and High definition for broadcast, analog and digital for radio, and a range of downloadable or streamable online video, audio and graphical formats.

　　Many broadcasters adopt a "pivotal" format for production and then transcode to any useful format for any distribution channel. It is a frequent practice to create one or more low resolution versions at the ingest stage for multimedia distribution, browsing and editing.

　　A crucial question is which format to archive in, to maximise future-proofing while creating the most savings on disk capacity and physical space.

　　In search of interoperability, an significant percentage of broadcasters use BWF for audio and MXF for video.

　　Video format choices are driven by production, as broadcasters tend to minimise the requirement for transcoding.

　　Half of the Kane's panel already broadcast at least one TV-channel in HDTV. In Japan, Korea and China this percentage reaches 100%. As a result broadcasters also store a HD version when produced natively in this format. The HD format is becoming "pivotal".

Deep archive recovery
　　Every broadcaster, especially those with a long history and heritage, has thousands of hours of video and audio archive to preserve and digitize. The BBC's patrimony is in the millions of hours.

　　Physical deterioration and obsolescent playback devices have already been mentioned as issues; the costs of digitization and indexing are also a significant issue.

　　Although all of the Kane's panel is storing new and current production digitally, 80% of them lack the budget to fully recover past content. Some hope to finance digitization through archive sales, but this is viable for a small percentage of patrimony, the so called "Crown Jewels" and this is likely to range from 1% to 5% of the total archive.

　　Although only 20% have already started a systematic recovery process, many historic broadcasters intend in the future to recover some patrimony through additional funding. Broadcasters tend to adopt a three-part selection process for the preservation, digitization and "distribution" of archive material to either professional or public customers, based on sales value, heritage value and physical decay urgencies.

　　Many broadcasters implement internal workshops or laboratories to carry out "fossil format" recovery and digitization. During digitization, they will aim to recover as much information as possible from existing "manual" indexing systems to reduce the amount of work required.

　　Alternatively they subcontract these tasks to specific service providers or may create local joint ventures with them to reduce the cost of transfer.

Conclusion
　　The governments of almost every country in the world consider the digitization of historic archives by their Public Service Broadcasters to be a key element of cultural strategy. Some governments also consider this as a support to companies engaged in high technology and as a possible generator of new jobs.