Archive, backup and data protection for media professionals (Part 2-3)

The word Archive is actually very old. The Romans called a building where they stored scripture rolls that were no longer needed for daily work “archivum”. Today´s digital file archive does the same. It holds all finalized productions, projects and assets in one place. This way it becomes the only place to look for finalized files. This is what is called a "Single Source of Truth“.

Read part 1 here.

The Media Archive serves:

• Returning customers with their requests of former productions 
• Reduction of search time for existing assets 
• Building a reference of media assets and projects
• Preservation of media assets and files (that may become or historic value)

Let´s have a look at what is not an Archive.

Misunderstandings of what an Archive is

• A directory „/Archive“ on a disk
• Loose hard disks on a shelf
• DVDs, CDs and Blu-rays without a Backup 
• All storage media without a catalog
• NAS storage without additional Backup
• All storage media without Metadata
• All storage media without migration support

These examples put files at risk and become obstacles along the way. A professional media archive has some requirements to provide robust protection and access over a long time. 

Basic principles when planning an Archive

1. Thorough planning
2. Include multiple perspectives and experiences
3. Soft- and Hardware that is fit for the job
4. Storage and storage media that fit requirements
5. Consistent follow through with workflow steps

If multiple persons are required to archive and/or restore, including their perspective early on improves the project. The workflow will profit immensely from multiple perspectives.

Metadata as key to the Archive

When years later files of a specific type, content or production are needed, metadata play an important part. Metadata describes the kind and contents of the actual files: What was captured, where and how? With a keyword, technical specifics or a description, files are a lot easier to find.

There are two kinds of metadata; technical and descriptive.

Often, recorders and cameras create technical metadata. These might include camera make and model, resolution, lens etc. Descriptive metadata describes who or what was captured, why and for what project etc. 

While technical metadata can be imported from a file, descriptive metadata has to be entered manually.

It is important to understand that tasks around archiving, due to their long-term implications, need awareness and responsibility. Any inconsistency or errors make finding files later more difficult or even impossible.

The file name can keep metadata

The filename is the most robust and simplest place for metadata. Important criteria and parameters can have their place here. This can survive transport from one platform to another or migration between storage generations. Even reassigning a misplaced file becomes easier this way.

Not all operating and file systems support the same characters. Avoiding incompatible characters is therefore important.

Blanks as well as the following characters should be avoided  / : * ? “ < > | [ ] & $

Some OS systems replace blanks with “%20” making reading filenames that contain them difficult. That´s why blanks should be replaced by  “_” (underscore) such as when used to separate categories as per the example below.

20200623_production_title_customer.mov

Copyright © and Trademark TM are other characters that are not supported on some platforms. To keep things simple capitalization should be avoided, too.

Numbers, dates and sorting

If the date is important for production it should be included in the filename. To enable easy sorting, the technical date format is the most helpful, and should be placed at the beginning of the filename.

YEAR-MONTH-DAY with the year in four digits: 

2021-01-23 or 

20210123 

for January 23, 2021.

For the same reason, all numbers should contain leading zeros for their maximum number of digits. Misunderstandings with big numbers can be avoided this way. It makes sense to be generous and think ahead concerning the number of leading zeros because they cannot be adjusted later without conflicts, e.g.:

buildings_farm_00014.jpg

Creating metadata on the Mac

Unknown to many people is the ability to re-name multiple files in one step.

After selecting multiple files (with similar content or criteria) go to the File menu and choose "Rename ... Items“.

This way you can easily add a keyword or description to a filename while keeping the leading number. This is a quick and efficient way to structure folders containing many files. All added keywords could later be used in a search.

Re-name files in the Finder.

The collection of metadata categories is individual for each production or company. Every company has its own focus that needs to be accommodated to enable productive work with the archive later.

While it might be customer names for one company it might be product names or numbers that are most relevant for another. All criteria that are relevant for later searches belong into the metadata scheme.

Check available metadata on a Mac

In the file header of each file, metadata is stored. This can easily be made visible on the Mac using the terminal.

All metadata in the file headers can automatically be imported into the metadata fields of the already mentioned P5 Archive. 

Here´s how to expose the metadata:

1. Start “Terminal” in the folder “Utilities”
2. Enter “mdls” (without curly quotes, short for metadata list)
3. Put a blank behind
4. Drag a media file (like a jpg file) into the terminal window to enter its path (a path can also be entered manually)
5. Press “return” to see the list of available metadata of that file
   
   

Terminal Screenshot mdls.

Example of a metadata scheme

As real-world example I've used the P5 data management platform by Archiware. One product is P5 Archive, the module for file and media archiving. Within P5 Archive one can easily build an individual metadata schema that can contain text fields and menus. They can be searched later.

This way, locations, scene descriptions, involved persons, technical specifications and more can describe each file. Additionally media thumbnails, poster frames and proxy clips can be displayed for media files to allow visual browsing.

Since media archives and asset collections can reach a tremendous size, while each file might be accessed only rarely, archiving pays off. For the same reason LTO tape offers many benefits and reduces storage cost. It has a shelf life of decades and is extremely cheap. More on that later.

Example of an individual metadata scheme in Archiware´s P5 Archive.

When filling metadata fields, consistency is crucial. Actually all files should use all metadata fields. This is the only way to find 100% of the relevant files in a later search.

As a consequence it is better to have less metadata fields that are filled consistently. Better to have 5 fields filled with all files than to have 25 fields that are only partially filled with input.

You can call this approach “modest metadata”. Foresight and sense of proportion are important.

A checklist can help building a cohesive archive.