Using XMP Metadata Effectively with Lightroom

Yesterday I had an interesting discussion about the upcoming Adobe Lightroom 4 with a reader of this blog. One of the questions that came up was: "What are XMP files?"

That's an interesting question about a subject that's often overlooked. Many Lightroom users, in fact, need not concern about this very often. However, they're a great tool you should know about since they can dramatically improve your user experience and also show the way to a more proficient use of Lightroom (as well as any other programs understanding this kind of files).

What Is XMP?

XMP stands for eXtensible Metadata Platform, a standard proposed by Adobe Systems, Inc. to define, create, store and process extensible file metadata.

Many file formats already contemplate some mechanisms to store metadata, even forms of extensible metadata. The advantage of being able to embed XMP data into a file is a huge leap towards standardization, since you will be able to take advantage of it in different applications. Also, compliant applications will simply ignore the content they can't handle.

At times, however, embedding XMP data into an existing file isn't an option: that is the case, for example, when dealing with proprietary RAW file formats (such as Nikon's NEF). You can obviously save XMP data for such a file too, but you'll have to use a so-called sidecar file, that is, a distinct XMP file provided alongside the original file. This solution is more cumbersome and error prone, but some people will argue that it has its own advantages such as leaving the original file unchanged, thus avoiding potential file corruption.

Why Is This Important to Me as a Photographer?

For all your practical purposes, Lightroom is able to save all the information provided by the user during the development process, including Adobe Camera RAW settings, in an XMP-compliant way, either embedded (when the file format allows it) or in a sidecar file.

If your workflow is performed in just one application and one machine, you won't probably benefit so much from this feature. However, as soon as you need to move your images to another application, you will probably face the problem of preserving adjustments (which can be thought of as metadata themselves).

Even a simple scenario such as test driving another program (such as the upcoming Lightroom 4) or moving the files to another machine will require you to migrate metadata as well. If your application isn't storing metadata in the source file itself, or if you choose not to do so, you will need to export it and make it available to the target application.

Migrating Images And Metadata

The simplest scenario is migrating an entire Lightroom catalog from a Lightroom instance to another, as in the case of a Lightroom upgrade. Since Lightroom stores the information of all your photos into the catalog itself, which act as a centralized database, it's sufficient to perform a copy of it and simply import it into another Lightroom instance. This operation is straightforward and no further option will be required for you to preserve all your metadata.

Even if you need to migrate only a subset of your catalog to another Lightroom instance there's an easy way to go. Select the images you want to migrate and choose the File/Export as Catalog menu item. Lightroom will generate a brand new catalog for you to migrate:

Export as Catalog Dialog Box

Things get slightly more complicated when you need to move images to a different application and metadata cannot be embedded into the original file. In this case, you have to tell Lightroom to generate the relevant XMP sidecar files for the images to be migrated and move them alongside the originals. You've got two choices:

• Have Lightroom alway save an XMP sidecar file for every photo.
• Have Lightroom generate on the fly XMP sidecar files only for the photos you need to migrate.

Unless your requirements justify the choice, I would always choose the second option since the first will introduce a significant, and often unnecessary, overhead.

To have Lightroom always generate XMP sidecar files, you need to select the Automatically write changes into XMP option in the Metadata tab of the Catalog Settings window:

Catalog Settings Window

Since I always leave this option off, as you can see in the previous screenshot, Lightroom is warning me that "changes made in Lightroom will not automatically be visible in other applications". If you select it, Lightroom will generate a sidecar file for every photo you process and keep it in sync.

To generate an XMP sidecar file on the fly, all you need to do is using the Metadata/Save Metadata to Files menu item (or use the Command-S shortcut if you're on a Mac).

Either way, Lightroom will generate the relevant XMP files alongside your original images, as you can see in the following screenshot:

XMP Files Together With Their Originals

No matter how you decide to generate them, you can now take your images and use them into another application. All the metadata that the target application is able to understand will be processed and the corresponding changes reapplied.

This mechanism is interesting also in the case you're often working with more than one computer. You could keep the catalog in sync, as explained in another post, or just keep XMP files in sync. Once you move an XMP file, just put it alongside the original and have your application read the metadata from it. In the case of Lightroom, all you have to do is using the Metadata/Read Metadata from Files command.

You could argue XMP metadata is easier to backup than an entire Lightroom catalog. Well, I bet it is. However, you Lightroom catalog is more than that and chances are you're backing it up regularly as well, thus defeating this potential advantage. 

Is There a Simpler Way?

This procedure is pretty simple and many XMP compliant applications (such as Lightroom, Camera RAW and Photoshop) will do the heavy lifting for you. Provided the XMP file is distributed with the original, they won't even ask you anything: they'll just read the metadata and apply the relevant settings.

As explained in the previous sections, you rely on XMP files in two cases:

• You cannot modify the original image and embed metadata into it.
• You don't want to modify the original image.

If you want to preserve your original files, there's no other option but using sidecar XMP files. If you cannot modify the original image (probably because it's written in a proprietary RAW format) but would willingly have Lightroom modify it if it could, you should consider converting your images to the DNG format (either during the import phase or afterwards with a simple export operation).

DNG stands for Digital Negative and it's a file format from Adobe, Inc. whose purpose, amongst others, is standardizing a RAW format and enable multi-vendor interoperability. However, DNG is much more than that:

• It's based on TIFF.
• It's both a container of RAW images and processed images.
• Metadata can be embedded in a variety of formats, including XMP and EXIF.

Some camera manufacturers already support on-camera DNG. If your camera does not, you can use a converter (such as Lightroom) to generate a DNG file from a proprietary RAW file. You can even choose to embed the original RAW file into the DNG, which is something I would recommend you do since sometimes some proprietary RAW metadata might be lost during the conversion.

Since XMP metadata can be embedded into a DNG file and Lightroom will keep it in sync, you will need neither no sidecar XMP file to migrate your photos. To some extent, you won't even need to rely on the very Lightroom catalog itself.

If you want to help me keep on writing this blog, buy your Adobe Photoshop licenses at the best price on Amazon using the links below.