Lost in…

….translation.

“Lost they get, things in translation”,

as Yoda might have said, in a galaxy long, long ago and far, far away where they all had human-type vocal cords and all spoke English. And all breathed the same oxygen/nitrogen mix.

So… to your camera. Assume that you shoot in RAW or JPG format—which is the case for almost all cameras today. Let’s assume RAW.

When you translate that original file to what you see on the screen, you are doing exactly that: interpreting and translating that RAW file. And translations and interpretations always bring inaccuracies. They will rarely if ever improve your file; they may decrease the quality (and often do). When I say never improve, I mean that you cannot get extra information out of a file by interpreting it. You can’t pluck a bald chicken, as the Dutch saying goes.

When you import into Lightroom, you can change the translation. In the DEVELOP module, bottom right, you get CAMERA CALIBRATION options like these, for example:

All those are just different ways of interpreting the RAW data. “Portrait” is a little less sharp with more emphasis on skin tones; “landscape” sharper, with more greens. And according to Adobe’s best reverse engineering of what Canon or Nikon do, when you select thatsetting on your camera.

Now let’s get to the point.  Some people say “Adobe’s DNG format rocks: it is the standard”. Lightroom offers you the option to automatically change the original file into a .DNG upon import.

Here’s the problem with that, in my view: by doing this, you are throwing out the original data (this goes squarely against the face of non-destructive editing) and trusting Adobe’s interpretation of it; what’s more, you are trusting Adobe’s interpretation of it more than you are trusting your camera’s maker.

And then when exporting, you will in most cases once again make a translation, this time from DNG to JPG or perhaps TIFF, or directly to your printer driver via the printer profile, if you print directly from Lightroom.

And each translation brings with it the danger of misinterpretation. Like that game where you whisper into the ear of the person next to you, who does the same with the person next to them, etc, and by the time it comes back to you, “my mother is wearing a red coat” has turned into “why are the ancient Greeks developing antiserum in the library?”

It is nice that DNG is a standard, but it’s not a generally accepted one yet, and the advantage of that move to standard is not worth throwing away the original data that your camera produces. So until cameras themselves produce DNG files (as indeed some already do), my advice is: no, do not convert your files to DNG upon import. Leave them in your camera’s RAW format.

 

7 thoughts on “Lost in…

  1. Hi Michael:

    I am a little confused here. Are you referring to converting Canon “.cr2” files into the “.dng” format upon importing OR Camera Profiles? or both?

    I didn’t get the impression that converting .cr2 into .dng threw away any digital information. It was a means of eliminating the pesky “.xmp” sidecar files that lived side-by-side with your .cr2 files.

    Camera Profiles appear to be a different issue. I am shooting with an old Digital Rebel Xsi and Lightroom defaults to the “ACR 4.4” profile. When I click around and choose Adobe Standard or Camera Faithful or Camera Portrait the tones do seem to change (from good to bad depending on your opinion).

    So, what profile would you suggest ACR 4.4? (or equivalent)

    Thanks!

  2. DNG is not an “interpretation”, nor is it lossy. The pixel data in the DNG file is, bit for bit, identical to the pixel data in the original camera raw (CR2, NEF, etc.) file.

    Compared to the original raw file, DNG makes the following changes:

    – The image data and metadata are in a standard TIFF/EP container, using standard Exif, XMP and IPTC tags and standard CIE and ICC colour reference data. (In a camera raw file, the format is proprietary and usually undocumented, and any third-party program that wants to add metadata must put it in a separate “sidecar” file to avoid the risk of corrupting the cr2 or nef file.)

    – All relevant characteristics of the camera, including its endianness, readout sequence and its exact pixel and Bayer-filter grid layout patterns, are included in the DNG file. (A cr2 or nef file, by comparison, just states the camera model, and the program opening the file must look that up in a separate database to get the actual image format descriptors. If that program’s database doesn’t have that specific camera model, the file is unreadable.)

    In other words: the DNG file contains the exact same data as the original raw file, but in a format that will still be usable decades from now without any proprietary, camera-specific tools. This is why many best-practices guides, including those of the Library of Congress and of the American Society of Media Photographers, specify that camera raw files be converted to DNG early in the workflow.

    • I think this is a little more complex. From digitalpreservation.gov (http://www.digitalpreservation.gov/formats/fdd/fdd000188.shtml):

      “The raw image data is processed for placement in the DNG file wrapper by software applications like Adobe Camera Raw, a plug-in for PhotoShop. These applications give users options for processing the raw image data into various versions, as indicated by the “may contain” relationships in this description and discussed in Notes below. The “default” action usually produces a version of the still-mosaiced raw data compressed with JPEG_orig_LL.”

      …and:

      “The raw image data is processed for placement in the DNG file wrapper by software applications like Adobe Camera Raw, a plug-in for PhotoShop. These applications give users options for processing the raw image data into various versions, as indicated by the “may contain” relationships in this description”

      “The conversion applications also generate special metadata about image characteristics that is recorded under DNG tags that represent an extension to the normal TIFF set; see Notes below. The special metadata is later used in various applications that interpret the camera raw data to produce a variety of image outputs.”

      “All the operations that take place during demosaicing are set in stone and can’t be redone.'”

      “The specification also describes how raw camera sensor values are mapped to linear reference values (p. 45) and how camera color space is mapped to CIE XYZ color space (pp. 47-48).”

      “Mapping” means interpreting. Also notice all the “usually”, “interpret” and “may” disclaimers. It is not at all clear that all pixel-level info is placed uninterpreted and unedited into the DNG file, Matt; not to me anyway. If you can take a DNG and revert it back to its RAW file, that’s when I would believe that it is not interpreted. But reading the above, it seems clear to me that this is not the case. Do you know otherwise?

    • Also, even if it is lossless, which I debate, it is still an interpretation; and the interpretation of a DNG will differ from the interpretation of its DNG. “Interpretation” does not necessarily imply loss. It does, however, imply change.

      • The DNG spec and reference implementation are freely available: https://helpx.adobe.com/photoshop/digital-negative.html

        The format as used by Adobe products. can be summed up as:
        – Metadata, as standard Exif / XMP / IPTC tags.
        – An image data container, usually lossless compressed. When you convert to DNG using Lightroom etc., this is the raw bitstream from the camera sensor, copied bit for bit from the CR2 or NEF file.
        – Metadata describing exactly how to map that bitstream into an image. This includes the description of the camera’s colour filter array pattern, the exact colours of those filter elements, the the sensor’s dark current floor, etc.

        That last component is needed whether you are working with a CR2, NEF, etc. file or with a DNG. The difference is that in a DNG, it is embedded in the file itself, while in native formats it must be looked up in the converter program’s database every time you load the file.

        It is completely possible, in theory, to convert a DNG back to the original fresh-out-of-camera raw file. No tools exist to do this because (a) there is no need to, and (b) nobody other than Canon knows exactly how to create a valid CR2 file. The only thing you might lose are a few rare proprietary metadata tags, if such exist, that aren’t used or supported by anything other than that specific camera.

        It is possible and completely legitimate to put a lossy compressed bitstream, or a demosaiced, linearized, compressed or uncompressed image in the DNG instead of the fresh-off-the-sensor bitstream. Lightroom has an option to save a lossy compressed, demosaiced image instead of the raw sensor data, and also to embed the entire original raw file as a binary blob inside the DNG so that you can pull it out and run the original Canon converter on it if you ever need to. These options are rarely used (and, IMHO, should be removed.)

        If you’re happy with CR2 or NEF, then by all means stick with it. Converting to DNG is only required if you want / need the file to work exactly the same in 2037’s software as it does in 2015’s software.

        • If you doubt this, by the way, try the following:
          Take a CR2 or NEF file. Have Lightroom make a DNG copy of it, and import that, so the original and the DNG are both in the library.
          Select both, make sure Auto Sync is on, and do your processing. Export both to a lossless format (TIF).
          Load the two TIFs into Matlab, ImageJ, etc. and diff them. You will get an array where all values are 0, indicating that the images are bit-for-bit identical whether you started with the DNG or the CR2.

Leave a Reply to Robert Cancel reply

Your email address will not be published. Required fields are marked *