Processing RAW files in Linux

[ Sunday, 15 June 2008, michalrz ]


In this article I’m going to show the benefits of storing the photos in RAW “format”. I’ll be also decribing GNU/Linux tools that can help you with processing such images.

Dragonia Magazine

I framed the word “format” with quotation marks on purpose, since it’s not a particular way of saving an image; it is rather an umbrella term for many formats, implemented by digital camera manufacturers in their products. Nikon makes use of the nef format with partially hidden info about correction curves, Fuji has its raf format, etc.

The term RAW, used in the field of digital photography, comes from English and has the same meaning as “raw”. It is common for manufacturers to supply customers with software for decoding their proprietary software.

Decoding is not an easy thing to do, because the light-sensitive points on an array (CMOS or CCD) do not sense colors, only light intensity (the light is filtered separately for each primary color), which means creating a “mosaic” image in the camera. The camera then has to process the image in such manner, as to provide each pixel with full information about the color (based on the neighboring monochromatic points). It is also necessary to correct the white balance, and some cameras also adjust the contrast and saturation. The electronic hardware in the camera has thus a plenty of complex operations to accomplish after the shutter button is pressed.

Because the computational power of even the most expensive cameras is limited, and an important feature affecting the choice of a camera is the number of photos per second and the overall operation speed. Manufacturers often take the shortcut and implement such processing and compression algorithms (to JPEG or TIFF), which do not optimize the picture’s quality. They even use tricks such as ignoring data from pixels on the edge, since they are computationally “difficult”. Meanwhile we could perform all these actions ourselves using much more powerful processors (present in a typical desktop computer), while preserving better image quality.

Advantages of RAW in a nutshell

  • Ability to use a better algorithm for converting colored dots into pixels while processing the pictures using a computer.
  • Better control over white balance (while correcting the balance of a JPEG we are working on a picture already “spoiled” by the camera).
  • JPEG is limited to 8 bits for each color channel (256 levels), while the camera usually records 12 bits (4096 levels). This data is accessible in RAW.
  • Better exposition correction than simply darkening or brightening of the picture, or gamma correction.

The basic and the most obvious drawback is the size of the resulting file (e.g. to a 2 MB JPEG there may correspond 15 MB of losslessly compressed RAW) and the need to convert the image before previewing in popular image viewers.

The problem for many camera owners is that not all manufacturers support RAW photography; sometimes they even give up TIFF in favor of JPEG in several quality variants.

In the following part of this article we will focus on a couple of GPL-licensed applications capable of handling these digital negatives.

The dcraw tool

This command line program authored by Dave Coffin and constantly enriched with support for new formats is a basis for many other GNU/Linux applications. The documentation is available multiple languages and the program itself allows you to use several interpolating algorithms (changing the “mosaic” of points into pixels), reconstructing details in overbrightened areas, noise reduction, choosing color space (sRGB, Adobe RGB, XYZ…), etc.

One serious drawback is the lack of preview, which means converting to ppm or tiff each time you want to view changes made (and this takes a couple of seconds).

The upper photo in figure 1 shows the result of running the command:

dcraw -n 150 -H 5 -w -q 3 -b 2.5 photo.raw

and saving the result as JPEG using GIMP, while the lower image has been cut out of a lossily compressed photo created by the camera. The option -n sets the level of noise reduction, -H – the way of interpreting very bright areas (0 – reduction to whiteness, 5 – reconstruction), -w forces the use of white balance info supplied by the camera.

Figure 1 – Upper photo – conversion from RAW using dcraw, lower – JPEG from a camera (Panasonic DMC-FZ30). A section (1/5 of the original) framed from the middle.

The interpolation algorithm is chosen by the -q switch (where 0 – low quality, 3 – AHD algorithm). It may be necessary to set the brightness level by trial and error (adding a number after -b)

Adding a -v switch will make the program verbose. For the sake of comparison the upper photo has been saturated with color (to make them similar). The colors of the photo created by the camera are rich, but even with the highest quality set, there are typical JPEG artifacts visible. Image created with dcraw out of RAW has more details in the bright areas, the colors are more balanced but also more realistic (lacking the spots visible on JPEGs, especially on tree trunks).

Figure 2 presents one of dcraw functions – restoration of overbrightened areas. Restoring areas from too dark or too bright will be much more successful using RAW and curve correction, than the similar correction done in GIMP on a lossy JPEG image.

Figure 3 shows restoring details in RAW (upper) and JPEG. In both cases a similar flattening correction curve was used.

The upper image appears to have more different colors, but with less saturation. Green in the lower photo is more juicy, but practically homogenous. Moreover, the reddish clouds on the left have much less details in the lower photo. The lower, in turn, has sharper edges and more saturated colors (but less hues). The original has been marked in figure 2 by a red stripe.

Rawstudio graphics editor

For much easier processing of the RAW pictures we can use a program called Rawstudio. For opening images it utilizes dcraw behind the curtains, thus granting support for a broad spectrum of popular digital cameras. However it is equipped with a very friendly, dark (facilitating working with graphics) interface made in GTK and the ability of opening the entire directory with photos simultaneously. In the upper part of the interface there is a toolbar displayed with thumbnails allowing you to switch between the pictures. Corrections may be made to any of the photos and immediately after that one may proceed with another (The program records changes on the run in the XML files placed in the picture directory). Controls altering the images are arranged in three tabs, enabling quick comparison between different configurations. Additionally, there is a split-screen mode, juxtaposing two versions of the same photo.

Among other things one may adjust correction curves and color saturation; exposition and modifications are made in a 16-bit color depth. Images can be queued to launch automatic modifications according to a predefined pattern (including a pattern for file naming).

Parameters can be copy-pasted between photos. A bit annoying detail is the fact that while using the correction curve some adjustments may yield a total blackening of the picture. But, that aside, Rawstudio is a very potent and convenient application, undergoing constant development. What may seem strange is that it lacks noise reduction, despite it being supported by dcraw.

Figure 2 – Uppermost photo – conversion from RAW using dcraw with -H9 option (emphasis on bright bright areas), lower – attempts at achieving similar results using gamma correction and JPEG exposition; red frame marks the original.

Figure 3 – Upper – curve correction in Rawstudio (working on a RAW file), lower – similar curve on JPEG in GIMP.

There may also be problems opening pictures taken by some Fuji cameras, since an unorthodox layout of the array points results in the picture being rotated. Such pictures may be properly opened using dcraw, although they get rotated counterclockwise, but that may be handled easily (-t option in dcraw). Rawstudio, in turn, shows the photo in a very weird position, which (as for version 1.0) I haven’t been able to handle properly.

Figure 4 – Rawstudio interface, split-screen view.

UFRaw

This is yet another free solution, operating either as a plugin for GIMP, or a stand-alone application.

Launched, the program welcomes the user with a picture choice window, which, even after opening an image for editing, remains present (but inactive) under the main window, allowing to open another file when needed. It lacks the photo directory browser known from Rawstudio.

The program supports (among others) Polish language interface, but it still requires many fixes. Nonetheless one quickly gets accustomed to the manifold of different options. These options are categorized under particular concepts like white balance, curve, color management, framing and rotating.

The program displays a lot of useful information like dimensions, EXIF data, histogram. It also lets the user choose details such as interpolation algorithm (just like dcraw).

In general it is much more mature than Rawstudio. The interface lacks directory preview and it’s bright, but instead stuffed with more options.

Figure 5 – UFRaw.

All these programs allow compressing raw data into a lossy format (usually JPEG), and certainly grant the user much more control over retouching the photo, offering more details to work with. What’s more, it can often be done much better (closer to our expectations), than by the camera.

Certainly, not every picture requires such elaborate processing, and hard drives tend to fill quickly if we decide to keep photos exclusively in RAW. Some cameras (like Panasonic FZ30 used for our tests), even when set to RAW, make an additional JPEG file to allow for later choice which one to keep for further modifications.

Links:

About the Author

Michał Rzepka

Dragonia Magazine editor, proofreader, translator. KateOS user.

New AdTaily ads!

Are you a film buff?

film buffs community, movie recommendations and reviews

RSS: Comments

You can follow the comments to this article through a special channel RSS 2.0 .

Related articles: Graphics

 more »

PolishLinux Top Content


Become our fan on Facebook!

PolishLinux.org on Facebook

Follow PolishLinux on Twitter!

Follow polishlinux on Twitter

Google Ads