<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<HTML>
<HEAD>
<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
<TITLE>Raw Digital Photo Decoding in Linux</TITLE>
</HEAD>
<BODY>
<a href="../index.html">Back to Dave Coffin's Home Page</a>
<center>
<H1>Raw Digital Photo Decoding in Linux</H1>
<p>
<b>
Not affiliated with
<a href="http://www.canon.com/">Canon</a>,
<a href="http://www.nikon.com/">Nikon</a>,
<a href="http://www.kodak.com/">Kodak</a>,
or any other company named herein.
</b>
</center>
<p>
Welcome! If you are wondering how to connect your digital
camera and download images to a Linux PC, go to the
<a href="http://gphoto.sourceforge.net/">gPhoto homepage</a>.
My software is for processing those images <em>after</em>
downloading them.
<p>
If you're downloading JPEG files, you don't need my software
at all. The image has already been processed inside the
camera. All digital cameras made since 1997 produce JPEG
images, so why would you want to do it any other way?
<p>
Well, despite the convenience and ubiquity of JPEG, there are
some disadvantages. JPEG is a lossy format -- to fit a big
image into a small file, a lot of information is thrown away.
That's why midrange and high-end digital cameras offer an
alternative: Raw, unprocessed CCD data files, for which the
camera manufacturer provides special decoding software.
<p>
Of course this software is for Windows and Macintosh only,
and there's no source code. So it's useless to users of other
operating systems, programmers hoping to design a better
interpolation algorithm, and historians not yet born in an
era when the only Windows machines will be in museums.
<p>
So here is my mission: Write an ANSI C program that decodes
any raw image from any digital camera on any computer running
any operating system.
<p>
That program is called <a href="dcraw.c">dcraw.c</a>,
and it's become a standard tool within and without the
Open Source world. It's small (about 3000 lines), portable
(standard C libraries only), free (both "gratis" and "libre"),
and when used skillfully, produces
<a href="http://www.insflug.org/raw/">better-quality output</a>
than the tools provided by the camera vendor.
<p>
<a href="../resume.html">Here's my resume</a>.
I do freelance consulting related to dcraw, and I'm also
available for full-time work in BIOS engineering or UNIX
system administration.
<p>
I can be reached by sending e-mail to cybercom dot net with
the username "dcoffin".
<H3>Free Software</H3>
There are no restrictions on this code, although I would like
to receive credit for it. Donations are welcome, especially
if you're using my code commercially.
<ul>
<li>
<a href="dcraw.c">dcraw.c -- Convert raw photos to PPM</a>
<br>Supports 95 cameras at last count.
Compile with "<b>gcc -o dcraw -O3 dcraw.c -lm</b>".
Run with no arguments to see a usage message.
Don't complain that 48-bpp output is too dark -- read the FAQ!
<li>
<a href="old/rawphoto.c">rawphoto.c -- plugin for GIMP 1.2</a>
<br><a href="rawphoto.c">rawphoto.c -- plugin for GIMP 2.0</a>
<br>After installing "<b>dcraw</b>",
do "<b>gimptool --install rawphoto.c</b>".
Now the GIMP can load raw photos without creating a PPM file.
For programmers, rawphoto.c shows how to create a graphical
interface to dcraw without modifying it.
<li>
<a href=".badpixels">.badpixels -- my camera's "hot pixels"</a>
<br>This file tells dcraw which pixels have died and when,
so that it can interpolate around them.
<li>
<a href="dcraw.1">dcraw.1 -- UNIX manpage for dcraw</a>
<br>Contains details and helpful hints not found elsewhere.
<li>
<a href="dcwrap">dcwrap -- example wrapper script</a>
<br>This is what I use with my PowerShot G2.
<li>
<a href="RCS/dcraw.c,v">dcraw.c,v -- complete unabridged RCS file</a>
<br>In case you want to learn more about the history of dcraw.c.
<br>If you don't have the RCS toolkit,
<a href="http://www.cs.purdue.edu/homes/trinkle/RCS/">
download it here</a>.
<li>
<a href="parse.c">parse.c -- read image data structures</a>
<br>This program displays CIFF and TIFF data structures in a very
cryptic format. It also extracts the thumbnail, if the image has
one. Compile with "<b>gcc parse.c</b>".
<li>
<a href="fixdates.c">fixdates.c -- set time on CRW files</a>
<br>Sets the time on CRW files to show when the photo was taken,
assuming that the camera's clock was set to Universal Time.
</ul>
For hackers only:
<ul>
<li><a href="decompress.c">decompress.c</a>
is a simple reference decompressor for CRW files.
<li><a href="pgm.c">pgm.c</a>
converts the decompressed data into an 8-bit grayscale image.
<li><a href="sony_clear.c">sony_clear.c</a>
decrypts SRF files from the Sony DSC-F828.
</ul>
<H3>Commercial Software</H3>
The following image viewers use dcraw or pieces of dcraw.c
to decode raw digital camera images:
<ul>
<li><a href="http://www.adobe.com/products/photoshop/cameraraw.html">Adobe Photoshop</a>
<li><a href="http://www.bibblelabs.com/">Bibble by Eric Hyman</a>
<li><a href="http://www.breezesys.com/">BreezeBrowser by Chris Breeze</a>
<li><a href="http://www.conceiva.com/">Conceiva Lightbox</a>
<li><a href="http://cpicture.de/en/t_raw.html">cPicture by Jürgen Eidt</a>
<li><a href="http://frostyplace.com/dcraw/">dcRAW-X by Bryan Chang</a>
<li><a href="http://www.lemkesoft.com/">GraphicConverter by Thorsten Lemke</a>
<li><a href="http://www.irfanview.com/">IrfanView by Irfan Skiljan</a>
<li><a href="http://www.astrosurf.com/buil/us/iris/iris.htm">IRIS image processor for astronomers</a>
<li><a href="http://www.lightboxsoftware.com/">Lightbox by Josh Anon</a>
<li><a href="http://www.wildcape.com/">Photo Companion by Jeff Moore</a>
<li><a href="http://www.polybytes.com/">PolyView by Polybytes</a>
<li><a href="http://lens.liteserv.com/download/powershovel2.php">PowerShovel-II by Luc Minnebo</a>
<li><a href="http://www.silverfast.com/">SilverFast DCPro by LaserSoft Imaging</a>
<li><a href="http://www.hamrick.com/">VueScan by Ed Hamrick</a>
</ul>
<p>
<H3>Frequently Asked Questions</H3>
<dl>
<dt><b>I don't have a C compiler. Could you send me an executable?</b>
<dd>No, but Francisco Montilla provides up-to-date Mac OS and
Windows executables
<a href="http://www.insflug.org/raw/">
on his website</a>.
And Benjamin Lebsanft has volunteered to maintain
<a href="http://home.arcor.de/benjamin_lebsanft/">
Windows executables optimized for specific CPUs</a>.
<p>
If you're familiar with the DOS command line but don't know C,
you can install this <a href="http://www.delorie.com/djgpp/">
free C compiler for Windows</a> and compile dcraw.c quite easily.
<p>
<dt><b>How can I read the EXIF data (shutter speed, aperture, etc.)?</b>
<dd>For JPEG files, Matthias Wandel wrote
<a href="http://www.sentex.net/~mwandel/jhead/">
an excellent utility called "jhead"</a>.
For Canon CRW files, try
<a href="http://neuemuenze.heim1.tu-clausthal.de/~sven/crwinfo/">
this tool by Sven Riedel</a>. Also
<a href="http://www.wonderland.org/crw/">
Peter Galbavy's website and mailing list</a>
have a lot of information about CRW format.
<p>
<dt><b>Why is 48-bpp output dark, flattened, or unreadable?</b>
<dd>If you want pretty pictures straight out of dcraw, stay with
24-bpp output. 48-bpp linear output is the best raw material
for professional image editors such as
<a href="http://www.adobe.com/products/photoshop/main.html">
PhotoShop</a> and
<a href="http://cinepaint.sourceforge.net/">CinePaint</a>,
but it's no good for most image viewers.
<p>
<dt><b>Could you please add an option for TIFF / PNG / BMP / JPEG output?</b>
<dd>No. Dcraw writes PPM only. To convert to other formats:
<pre>
dcraw -c crw_0001.crw | pnmtotiff -truecolor > crw_0001.tiff
dcraw -c crw_0001.crw | pnmtopng > crw_0001.png
dcraw -c crw_0001.crw | ppmtobmp > crw_0001.bmp
dcraw -c crw_0001.crw | cjpeg > crw_0001.jpeg
</pre>
I used the
<a href="http://netpbm.sourceforge.net/">Netpbm toolkit</a>
in these examples.
<a href="http://www.imagemagick.org/">ImageMagick</a>
also does command-line format conversions. Both are free.
<p>
<dt><b>Why don't your images look the same as Canon's?</b>
<dd>Such differences nearly always relate to brightness, gamma,
and white balance, which can all be set on the command line. See
<a href="rpd_rico">this posting on rec.photo.digital</a>.
<p>
<dt><b>What does the "-f" option do? What's "four-color RGB"?</b>
<dd>If you see <a href="screendoor.png">patterns like this</a>
in your output images, use the "-f" option. Otherwise, don't.
<p>
<dt><b>Why are there false colors along edges within the image?</b>
<dd>Because of interpolation. This is a hard problem, easily defined:
<ol>
<li>Take a three-color RGB image. At each pixel, set two color
values to zero.
<li>Reconstruct the original three-color image as best you can
from the remaining one color per pixel.
</ol>
<a href="http://www-ise.stanford.edu/~tingchen/">
This research paper by Ting Chen</a>
compares various color-interpolation algorithms. Dcraw uses his
favorite, the Variable Number of Gradients Method, with Bilinear
as an option for quick test images.
<p>
Other methods are described in
<a href="http://www4.ncsu.edu:8030/~rramana/research.html">
the research of Rajeev Ramanath</a>
and at <a href="http://dsplab.ece.cornell.edu/">
http://dsplab.ece.cornell.edu/</a>.
<p>
<a href="http://www.dpreview.com/news/0202/02021101foveonx3.asp">
The Foveon X3 Capture chip</a> requires a different kind of interpolation.
Unlike CCD arrays, it captures three colors at every pixel location.
But the colors are not well separated, so the raw data looks very gray.
Enhancing the color without also enhancing noise is very difficult.
<p>
<dt><b>Why are my Fuji images rotated 45 degrees?</b>
<dd>Interpolation artifacts are more noticeable along orthogonal
edges than diagonal ones. Thus Fuji decided to rotate the pixel
array 45 degrees, to favor orthogonal edges. You must rotate
dcraw's output back to normal. Some detail is lost, but the final
result shows very few artifacts.
<p>
I recommend the following commands for best results:
<pre>
dcraw -c dscf0000.raf | fujiturn | pnmscale 0.70710678 > dscf0000.ppm
</pre>
<a href="fujiturn.c">Download the source code for "fujiturn" here.</a>
<p>
<dt><b>How do I get my camera to take raw photos?</b>
<dd>
<p>
For some Nikon Coolpix cameras, you need to enable a
<a href="http://e2500.narod.ru/raw_format_e.htm">
special "DIAG RAW" mode</a>.
<p>
For Casio cameras, see
<a href="http://www.inweb.ch/foto/rawformat.html">
Maurice Delaney's website</a>.
<br>His server seems to have disappeared, so read
<a href="http://forums.dpreview.com/forums/read.asp?forum=1015&message=4961779">
this discussion on dpreview</a> instead.
<p>
For other cameras, refer to the User's Manual.
<p>
<dt><b>Does dcraw work with my camera?</b>
<dd>Most likely, yes. The following cameras are supported:
<ul>
<li>Canon PowerShot 600
<li>Canon PowerShot A5
<li>Canon PowerShot A50
<li>Canon PowerShot Pro70
<li>Canon PowerShot Pro90 IS
<li>Canon PowerShot G1
<li>Canon PowerShot G2
<li>Canon PowerShot G3
<li>Canon PowerShot G5
<li>Canon PowerShot S30
<li>Canon PowerShot S40
<li>Canon PowerShot S45
<li>Canon PowerShot S50
<li>Canon PowerShot Pro1
<li>Canon EOS D30
<li>Canon EOS D60
<li>Canon EOS 10D
<li>Canon EOS 300D
<li>Canon EOS DIGITAL REBEL
<li>Canon EOS D2000C
<li>Canon EOS-1D
<li>Canon EOS-1DS
<li>Canon EOS-1D Mark II
<li>Casio QV-2000UX
<li>Casio QV-3000EX
<li>Casio QV-3500EX
<li>Casio QV-4000
<li>Casio QV-5700
<li>Contax N DIGITAL
<li>Creative PC-CAM 600
<li>Fuji FinePix S2Pro
<li>Fuji FinePix S5000
<li>Fuji FinePix S7000
<li>Fuji FinePix F700
<li>Imacon Ixpress
<li>Kodak DCS315C
<li>Kodak DCS330C
<li>Kodak DCS420
<li>Kodak DCS460
<li>Kodak DCS460A
<li>Kodak DCS520C
<li>Kodak DCS560C
<li>Kodak DCS620C
<li>Kodak DCS620X
<li>Kodak DCS660C
<li>Kodak DCS660M
<li>Kodak DCS720X
<li>Kodak DCS760C
<li>Kodak DCS760M
<li>Kodak EOSDCS1
<li>Kodak EOSDCS3B
<li>Kodak NC2000F
<li>Kodak ProBack
<li>Kodak PB645C
<li>Kodak PB645H
<li>Kodak PB645M
<li>Kodak DCS Pro 14n
<li>Minolta DiMAGE 5 (untested)
<li>Minolta DiMAGE 7 (untested)
<li>Minolta DiMAGE 7i
<li>Minolta DiMAGE A1
<li>Minolta DiMAGE A2
<li>Nikon D1
<li>Nikon D1H
<li>Nikon D1X
<li>Nikon D100
<li>Nikon D2H
<li>Nikon D70
<li>Nikon E950 ("DIAG RAW" hack)
<li>Nikon E990 ("DIAG RAW" hack)
<li>Nikon E995 ("DIAG RAW" hack)
<li>Nikon E2100 ("DIAG RAW" hack)
<li>Nikon E2500 ("DIAG RAW" hack)
<li>Nikon E4300 ("DIAG RAW" hack)
<li>Nikon E4500 ("DIAG RAW" hack)
<li>Nikon E5000
<li>Nikon E5700
<li>Olympus C5050Z
<li>Olympus C5060WZ
<li>Olympus C8080WZ
<li>Olympus E-1
<li>Olympus E-10
<li>Olympus E-20
<li>Pentax *ist D
<li>Pentax Optio S
<li>Pentax Optio S4
<li>Phase One LightPhase
<li>Phase One H10
<li>Phase One H20
<li>Phase One H25
<li>Rollei d530flex
<li>Sigma SD9
<li>Sigma SD10
<li>Sinar 12582980-byte
<li>Sony DSC-F828
</ul>
</dl>
<p>
<H3>History</H3>
It started in February 1997, when I bought a Canon PowerShot 600.
Decoding the raw data was more difficult than I had expected,
knowing nothing about filter arrays, colorspace conversion, etc.
But in August 1997, I found a decent interpolation technique,
and finally was able to create images comparable in quality to
Canon's.
<p>
Word slowly spread, and people asked me to do other cameras,
sending me sample images to decode. I added support for the
PowerShot A5 in May 1999 and the PowerShot A50 and Pro70 in
May 2000.
<p>
In late September 2001, after months of effort, I finally
figured out the lossless compression algorithm used by the
PowerShots Pro90, G1, G2, S30, S40, and EOS D30/D60 cameras.
<p>
I solved the Canon EOS-1D on Jan 28, 2002 and the Nikon
compressed NEF format on March 24, 2002. Olympus ORF format
is not compressed, so it's much easier to decode.
<p>
On November 19, 2002, I was laid off. During that month, I
added nineteen Kodak cameras, the PowerShot G3/S45, the Canon
EOS-1DS, the Fuji S2, and the Minolta DiMAGE 7. In early
December, I replaced the whole color-interpolation system,
yielding sharper images for all cameras.
<p>
On December 10, I attacked the Sigma SD9. I solved the
compression algorithm on December 31, then spent another
six weeks constructing a Foveon-specific interpolation
routine to enhance color and reduce noise.
<p>
<H3>Misc. Stuff</H3>
<a href="secrets.html">A detailed description</A>
of the PowerShot 600 written in 1997.
<p>
<a href="renum">renum</A>
A bash script to rename all files in the current directory,
numbered by date of last modification. For example,
<pre> renum rover 56</pre>
<p>
will rename the files to "rover-0056.jpg", "rover-0057.jpg",
"rover-0058.jpg"...
<p>
<a href="lcfile">lcfile</A>
A Python script to rename files and directories to lowercase.
<H3>Patches for XV 3.10a</H3>
XV is still my favorite image viewer, even though its last
official release was in 1994. In case you're still using
this antique, here are some useful patches for it:
<p>
To install them, unpack the
<a href="http://trilon.com/xv/index.html">xv-3.10a</a>
distribution. From the parent (not the xv-3.10a subdirectory) do:
<p>
<tt>cat patch1 patch2 | patch -p0</tt>
<ul>
<li>
<a href="xv_smooth.patch"> Smooth patch</a>
Using XV to expand an image from 832x608 to 832x624 blurs it badly.
This patch does the interpolation correctly, and twice as fast.
<a href="xv-bug.txt">This bug report explains the fix</a>
<li>
<a href="xv_ps_save.patch"> PostScript free() patch</a>
Prevents a segmentation fault when saving a color-adjusted image
in PostScript format.
<li>
<a href="xv_jpeg.patch"> JPEGlib patch</a>
Changes a couple of JPEG library defaults.
<li>
<a href="xv_png_crw.patch"> My XV super-patch</a>
Adds PNG support, raw photo decoding, non-destructive deletion,
simulated color-blindness, plus all of the above patches.
</ul>
<hr>
</HTML>
