Scanning on the HP5470

Up: Martijn's Homepage  
Prev: GnuCash Portfolio Report   Next: Debian package for OpenH323 proxy  

Current status: Current standalone correct the color shifting. Calibration still needs to be done. A SANE backend has been started and can get preview images but is experiencing difficulties. An example of the current output is here.

Note also the current code is now kept in CVS on sourceforge. What is below is merely snapshots. Please check there for latest info.

Well, I managed it. After researching which scanners would work under Linux and which not I had it all worked out. When I finally got round to ordering it it was out of stock. They had an alternative which had all the same features and the model number was only slightly different. I unthinkingly said yes. How different could it be?

Anyway, it's quite different as many people have discovered. I ignored the issue for a while but finally decided I should work it out, after I found out you could trace USB communications in Windows without extra hardware.

I would like to thank Bertrik Sikken for his coverage of much of the protocol, as well as example code. He wasn't totally right but he did pretty well. To help with my work I wrote some perl scripts to convert the output of sniffusb (homepage?) output something like the following:

72      39.42219680     C0 04 00 C5 00 00 02 00 R       1B 00
73      39.43099200     C0 04 00 34 00 00 10 00 R       40 00 00 01 C2 00 00 00 02 D0 00 A0 00 00 00 00
74      39.43574720     C0 04 00 C5 00 00 02 00 R       34 00
75      39.43673360     40 0C 87 00 80 44       W       14
76      39.44080080     40 0C 83 00 90 EF       W       24
77      39.44445360     40 04 82 00 8C EF       W       00 00 00 00 00 F0 00 00
78      40.02064400         R(bulk) 61440 elements
79      40.02324960     40 04 82 00 00 F0       W       00 00 00 00 00 F0 00 00
80      40.60471840         R(bulk) 61440 elements
81      40.60962800     40 0C 01 1B 00 00       W       40
82      40.61917120     C0 04 00 C5 00 00 02 00 R       1B 00
83      40.62331360     40 04 00 E7 00 00       W       00 54 02 80
84      40.71464000     C0 04 00 C5 00 00 02 00 R       E7 00
85      40.71496880     40 04 00 15 00 00       W       01 2C
86      40.72302480     C0 04 00 C5 00 00 02 00 R       15 00
Which is much easier to read. If you got lost in all the detail, these scripts may be useful for you to get a handle on what's going on.

The logs I've captured so far are all in the package also. Most of them were captured with a patch that excluded the contents of all bulk data transfers. If I didn't do that I'd lose URBs at random. At least this way I get them all! If someone knows a way to get all the URBs and all the data, let me know.

Protocol

In addition to Bertrik's info, I've worked out the following:

As can be seen from the -fix file, URBs 57 to 78, the second value returned from the C500 command is an error code, where zero is normal and anything else is an error. After an error you can do a read command on 0300 which returns three bytes which, presumably, give more details...

Here I found out that the scanner can actually produce errors. Not that we can work out what they mean ... :(

To warm up the lamp, the system writes 01 to command 0000. When it succeeds it continues. The error from the C500 is 02. The 0300 returns 0B 80 01.

Sending the 2500 for the scan parameters is not sufficient to trigger a scan. You need to send a 1B05 command before and a 1B01 afterwards. What you send is either a 0x40 or a 0x00. It seems to be mostly 0x40 except for that calibration scan where it is 0x00. This is the scan that seems to return 48bpp. It's my belief that it controls the lighting, since the scan where it is zero appears to be the scan where it is looking for the minimums.

There appears to be some kind of gamma correction table written using the 2Axx commands. There are three 2A01, 2A02 and 2A03 presumably for red, green and blue. Each of these takes a 128KiB transfer as input (the figure 0x020000 is given with the command). It doesn't appear to need to enable the table, there is no command between the upload and the next scan.

Prior to these tables there is another table uploaded with command E603 (size 32792 bytes). The 0x8018 is given with the command. The remaining bytes are of unknown meaning. The data looks like more calibration. The scanner is 2690 pixels wide. It appears to send 12 bytes for each CCD pixel. 42 pixels are grouped with 8 pad bytes to produce blocks of exactly (42 * 12 + 8 =) 512 bytes. 64 of these blocks are sent plus the final 2 pixel to produce (64 * 512 + 2 * 12 =) 0x8018/32792 bytes. 42 * 64 + 2 = 2690. Each block of 12 appears to be three high 16 integers plus three low 16 integers. Limits?

What works

Currently the program finds the scanner, warms up the lamp, performs two calibration scans and then performs the preview scan. The calibrations produce useful data (in files calibrate1.dat and calibrate2.dat). The preview scan dumps stuff to output.dat. It produces something bitmap-like, I'm yet to convert to image. Coming soon!

The calibration files

Currently I'm not entirely sure what to do with these files. All I know is it's a bitmap of the size given by the scan info. On disk the rows are 6*2690+3 bytes long, divided into R, G and B sections. This command is good for getting an overview:
perl -e 'while( read STDIN, $buffer, 2690*2+1 ) { my @a = unpack("n*",$buffer."\x00"); print join(" ", map { sprintf "%04X", $_ } @a ),"\n" }' < calibrate1.dat |less

The code

The code is quite a bit more longer than Bertrik's version but I tried to cover everything to make the scanner work. It's hopefully fairly readable. Contact me if you need help. Note that I have a HP 5470C. I'm not sure how close this is to the HP 5400.

Note this version produces lots of output. Don't be afraid of the several pages of output produced.

There is a README within describing the contents in more detail.

hp5470-20030226.tar.gz - Source, cleanups. SANE backend works (dynamic only) (79K)
hp5470-20030224.tar.gz - Source, some code rearrangements (78K)
hp5470-20030223.tar.gz - Source, prototype SANE backend (78K)
hp5470-20030221.tar.gz - Source, new writes PPMs (56K)
hp5470-20030220.tar.gz - Source with processed logs (old version) (35K)
hp5470-20030220-logs.tar.gz - Full captured logs (2.7M)

There is a project to build a SANE backend but I havn't looked into that.


Up: Martijn's Homepage  
Prev: GnuCash Portfolio Report   Next: Debian package for OpenH323 proxy  
By Martijn van Oosterhout (kleptog (at) svana.org)
Copyright © 2000-2006 - Last modified 25/02/2003