Using Sets in reduction tasks

This section provides a brief explanation of what the CCDPACK reduction tasks (MAKEBIAS, MAKECAL, MAKEFLAT, DEBIAS, CALCOR, FLATCOR) do when they encounter input files with Set header information. It assumes that the global USESET parameter is TRUE, so you should first have set it using a command like

% ccdsetup useset=true accept

The reduction tasks considered here are all concerned with constructing calibration frames from a stack of input frames (MAKEBIAS, MAKECAL, MAKEFLAT) or with applying those calibration frames to a group of data frames (DEBIAS, CALCOR, FLATCOR). In the case of a single-chip instrument it is normally adequate to bung all the input files together and combine them or correct them accordingly. In a multi-chip context however, it is necessary to perform the make/apply calibration cycle separately for the data from each chip.

So the combination programs, which take multiple input images and produce one output image for each chip, do something like this:

1. Read in all the input files
2. Group them according to Set Index attribute
3. For each Set Index value represented in the input files:
   • Get values of any global parameters specific to this Set Index
   • Combine the data from those of the input files with this Set Index to produce a calibration frame
   • Write out the calibration frame with a suitable (new) Set Name attribute and the same Set Index attribute as this group of input files

If there is more than one Set Index value represented in the input files therefore, more than one calibration frame gets written out. In all cases, the OUT parameter of these programs refers to a single filename, so the programs write a new HDS container file of this name and store one NDF structure inside for each Set Index.

The correction programs, which take one calibration image for each chip and use it to turn multiple input files into corrected output files, then do this:

1. Read in all the input files
2. Group them according to Set Index attribute
3. For each Set Index value represented in the input files:
   • Get values of any global parameters specific to this Set Index
   • Read in the calibration frame with the corresponding Set Index from the calibration file
   • Go through all the input files with this Set Index, correct them using the corresponding calibration frame, and write them out with the same Set headers as the input files

So if you have several observations from a three-chip mosaic camera sorted into Sets to debias, you can replace a sequence like this:

% makebias in='bias*_chip1' out=master_chip1
% makebias in='bias*_chip2' out=master_chip2
% makebias in='bias*_chip3' out=master_chip3
% debias in='data*_chip1' bias=master_chip1 out='debias_*'
% debias in='data*_chip2' bias=master_chip2 out='debias_*'
% debias in='data*_chip3' bias=master_chip3 out='debias_*'

by this one, which does just the same thing:

% makebias in='bias*_chip?' out=master 
% debias in='data*_chip?' bias=master out='debias_*'

You may notice from the scheme above that the output frames have exactly the same Set headers (i.e. the same Set Index and Set Name) as the input frames. As long as pre-calibration and post-calibration frames will not be presented to the same CCDPACK program at the same time, which is normally the case, this will cause no trouble. If for some reason they will be fed together to the same CCDPACK Set-aware program though, you may need to doctor the Set Name headers of one or other Set (use the NAME parameter of MAKESET).

These programs assume that only one type of Set (i.e. Set data from only one instrument) is being considered at once -- if two images have the same Set Index attribute they are assumed to come from the same chip, so you should not mix data from different types of Set in the same invocation of a CCDPACK reduction command.

As explained in the previous section, certain parameters may need to be different for the different members of each Set (i.e. they are specific to a given Set Index). If they have been configured globally using CCDSETUP then the appropriate values will be used. Otherwise, you may be prompted for them. In this case you will be prompted for the value once for each Set Index represented in the input files. Note that if you supply these parameters on the command line or use the ACCEPT keyword without using CCDSETUP to assign global values, they will take the same value for all the files.

As with the other Set processing aspects, for the most part it's not necessary to understand the technicalities to use these programs, the upshot is that although multiple calibration frames, one for each chip, are constructed, they are all kept in a single file (an HDS container file), and this can be passed using just the single filename to the correction programs. Both lots of programs will keep track of which frames have come from which chip and use them in the correct places. If you look at the output of the various calibration programs operating on data with Set headers, the way the above scheme works should be clear. To see it in action, you can look at the log output of the setexercise script (see section ).