XMM-Newton Science Analysis System


evlistcomb (evlistcomb-4.20) [xmmsas_20230412_1735-21.0.0]

Description

The MOS, PN or RGS chains are applied separately to event lists attached to a single CCD (or CCD/node for MOS). evlistcomb builds a global calibrated event list (PPS product) for one EPIC or RGS camera from their outputs. This is done by taking in all input files the extension whose name is specified by the maintable parameter (called 'merged extension' in what follows) and merging the columns into a single output extension of the same name. A CCDNR column is added to the extension to keep track of all events' origin. Its contents is taken from the CCDID keyword (and QUADRANT for EPIC PN). For EPIC MOS only (not RGS) the node (CCDNODE keyword) is coded into the CCD number by adding 10 if the redundant node is used.
Actually two calibrated event lists are built if all CCDs are not operated in the same mode. The mode is recognized by the DATAMODE keyword, which must be one of both values specified in the xxxdatamodes parameter. The first DATAMODE keyword is associated with the imagingset, xxximgcolnames) and xxximgcoltypes parameters (it need not be IMAGING, for example it is used for RGS SPECTROSCOPY mode). The second keyword in xxxdatamodes is associated with the timingset, xxxtimcolnames) and xxxtimcoltypes parameters.
The xxxdatamodes parameter is only a way to tell evlistcomb the name of the modes. It cannot be used to select which mode you want in output, and should not be set differently in the various instrument modes.

When used in default mode, evlistcomb propagates only that subset of the original columns, specified in the Data Products ICD. The columns are converted into the type of the ICD if need be. This is done by specifying the instrument parameter to 'emos', 'epn' or 'rgs'. The task will then use the default list of modes (xxxdatamodes), column names (xxxyyycolnames) and types (xxxyyycoltypes), where xxx is the 3 or 4-letter instrument symbol and yyy the mode symbol ('img' or 'tim'). The columns appearing in the list but not present in the input files are not created. When transtyping evlistcomb checks for overflow and sets to null the data which overflow the output type.

In order to generate an output file with other columns, one needs to specify by hand the xxxyyycolnames and xxxyyycoltypes parameters for the instrument and mode(s) one is interested in. The correspondance between the column names and types is done simply by order of appearance.

The task also copies over the secondary extensions specified by the othertables parameter for all files into extensions with the same name (truncated to 6 characters) followed by nn (the 2-digit CCDNR). This works both for tables and arrays. Except their name, those extensions are copied without change and entirely (data and keywords).

The compatibility of the files is checked through a number of primary keywords, specified by the primarychecks parameter. Those keywords must exist in all files. All files in the list which do not share the first file's setting are rejected. All keywords (not only those in primarychecks) present in the primary header of any of the input files are propagated to the primary header of the output file, except FILENAME which is clearly file-specific. In practice this means a keyword takes the value it has in the last file of the list (of that mode) where it is present. All those (global) keywords are also copied to all output extensions, except those specified in primaryonly.

Some keywords in the merged extensions (extensionchecks parameter) may also be checked for compatibility between files. Those do not have to exist, but will be propagated if they do. Other keywords (specified by the mainattributes parameter) may also be propagated to the merged extensions. Those will take the value they have in the last file of the list where they are present. Yet other keywords in the merged extensions may be maximised (maxattributes) or minimised (minattributes).
The four sets of parameters (extensionchecks, mainattributes, maxattributes, minattributes) may include column specific keywords as well.

All keywords in the merged extensions are not automatically propagated, as those keywords are usually different for each CCD/node. If CCD-dependent keywords are needed down the line, they must be propagated by means of one of the secondary extensions (evlistcomb does not do that automatically).

The standard column specific keywords (TNULL, TUNIT) are taken from the first valid file where they are set. All subsequent files which have those keywords set to a different value are rejected (files with keywords not set are accepted). To propagate other column specific keywords, they must be specified manually via the extensionchecks, mainattributes, maxattributes and minattributes parameters (see above).

It is possible to merge several extensions (maintable may be a list). In that case all keyword operations (extensionchecks, mainattributes, maxattributes, minattributes) are done on all extensions. It is not possible to specify a specific list of columns for each of the extensions to merge. All columns to be merged (in all extensions) must appear in the xxxyyycolnames and xxxyyycoltypes parameters (but they don't have to exist).

evlistcomb will also accept in input files generated by a previous call to evlistcomb. In that case the CCDNR column will be propagated as a normal column (but need not be specified in the list of columns to propagate), and all the secondary extensions whose first 6 characters are common with one of othertables will be propagated.

In the PPS, evlistcomb must be followed by a call to evselect which will apply the selection on Good Time intervals and add the EXPOSURE keyword. evlistcomb can be applied as is to slew data.