When used in default mode, evlistcomb propagates only that subset of the original columns, specified in the Data Products ICD. The columns are converted into the type of the ICD if need be. This is done by specifying the instrument parameter to 'emos', 'epn' or 'rgs'. The task will then use the default list of modes (xxxdatamodes), column names (xxxyyycolnames) and types (xxxyyycoltypes), where xxx is the 3 or 4-letter instrument symbol and yyy the mode symbol ('img' or 'tim'). The columns appearing in the list but not present in the input files are not created. When transtyping evlistcomb checks for overflow and sets to null the data which overflow the output type.
In order to generate an output file with other columns, one needs to specify by hand the xxxyyycolnames and xxxyyycoltypes parameters for the instrument and mode(s) one is interested in. The correspondance between the column names and types is done simply by order of appearance.
The task also copies over the secondary extensions specified by the othertables parameter for all files into extensions with the same name (truncated to 6 characters) followed by nn (the 2-digit CCDNR). This works both for tables and arrays. Except their name, those extensions are copied without change and entirely (data and keywords).
The compatibility of the files is checked through a number of primary keywords, specified by the primarychecks parameter. Those keywords must exist in all files. All files in the list which do not share the first file's setting are rejected. All keywords (not only those in primarychecks) present in the primary header of any of the input files are propagated to the primary header of the output file, except FILENAME which is clearly file-specific. In practice this means a keyword takes the value it has in the last file of the list (of that mode) where it is present. All those (global) keywords are also copied to all output extensions, except those specified in primaryonly.
Some keywords in the merged extensions (extensionchecks parameter)
may also be checked for compatibility between files. Those do not have
to exist, but will be propagated if they do.
Other keywords (specified by the mainattributes parameter)
may also be propagated to the merged extensions.
Those will take the value they have in the last file of the list
where they are present.
Yet other keywords in the merged extensions may be maximised
(maxattributes) or minimised (minattributes).
The four sets of parameters (extensionchecks, mainattributes, maxattributes, minattributes) may include column specific keywords as well.
All keywords in the merged extensions are not automatically propagated, as those keywords are usually different for each CCD/node. If CCD-dependent keywords are needed down the line, they must be propagated by means of one of the secondary extensions (evlistcomb does not do that automatically).
The standard column specific keywords (TNULL, TUNIT) are taken from the first valid file where they are set. All subsequent files which have those keywords set to a different value are rejected (files with keywords not set are accepted). To propagate other column specific keywords, they must be specified manually via the extensionchecks, mainattributes, maxattributes and minattributes parameters (see above).
It is possible to merge several extensions (maintable may be a list). In that case all keyword operations (extensionchecks, mainattributes, maxattributes, minattributes) are done on all extensions. It is not possible to specify a specific list of columns for each of the extensions to merge. All columns to be merged (in all extensions) must appear in the xxxyyycolnames and xxxyyycoltypes parameters (but they don't have to exist).
evlistcomb will also accept in input files generated by a previous call to evlistcomb. In that case the CCDNR column will be propagated as a normal column (but need not be specified in the list of columns to propagate), and all the secondary extensions whose first 6 characters are common with one of othertables will be propagated.
In the PPS, evlistcomb must be followed by a call to evselect which will apply the selection on Good Time intervals and add the EXPOSURE keyword. evlistcomb can be applied as is to slew data.
XMM-Newton SOC -- 2021-11-30