XMM-Newton Science Analysis System: User Guide

6.5 OM SAS processing products

This section provides a general description of the final products obtained when running the SAS chains on OM data.

The chains produce intermediate files that are sometimes overwritten by different tasks. They are recognised by the first letter of their name ("I" in omichain, "F" in omfchain, "g" and "u" in omgchain). Their detailed descriptions can be found in the on-line documentation.

By running the tasks interactively, the user can maintain these intermediate products for a better understanding of the whole process.

OM processing chains and pipeline products are named as zooooooooooOMueeettttttsxxx.fff, where:

• z denotes final product (P: image or fast, p: grism) or intermediate file (I, F, g, u) as defined above
• oooooooooo stands for the observation ID,
• u is the exposure flag (S:scheduled, U:unsched, X:not applicable),
• eee is the exposure ID,
• tttttt is the data type,
• s is the OM window within the exposure,
• xxx is the source number within the window,
• fff is the file type (FIT, FTZ, PDF, PNG etc.).

In what follows, a data type by the tttttt identifier, e.g. tttttt= SIMAGE, is used for an OM sky image product. The window and source number identifier (sxxx) is omitted.

In combined images resulting from ommosaic or after processing the full frame image produced by omcomb, files with data type FIMAG_, FSIMAG, HSIMAG, LSIMAG and RSIMAG, "s" indicates the corresponding filter used. The optical filters are noted V, B and U, while L, M and S represent UVW1, UVM2 and UVW2 respectively.

There are two products associated with the tracking history files, namely the TSHPLT.PDF and the TSTRTS.FIT file. There are three data products per observing science window (OSW), namely IMAGE_.FIT, SIMAGE.FIT and SWSRLI.FIT. If fast mode data are present, there are two products per fast OSW, namely TIMESR.FIT and its graphics version TIMESR.PDF. There are two products generated on a per observation basis, which are FLAFLD.FIT and OBSMLI.FIT.

The standard pipeline does not run the chains omichain, omfchain, omgchain included in the SAS system. Similar chains using the same tasks, sometimes with different parameters, are run instead.

It should be noted that the pipeline produces compressed FITS files named .FTZ while the output obtained when the SAS chains are run by the user is not compressed, .FIT.

A brief description is given of the main files generated when running SAS. The user should be aware that there may be some differences in the naming of files produced by the standard pipeline (so-called pps products) run by the SOC and included into the XMM-Newton Science Archive. Also, intermediate files are deleted in the pipeline processing. Table 12 shows the current equivalence between both types of files.

• TSHPLT.PDF file: It provides a visualisation of the tracking history file. It gives an overview of the pointing stability in the course of an exposure. The drift history of an exposure is displayed as a vector diagram. Each data point represents the average attitude solution of one tracking frame, lasting typically 20 s. The pointing drift is calculated with respect to the OM attitude at the time where the reference frame was defined. The reference attitude of an exposure is determined at the beginning of a science exposure. Histograms at the side of the vector diagram show the projection of the OM pointing drift onto the x- and y-direction. The second output page shows the incremental OM drift between 2 subsequent tracking frames. A clustering of points at one location would indicate a systematic drift into one direction. Typically the pointing stability is better than 1 arcsec during one exposure, which corresponds to 2 pixels on the diagrams. There is one TSHPLT PDF file produced for each THX file in the ODF.

• TSTRTS.FIT file:. There is one TSTRTS FITS file produced for each THX file. The binary extension comprises $n$ columns, i.e. one for each tracking star used. Each column lists the count rate (cts/s) of a tracking star averaged over the tracking frame duration. Each row indicates a new tracking frame, and one column corresponds to a time series of a specific star. The number of columns corresponds to the number of tracking stars used in an exposure. It can be any number up to 15.

• IMAGE_.FIT file: The Primary array contains the flatfield and mod8 corrected image. The header contains the WCS parameters applied to the raw image. The MODES extension lists the details of the window configuration. The QUALITY extension contains in an image the quality array. Any non-zero entry in the quality array reflects a bad pixel notified in the calibration bad pixel map or any location of the image where problems were encountered during the image processing. The pseudo images generated for fast mode windows follow the same naming.

• QIMAG_.FIT, FQIMA_ files: Produced since SAS v13.0, these files have the same structure as the previously defined IMAGE_.FIT. The QUALITY extension here contains much more complete information. These files are used in omdetect and they replaced, from SAS v14.0, the former ones, which were maintained for comparison.

• SIMAGE.FIT file: The primary array of this dataset contains the north-aligned sky image. The image is flatfielded, mod8 corrected, resampled and distortion corrected. WCS coordinates are contained as header keywords. Coincidence losses and dead time are not corrected in this image. The north-aligned sky pseudo images generated for fast mode windows and the sky aligned grism images follow the same naming.

• SWSRLI.FIT file: The primary header of an OSW source list file contains, besides the definition of the OSW within the exposure and the observation, the sensitivity limit of the plate expressed in count rates and in instrumental magnitudes. The binary extension holds the list of detected sources. There are the following entries for each detected source: identifier within the OSW, source position in pixel, source position in RA/Dec and galactic coordinates, the positional uncertainty, the extracted source count rates and its error estimate, the significance of the source detection, the aperture size used and the value of the PSF and coincidence loss corrections, the brightness in instrumental magnitudes, the shape of the source expressed as the two semi-axes of an ellipse (for point sources the two axes should agree), the orientation of the ellipse and three quality flags, describing whether the source is thought to be extended, confused or affected by bad pixels. The last entry is the source identifier in the final combined source list.

  In case of grisms the SWSRLI file contains the position and astronomical coordinates of the detected zero and first orders and their correlation if it has been found.

• FLAFLD.FIT file: This image is either part of the pipeline products or generated by omflatgen. There is one file per observation containing an image extension. The image describes the flatfield response covering the whole detector at coarse resolution. Since flat field correction is not needed, all pixels values are set to unity.

• EVLIST.FIT file: This is an intermediate events list produced when processing fast mode data

• TIMESR.FIT file: The time series contains source (background subtracted) and background rates and their error estimates as a function of time, within the specified time sampling. These time series are produced only for fast mode data.

• TIMESR.PDF or .PS file: The time series is plotted here to produce a graphic light curve, together with some statistics performed on the data.

• OBSMLI.FIT file: This file contains the combined source list for all OSWs of an observation. Information of sources whose spatial position coincides within the specified nsigma are merged together. An observation source identifier is assigned to each source, which can be used to look up the sources in OSW source lists (SWSRLI files). The list contains: the observation source identifier, the position in RA/Dec and galactic coordinates, the positional uncertainty, the detection significance and corrected count rates in the different filters, the source brightness expressed in instrumental magnitudes and its uncertainty, the quality, source extension and confusion flag for each filter, the characterisation and orientation of the spatial extent (described as in the SWSRLI files by the two semi-axes and the orientation angle with respect to the image x-axis) for each filter, and finally, if a transformation was possible, the source brightness in a standard photometric system, as well as the colour indices.

  AB magnitudes and absolute fluxes for all filters are included as well.

  If omvariability has been used, then two columns, filter_CHI2 and filter_MAXDEV provide variability information for detected sources.

• omvariability.ps file: omvariability produces this file containing plots showing the light curves of detected variable sources.

• (FIMAG_, FSIMAG, HSIMAG, LSIMAG, RSIMAG, USIMAG).FIT files: These are combined full frame or mosaiced images produced by merging images of different exposures obtained with the same filter.

• (HSISWS, LSISWS, RSISWS, USISWS)x.FIT files: Where x can be V, B, U, L, M, S are the equivalent to SWSRLI but obtained when running omdetect on mosaiced or stacked images of the filter x.

• OBSMOS.FIT file: Equivalent to OBSMLI but obtained when running omsrclistcomb on the RSISWSx source files obtained from mosaiced or stacked images.

• OBSMER.FIT file: Final combined source list obtained with ommergelists, where the new detections in OBSMOS are added to the original OBSMLI list.

• RIMAGE_.FIT file: The undistorted and rotated image produced by omgprep when processing grism data. It is the basis for the automatic spectral detection and extraction, and for interactive extraction with omgsource.

• REGION.ASC file: This is an ASCII version of the SWSRLI source list file (only the source positions) for use with ds9 display.

• SPECLI.FIT file: This is similar to the SWSRLI file, but it contains only the zero and first order positions of the main target extracted spectrum, or if the parameter extractfieldspectra=yes was used, all final extracted spectra. For each extracted spectrum, the nature of the source is also given (Target, Field Object, Default Extraction,...)

• SPCREG.ASC file: This is an ASCII version of the SPECLI source list file (position, size and shape of the zero and first orders) for use with ds9 display. A file SPCREG.PS shows the grism rotated image (RIMAGE) with the over-imposed extraction regions contained in the SPCREG.ASC file.

• SPECTR.FIT file: The extracted and calibrated spectra are written in this file. For each spectrum a table extension is given. In the table columns one can find:

       -  wavelength
       -  net spectrum (count rate)
       -  error in net spectrum
       -  background rate
       -  error in background
       -  flux calibrated spectrum
       -  error in flux
  

• SPECTR.PDF / SPECTR.PS files: They contain plots of the extracted spectra in the corresponding SPECTR.FIT file.

Table 12: File name equivalence between direct SAS and Pipeline products for OM.

SAS name	PPS name	Description
TSHPLT	TSHPLT	OM tracking history plot
TSTRTS	TSTRTS	OM tracking star time series
IMAGE_	IMAGE_	OM OSW image (any filter or grism)
IMAGE_	IMAGEF	OM fast mode OSW image
SIMAGE	SIMAGE	OM OSW sky aligned image
SIMAGE	SIMAGF	OM fast mode OSW sky aligned image
REGION	SWSREG	OM OSW sources region file
REGION	SFSREG	OM fast mode OSW sources region file
SWSRLI	SWSRLI	OM OSW sources list
SWSRLI	SFSRLI	OM fast mode OSW sources list
TIMESR	TIMESR	OM fast mode OSW source time series
OBSMLI	OBSMLI	OM combined observation source list
FIMAG_	FIMAG_	OM combined full-frame image
FSIMAG	FSIMAG	OM combined full-frame sky image
HSIMAG	HSIMAG	OM full-frame HIRES sky image mosaic
LSIMAG	LSIMAG	OM full-frame LORES sky image mosaic
RSIMAG	RSIMAG	OM default mode sky image mosaic
USIMAG	USIMAG	OM user windows sky image mosaic
RSISWS	RSISWS	OM default mode source list from image mosaic
HSISWS	HSISWS	OM full-frame HIRES source list from image mosaic
LSISWS	LSISWS	OM full-frame LORES source list from image mosaic
USISWS	USISWS	OM user windows source list from image mosaic
OBSMOS	OBSMOS	OM mosaic merged sources list
OBSMER	none	OM final combined sources list: OBSMLI+OBSMOS
RIMAGE	GIMAGE	OM grism rotated image
SIMAGE	SIMAGE	OM grism OSW sky aligned image
SPECLI	SPECLI	OM grism spectra list
REGION	SGSREG	OM grism DS9 regions
SPCREG	SPCREG	OM grism DS9 spectrum regions
SPECTR	SPECTR	OM source extracted spectra
SWSRLI	SGSRLI	OM grism OSW sources list