1 PCMS Processing ----------------- Full release note for prod8.1 configuration 07000001_20090303.144400 ==================================================================== PCMS Infrastructure Version: 8.1 Pipeline Version: 07000001_20090303.144400 Pipeline Name: prod8.1 Release Date: 2009-04-15 SAS Manifest: xmmsas_20090112_1802 SAS Version: 8.0.1 ppstrack Summary ======= This pipeline represents a large change with respect to the previous one (pipeline version:02001007_20070918.154623), embodying both substantial (mainly infrastructural) changes to the PCMS processing system itself and incorporation of a slightly upgraded version of the SAS 8.0.1 patched version of the SAS. As such, it represents a marked step forward from SAS 6.9.3 used in the last version of the processing pipeline and bypasses SAS 7 software. The many SAS changes that have taken place between SAS 6.9 and 8.0 are detailed at https://www.cosmos.esa.int/web/xmm-newton/releasenotes-generation?p_p_id=com_liferay_iframe_web_portlet_IFramePortlet_INSTANCE_2lcw40543LQS&_com_liferay_iframe_web_portlet_IFramePortlet_INSTANCE_2lcw40543LQS_iframe_v=8_0_0 although many of the changes listed there are of minor importance or do not impact on the pipeline. Key changes include: 1) PN exposure maps are now made using the band 2 image as a template for bands 1 and 2, and the band 3 image for bands 3, 4 and 5. The band 8 exposure map is an average of the band exposure maps. Previously the band 8 image was used as the template for all bands. 2) PN image mode event times are randomized within the frame duration, avoiding aliasing effects. 3) OM images are, where possible, rectified against the USNO-B1 catalogue (previously only source source lists were rectified). 4) Changes employed in OM tasks (e.g. source detection and flagging improvements) needed in making the OM catalogue, are propogated. Note concerning leap second corrections ======================================= This observation has been processed with the prod8.1 pipeline which uses SAS manifest, xmmsas_20090112_1802. However, the leapsecs.dat file in the utils package has been replaced with the latest version which includes the extra leapsecond added on 2009-01-01 - it was (erroneously) absent from the leapsecs.dat file in the original xmmsas_20090112_1802 manifest. See http://xmm2.esac.esa.int/sas/8.0.0/watchout/8.0.0/leapsec_update.shtml Documentation ============= Documentation for this pipeline and pipeline products can be found here: Pipeline Summary ================ i. Products Created. ii. SAS Tasks. iii. Calibration Files. iv. Response Matrices. v. Energy Bands. vi. Energy Conversion Factors. vii. Source Position Rectification. i. Products Created =================== The following table summarises all possible product files created by this pipeline. Withdrawn products are included and marked for completeness. Product file names are subdivided into components: Pooooooooooiiseeeccccccbnnn.ttt where: oooooooooo is the Observation Identifier. ii is the instrument. s is the scheduled flag. eee is the exposure number. cccccc is the file identifier given in the table below. b is the instrument specific field (ISF): Energy band for EPIC instruments. OM science window or OM filter code for OM products. Spectral order for RGS products. nnn is the hexidecimal source number or 000. ttt is the file type. |=============|========================================|===================|=========|==============|========|==============| | Status | Product Index Description | File Identifier | Class | File Types | ISF | Per Source | |=============|========================================|===================|=========|==============|========|==============| | | MAIN CROSS CORRELATION PAGE | XCORRE | CATOBS | HTM | | | | | EPIC OBSERVATION BOX-LOCAL SOURCE LIST | OBLSLI | EPICOBS | FIT | | | | | EPIC OBSERVATION BOX-MAP SOURCE LIST | OBMSLI | EPICOBS | FIT | | | | | EPIC OBSERVATION SENSITIVITY MAP | OSNSMP | EPICOBS | FIT | BAND | | | | EPIC MOS IMAGING MODE EVENT LIST | MIEVLI | EPICEXP | FIT | | | | | EPIC PN IMAGING MODE EVENT LIST | PIEVLI | EPICEXP | FIT | | | | | EPIC TIMING MODE EVENT LIST | TIEVLI | EPICEXP | FIT | | | | | EPIC FLARE BACKGROUND TIMESERIES | FBKTSR | EPICEXP | FIT,PDF | | | | | EPIC EXPOSURE MAP | EXPMAP | EPICEXP | FIT,PNG | BAND | | | | EPIC IMAGE | IMAGE_ | EPICEXP | FIT,PNG | BAND | | | | EPIC SOURCE DS9 REGIONS | REGION | EPICOBS | ASC | | | | | EPIC OBSERVATION ML SOURCE LIST | OMSRLI | EPICOBS | FIT | | | | | EPIC SUMMARY SOURCE LIST | OBSMLI | EPICOBS | FIT,HTM | | | | | EPIC DETECTION MASK | DETMSK | EPICOBS | FIT,PNG | | | | | EPIC MERGED BACKGROUND MAP | BKGMAP | EPICOBS | FIT,PNG | BAND | | | | EPIC OBSERVATION BACKGROUND MAP | OBKGMP | EPICOBS | FIT,PNG | BAND | | | | EPIC OBSERVATION EXPOSURE MAP | OEXPMP | EPICOBS | FIT,PNG | BAND | | | | EPIC OBSERVATION IMAGE | OIMAGE | EPICOBS | FIT,PNG | BAND | | | | EPIC OBSERVATION SUMMARY | SUMMAR | EPICOBS | HTM | | | | | EPIC SOURCE DS9 REGION | SRCREG | EPICSRC | ASC | | Y | | | EPIC ANCILLARY RESPONSE FUNCTION | SRCARF | EPICSRC | FIT | | Y | | | EPIC SOURCE BACKGROUND SPECTRUM | BGSPEC | EPICSRC | FIT | | Y | | | EPIC SOURCE SPECTRUM | SRSPEC | EPICSRC | FIT | | Y | | | EPIC SOURCE TIMESERIES | SRCTSR | EPICSRC | FIT | BAND | Y | | | EPIC SOURCE FFT PLOT | SFFTPL | EPICSRC | PDF | BAND | Y | | | EPIC SOURCE SPECTRUM PLOT | SPCPLT | EPICSRC | PDF | | Y | | | EPIC SOURCE TIMESERIES PLOT | STSPLT | EPICSRC | PDF | BAND | Y | | | OM FULL-FRAME IMAGE | FIMAG_ | OMEXP | FIT | | | | | OM TRACKING STAR TIMESERIES | TSTRTS | OMEXP | FIT,PDF | | | | | OM FULL-FRAME SKY IMAGE | FSIMAG | OMEXP | FIT,PNG | | | | | OM TRACKING HISTORY PLOT | TSHPLT | OMEXP | PDF | | | | WITHDRAWN | OM FLATFIELD | FLAFLD | OMOBS | FIT | | | | | OM OBSERVATION SOURCE LIST | OBSMLI | OMOBS | FIT | | | | | OM FULL-FRAME HIRES SKY IMAGE MOSAIC | HSIMAG | OMOBS | FIT,PNG | FILTER | | | | OM FULL-FRAME LORES SKY IMAGE MOSAIC | LSIMAG | OMOBS | FIT,PNG | FILTER | | | | OM RUDI-5 SKY IMAGE MOSAIC | RSIMAG | OMOBS | FIT,PNG | FILTER | | | | OM USER WINDOWS SKY IMAGE MOSAIC | USIMAG | OMOBS | FIT,PNG | FILTER | | | | OM OBSERVATION SUMMARY | SUMMAR | OMOBS | HTM | | | | | OM OSW SOURCE TIMESERIES | TIMESR | OMSRC | FIT,PDF | OSW | Y | | | OM GRISM DS9 REGIONS | SGSREG | OMSW | ASC | OSW | | | | OM GRISM DS9 SPECTRUM REGIONS | SPCREG | OMSW | ASC | OSW | | | | OM OSW FAST REGION FILE | SFSREG | OMSW | ASC | OSW | | | | OM OSW REGION FILE | SWSREG | OMSW | ASC | OSW | | | | OM FAST MODE OSW IMAGE | IMAGEF | OMSW | FIT | OSW | | | | OM FAST MODE OSW SKY IMAGE | SIMAGF | OMSW | FIT | OSW | | | | OM GRISM SPECTRA LIST | SPECLI | OMSW | FIT | OSW | | | | OM OSW FAST SOURCE LIST | SFSRLI | OMSW | FIT | OSW | | | | OM OSW GRISM SOURCE LIST | SGSRLI | OMSW | FIT | OSW | | | | OM OSW IMAGE | IMAGE_ | OMSW | FIT | | | | | OM OSW SOURCE LIST | SWSRLI | OMSW | FIT | | | | | OM GRISM SOURCE SPECTRUM | SPECTR | OMSW | FIT,PDF,PNG | OSW | Y | | | OM GRISM-ALIGNED IMAGE | GIMAGE | OMSW | FIT,PNG | OSW | | | | OM OSW SKY IMAGE | SIMAGE | OMSW | FIT,PNG | | | | | PPS RUN MESSAGE | PPSMSG | PPSOBS | ASC | | | | | PPS SCRIPT LOG | SCRLOG | PPSOBS | ASC | | | | | ATTITUDE TIME SERIES | ATTTSR | PPSOBS | FIT | | | | | CALIBRATION INDEX FILE | CALIND | PPSOBS | FIT | | | | | PPS PRODUCT INDEX | PINDEX | PPSOBS | FIT | | | | | PPS OBSERVATION SUMMARY | SUMMAR | PPSOBS | HTM | | | | | PPS RUN SUMMARY | PPSSUM | PPSOBS | HTM | | | | | SSC LOGO 1 | SSCLG1 | PPSOBS | PNG | | | | | SSC LOGO 2 | SSCLG2 | PPSOBS | PNG | | | | | RGS BACKGROUND MODEL | BGMODL | RGSEXP | FIT | ORDER | | | | RGS EVENT LIST | EVENLI | RGSEXP | FIT | | | | | RGS EXPOSURE MAP | EXPMAP | RGSEXP | FIT | | | | | RGS SOURCE LIST | SRCLI_ | RGSEXP | FIT | | | | | RGS WHOLE FIELD RESPONSE MATRIX | WREMAT | RGSEXP | FIT | ORDER | Y | | | RGS ENERGY-DISPERSION IMAGE | ORDIMG | RGSEXP | FIT,PNG | | | | | RGS FLARE BACKGROUND TIMESERIES | FBKTSR | RGSEXP | FIT,PNG | | | | | RGS IMAGE | IMAGE_ | RGSEXP | FIT,PNG | | | | * | RGS WHOLEFIELD SPECTRUM | WFSPEC | RGSEXP | FIT,PDF | ORDER | | | | RGS FLUXED SPECTRUM | FLUXED | RGSOBS | FIT,PDF | ORDER | Y | | | RGS OBSERVATION SUMMARY | SUMMAR | RGSOBS | HTM | | | | | RGS CROSS-DISPERSION HISTOGRAM | DSPHIS | RGSSRC | FIT | ORDER | Y | | | RGS RESPONSE MATRIX | RSPMAT | RGSSRC | FIT | ORDER | Y | | | RGS SOURCE BACKGROUND SPECTRUM | BGSPEC | RGSSRC | FIT | ORDER | Y | | | RGS SOURCE SPECTRUM | SRSPEC | RGSSRC | FIT,PDF | ORDER | Y | | * | RGS BACKGROUND SPECTRUM | SBSPEC | RGSSRC | FIT | ORDER | Y | |=============|========================================|===================|=========|==============|========|==============| * - New in bulk reprocessing pipeline (06000520_20070308.163249) and present in following pipeline (02001007_20070918.154623) but erroneously omitted from corresponding release note tables. ii. SAS Tasks ============= The table below summarises differences between the base public SAS (8.0 - manifest xmmsas_20080701_1801) and the distinct (PPS) manifest of the SAS (xmmsas_20090112_1802) used in this pipeline. The PPS manifest contains a small number of task revisions that were included in the public SAS 8.0.1 (xmmsas_20081024_1801) patch release, plus one more that was made subsequent to that. Older, identical and newer task versions used in the pipeline cf. the public SAS are flagged by the <, = and > symbols in the 2nd column of the table. |===============|===|=========|=========| | TASK | | (PPS) | (8.0) | |===============|===|=========|=========| |addattribute | = | 2.1 | 2.1 | |arfgen | > | 1.75.6 | 1.75.5 | |attcalc | = | 4.32 | 4.32 | |attfilter | = | 1.3.1 | 1.3.1 | |atthkgen | = | 1.20.1 | 1.20.1 | |badpix | = | 2.32 | 2.32 | |badpixfind | = | 1.37 | 1.37 | |cifbuild | = | 4.7.1 | 4.7.1 | |dal | = | 1.182.1 | 1.182.1 | |daltools | = | 1.9 | 1.9 | |dpssflag | = | 10.2.1 | 10.2.1 | |dpsssrc | = | 1.1 | 1.1 | |dscalc | = | 2.6 | 2.6 | |eboxdetect | = | 4.19.2 | 4.19.2 | |eexpmap | = | 4.4.1 | 4.4.1 | |efftplot | = | 2.17 | 2.17 | |ekstest | = | 2.2.5 | 2.2.5 | |elcplot | = | 2.5 | 2.5 | |emask | = | 2.12 | 2.12 | |embadpixfind | > | 2.3 | 2.2 | |emenergy | = | 8.6 | 8.6 | |emevents | = | 8.4 | 8.4 | |emeventsproj | = | 1.5.1 | 1.5.1 | |emframes | = | 5.8 | 5.8 | |emldetect | = | 4.60.1 | 4.60.1 | |emosaic | = | 1.15 | 1.15 | |epevents | = | 6.43 | 6.43 | |epframes | = | 8.87.1 | 8.87.1 | |eposcorr | = | 3.12.1 | 3.12.1 | |esensmap | = | 3.11 | 3.11 | |esources | = | 1.2 | 1.2 | |especget | > | 1.27.7 | 1.27.6 | |esplinemap | = | 4.4 | 4.4 | |evalcorr | = | 0.5 | 0.5 | |evlistcomb | = | 4.19.2 | 4.19.2 | |evselect | = | 3.60.2 | 3.60.2 | |gtialign | = | 2.9 | 2.9 | |gtimerge | = | 1.7.2 | 1.7.2 | |hkgtigen | = | 1.14.2 | 1.14.2 | |implot | = | 2.17.1 | 2.17.1 | |lccorr | = | 2.28 | 2.28 | |lccorr_pcms | = | 1.5.1 | 1.5.1 | |lcplot | = | 1.19.1 | 1.19.1 | |merge | = | 0.13 | 0.13 | |odfingest | = | 3.28 | 3.28 | |omatt | = | 2.46.2 | 2.46.2 | |omcomb | = | 2.4.2 | 2.4.2 | |omcosflag | = | 1.26 | 1.26 | |omdetect | > | 5.16 | 5.14.1 | |omdrifthist | = | 1.27.1 | 1.27.1 | |omfastflat | = | 1.16.2 | 1.16.2 | |omfastshift | = | 1.25.1 | 1.25.1 | |omflatfield | = | 2.5 | 2.5 | |omflatgen | = | 1.12 | 1.12 | |omgprep | > | 1.6.1 | 1.6 | |omgrism | > | 1.24 | 1.22 | |omgrismplot | = | 1.3 | 1.3 | |omlcbuild | = | 1.31.1 | 1.31.1 | |ommag | = | 3.8 | 3.8 | |ommodmap | = | 2.17.2 | 2.17.2 | |ommosaic | > | 1.18.1 | 1.18 | |omprep | = | 2.36 | 2.36 | |omregion | = | 2.11.1 | 2.11.1 | |omsrclistcomb | = | 2.11.4 | 2.11.4 | |omthconv | = | 1.32 | 1.32 | |ppssumm | = | 3.4.5 | 3.4.5 | |region | = | 7.9.3 | 7.9.3 | |rgsangles | = | 4.6.3 | 4.6.3 | |rgsbadpix | = | 2.18.2 | 2.18.2 | |rgsbkgmodel | = | 1.3.2 | 1.3.2 | |rgsenergy | = | 2.0.4 | 2.0.4 | |rgsevents | = | 3.12.3 | 3.12.3 | |rgsfilter | = | 1.17.1 | 1.17.1 | |rgsfluxer | = | 1.7.4 | 1.7.4 | |rgsframes | = | 3.17.2 | 3.17.2 | |rgsimplot | > | 1.17 | 1.15 | |rgsoffsetcalc | = | 1.18.1 | 1.18.1 | |rgsregions | = | 1.19.2 | 1.19.2 | |rgsrmfgen | = | 1.10.1 | 1.10.1 | |rgssources | = | 6.0.1 | 6.0.1 | |rgsspecplot | = | 2.18.4 | 2.18.4 | |rgsspectrum | = | 2.6.3 | 2.6.3 | |slconv | = | 2.2 | 2.2 | |srcmatch | = | 3.15.20 | 3.15.20 | |tabgtigen | = | 2.12.3 | 2.12.3 | |tools | = | 1.63 | 1.63 | |===============|===|=========|=========| iii. Calibration Files ====================== Pipeline processing generally accesses an evolving set of CCFs (current calibration files). For each observation, a subset of the full set of CCFs is used in processing that pertains to the epoch of the observation; the CCF components/versions used are identified in the calibration index file (PooooooooooOBX000CALIND0000.FIT) that is included in the delivered observation data product set. iv. Response Matrices ===================== The pipeline uses a set of EPIC canned RMFs which are updated from time to time. At inception, this pipeline accesses version 7.0 of the PN RMFs and version 1.2 of the MOS RMFs. The latest available canned RMFs are described and available from http://xmm2.esac.esa.int/external/xmm_sw_cal/calib/index.shtml Note that for the MOS instruments, the latest available RMFs are position dependent and supercede those used in the pipeline. Users analysing MOS spectra of sources on or near the MOS 'patch' may wish to obtain and use the newer RMF files. v. Energy Bands =============== |======|===================| | Band | Energy Range (eV) | |======|===================| | 1 | 200 - 500 | | 2 | 500 - 1000 | | 3 | 1000 - 2000 | | 4 | 2000 - 4500 | | 5 | 4500 - 12000 | | 6 | 200 - 2000 | | 7 | 4500 - 12000 | | 8 | 200 - 12000 | | 9 | 500 - 4500 | |======|===================| vi. Energy Conversion Factors ============================= EPIC energy conversion factors (ECFs) are computed using the instrument response files for an assumed absorbed power law model with Gamma=1.7 and an absorbing column density of 3x10^20 /cm2. They are provided for each EPIC instrument and filter, for each energy band. In this pipeline, the following fixed energy conversion factors (ECFs) are used: |============|========|=========================================================================| | Instrument | Filter | Band | | | | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | |============|========|=========|=========|=========|==========|==========|===|===|===|=========| | epn | Open | 16.5304 | 10.2555 | 6.16611 | 1.98190 | 0.556273 | - | - | - | 5.13134 | | epn | Thin | 9.05601 | 8.25349 | 5.87836 | 1.95026 | 0.555226 | - | - | - | 4.58257 | | epn | Medium | 7.88027 | 7.99516 | 5.77852 | 1.92694 | 0.554529 | - | - | - | 4.48375 | | epn | Thick | 4.69897 | 6.13757 | 4.99916 | 1.82646 | 0.547553 | - | - | - | 3.78153 | | emos1 | Open | 3.09427 | 2.19119 | 2.12500 | 0.757786 | 0.143898 | - | - | - | 1.51194 | | emos1 | Thin | 1.72543 | 1.80741 | 2.02768 | 0.746683 | 0.143585 | - | - | - | 1.39049 | | emos1 | Medium | 1.51415 | 1.75577 | 1.99329 | 0.737800 | 0.143375 | - | - | - | 1.36449 | | emos1 | Thick | 0.98524 | 1.42597 | 1.77517 | 0.707822 | 0.141457 | - | - | - | 1.20820 | | emos2 | Open | 3.14224 | 2.19588 | 2.13171 | 0.762227 | 0.151536 | - | - | - | 1.51740 | | emos2 | Thin | 1.73763 | 1.80975 | 2.03440 | 0.750569 | 0.151222 | - | - | - | 1.39543 | | emos2 | Medium | 1.52095 | 1.75929 | 1.99916 | 0.742242 | 0.151013 | - | - | - | 1.36917 | | emos2 | Thick | 0.98324 | 1.42832 | 1.78104 | 0.711708 | 0.148990 | - | - | - | 1.21236 | |============|========|=========|=========|=========|==========|==========|===|===|===|=========| vii. Source Position Rectification ================================== Source position rectification was performed against extracts from the USNO-B1 catalogue.