7.
GOME-2 Products Processing Algorithms |
This section briefly describes the major algorithmic functions of the GOME-2 Level 0 to 1b processor. Only algorithmic functions are described. Those related to data handling are excluded. Functions consisting of a Calculate Correction and an Apply Correction part (as shown in the previous functional decomposition diagrams) will be described together. Module numbers used in the GOME-2 Level 1 Product Generation Specification [RD7] (where exhaustive information, including variable lists, full algorithm descriptions and data handling modules can be found) are provided.
Functions will be presented in six groups: initialisation and preprocessing, geolocation, in-flight calibration parameters, scene-dependent corrections, absolute radiometric calibration, and quality flagging. For each function the type of data affected is indicated: radiometric (i.e., "signals"), spectral (i.e., "wavelengths"), housekeeping data, geolocation, quality flags, or other. Several functions affect more than one of these types.
Initialise
orbit propagator |
Geolocation |
||
| Modes | Calibration key data / MME | PGS reference | |
| Calculate from | N/A | N/A | AG.1 |
| The orbit propagator function is initialised
by calling it in its initialisation mode. A Cartesian orbit state
vector in Earth-fixed coordinates close to Ascending Node Crossing
(ANX) is required on input. UTC and mean Kepler elements at ANX are
provided as output. This function is executed once at PPF start-up. |
|||
| Preprocess
Müller matrix elements |
Radiometric |
||
| Modes | Calibration key data / MME | PGS reference | |
| Calculate from | N/A | Radiance response Irradiance response Polarisation sensitivity |
A2.1 (AG.2) |
| Calibration key data on radiance, irradiance
and polarisation response are converted to a representation as Müller
matrix elements M. In this formalism, used throughout
[RD7], elements M1 represent the radiometric response, M2 the
s/p polarisation sensitivity, and M3 the +45°/-45° polarisation
sensitivity. Conversion is performed to a wavelength and angular grid
defined as initialisation parameters. This function is executed once at PPF start-up. |
|||
| Receive and
validate Level 0 data |
Quality |
||
| Modes | Calibration key data / MME | PGS reference | |
| Calculate from | All | N/A | A1.1 |
| Basic integrity checks are performed on
the incoming Level 0 data. The GOME-2 Level 0 data flow is checked
in three steps, of which the first two are related to the integrity
of individual packets, and the last one to the integrity of the sequence
of packets. GOME-2 data packets are identified in the data flow via their fixed fields. The length of the data packet is checked. For each packet, the checksum is recalculated and compared against the checksum contained in the packet. A basic check for duplicate packets and the time order of the packets is performed. In the event of problems, error messages are issued. The normal quality flagging mechanism via PCDs cannot be used here because corrupted packets have to be excluded from further processing. This function is executed for every scan. |
|||
| Determine observation
mode and viewing angles |
Housekeeping
Quality |
||
| Modes | Calibration key data / MME | PGS reference | |
| Calculate from | All | N/A | A2.3 |
| Data from different observation modes will
be sent to different branches of the processing. This function derives
the observation mode for a scan from a combination of housekeeping
data and the scanner viewing angles. The viewing angles themselves
are calculated from the scan mirror readings in the data packet. Scans
which do not match any of the available observation modes are classified
as "invalid". Furthermore, both the PMD transfer and readout
mode are determined, and the UTC times corresponding to the scan mirror
positions are calculated. This function is executed for every scan. |
|||
| Convert housekeeping
data |
Housekeeping
Quality |
||
| Modes | Calibration key data / MME | PGS reference | |
| Apply to | All | N/A | AG.3 |
| Selected housekeeping data, e.g., temperatures,
lamp currents and voltages, are converted from raw instrument binary
units into physical units. The conversion uses a polynomial expansion
which is usually linear (two coefficients). Polynomial coefficients
may vary between models and are defined as initialisation parameters. Converted temperatures, voltages and currents are checked against thresholds defined as initialisation parameters and quality flags are raised as needed. This function is executed for every scan. |
|||
| Prepare PMD
data |
Radiometric |
||
| Modes | Calibration key data / MME | PGS reference | |
| Apply to | All | N/A | AG.4 |
| For each PMD band, the GOME-2 on-board
processor co-adds individual detector readouts (16-bit values) into
a band sum, then divides the sum by a scale factor such that it fits
again into a 16-bit value. Therefore, PMD signals in binary units
have to be reconstructed from the scaled values and scale factors
found in the Science Data Packet which is the task of this function. This function is executed for every scan containing PMD data in band transfer mode. |
|||
| Calculate geolocation
for fixed grid |
Geolocation |
||
| Modes | Calibration key data / MME | PGS reference | |
| Calculate from | All | Instantaneous field of view | A2.6 |
| A number of geolocation parameters are
calculated from an orbit state vector, the UTC, and scanner viewing
angles. Most geolocation parameters are calculated in granules of
the shortest effective integration time for the main channels (187.5
ms, 32 times per scan), independent of the actual integration time
in the main channels. The remaining geolocation parameters are calculated
once per scan, either because their variation within a scan is small,
or because they are not needed on a finer grid by higher-level processing. Basic geolocation parameters such as the sub-satellite point and solar angles at the satellite are calculated for all measurement modes. In addition, mode-specific geolocation parameters are calculated for Earth, sun and moon modes: Earth mode: solar and line-of-sight zenith and azimuth angles at a given height, corner and centre coordinates (latitude/longitude) of the ground pixel at ground level, satellite height and Earth radius. Sun mode: distance between satellite and sun, and relative speed of satellite and sun. Moon mode: Lunar elevation and azimuth angles, sun-moon distance, satellite-moon distance, lunar phase angle, illuminated fraction of lunar disc. |
|||
| Calculate geolocation
for actual integration times |
Geolocation |
||
| Modes | Calibration key data / MME | PGS reference | |
| Calculate from | Earth | N/A | A3.2 (AG.19) |
| Geolocation parameters for the actual integration
times in a scan are calculated from the geolocation parameters for
the fixed 187.5 ms grid. This is done only for integration times resulting
in "simple" ground pixel shapes (below 6 s: no change in
scanning direction during integration; 6 s and higher: only full scans). The synchronisation between main channel readouts and 187.5 ms ground pixels is taken into account. Geolocations refer to the first detector pixel read in a main channel. |
|||
A number of calibration parameters can be derived from the dedicated in-flight calibration measurements described in In-Flight Characterisation and Calibration. These are: dark signal, pixel-to-pixel gain (PPG) correction, spectral calibration parameters, Etalon correction, and the Sun Mean Reference spectrum. The frequency of in-flight calibrations varies between once per orbit (e.g. dark signal) and once per month (e.g. PPG correction). Corrections are generally sufficiently stable to be valid for more than one orbit.
In-flight calibration parameters are not immediately used or stored in a product after they have been calculated. Instead they are stored first in a database. When an Apply Correction step requires in-flight calibration parameters, they are selected from the database according to certain rules. In particular, parameters are only updated close to the northern hemisphere terminator. This ensures that in-flight calibration parameters are not changed on the dayside which could lead to artificial steps in higher level processing. Only those in-flight calibration parameters selected to be used are stored within the Level 1a product.
If a given in-flight calibration parameter depends on a parameter or parameter combination, this is indicated in the tables below. It is then calculated, stored, and applied separately for each parameter combination. For integer parameters such as integration time and PMD modes this is straightforward. For temperature values, bins are used whose width is defined in the initialisation parameters such that the change of the in-flight calibration parameter within a given bin can be tolerated.
Examples of in-flight calibration parameters (dark signal correction, PPG correction, spectral calibration and etalon correction) and their effect on the spectrum to which they are applied are shown in In-Flight Characterisation and Calibration.
| Dark Signal Correction |
Radiometric Quality |
||
| Modes | Calibration key data / MME | PGS reference | |
| Calculate from | Dark | N/A | A2.8 |
| Apply to | All above Dark | N/A | AG.10 |
| Depends on | FPA and PMD: Integration time, detector temperature PMD: PMD readout and transfer modes |
||
| The dark signal for each parameter combination
is calculated as an average of measurements in dark mode during eclipse.
Applying the dark signal correction means subtracting the dark signal
from a spectrum. This is the most basic correction and the first of
all the corrections to be applied in Level 0 to 1 processing. The dark signal has two components: the integration-time independent offset of typically 1500 BU, and the integration time and detector temperature dependent leakage current of typically 0.7 BU/s at a detector temperature of 235 K. When calculating and applying the dark signal correction, only the total dark signal is considered. There is no need to split the dark signal into its components. This is only done for long-term monitoring purposes, see Sensor Performance Assessment. |
|||
| Pixel-to-pixel
gain (PPG) correction |
Radiometric Quality |
||
| Modes | Calibration key data / MME | PGS reference | |
| Calculate from | LED (WLS) | N/A | A2.11 |
| Apply to | All above Dark | N/A | AG.12 |
| Given a uniform illumination over the detector
array, dark-signal corrected signals vary slightly between detector
pixels, mainly because of small differences in pixel width. To compensate
for this effect, the variation in pixel-to-pixel gain is determined
from measurements in LED mode which provide a fairly uniform illumination.
A PPG correction spectrum is obtained by applying a triangular smoothing
to the LED (fall back: WLS) measurements and dividing the measured spectrum
by the smoothed spectrum. Applying the PPG correction means dividing
a spectrum by the PPG correction spectrum. The PPG correction is of the order of 10-4 relative. |
|||
| Spectral calibration |
Spectral Quality |
||
| Modes | Calibration key data / MME | PGS reference | |
| Calculate from | SLS | FPA: SLS line positions PMD: PMD slit function, SLS Stokes fractions, FPA overlap wavelengths, MME |
A2.13 A2.14 |
| Apply to | N/A | AG.13 | |
| Depends on | Predisperser prism temperature | ||
| Spectral calibration is the assignment of
a wavelength value to each detector pixel. For each GOME-2 channel,
a low order polynomial approximation will be used to describe wavelength
as a function of detector pixel. Polynomial coefficients are derived
from preprocessed spectra of the on-board PtCrNeAr Spectral Light Source
(SLS) which provides a number of spectral lines at known wavelengths
across the GOME-2 wavelength range. Different algorithms are used for FPA and PMD channels because their spectral resolution is different. Individual spectral lines can only be resolved in the FPA channels. Positions of individual lines from a predefined set are determined using a Falk centre-of-gravity algorithm. For the PMD channels an iterative cross-correlation algorithm is used. The expected PMD signal is calculated from the measured FPA signal, taking into account the PMD slit function and ratio of radiometric response between PMD and FPA channels. The expected PMD signal is then spectrally matched with the measured PMD signal using cross-correlation. The detector pixel onto which light of a given wavelength is impinging depends on instrument temperature. The temperature at the predisperser prism is used as a reference. Typical shifts with temperature are of the order of 0.01 pixel / K for the FPA channels, but depend on detector pixel. |
|||
| Etalon correction |
Radiometric Quality |
||
| Modes | Calibration key data / MME | PGS reference | |
| Calculate from | WLS (Sun) | Reference WLS spectrum | A2.16 |
| Apply to | AG.14 | ||
| Interference in the thin detector coating
layer causes a wave-like pattern on the radiance response (fixed etalon).
When deposits settle on the detector coating, the interference pattern
is changed (variable etalon). The etalon correction accounts for changes
in the variable etalon between on-ground calibration (reference etalon)
and the in-orbit situation. It is calculated from preprocessed spectra
of the on-board quartz tungsten halogen white light source (WLS). The
in-orbit WLS spectrum is ratioed to a reference WLS spectrum representative
of the on-ground calibration of the radiance response. For each channel,
a bandpass filter is applied to the ratio. Spectral components within
the bandpass (typically 4-10 oscillations per channel) are
declared to be the Etalon correction. The remainder is called Etalon
residual. Applying the Etalon correction means dividing a spectrum by
the Etalon correction spectrum. The Etalon correction typically is of the order of 10-2 relative. |
|||
| Sun mean reference
(SMR) spectrum |
Radiometric Spectral
Quality |
||
| Modes | Calibration key data / MME | PGS reference | |
| Calculate from | Sun | Irradiance response MME | A2.20 (AG.17, AG.18) |
| Apply to | Earth | N/A | A3.11 |
| A fully calibrated solar spectrum is calculated
from preprocessed spectra acquired in sun mode. Spectra within the central
part of the sun field-of-view are absolutely radiometrically calibrated
taking into account solar elevation and azimuth angles for each spectrum,
and then averaged into a sun mean reference (SMR) spectrum. The radiometric
calibration is performed for the actually measured spectra, i.e., SMR
intensities are not normalised to an Earth-sun distance of 1 AU. The
SMR spectrum is also spectrally calibrated, correcting the Doppler shift
due to the movement of the satellite towards the sun. Applying the SMR
spectrum means dividing an Earth-radiance spectrum by the SMR spectrum. The SMR irradiance is of the order of 5·1014 photons / (s cm2 nm) at 550 nm. |
|||
| Polarisation
Correction |
Radiometric Quality |
||
| Modes | Calibration key data / MME | PGS reference | |
| Calculate from | Earth | Polarisation MME | A2.21 |
| Apply to | Earth | N/A | A3.10 |
| GOME-2 is a polarisation-sensitive instrument:
the measured signal depends on the polarisation state of the incoming
light which in turn depends on the observed scene, in particular observation
geometry and cloudiness. In the Müller matrix formalism, the polarisation
state is characterised by Stokes fractions q (for the s/p component)
and u (for the +45°/-45° component). For each ground pixel,
these Stokes fractions are determined for a number of wavelengths from
the observation geometry (single scattering value for the short wavelength
end) and ratios of PMD s and PMD p band measurements. In contrast to
GOME-1, FPA measurements are not used in this step. Stokes fractions
are then interpolated to the wavelengths of the FPA detector pixels,
and for each pixel a polarisation correction factor is calculated from
Stokes fractions and polarisation MMEs. FPA signals are polarisation
corrected by dividing them by the polarisation correction factor. The polarisation correction typically is of the order of 10-2 to 10-1 relative, but may be larger for strongly polarised scenes. |
|||
| Stray light correction | Radiometric | ||
| Modes | Calibration keydata / MME | PGS reference | |
| Calculate from | Earth, Sun | Uniform stray light fraction Ghost stray light intensity and position polynomial coefficients |
AG.15 |
| Apply to | Earth, Sun | N/A | AG.16 |
| Spectral stray light is (unwanted) light
from other than the nominal wavelength measured by a given detector
pixel. We distinguish between uniform stray light from diffuse scatter
and ghost stray light from specular reflections. Both types have been
characterised during on-ground calibration. It turned out that only
intra-channel stray light has to be considered, the most important one
being the uniform stray light in channel 1. For each ground pixel the
uniform stray light contribution is calculated by weighting the sum
of the measured signals in a channel with the corresponding stray light
fraction from the calibration key data. The ghost stray light position
and intensity is calculated from the "parent" position and
intensity using polynomial coefficients from the key data. Applying
the stray light correction means subtracting the total stray light (sum
of uniform and ghost stray light) from the measured signals. The stray light correction typically is smaller than 10-3 relative. |
|||
| Spatial aliasing
correction |
Radiometric |
||
| Modes | Calibration key data / MME | PGS reference | |
| Calculate from | Earth | N/A | A3.14 |
| Apply to | |||
| Spatial aliasing is caused by the finite
readout time of FPA and PMD detector pixel arrays. The detector pixels
are read consecutively while the scan mirror is moving and therefore
each pixel observes a ground scene that is slightly shifted in space
compared to the previous pixel. E.g., with an FPA integration time of
187.5 ms and a detector readout time of 46.875 ms, the shift between
the first and last detector pixel amounts to one quarter of the observed
scene. This will alias spatial patterns into spectral patterns whose
magnitude will depend on scene variability (cloud/no cloud, land/water,
etc.). At those main channel boundaries where the last pixel of a channel
and the first pixel of the next channel are read at different times,
spatial aliasing results in intensity jumps. Spatial aliasing correction makes use of the higher spatial resolution of the PMD channels as compared to the FPA channels. FPA signals are corrected to the time of the readout of the detector pixel at the short wavelength end of the detector. When calculating the correction factor, the relative timing between PMD and FPA readouts is considered. For each FPA channel, only one PMD band is selected, and the correction is interpolated in wavelength between the selected PMD bands to avoid spectral artifacts being introduced by the correction. The spatial aliasing correction typically is of the order of 10-1 relative. |
|||
| Calculate fractional
cloud cover and cloud top pressure |
Other Quality |
||
| Modes | Calibration key data / MME | PGS reference | |
| Calculate from | Earth | N/A | A3.15 |
| An effective cloud fraction and cloud top
pressure are retrieved for each GOME-2 ground pixel using the Fast Retrieval
Scheme for Clouds from the Oxygen A band (FRESCO) developed by KNMI.
The FRESCO retrieval method is based on a comparison of measured and
simulated reflectivities in three approximately 1 nm wide spectral windows
in and around the oxygen A band (758, 761, 765 nm). Information on cloud
fraction is mainly coming from the continuum (high intensity for high
cloud cover). Information on cloud top pressure is mainly coming from
the oxygen A band itself (high absorption for low cloud). Cloud parameters derived in this function are reported in the products for use in higher level processing. They are not used any further in the Level 0 to 1 processor. |
|||
| Normalise to
integration time of 1 second |
Radiometric |
||
| Modes | Calibration key data / MME | PGS reference | |
| Apply to | All | N/A | AG.11 |
| Preprocessed signals are normalised to an integration time of one second by dividing them by the integration time. For the PMD channels the actual integration time for each block, as determined by the readout and transfer modes, is used. | |||
| Apply radiance
response |
Radiometric |
||
| Modes | Calibration key data / MME | PGS reference | |
| Apply to | Earth | Radiance response MME | A3.11 |
| Absolutely calibrated radiances in physical
units [photons / (s cm2 nm sr)] are calculated from preprocessed
signals in instrument units [BU/s]. This is done by dividing the preprocessed
signals by the radiance response MME for the actual scanner viewing
angle. For calculating absolute radiances from the PMD channels, signals
from PMD p and PMD s channels are combined. Radiances are strongly dependent on observation geometry and ground scene. Even for a fixed geometry a variation of one order of magnitude between a cloudy and a cloud-free scene is common. For a solar zenith angle of 60° typical radiance values are between 1013 and 1014 photons / (s cm2 nm sr). |
|||
| Apply irradiance
response |
Radiometric |
||
| Modes | Calibration key data / MME | PGS reference | |
| Apply to | Sun | Irradiance response MME | AG.17 |
| Absolutely calibrated irradiances in physical
units [photons / (s cm2 nm)] are calculated from preprocessed
signals in instrument units [BU/s]. This is done by dividing the preprocessed
signals by the irradiance response MME for the actual solar elevation
and azimuth angle combination. As stated above for the SMR, the solar irradiance is of the order of 5·1014 photons / (s cm2 nm) at 550 nm. |
|||
The task of product quality control is shared between the Level 0 to 1 processor and the Product Quality Evaluation (PQE) function. The Level 0 to 1 processor condenses information relevant for product quality monitoring into Product Confidence Data (PCD). These PCD records are then used by the PQE function to derive quality reports and statistics.
PCDs are provided at three levels:
A number of functions create their own PCD records, as indicated in the function descriptions above. In addition, there are two dedicated functions performing specific quality checks and generating additional PCD records.
| Determine PCDs
from raw intensity |
Quality |
||
| Modes | Calibration key data / MME | PGS reference | |
| Calculate from | All | N/A | A2.4 (AG.5, AG.6) |
Raw signals are checked for the following conditions
affecting their quality, and flags are raised accordingly:
|
|||
| Determine PCDs
from geolocation |
Quality |
||
| Modes | Calibration keydata / MME | PGS reference | |
| Calculate from | All | N/A | A2.7 (AG.7, AG.8, AG.9) |
Geolocation parameters are checked for a number
of conditions affecting data quality, and flags are raised accordingly:
|
|||
The Sensor Performance Assessment (SPA) function is used for long-term monitoring of instrument performance. Performance-related monitoring parameters are: instrument housekeeping data, spectral data (in particular in-flight calibration parameters) and polarisation data. SPA functionality comprises extraction, preprocessing and analysis of the monitoring parameters. The SPA function consists of two components:
A typical use of the SPA function would be to extract and preprocess a selected monitoring parameter, e.g. leakage current, and to visually and statistically analyse its time series (trends, oscillations, jumps, etc.).
| Extract and pre-process
monitoring data |
SPA |
||
| Modes | Data source | PGS reference | |
| Calculate from | All | 1a, 1b, in-flight calibration | A4.1 |
| Monitoring data from the Level 1a and 1b data
products and the in-flight calibration database is extracted, preprocessed
and written to the SPA database. Preprocessing reduces the volume of monitoring data, which is essential if time series over several years have to be analysed. The following preprocessing steps are applied to the extracted monitoring data:
|
|||
| Perform statistical
analysis on time series |
SPA |
||
| Modes | Data source | PGS reference | |
| Calculate from | All | SPA database | A4.2.1 |
| This function provides basic statistical functions for the higher level functions described below. | |||
| Monitor housekeeping
data |
SPA |
||
| Modes | Data source | PGS reference | |
| Calculate from | All | SPA database | A4.2.3 to A4.2.6 |
| Monitoring of housekeeping data covers thermal,
electrical and mechanical instrument performance. For life-limited items such as on-board lamps, scanner and shutter, usage statistics are derived. The total time per instrument mode is calculated. Selected temperatures are statistically analysed, in particular with respect to orbital and annual periodicities and trends. SLS and WLS voltages are statistically analysed, per switch on-period and as a long-term time series. This includes monitoring of SLS ignition delay and ignition voltage. Scanner positions across the scan are compared to the nominal values, and the timeseries of deviations is monitored. |
|||
| Monitor spectral
data |
SPA |
||
| Modes | Data source | PGS reference | |
| Calculate from | All | SPA database | A4.2.7 to A4.2.14 |
The variation of spectral data over time is
analysed in order to monitor detector performance, overall instrument
throughput and stability, and sun diffuser reflectivity. Time series of
the following parameters can be statistically analysed:
|
|||
| Monitor polarisation
data |
SPA |
||
| Modes | Data source | PGS reference | |
| Calculate from | Sun, Earth | SPA database | A4.2.15 to A4.2.16 |
Selected Stokes fractions are monitored in
order to detect changes in the instrument polarisation response and the
performance of the PMD channels. Stokes fractions are selected such that
they should be zero due to the measurement geometry. Time series of the
following parameters can be statistically analysed (see
[SCD20]):
|
|||
The Product Quality Evaluation (PQE) function provides information about the quality of the generated Level 1 products. It uses PCD records from the Level 1a and 1b data products to derive quality reports, statistics and quick-look data. The PQE function consists of two components:
| Extract quality
data |
PQE |
||
| Modes | Data source | PGS reference | |
| Calculate from | All | Level 1a and 1b products | A5.1 |
| PCD records are extracted from Level 1a and Level 1b products and stored in the PQE database. | |||
| Generate daily
quick-look data |
PQE |
||
| Modes | Data source | PGS reference | |
| Calculate from | Earth | PQE database | A5.2.1 |
The following parameters are calculated (usually
from 1 day of data, i.e., 14 orbits) and visualised on a global map:
|
|||
| Generate product
quality summaries |
PQE |
||
| Modes | Data source | PGS reference | |
| Calculate from | All | PQE database | A5.2.2 |
| PCDs for a specified time interval (day, week,
month, year) are condensed as follows: PCDs describing an occurrence number (counters) are summed. PCDs describing mean values are averaged. Flags on non-nominal situations are counted. A report with the condensed information is issued. |
|||