6. IASI Level 2 Product Formats and Dissemination
|
|
|
6. IASI Level 2 Product Formats and Dissemination |
A description of the dissemination means for EPS products and formats is provided in the following paragraphs, focusing down on IASI products and their formats.
Global EPS products at different levels will be distributed in near real-time via EUMETSAT's Data Distribution System (EUMETCast). EUMETCast utilises the services of a satellite operator and telecommunications provider to distribute data files using Digital Video Broadcast (DVB) to a wide audience located within the geographical coverage zone which includes most of Europe and certain areas in Africa.
Within the current EUMETCast configuration, the multicast system is based upon a client/server system with the server side implemented at the EUMETCast uplink site (Usingen, Germany) and the client side installed on the individual EUMETCast reception stations. The telecommunications suppliers provide the DVB multicast distribution mechanism. Data/product files are transferred via a dedicated communications line from EUMETSAT to the uplink facility. These files are encoded and transmitted to a geostationary communications satellite for broadcast to user receiving stations. Each receiving station decodes the signal and recreates the data/products according to a defined directory and file name structure. A single reception station can receive any combination of the provided services.
A typical EUMETCast reception station comprises of a standard PC with DVB card inserted and a satellite off-set antenna fitted with a digital universal V/H LNB. In addition, users require the multicast client software, which can be obtained via the EUMETSAT User Services.
More detailed information on this service can be found in the EUMETSAT webpage EUMETCast Dissemination Scheme.
Products distributed on EUMETCast can be formatted in a variety of formats, including EPS native format and the WMO formats (BUFR and GRIB).
A subset of EPS products will be disseminated additionally in near real-time via the Global Telecommunication System (GTS). GTS is the World Meteorological Organization integrated network of point-to-point circuits, and multi-point circuits which interconnect meteorological telecommunication centres. Its purpose is to enable an efficient exchange of meteorological data and products in a timely and reliable way to meet the needs of World, Regional and National Meteorological Centres. The circuits of the GTS are composed of a combination of terrestrial and satellite telecommunication links. Meteorological Telecommunication Centres are responsible for receiving data and relaying them selectively on GTS circuits. The GTS is organised on a three-level basis, namely:
More detailed information on this service can be found on the WMO website www.wmo.int.
Products distributed on the GTS are in official WMO formats, namely BUFR or GRIB.
All EPS products and auxiliary data are normally archived and made available to users from the EUMETSAT Data Centre (formerly known as the UMARF or Archive Services) upon request.
The Data Centre can be accessed through the EUMETSAT webpage EUMETSAT Data Centre. Access is through a Web interface, the Online Ordering Application, through which the users are able to browse and order products, manage their user profile, retrieve products, documentation and software libraries, get help, etc.
The Data Centre features include geographical and time sub-setting and image preview. EPS products archived in the Data Centre can be accessed in a variety of formats, including EPS native format and HDF5.
Table 6.1 summarises the different dissemination means and formats for IASI Level 2 products.
| Format | Real-Time Direct Broadcast | Near-Real-Time dissemination on EUMETCast (timeliness) | Near-Real-Time dissemination on GTS (timeliness) | EUMETSAT Data Centre retrieval (timeliness) | FTP Access |
| EPS native format | – | – |
– |
IASI Level 2 (8-9 h) |
– |
| HDF5 | – |
– |
– |
IASI Level 2 (8-9 h) |
– |
| WMO (BUFR) | – |
IASI Level 2 products (3 h) |
IASI Level 2 products (3 h) |
-- |
– |
| NetCDF | – | – | – | IASI SST L2Pcore product (8-9 h) | IASI SST L2Pcore product (4-6 h) |
'Timeliness' refers to the elapsed time between sensing and dissemination.
Table 6.1: Summary of dissemination means and formats for IASI Level 2 products
The IASI Level 2 products disseminated to users in near real-time are:
The data are disseminated in WMO (BUFR) format, as per Section 6.5 below. A full description of the IASI Level 2 BUFR format is given in Section 10.
The IASI Level 2 products available from the EUMETSAT Data Centre are:
The products are archived as full-dump products, but sub-setting capabilities are provided to the user in the retrieval step. The products are available for the users in the EUMETSAT Data Centre 8 to 9 hours after sensing, either in EPS native format or in HDF5 format.
The IASI Level 2Pcore SST products will be available on FTP. Access to the FTP server requires user registration via the EUMETSAT Earth Observation Portal: www.eumetsat.int/Home/Main/DataAccess/EOPortal.
6.3.1.1 General overview of the EPS generic product format
All products in EPS native format are structured and defined according to an EPS Generic Product Format. This format is not IASI specific. The general product section breakdown is given, and the following sections will focus on how this generic format is further applied to IASI products.
This description is not aimed at supporting the writing of reader software for the IASI or other EPS products, because readers and product extraction tools are already available (see Section 5). The intention of this and the following sections is to provide enough information to be able to use such available tools and to interpret the retrieved information.
For users interested in writing their own product readers for one or several IASI products in EPS native format, we refer them to the detailed format specifications provided in [RD11] and [RD12].
The general structure of the products is broken down in sections, which contain one or more records of different classes. Every single record is accompanied by a Generic Record Header (GRH), which contains the metadata necessary to uniquely identify the record type and occurrence within the product. The following general structure is followed by all EPS products, where all the sections occur always in the given order.
Header Section, containing metadata applicable to the entire product. The header section may contain two records, the Main Product Header Record (MPHR) and the Secondary Product Header Record (SPHR).
Pointer Section, containing pointer information to navigate within the product. It consists of a series of Internal Pointer Records (IPR), which include pointers to records within the Global Auxiliary Data, Variable Auxiliary Data and Body Sections that follow.
Global Auxiliary Data Section, containing information on the auxiliary data that have been used or produced during the process of the product and applies to the whole length of the product. There can be zero or more records in this section, and they can be of two classes: Global External Auxiliary Data Record (GEADR), containing an ASCII pointer to the source of the auxiliary data used, and Global Internal Auxiliary Data Record (GIADR), containing the auxiliary data used itself.
Variable Auxiliary Data Section, containing information on the auxiliary data that have been used or produced during the process of the product and may vary within a product, but with a frequency in any case less than the measurement data itself. There can be zero or more records in this section, and they can be of two classes: Variable External Auxiliary Data Record (VEADR), containing an ASCII pointer to the source of the auxiliary data used, and Variable Internal Auxiliary Data Record (VIADR), containing the auxiliary data used itself.
Body Section, which is usually the bulk of the product and contains the raw or processed instrument data and associated information. This section contains time-ordered Measurement Data Records (MDR). A particular type of MDR can occur to indicate the location of an unexpected data gap within any product, the Dummy Measurement Data Record (DMDR).
The format of the MPHR, IPRs, GEADR, VEADR and DMDRs is common to all products, while the other records can be of different formats and contents. Sub-classes have an instrument-specific content and format. Every record consists of a series of fields, which can have different data types. See Appendix C for all possible data types.
It is important to note that GEADR and VEADR records are included in the products to support processing configuration control for EUMETSAT at product level. They point to the name of auxiliary data files used in the processing, but they are not of any interest or use to the end-user for the utilisation of the products.
Two types of records deserve special description, because they are key to navigate within the products, namely the GRH and the IPR. Their format and the meaning of their fields are detailed in Appendix D.
Table 6.2 gives an example of general structure of the Generic Product Format.
| Section | RECORD CLASS | RECORD SUBCLASS | START TIME | STOP TIME | ||||||||||||||||||||||||||||||||||||||||||||||||
| HEADER SECTION |
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||
| INTERNAL POINTER SECTION |
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||
| GLOBAL AUXILIARY DATA SECTION |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||
| VARIABLE AUXILIARY DATA SECTION |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||
| BODY SECTION |
|
|
|
|
Table 6.2: Generalised schematic of the generic product format
6.3.1.2 Granularity of the EPS products
The Full EPS product is produced by processing a so-called granule of data. One granule contains 3 minutes of instrument data and one granule of IASI data, therefore 22 or 23 IASI scan lines. This is the product size which is also used to archive the data in the EUMETSAT Data Centre. The length in time of the product is contained in the MPHR.
6.3.1.3 Product format version control
Every record class and sub-class has an associated record version number contained in its corresponding GRH. In addition, each product has a format version number, which is stored in the MPHR.
6.3.1.4 Product naming convention
File naming convention for EPS products in EPS native format provides a product name that uniquely identifies any product and provides a summary of its contents. The field contents in a product name correspond to those in the MPHR.
<INSTRUMENT_ID>_<PRODUCT_TYPE>_<PROCESSING_LEVEL>_<SPACECRAFT_ID>_
_<SENSING_START>_<SENSING_END>_<PROCESSING_MODE>_<DISPOSITION_MODE>_
_<PROCESSING_ TIME>
| Product Name Field / MPHR Field | Description | Size in Characters |
| INSTRUMENT_ID | Instrument identification | 4 |
| PRODUCT_TYPE | Product Type | 3 |
| PROCESSING_LEVEL | Processing Level Identification | 2 |
| SPACECRAFT_IUD | Spacecraft identification | 3 |
| SENSING_START | UTC Time of start of Sensing Data | 15 |
| SENSING_END | UTC Time of end of Sensing Data | 15 |
| PROCESSING_MODE | Identification of the mode of processing | 1 |
| DISPOSITION_MODE | Identification of the type of processing | 1 |
| PROCESSING_TIME | UTC time at start of processing for the product | 15 |
Table 6.3: EPS product name fields and their correspondence with MPHR fields
For the IASI Level 2 product, the resulting product file names are as follows:
| Product | Product name |
| IASI Level 2 | IASI_SND_02_Mnn_<...> |
Table 6.4: Generic IASI Level 2 product names
Records to be found in the IASI Level 2 products are:
| Record name | Description | Usage | Subclass ID |
| MPHR | Main Product Header Record | Main product identification details | 0 |
| GEADR | Global External Auxiliary Data Record | Global auxiliary data set used for the generation of the IASI L2 product not written into the product | 0 |
| GIADR | Global Internal Auxiliary Data Record | Pressure levels for temperature, humidity and ozone profiles | 0 |
| VEADR | Variable External Auxiliary Data Record | Variable auxiliary data set used for the generation of the IASI L2 product not written into the product | 0 |
| MDR | Measurement Data Record Level 2 | IASI Level 2 products, processing and quality information | 0 |
Table 6.5: Records found in the IASI L2 format
The IASI Level 2 products are organised as successive scan lines. Each IASI Level 1 MDR product contains one IASI scan line. The IFOV within one scan line are referenced to by the geolocation. The START/STOP timeshare is indicated in the MPHR.
The occurrence of the different records in the Level 2 products is as follows:
| Record | Occurrence |
| MPHR | Once per product |
| GEADR | Once per product |
| GIADR | Once per product |
| VEADR | As required |
| MDR | Once per scan line |
Table 6.6: Occurrence of the different records in the IASI L2 format
See Section 9 for more details on the contents and EPS format of the IASI Level 2 products.
Detailed format descriptions are not currently available. HDF formatted products retrieved from the archive are named like the EPS formatted products (see Section 6.2 above) with an ‘.h5’ extension. The products can be read using standard HDF libraries. For more information on HDF5 formats in general, see the HDF5 webpages.
The IASI Level 2 products made available in WMO (BUFR) format are summarised in the table below.
| Product | Bulletin header | Originating station | Descriptor sequence |
| Atmospheric temperature and water vapour | IEDX[01-19] | EUMP | 0‑01‑007, 0‑01‑031, 0‑02‑019,
0‑02‑020, 0‑04‑001, 0‑04‑002, 0‑04‑003,
0‑04‑004, 0‑04‑005, 0‑04‑006, 0‑05‑040,
2‑01‑133, 0‑05‑041, 2‑01‑000, 0‑05‑001,
0‑06‑001, 0‑05‑043, 0‑13‑038, 0‑08‑012,
0‑08‑013, 0‑02‑022, 0‑08‑065, 0‑40‑192,
0‑40‑193, 0‑40‑194, 0‑40‑195, 0‑40‑196,
0‑40‑197, 0‑40‑198, 0‑07‑024, 0‑05‑021,
0‑07‑025, 0‑05‑022, 0‑08‑003, 0‑10‑004,
0‑08‑049, 0‑12‑061, 0‑08‑049, 2‑02‑126,
0‑07‑001, 2‑02‑000, 0‑08‑003, 1‑06‑000,
0‑31‑001, 2‑02‑131, 2‑01‑138, 0‑07‑004,
2‑01‑000, 2‑02‑000, 0‑12‑101, 1‑10‑000,
0‑31‑001, 2‑02‑131, 2‑01‑138, 0‑07‑004,
2‑01‑000, 2‑02‑000, 2‑02‑129, 2‑01‑134,
0‑13‑002, 2‑01‑000, 2‑02‑000 See also [RD25]: BUFR Descriptors for IASI L2 Data for more details |
| Atmospheric ozone | IEDX[21-39] | EUMP | 0‑01‑007, 0‑01‑031, 0‑02‑019, 0‑02‑020, 0‑04‑001, 0‑04‑002, 0‑04‑003, 0‑04‑004, 0‑04‑005, 0‑04‑006, 0‑05‑040, 2‑01‑133, 0‑05‑041, 2‑01‑000, 0‑05‑001, 0‑06‑001, 0‑05‑043, 0‑13‑038, 0‑08‑012, 0‑08‑013, 0‑02‑022, 0‑08‑065, 0‑40‑192, 0‑40‑193, 0‑40‑194, 0‑40‑195, 0‑40‑196, 0‑40‑197, 0‑40‑198, 0‑07‑024, 0‑05‑021, 0‑07‑025, 0‑05‑022, 2‑02‑126, 0‑07‑001, 2‑02‑000, 1‑07‑000, 0‑31‑001, 2‑02‑131, 2‑01‑138, 0‑07‑004, 0‑07‑004, 2‑01‑000, 2‑02‑000, 0‑15‑020 |
| Trace gases | IEDX[41-59] | EUMP | 0‑01‑007, 0‑01‑031, 0‑02‑019, 0‑02‑020, 0‑04‑001, 0‑04‑002, 0‑04‑003, 0‑04‑004, 0‑04‑005, 0‑04‑006, 0‑05‑040, 2‑01‑133, 0‑05‑041, 2‑01‑000, 0‑05‑001, 0‑06‑001, 0‑05‑043, 0‑13‑038, 0‑08‑012, 0‑08‑013, 0‑02‑022, 0‑08‑065, 0‑40‑192, 0‑40‑193, 0‑40‑194, 0‑40‑195, 0‑40‑196, 0‑40‑197, 0‑40‑198, 0‑07‑024, 0‑05‑021, 0‑07‑025, 0‑05‑022, 2‑02‑126, 0‑07‑001, 2‑02‑000, 0‑40‑199, 0‑40‑200, 0‑40‑201, 0‑40‑202 |
| Cloud parameters | IEDX[61-79] | EUMP | 0‑01‑007, 0‑01‑031, 0‑02‑019, 0‑02‑020, 0‑04‑001, 0‑04‑002, 0‑04‑003, 0‑04‑004, 0‑04‑005, 0‑04‑006, 0‑05‑040, 2‑01‑133, 0‑05‑041, 2‑01‑000, 0‑05‑001, 0‑06‑001, 0‑05‑043, 0‑13‑038, 0‑08‑012, 0‑08‑013, 0‑02‑022, 0‑08‑065, 0‑40‑192, 0‑40‑193, 0‑40‑194, 0‑40‑195, 0‑40‑196, 0‑40‑197, 0‑07‑024, 0‑05‑021, 0‑07‑025, 0‑05‑022, 2‑02‑126, 0‑07‑001, 2‑02‑000, 0‑08‑003, 1‑12‑003, 2‑02‑129, 2‑01‑131, 0‑07‑004, 2‑01‑000, 2‑02‑000, 0‑12‑101, 2‑02‑130, 2‑01‑135, 0‑20‑081, 2‑01‑000, 2‑02‑000, 0‑20‑056 |
| Surface emissivity | IEDX[81-99] | EUMP | TBD (not disseminated) |
Table 6.7: IASI Level 2 products made available in WMO (BUFR) format
The full format description of these products is available in the WMO Manual on Codes [RD46].
The names of the IASI Level 2 products distributed on EUMETCast are specified in [RD24]. They follow the pattern:
iasi_yyyymmdd_hhmmss_metopa_nnnnn_eps_o_<product code>.l2_bufr
where:
yyyymmdd stands for the UTC year, month, day of the data start sensing time hhmmss stands for the UTC hour, minute, second of the data start sensing time nnnnn is the orbit number product code = ‘twt‘ for atmospheric temperature and water vapour
= ‘ozo‘ for atmospheric ozone
= ‘trg‘ for trace gases
= ‘ems‘ for emissivity
= ‘clp‘ for cloud parameters
The IASI L2Pcore contains the fields identified in Table 6.8.
| Field | Description |
| Latitude | Latitude (degrees North) |
| Longitude | Longitude (degrees East) |
| Time | Reference time of SST data array |
| Sea_surface_temperature | Skin SST measurement values (full resolution) |
| Sst_dtime | Deviation of observation time from reference time |
| L2P_flags | Flag qualifying SST input data |
| SSES_bias | Single Sensor Error Statistics bias |
| SSES_standard _deviation | Single Sensor Error Statistics standard deviation uncertainty |
| Satellite_zenith_angle | Satellite zenith angle (degrees) |
| Solar_zenith_angle | Solar zenith angle (degrees) |
| Quality_level | L2P data quality, from 2 (= first useable data) to 5 (= best quality) |
| Wind_speed | 10-metre ECMWF wind speed (m/s) |
Table 6.8: IASI L2Pcore SST field description