NWP SAF SDP Scatterometer Data Processor User Manual and Reference Guide

NWP SAF
Satellite Application Facility
for Numerical Weather Prediction
Document NWPSAF-KN-UD-002
Version 1.5
24-07-2007
SDP User Manual and Reference Guide
Scat group
Jos de Kloe, Marcos Portabella, Ad Stoffelen, Anton Verhoef, Jeroen Verspeek,
and Jur Vogelzang
KNMI, De Bilt, The Netherlands
SDP User Manual and
Reference Guide
NWP SAF
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
SDP User Manual and Reference Guide
Scat group
Jos de Kloe, Marcos Portabella, Ad Stoffelen, Anton Verhoef, Jeroen Verspeek,
and Jur Vogelzang
KNMI, De Bilt, The Netherlands
This documentation was developed within the context of the EUMETSAT Satellite
Application Facility on Numerical Weather Prediction (NWP SAF), under the
Cooperation Agreement dated 16 December, 2003, between EUMETSAT and the Met
Office, UK, by one or more partners within the NWP SAF. The partners in the NWP
SAF are the Met Office, ECMWF, KNMI and Météo France.
Copyright 2006, EUMETSAT, All Rights Reserved.
Change record
Remarks
Version
Date
Author /
changed by
0.0
1.0
1.1
1.2
1.3
Oct 2004
May 2005
11-01-2006
27-03-2006
04-09-2006
Hans Bonekamp
Hans Bonekamp
Jur Vogelzang
Jur Vogelzang
Jur Vogelzang
1.4
1.4a
05-04-2007
09-05-2007
Jur Vogelzang
Jur Vogelzang
1.5
24-07-2007
Jur Vogelzang
First draft
Beta release
Beta release
First public release
Routines moved from SwsSupport to genscat; index
types removed; some typo’s corrected.
New 2DVAR, improved inversion
Improved description of KNMI BUFR format and flag
handling
Rewrote use under Windows; rewrote sections3.2
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1 Aims and scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Development of SDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3 Testing SDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.4 User Manual and Reference Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.5 Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
9
9
10
10
11
2
SDP User Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1 Why using the SDP program ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 Modes of using SDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3 Installing SDP . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3.1 Directories and files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3.2 Environment variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3.3 Installing BUFR library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3.4 Compilation and linking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.4 Command line options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.5 Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.6 Testruns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.7 Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
13
18
18
19
20
21
22
24
27
27
29
3
SDP Product Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1 Purpose of program SDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Output specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Input specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4 System requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31
31
31
33
33
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
3.5` Details of functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5.1 BUFR IO and coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5.2 Output resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5.3 Quality Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5.4 Inversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5.5 Ambiguity Removal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5.6 Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6 Details of performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
34
34
34
34
35
35
36
36
4
Program Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1 Top Level Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1.1 Main program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1.2 Layered model structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1.3 Data structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1.4 Quality flagging and error handling . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . .
4.1.5 Verbosity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2 Module Design for genscat layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2.1 Module inversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2.2 Module ambrem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2.3 Module Bufrmod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2.4 Support modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3 Module Design for SeaWinds layer . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3.1 Module SwsData . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3.2 Module SwsBufr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3.3 Module SwsSupport . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4 Module design for process layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4.1 Module SdpSupport . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4.2 Module SdpIO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4.3 Module SdpPrePost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4.4 Module SdpInversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4.5 Module SdpAmbrem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.5 Flag use
........................................................
37
37
37
38
41
42
42
43
43
43
43
43
44
44
51
52
53
53
53
54
56
56
57
5
Inversion module class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.2 Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.3 Antenna direction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
59
59
60
61
6
Ambiguity Removal module class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.1 Ambiguity Removal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.2 Module Ambrem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
63
63
64
2
NWP SAF
6.3
6.4
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Module BatchMod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The KNMI 2DVar scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.4.2 Data structure, interface and initialization . . . . . . . . . . . . . . . . . . . . . . . . . .
6.4.3 Reformulation and transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.4.4 Module CostFunc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.4.5 Adjoint method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.4.6 Structure Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.4.7 Minimisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.4.8 MultiFFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The PreScat scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
64
67
67
68
70
71
71
72
72
73
74
Module BufrMod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.2 Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.3 Data structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.4 Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.4 BUFR table routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.5 Center specific modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
75
75
75
77
79
80
80
6.5
7
SDP User Manual and
Reference Guide
References
..............................................................
81
Appendix A
Calling tree for SDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
83
Appendix B1 Calling tree for inversion routines
..............................
93
Appendix B2 Calling tree for AR routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
97
Appendix B3 Calling tree for BUFR routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
Appendix C1 NOAA BUFR output file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
Appendix C2 KNMI BUFR output file
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Appendix D
ECMWF BUFR data routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
Appendix E
Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
3
NWP SAF
SDP User Manual and
Reference Guide
4
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Preface
Preface to version 1.0
Software code for processing satellite data may become very complex. On the one hand, it
consists of code related to the technical details of the satellite and instruments, on the other hand,
the code drives complex algorithms to create the physical end products. Therefore, the
EUMETSAT Satellite Application Facility (SAF) project for Numerical Weather Prediction
(NWP) has included some explicit activities aiming at enhancing the modularity, readability and
portability of the processing code.
For several years, the KNMI observation research group has been developing processing code to
supply a Near Real Time (NRT) level 2 surface wind product based on the SeaWinds
Scatterometer level 1 Normalized Radar Cross Section data (σ0). This work is coordinated and
supervised by Ad Stoffelen. In the beginning only an adaptation of his ERS code existed. Later
Marcos Portabella and Julia Figa added modifications and extensions to improve, e.g., the wind
retrieval and quality control algorithms. In 2003, John de Vries finished the first official release of
a processor within the NWP SAF. This processor is called the QuikSCAT Data Processor (QDP).
QDP is available for the meteorological community since spring 2004. Several users run QDP
operationally. At KNMI, Anton Verhoef is running QDP and providing support for QDP as part
of Initial Operational Phase (IOP) of the Ocean Sea-Ice (OSI) SAF wind product.
Meanwhile, Jos de Kloe has been updating the code for ERS and ASCAT scatterometer wind
processing. For many parts of the process steps (e.g., the BUFR handling and part of the wind
retrieval) a large overlap with SeaWinds Data processing coding exists. The KNMI SCAT group
is working towards generic NRT scatterometer processing. As a result, a new modular processing
code for SeaWinds data has been developed within the NWP SAF IOP. The working name of this
code is currently the SeaWinds Data Processor (SDP). This document is the corresponding
reference manual. I hope this manual will strongly contribute to the comprehension of future
developers and of users interested in the details of the processing.
Hans Bonekamp, October 2004
5
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Preface to version 1.1
This is the first version of the SDP User Manual and Reference Guide that will be distributed to a
larger audience, as deliverable in the NWP SAF project. After Hans Bonekamp left to
EUMETSAT, Jos de Kloe, Marcos Portabella, and Anton Verhoef (as beta tester) continued
working on the SDP code. They removed a number of bugs and made a lot of improvements:
memory management was revised and the Generic Wind Section BUFR format was introduced.
My role was to adapt the first draft of this document. With the help of Jos and Anton I found my
way into the code. I made a number of adaptations and extensions to the original text, but left the
underlying structure of the document unchanged.
The reader is kindly invited to give his comments in order to improve future versions of this
document.
Jur Vogelzang, September 2005
Preface to version 1.2
Version 1.2 will be the first public version of SDP. The recommendations made by EUMETSAT
during the Delivery Readiness Inspection in November 2005 were all implemented. Moreover,
almost all known problems have been solved. The reader is kindly invited to give his comments
in order to improve future versions of this document.
Jur Vogelzang, March 2006
Preface to version 1.3
Version 1.3 is an update of the first public version of SDP. Some routines in modules SwsSupport
and Ambrem2DVAR were moved to genscat, which led to some differences in the program
structure. The index_type datatype is no longer needed and has been removed. The importance of
setting the environment variables to their proper values during compilation and linking has been
stressed. The inversion module has been improved at very low wind speed and flag management
has been revised. Some typo’s were corrected.
Jur Vogelzang, September 2006
6
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Preface to version 1.4
In version 1.4 of SDP the two dimensional variational ambiguity removal method (2DVAR) has
been completely revised. Some serious errors have been corrected. As a consequence, chapter 6
has been adapted. The presentation of the equations governing 2DVAR has been moved to a
separate report. The inversion has been improved for small wind speeds. The program structure
and organization has been changed. Some small changes were made in the command line
arguments of SDP: some unused commands are removed and a new one is added for reading the
2DVAR parameter values from file.
Jur Vogelzang, April 2007
Preface to version 1.4a
The description of the KNMI BUFR output file format is improved. The differences between the
NOAA file format and the KNMI format are stressed, notably regarding the solution probability.
A description of the use of flags by SDP is included. Some minor changes in the code structure
are documented. All references to web addresses were checked and corrected where needed.
Jur Vogelzang, May 2007
Preface to version 1.5
The difficulties with installing SDP under Cygwin have been solved to a sufficient level. Though
the minimalization strategy has been altered substantially, this does not have any impact on the
documentation. For the sake of completeness, the –ocf option, a new command option for
dumping the observation cost function, has been included in section 2.5. Like the –ana option,
the –ocf option is intended for research purposes. A description of how the lookup tables for the
expected MLE’s were obtained has been added to section 2.3.2. Section 3.2 (output specification)
has been updated and corrected.
Jur Vogelzang, July 2007
7
NWP SAF
SDP User Manual and
Reference Guide
8
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Chapter 1
Introduction
1.1
Aims and scope
The SeaWinds Data Processor (SDP) is a software package written in Fortran90 for handling data
from the SeaWinds scatterometer instruments. Details of these instruments can be found on
several sites and in several other documents. Important references are listed at the end of this
section.
SDP generates surface winds based on SeaWinds data. In particular, it allows performing the
ambiguity removal with the 2DVar method and it supports the MSS scheme, as an alternative to
the DIRTH scheme employed by NOAA. The output of SDP consists of wind vectors which
represent surface winds within the ground swath of the scatterometer. Input of SDP are
Normalized Radar Cross Section (NRCS, σ 0 ) data. These data may be real-time. The input and
output files of SDP are in BUFR format.
For SeaWinds on QuikSCAT the data are available for several years. Unfortunately, due to its
failure after 9 months, a ready to use real-time (BUFR, see subsection 3.5.1) product for
Seawinds on Adeos II is not available.
More information can be found in [Kerkmann, 1998; Leidner et al., 2000; Portabella, 2002;
Stoffelen, 1998].
1.2
Development of SDP
SDP is developed within the NWP SAF IOP program as code which can be run in an operational
setting. The coding is in Fortran 90 and has followed the procedures specified for the NWP SAF.
9
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Table 1.1 provides an overview of the persons involved in the development. Special attention has
been paid on robustness and readability. SDP may be run on every modern UNIX or LINUX
machine. SDP can also be run on a Windows machine under Cygwin as UNIX emulator (MinGW
proved to be too limited to support SDP).
Task
Coordinator
Lead Investigator
Development Team
Integrator
Project Team Leader
Beta testers
Reviewers
Person
Ad Stoffelen
Hans Bonekamp, Jur Vogelzang
Hans Bonekamp, Jos de Kloe, Anton Verhoef, Jur Vogelzang
Hans Bonekamp, Jur Vogelzang
Ad Stoffelen
Marcos Portabella, Anton Verhoef, Ferry van Geffen
Ad Stoffelen, Jos de Kloe, Marcos Portabella
Table 1.1 Overview of development tasks.
1.3
Testing SDP
Modules are tested by test programs and test routines. Many test routines or test support routines
are part of the modules themselves. Test programs can be compiled separately. For the SDP
program, the description of the test programs and the results of the testing are reported in [SCAT
group, 2005].
1.4
User Manual and Reference Guide
This document is intended as the complete reference book for SDP.
Chapter 2 is the user manual (UM) for the SDP program. This chapter provides the basic
information for installing, compiling, and running SDP.
Chapter 3 contains the Product Specification (PS) of the SDP program. Reading the UM and the
PS should provide sufficient information to the user who wants to apply the SDP program as a
black box.
The subsequent chapters are of interest to developers and users who need more specific
information on how the processing is done. The Top Level Design (TLD) of the code and the
Module Design (MD) of the SDP code can be found in chapter 4.
Several modules are very generic for NRT scatterometer data processing. Examples are the
modules for the BUFR handling, ambiguity removal, and parts of the wind retrieval. These
generic modules are part of the genscat layer and are described in chapters 5, 6 and 7.
The appendices of this document contain a complete calling tree of the SDP program up to and
including the genscat layer. The appendices also contain a list of SeaWinds BUFR data
descriptors, a list of the ECMWF BUFR routines, and a list of acronyms.
Finally, many sections end with a remarks alinea. Mostly, the remarks contain some
10
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
recommendations for future development or explain the correspondence with the QDP code.
These remarks will be reconsidered in future versions of this reference book.
1.5
Conventions
Names of physical quantities (e.g., wind speed components u and v), modules (e.g. BufrMod),
subroutines and identifiers are printed italic.
Names of directories and subdirectories (e.g. /SDP/sdp), files (e.g. sdp.F90), and commands
(e.g. sdp -f input) are printed in Courier. When addressing software systems in general, the
normal font is used (e.g. SDP, genscat).
Hyperlinks are printed in blue and underlined (e.g. www.knmi.nl/scatterometer).
References are in square brackets with the name of the author italic (e.g. [Stoffelen, 1998]).
11
NWP SAF
SDP User Manual and
Reference Guide
12
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Chapter 2
SDP User Manual
This chapter is the user manual of the SDP program. The SDP program is the follow-up of the
QDP program. Therefore, the QDP user manual [de Vries et al., 2004] is in some cases
appropriate to understand the operations of SDP. However, SDP has extended capabilities, such
as higher resolution and the Multi Solution Scheme (MSS).
Section 2.2 provides information on how to install, compile, and link the SDP software. The
command line arguments of SDP are discussed in section 2.3. Section 2.4 gives information on
some scripts for running SDP that are part of this release.
2.1
Why using the SDP program ?
Scatterometers provide valuable observational data over the world's oceans. Therefore, successful
assimilation of scatterometer data in numerical weather prediction systems generally improves
weather forecasts. The SDP program has been developed to fully exploit scatterometer data. It is
meant to form the key component of the observation operator for surface winds in data
assimilation systems.
The general scheme of SDP (and any other wind scatterometer data processor is given in figure
2.1. The input of the SDP program is the NOAA SeaWinds level 2b BUFR wind product.
However, only the level 1 data contained in the NOAA (the σ0 values) are used in SDP.
The SDP processing chain contains five steps (see figure 2.1):
1.
Pre-processing. The input BUFR file is decoded and the σ0 values are written in the data
structures of SDP.
13
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
2.
Inversion. The σ0 values are compared to the Geophysical Model Function (GMF) by means
of a Maximum Likelihood Estimator (MLE). The wind vectors that give the best description
of the σ0 values (the solutions) are retained. The MLE is also used to assign a probability to
each wind vector. The normal scheme allows 4 solutions at most, but in the Multi Solution
Scheme (MSS) the maximum number of solutions is 144.
3.
Quality Control. Solutions that lie far away from the GMF are likely to be contaminated by
rain, sea ice, and/or confused sea state. During Quality Control these solutions are identified
and flagged.
4.
Ambiguity Removal. This procedure identifies the most probable solution using some form
of external information. SDP uses a two-dimensional variational scheme (2DVar) as default.
A cost function is minimized that consists of a background wind field and all solutions with
their probability, using mass conservation and continuity as constraints. The background
wind field is obtained from the NCEP model winds in the NOAA SeaWinds level 2b product
(the input file of SDP).
5.
Quality Monitoring. The last step is to write the results in BUFR format and to output quality
indicators.
Input
(σ0 values)
Pre-processing
Inversion
Quality Control
NWP
model
Ambiguity Removal
Quality Monitoring
Output
wind field
Figure 2.1 SDP processing scheme. When using MSS the wind vectors and their probabilities after
Quality Control may be fed directly in the Data Assimilation step of a Numerical Weather Prediction
model.
Step 1 and 5 of the processing chain are rather trivial; the real work is done in steps 2, 3, and 4.
Note that an inconsistency may arise if the output wind field is assimilated into a numerical
weather prediction (NWP) model: in the data assimilation step the scatterometer wind field will
14
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
be checked for mass conservation and continuity, but this has already been done in the 2DVar
Ambiguity Removal step! Therefore it is recommended to feed the wind solutions and their
probabilities directly into the NWP data assimilation step after Quality Control, as indicated in
figure 2.1.
As further detailed in chapter 3, SDP profits from developments in
• inversion and output of the full probability density function of the vector wind (Multi
Solution Scheme, MSS);
• rain detection and Quality Control (QC);
• meteorologically balanced Ambiguity Removal (2DVar);
• quality monitoring;
• variable resolution.
Figure 2.2 shows some example wind fields that demonstrate the improvements achievable with
SDP using MSS and 2DVar.
Another important - but not yet validated - aspect of the SDP program is the possibility to create
an output wind product with a different resolution. Figure 2.3 shows an example of a SDP result
at 25 km resolution. There is, of course, a trade-off between the output resolution and the output
accuracy. The SDP program may help to process the data in the most appropriate manner for the
application under consideration.
SDP yields wind fields with high accuracy. Table 2.1 shows the results of a study on its accuracy.
The table gives the mean root mean square difference with the ECMWF First Guess at
Appropriate Time (FGAT) for the SDP processed winds without and with MSS. As a reference,
the results for NCEP model winds are given in the last column. The MSS result is much better,
especially at nadir, and further improves on the NCEP winds.
Swath region
Sweet
nadir
SDP standard
2.48
2.98
SDP with MSS
2.23
2.45
NCEP
2.85
2.96
Table 2.1 Mean vector root mean square difference with ECMWF FGAT winds for SDP processed winds
and 1000 mb level NCEP model winds.
A complete specification of the SDP program can be found in the Product Specification in
Chapter 4. The program is based on generic genscat routines for inversion, ambiguity removal,
and BUFR file handling. These routines are discussed in more detail in chapters 5 – 7.
15
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Figure 2.2 An example of the advantages of SDP. The upper left image shows the wind field obtained
from SeaWinds using the standard NOAA processing. The field contains some errors at low wind speeds
and doesn’t look smooth. The upper right image shows the wind field obtained by running SDP in MSS
mode, retaining the most probable solution. As a reference, the lower image shows the ECMWF first guess
winds. The scatterometer fields contain more detail and, even more important for prediction, put the
structure at a different location.
16
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Figure 2.3 SDP wind field retrieved in MSS mode for January 31, 2005, at 25 km resolution, overlaid on
an IR satellite image. Only wind arrows 50 km apart are shown. The cold front on the left of the image is
clear and sharp. The yellow dots are rejected WVC’s, mostly because of rain.
17
NWP SAF
2.2
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Modes of using SDP
There are several modes to assimilate the SeaWinds data in NWP models using SDP. Anyway,
the first thing to assure oneself of is the absence of biases by making scatter plots between
SeaWinds and NWP model first guess for at least wind speed, but wind direction and wind
components would also be of interest to guarantee consistency.
The operational SDP SeaWinds product, available as a deliverable from the NWP SAF project,
could be the starting point for NWP assimilation:
1.
The unique solution at every WVC may be assimilated as if it were buoys. This is the fastest
way and one exploits the data to a large extend. For a small advantage, SDP could be
installed to provide 2D-VAR solutions based on the local first guess.
2.
The SDP software may be used to modify the 3D-VAR or 4D-VAR data assimilation system
to work with the ambiguous wind solutions and their probabilities at every WVC. This is
some investment, but is applicable for all scatterometer data. The advantage with respect to
1) occurs occasionally, but always in the dynamic atmospheric cases (storms/cyclones) that
are really relevant.
1) and 2) can be based on SDP in standard or MSS mode, and at various resolution. MSS is
somewhat more dependent on the first guess in 2D-VAR than the SDP standard, but much less
noisy (see above) A more noticeable advantage is thus obtained by using the local first guess and
potentially the full hi-res benefit of the SeaWinds data is achieved. At the moment, the 25-km
mode is experimental, since at KNMI we are now objectively evaluating the added value of MSS
and 2DVar at 25 km. Please contact the NWP SAF helpdesk if this mode will be implemented
(address: http://www.metoffice.com/research/interproj/nwpsaf/) The mode of using SDP thus
depends on the opportunities, experience, and time the user has to experiment with SeaWinds in
the NWP system under consideration. See also section 3.2.
The SDP program can, of course, also be used to create a stand-alone wind product. Such a standalone SeaWinds wind product is a deliverable of the OSI SAF project. More information on this
project can be found at the project web site, http://www.osi-saf.org/index.php.
2.3
Installing SDP
SDP is written in Fortran 90 (with a few low level modules in C) and is designed to run on a
modern computer system under LINUX or UNIX. SDP needs a Fortran 90 compiler and a C
compiler for installation. SDP comes along with a complete make system for compilation. The
makefile contains installation scripts which are written in Bourne shell to enhance portability.
When compiled, SDP requires about 60 Mb disk space.
SDP may also run under Windows. However, SDP needs the BUFR library from ECMWF, and
this poses some restrictions on the systems supported. Under Windows one must use Cygwin, a
free UNIX emulator (see http://www.cygwin.com/ for more information and download).
MinGW/MSYS (http://www.mingw.org/) proved to be too limited to support SDP.
18
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
To install SDP, the following steps must be taken:
1.
Copy the SDP package (file SDP_1.5.tar.gz) to the directory from which SDP will
be applied, and unzip and untar it. This will create subdirectories SDP and genscat that
contain all code needed (see 2.3.1).
2.
Download the ECMWF BUFR library file bufr_000320.tar.gz (or another version
not earlier than 000240) and copy it to directory /genscat/support/bufr. See also
2.3.3.
3.
Go to the work directory (the one above directories SDP and genscat and enter
InstallSDP. The script will ask for the compiler used and invoke the make system for
compilation and linking of the software (see also 2.3.4). For convenience, this script
checks if the BUFR library file is present.
SDP is now ready for use, provided that the environment variables discussed in section 2.3.2 have
the proper settings. See also 2.4 and 2.5.
2.3.1
Directories and files
All code for SDP is stored in a file named SDP_1.5.tar.gz that is made available in the
framework of the NWP SAF project. This file should be placed in the directory from which SDP
is to be run. After unzipping (with gzip -df SDP_1.5.tar.gz) and untarring (with tar xf SDP_1.5.tar), the SDP package is extracted in subdirectories SDP and genscat, which
are located in the directory where the original file SDP_1.5.tar.gz was located.
Subdirectories SDP and genscat each contain a number of files and subdirectories. A copy of
the release notes and the script InstallSDP can also be found in the directory containing SDP
and genscat.
Tables 2.1 and 2.2 lists the contents of directories SDP and genscat, respectively, together with
the main contents of the various parts.
Name
data
docs
exec
makefile
python
readme.txt
sdp
sws
test
Type
subdirectory
subdirectory
subdirectory
file
subdirectory
file
subdirectory
subdirectory
subdirectory
Contents
Look Up Table for the SeaWinds Geophysical Model Function (GMF)
Documentation, including this document
Shell scripts for running SDP with various input options
Makefile for compiling SDP under LINUX or UNIX
Python scripts for running SDP under various operating systems
Readme file with some information on SDP.
Source code for main SDP program and supporting routines
Source code for SeaWinds dependent routines
Example BUFR input and output files for testing purposes.
Table 2.1 Contents of directory SDP.
19
NWP SAF
SDP User Manual and
Reference Guide
Name
ambrem
ambrem/twodvar
inversion
main
Makefile
Objects.txt
Readme.txt
Set_Makeoptions
support
support/BFGS
support/bufr
Type
subdirectory
subdirectory
subdirectory
subdirectory
file
file
file
script file
subdirectory
subdirectory
subdirectory
support/datetime
support/file
support/multifft
use_g95
use_gfortran
use_ifort
use_pgf90
subdirectory
subdirectory
subdirectory
script file
script file
script file
script file
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Contents
Source code for ambiguity removal routines
Source code for KNMI 2DVar ambiguity removal routines
Source code for inversion routines
Dummy subdirectory to facilitate the make system
Makefile for compiling Gencat
Part of the makefile
Readme file with some information on genscat
Script needed by the make system.
Collection of general purpose routines sorted in subdirectories
Source code for minimization routines needed in 2DVar
BUFR tables (in subdirectories) and source code for BUFR file
handling routines
Source code for date and time conversion routines
Source code for file handling routines
Source code for FFT routines needed in minimization
Script for choosing the GNU g95 Fortran compiler
Script for choosing the GNU-GCC compiler collection
Script for using the Intel Fortran compiler
Script for using the Portland Fortran compiler
Table 2.2 Contents of directory genscat.
Directories SDP and genscat and their subdirectories contain various file types:
ƒ Fortran 90 source code, recognizable by the .F90 extension;
ƒ Files and scripts that are part of the make system for compilation like Makefile_thisdir,
Makefile, use_, Objects.txt and Set_Makeoptions (see 2.3.4 for more details);
ƒ Scripts for the execution of SDP in directories /SDP/exec and /SDP/python;
ƒ Look-up tables and BUFR tables needed by SDP;
ƒ Files with information like readme.txt.
After compilation, the subdirectories with the source code will also contain the object codes of
the various modules and routines.
2.3.2
Environment variables
SDP needs a number of environment variables to be set. These are listed in table 2.3 together with
their possible values.
The PLATFORM variable depends on the operating system used. It should be set to
big_endian under IRIX and SUN OS, and to little_endian under LINUX, OSF, and
Windows. The PLATFORM variable is needed to guide SDP to the correct version of the lookup
table containing the Ku-band Geophysical Model Function (GMF) needed for the inversion. These
tables are in binary form, and the various operating systems have different representations of
binary data.
20
NWP SAF
SDP User Manual and
Reference Guide
Name
PLATFORM
BUFR_TABLES
LUT_FILENAME_KU_HH
LUT_FILENAME_KU_VV
EXP_MLE_FILENAME
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Value(s)
big_endian
little_endian
genscat/support/bufr/bufr_tables/
SDP/data/${PLATFORM}/nscat2_250_73_51_hh.dat
SDP/data/${PLATFORM}/nscat2_250_73_51_hh.dat
Depends on resolution, see table 2.4
Table 2.3 Environment variables for SDP.
The BUFR_TABLES variable guides SDP to the BUFR tables needed to read the input and write
the output.
The variables LUT_FILENAME_KU_HH and LUT_FILENAME_KU_VV point SDP to the correct
Ku-band GMF lookup tables at HH and VV polarization, respectively. Note that these variables
contain the PLATFORM variable already discussed.
Resolution
(km)
25
50
100
Value of EXP_MLE_FILENAME
SDP/data/${PLATFORM}/mean_bufr_1r_mle_knmi9_25_r5_mm.dat
SDP/data/${PLATFORM}/mean_bufr_1r_mle_knmi9_50_r5_mm.dat
SDP/data/${PLATFORM}/mean_bufr_1r_mle_knmi9_100_r5_mm.dat
Table 2.4 Values of variable EXP_MLE_FILENAME for various resolutions.
The EXP_MLE_FILENAME variable points SDP to a lookup table containing the mean MLE’s as
a function of node number and wind speed [Portabella, 2002]. The mean MLE’s are needed for
quality control, see section 4.4.3. These LUT’s are resolution dependent, so the value of
EXP_MLE_FILENAME must agree with the resolution specified in the command line options of
SDP (see section 2.3). The possible values of EXP_MLE_FILENAME are shown in table 2.4.
The lookup tables for the expected MLEs which are provided with the SDP package are obtained
using the method described by Portabella [2002, page 39 third bullet and appendix B.4]:
QuikSCAT data of the first 21 days of 2001 were reprocessed using SDP and the MLE values
were calculated. Tables (matrices of node numbers and speed indexes) of mean MLE values were
created. The data of each matrix element (node number, wind speed) were filtered by repeatedly
throwing away all values higher than 5 times the mean value for that element. After 9 iterations,
the data sets appeared to converge and no more values were rejected. This procedure was done for
100 km, 50 km and 25 km resolutions, yielding the three tables with expected MLE values.
2.3.3
Installing BUFR library
SDP needs the ECMWF BUFR Library for its input and output operations. Only ECMWF is
allowed to distribute this software. It can be obtained free of charge from ECMWF at the BUFR
web page http://www.ecmwf.int/products/data/software/bufr.html. The package contains scripts
for compilation and installation. The reader is referred to this site for assistance in downloading
21
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
and installing the BUFR Library.
Directory genscat/support/bufr contains the shell script make.bufr.lib, which
unzips, untars, and compiles the BUFR library file downloaded from ECMWF. This script is part
of the genscat make system and is automatically invoked when compiling genscat. The current
version assumes BUFR version 000320, but later versions (or earlier, but not earlier than 000240)
can be used if the reference to file bufr_000320 is set to the appropriate file name in scripts
make.bufr.lib and make.clean.bufr.lib, that are both located in directory
genscat/support/bufr.
BUFR file handling at the lowest level is difficult to achieve. Therefore some routines were coded
in C. These routines are collected in library BUFRIO (see also section 7.4). Its source code is
located in file bufrio.c in subdirectory genscat/support/bufr. Compilation is done
within the genscat make system and requires no further action from the user (see 2.3.4).
2.3.4
Compilation and linking
Compilation and linking of SDP under LINUX or UNIX is done in three steps by the script
InstallSDP:
1. Set the compiler environment variables according to the choice entered on request. This is
equivalent of running the appropriate use_* scripts in directory genscat;
2. Go to directory genscat and invoke the make system;
3. Go to directory SDP and invoke the make system to produce the executable sdp in directory
SDP/sdp.
Before activating the make system, some environment variables identifying the compiler should
be set. These variables are listed in table 2.5. The environment variables in table 2.5 are set by the
script InstallSDP, but can also be set by using one of the use_* scripts located in directory
genscat. Table 2.6 shows the properties of these scripts. The scripts are in Bourne shell
(extension .bsh) and in C shell (extension .csh). Note that all scripts select the GNU gcc C
compiler.
Variable
GENSCAT_F77
GENSCAT_F90
GENSCAT_CC
GENSCAT_LINK
GENSCAT_CLINK
GENSCAT_SHLINK
Function
Reference to Fortran 77 compiler
Reference to Fortran 90 compiler
Reference to C compiler
Reference to linker for Fortran objects
Reference to linker for C objects
Reference to linker for shared objects
Table 2.5 Environment variables for compilation and linking.
22
NWP SAF
Script
use_g95
use_gfortran
use_ifort
use_pgf90
SDP User Manual and
Reference Guide
Fortran
compiler
g95
gfortran
ifort
g90
C
compiler
gcc
gcc
gcc
gcc
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Remarks
GNU compilers by A. Vaught
GNU-GCC 4.0 compiler collection
Intel Fortran compiler
Portland Fortran compiler
Table 2.6 Properties of the four use_* scripts.
Example: To select the GNU g95 compiler under Bourne shell type “. use_g95.bsh”, the dot
being absolutely necessary in order to apply the compiler selection to the current shell. Under C
shell the equivalent command reads “source use_g95.csh”.
If the user wants to use a Fortran or C compiler not included in table 2.6, he can make his own
version of the InstallSDP or use_* script, or include the environment variables for
compilation and linking in his startup file. The user must extend
SDP is delivered with a complete make system for compilation and linking under UNIX or
LINUX. The make system is designed as portable as possible, and system dependent features are
avoided. As a consequence, some tasks must be transferred to shell scripts. The make system
consists of two parts: one for SDP and one for genscat. The genscat part should be run first. For
compilation and linking of the genscat part, the user should move to the genscat directory and
simply enter make.
The Makefile refers to each subdirectory of genscat, invoking execution of the local
Makefile and, in cases where a subdirectory contains code as well as a subdirectory containing
code, Makefile_thisdir. The makefiles need supplementary information from the files
Objects.txt which are present in each directory containing code. The settings for the
compilers are located in file Makeoptions in directory genscat. This file is generated by the
Bourne shell script Set_Makeoptions which is called automatically by the genscat make
system. The local Makefile in subdirectory genscat/support/bufr calls the script
make.bufr.lib for compilation of the BUFR library (see 2.3.3). It also contains the Fortran
program test_modules that generates the binary BUFR tables B and D from the ASCII tables
already present, and is executed automatically by the make system. Program test_modules
can also be used to test the genscat BUFR module, see 2.7. The Makefile in subdirectory
genscat/support/bufr/bufr_tables
calls
the
shell
scripts
run_make_symlinks_for_first_table and run_make_all_needed_symlinks_
for_SEAWINDS. These scripts make copies of the generic binary BUFR tables B and D under
different names. There are four different naming conventions in BUFR version 000240 to
000280, and binary files are generated for each of them. The copies could be replaced by
symbolic links to save disk space, but this is not guaranteed to work on each system (symbolic
links are not understood by Cygwin under Windows XP). Further information on the make
system is given in the inline comments in the scripts and makefiles.
23
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Compilation and linking of the SDP part is done in a similar manner: go to the SDP directory and
enter make. As with genscat, the make system will execute makefiles in every subdirectory of
SDP. The result is the executable sdp in directory SDP/sdp. SDP is now ready for use. The
make system of SDP doesn’t need any further files except the genscat file Makeoptions. This
is the reason why genscat should be compiled first.
The GMF tables in SDP/data are set to read-only. Some systems (e.g. Cygwin) require write
permission for properly reading those tables. This should be done separately using the command
chmod u+w in the appropriate subdirectory.
When recompiling (part of) SDP or genscat with the make system, for instance when installing a
new version of the BUFR library, one should be sure that the proper environment variables for
compilation and linking are set. To recompile all of the software enter InstallSDP again.
To recompile part of the software invoke the make system where needed. Don’t forget to rerun
the use_* commands to select the right compiler.
2.4
Command Line Options
The SDP program is started from directory SDP/sdp with the command
sdp [options] < -f BUFRfile | -fl FileList >
with <> indicating obligatory input, [] indicating obligatory input, and | indicating alternatives.
The following command line options are available:
-f <BUFRfile>
Process a single BUFR input file with name BUFRfile.
The BUFR input file should have the NOAA format.
Example: sdp -f QS_D02001_S0006_E0120_B1320303 will
process this file. The results will be written on a file with the name
QS_D02001_S0006_E0120_B1320303~. In general, each output
BUFR file has the same name as the corresponding input file, but with a
tilde attached. Either this option or the next one is obligatory.
-fl <FileList>
Process a list of BUFR input files in the file named FileList. Either
this option or the previous one is obligatory.
-par <File>
Read the parameters of 2DVAR from a file with name (and path) File.
If absent, SDP assumes default values. See 6.4 for more information.
This option is included for research purposes.
-mss
Use the Multiple Solution Scheme for Ambiguity Removal.
If the Multiple Solution Scheme (MSS) is switched on, SDP internally
works with 144 different solutions for the wind vector. If MSS is
switched off, SDP calculates four solutions at most. MSS is switched off
as default.
24
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
-genericws <n>
Produce a second BUFR file in generic wind section format.
This option generates a second BUFR output file in the KNMI generic
wind section format not yet approved by the WMO. The number n
specifies the number of wind vector solutions written in the output file.
The number n should not exceed 144. The name of the output file is the
same as that of the input file, but with an extension ~.genws.
Example: the command sdp –genericws 144 –f QSExample
(without MSS switched on) will produce a second output file with name
QSExample~.genws. However, this file contains only 4 wind
solutions at most because MSS is switched off by default. The other 140
solutions are set to missing.
Without MSS switched on, it is more appropriate to set n equal to 4.
-ana
Dump 2DVAR analysis increments
This option dumps the 2DVAR analysis increments batch by batch in
ASCII format on a file with extension .ana. It is included for research
purposes.
-ocf
Dump 2DVAR observation cost function
This option dumps the values of the observational part of the 2DVAR
cost function batch by batch in ASCII format on a file with extension
.ocf. It is included for research purposes.
-resol <i>
Select output resolution.
The output resolution is controlled by a single resolution index i, with i
an integer from 0 to 15. The least significant two bits define the across
track in multiples of 25 km, and the next two bits similarly define the
along track resolution (see table 2.7). The default value is 15, i.e. 100 km
resolution.
Index
0
1
3
4
5
7
12
13
15
Resolution in km
Across
Along track
track
25
25
50
25
100
25
25
50
50
50
100
50
25
100
50
100
100
100
Implemented?
Yes (not yet validated)
No
No
No
Yes (not yet validated)
No
No
No
Yes (default)
Table 2.7 Resolution specification.
25
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Example: the command sdp -f QSExample -resol 0 will
process the SeaWinds file QSExample with a resolution of 25 km along
track and across track. The results are written on file QSExample~.
Warning: the environment variable EXP_MLE_FILENAME should have
the correct value corresponding to the resolution set with the -resol
command.
-qdp
Processing in QDP mode.
This mode of operation is equal to the old QDP scheme. MSS is switched
off and the inversion uses no parabolic fitting to find the minimum in the
cost function when determining the wind direction. The resolution is set
to 100 km (resolution index 15).
-noinv
Switch off inversion (default switched on).
-noamb
Switch off ambiguity removal (default switched on).
This option is useful when is run in MSS mode, and selection of the
scatterometer wind is left to the data assimilation procedure of the
Numerical Weather Prediction model. In other words: the NWP model is
fed with a large number of solutions and their probability, and finds the
best value when comparing with other data sources. This avoids too large
influence of the NWP model. Such a procedure will be implemented for
KNMI’s HIRLAM.
-nowrite
Do not produce BUFR output (default switched on).
-mon
Switch on the monitoring function.
The results are written on a file with the same name as the input file, but
with an extension .mon added. As default no monitoring file is
produced.
-mononly
Write the monitoring file without any processing.
The command sdp -mononly has the same effect as the command
sdp -mon -noinv -noamb –nowrite.
Warning: the –qdp option is switched off by the –mononly option.
-verbosity <l>
Set the verbosity level to l.
If the verbosity level is -1 or smaller, no output is written to the standard
output except error messages. If the verbosity level equals 0 only some
top level processing information is written to output. If the verbosity level
is 1 or greater, also additional information is given.
Running the command sdp without any command line options will yield the following output on
the console:
26
SDP User Manual and
Reference Guide
NWP SAF
Usage: sdp
with [.] :
<.> :
| :
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
[options] < -f BUFR file | -fl file list >
free options
mandatory options
choice between alternatives
Options:
-f <BUFRfile>
-fl <Filelist>
-par <File>
-mss
-genericws <N>
-
-resol <I>
-qdp
-noinv
-noamb
-nowrite
-mon
-mononly
-verbosity <L>
-
process file named BUFRfile
process list of BUFR files in Filelist
Read 2DVAR parameters from File
use Multiple Solution Scheme MSS
write second BUFR file with generic wind section
containing N wind solutions
set resolution index to value I
process in QDP mode
switch off inversion
switch off ambiguity removal
do not produce BUFR output
switch on monitoring
write monitoring info without processing
set verbosity level to L
Running the command sdp with an illegal option Illegal will produce the same output, but
preceded by the error message:
Invalid option Illegal
2.5 Scripts
Directory SDP/execs contains four Bourne shell scripts for running SDP with specific input
options and the correct environment variables. The reader is referred to the scripts themselves to
find out their use and operation.
Directory SDP/python contains Python scripts for execution of the SDP program on different
platforms (e.g., Linux, SGI, and Sun). The main goal of these scripts is to test the operation of the
program.
A dedicated Python package called seawindspy contains support data and procedures, see the
folder SDP/python/seawindspy. For example, it contains the Python Classes SdpClass
and QdpClass to operate SDP or QDP in Python scripts. Python is a freeware object-oriented
programming language. It can be obtained from www.python.org.
It is recommended to use shell scripts for running SDP to avoid errors caused by conflicting
values of environment variables and command line options.
2.6
Testruns
Directory SDP/tests contains four BUFR files for testing the SDP executable.. File
QS_D02001_S0006_E0120_B1320303 is an input file for SDP. Files SDP_Testrun_1,
SDP_Testrun_2 ,and SDP_Testrun_3 are SDP output files for the runs specified in table
2.8.
27
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Copying file QS_D02001_S0006_E0120_B1320303 to directory SDP/execs and running
one of the commands of table 2.8 will yield a BUFR output file with the default name
QS_D02001_S0006_E0120_B1320303~ which should contain the same results as one of
the three SDP_Testrun files, depending on which command is applied.
Command
sdp_025 -f ../tests/QS_D02001_S0006_E0120_B1320303
sdp_025 -f ../tests/QS_D02001_S0006_E0120_B1320303 -mss
sdp_qdp -f ../tests/QS_D02001_S0006_E0120_B1320303
Result identical with
SDP_Testrun_1
SDP_Testrun_2
SDP_Testrun_3
Table 2.8 SDP testruns.
Figure 2.4 shows the global coverage of the testrun. SeaWinds covered part of the Indian Ocean
southeast of India, part of the Barentz Sea north of Scandinavia, small parts of the Hudson Bay,
the Great Lakes, and the Gulf of Mexico, and a large strip in the Pacific west of South America.
The colors indicate the magnitude of the wind speed as indicated by the legendum. Figure 2.4
shows the results of testrun number 2, but the two other testruns will yield very similar results for
the magnitude of the wind speed. More information on these tests (and other tests) is given in the
SDP Test Report [SCAT group, 2005].
Figure 2.4 Global coverage of the testruns. Wind speed results for testrun 2 are shown.
Due to round-off differences, a simple file comparison may not be appropriate to test the SDP
output. It is then necessary to decode the BUFR files and compare the retrieved wind field with
the one in the SDP_Testrun file. BUFR decoding software is not part of the SDP package, but
may be obtained from KNMI if requested. See also below.
28
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Directory genscat/support/bufr contains a test program named test_modules. It is
invoked by the genscat make system to construct the BUFR tables required by SDP, but it can
also be used to test the genscat BUFR module. The program is used as follows:
test_modules [BUFRinput]
where BUFRinput is the BUFR input file.
If omitted, the program uses as default input the file testreading.bufr in directory
genscat/support/bufr. The output is written on the BUFR file named
testwriting.bufr. The directory also contains a shell script named run_test_modules
that sets the environment variables required and executes the program. Further information can be
found in the comment lines of the source code of test_modules.
Subdirectories convert, num, file and datetime of genscat/support contain test
programs for the module in that subdirectory. The test programs write their result to the standard
output. For comparison, a copy of the output is contained in the .output files. Table 2.9 gives
an overview of the genscat test programs.
Directory
genscat/support/bufr
genscat/support/convert
genscat/support/datetime
genscat/support/file
genscat/support/numerics
Program name
test_modules
test_convert
TestDateTimeMod
TestLunManager
test_numerics
Output file
testwriting.bufr
test_convert.output
TestDateTimeMod.output
TestLunManager.output
test_numerics.output
Remarks
Part of make system
Wind speed conversion
Date and time conversion
File management
Numerical issues
Table 2.9 Test programs in genscat.
2.7
Documentation
Directory SDP/docs contains some documentation on SDP, including this document and the
Test Report. Further information can be found in the readme.txt files, and in the comments in
scripts, makefiles and source code.
29
NWP SAF
SDP User Manual and
Reference Guide
30
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Chapter 3
SDP product specification
3.1
Purpose of program SDP
The SeaWinds Data Processor (SDP) program has been developed to fully exploit σ 0 data from
the SeaWinds scatterometer instruments on either the QuikScat or Adeos-II (Midori-II) satellites
to generate surface winds. SDP may be used real-time. The main application of SDP is to form
the core of an Observation Operator for SeaWinds Scatterometer data within an operation
Numerical Weather Prediction System.
Program SDP is also a level 2 data processor. It reads data from the NOAA SWS\_met product,
see [Leidner et al., 2000]. SDP applies improved algorithms for inversion, Quality Control, and
Ambiguity Removal at various spatial resolutions. These methods are mainly developed and
published by KNMI. The output of SDP is again a BUFR file.
3.2
Output specification
The wind vectors generated by SDP represent the instantaneous mean surface wind at 10 m
anemometer height in a 2D array of Wind Vector Cells (WVC's) with specified size (optionally
100 × 100 km2, 50 × 50 km2, or 25 × 25 km2). These WVC's are part of the ground swath of the
instrument and are numbered with revolution numbers, along-track row numbers, and acrosstrack node numbers. Therefore, every WVC is identified by a unique (lat, lon, time) triple or a
unique (revolution number, row number, node number) triple.
In conventional mode, the wind output for every WVC consists of up to 4 ambiguities (wind
vector alternatives, with varying probabilities). The selected wind vector is indicated by a
31
SDP User Manual and
Reference Guide
NWP SAF
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
selection index. For every WVC additional parameters are stored. These are e.g.: latitude,
longitude, and time information, revolution, row, and node numbers, background wind vector,
cell quality flag, and information on the scatterometer beams including σ 0 and K p data. The
output file is structured according to the conventions of the SWS\_met input product (NOAA
format). A full description is given in Appendix C1.
A second output file is produced if the genericws option is switched on. This file is in the socalled Generic Wind Section format or KNMI format. It contains up to 144 wind vector solutions
and their normalized MLE’s. This format is not yet approved by the WMO. A full description is
given in appendix C2.
At this point it is important to note some differences between the various BUFR output formats:
ƒ NOAA BUFR output without MSS applied. Up to 4 solutions are given. The probability of
each solution is given as a number between 0 and 1 with a resolution of 0.001. The probability
is normalized to 1, i.e., the sum of the probabilities over all solutions equals 1.
ƒ NOAA BUFR format with MSS. Only the selected solution is given with its normalized MLE.
The number of solutions is 0 or 1, the solution index is missing or 1.
ƒ KNMI BUFR format without MSS applied. Up to 4 solutions are given, like for the NOAA
BUFR format, but now for each solution both the base 10 logarithm of the normalized
probability and the normalized MLE are given.
ƒ KNMI BUFR format with MSS applied. Up to 144 solutions with both the base 10 logarithm
of their probability and their normalized MLE are written. The number of output solutions can
be determined with the –genericws command line option.
See table 3.1 for a summary.
Number of solutions
Probability
information
Intended use
NOAA BUFR format
No MSS
MSS
1-4
1
Normalized
Normalized MLE
probability
Stand-alone
product for
nowcasting
KNMI BUFR format
No MSS
MSS
1-4
1-144
Base 10 logarithm
Base 10 logarithm
of normalized
of normalized
probability
probability
Normalized MLE
Normalized MLE
Assimilation into
NWP models
Table 3.1 Differences between the various BUFR output formats.
When using MSS, SDP internally stores 144 solutions, each with a normalized probability.
Therefore the probability of the solution selected by the ambiguity removal can be much lower
than one may expect: a probability of 1% or lower is perfectly well possible. This applies
especially to cases in which the minimum of the inversion cost function is very broad. A standard
procedure would select only the minimum value (with relatively high probability), whereas MSS
takes all solutions around the minimum into account and divides the probability over them.
32
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
The definitions of the MLE, the normalized MLE and their relation with the probability are
described by Portabella [2002].
3.3
Input Specification
Input of SDP is the SeaWinds Scatterometer Near-Real-Time BUFR Geophysical Data Product,
or shortly the SWS\_Met Data Product. This product is created by NOAA. Though it is in fact
already a level 2 product in itself, it should be stressed here that only the basic level 1 data from
this NOAA product are used as input for the SDP program.
For SeaWinds on QuikSCAT the data have now been available for several years. It contains
WVC-composite σ 0 data based on slices of the scatterometer pulse footprint. Details of this
product can be found in [Leidner et al., 2000].
Unfortunately, the Adeos-II satellite collapsed after 9 months of operation. A similar data product
is not (yet) available for this satellite.
Remarks:
3.4
−
At KNMI, the data are gathered in a daily archive file. These SWS\_met files are stored
in the MOS system.
−
At ECMWF, the MARS system contains SWS\_met data stored in 6-hourly BUFR files.
These files are also suitable as input for the SDP program.
System requirements
Table 3.2 shows the platform and compiler combinations for which SDP has been tested. SDP is
designed to run on any UNIX (LINUX) based computer platform with a Fortran compiler and a C
compiler. The equivalent of a modern personal computer will suffice to provide a timely NRT
wind product. SDP requires about 80 MB disk space when installed and compiled.
Platform
Suse LINUX work station
SunOS UNIX
Windows XP PC with Cygwin
Fortran compiler
Portland pgf90
GNU g95
Sun Fortran
GNU g95
C compiler
GNU gcc
GNU gcc
GNU gcc
Table 3.2 Platform and compiler combinations for which SDP has been tested.
SDP may also run in other environments, provided that the environment variables discussed in
section 2.2 are set to the proper values, and that the BUFR library is properly installed. For
Windows one needs Cygwin as UNIX emulator. MinGW proved to be too limited to support
SDP.
33
NWP SAF
SDP User Manual and
Reference Guide
3.5
Details of functionality
3.5.1
BUFR IO and coding
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Data sets of Near Real Time meteorological observations are generally coded in the Binary
Universal Form for Representation, or shortly BUFR. BUFR is a machine independent data
representation system (but it contains binary data, so care must be taken in reading and writing
these data under different operating systems). A BUFR message (record) contains observational
data of any sort in a self-descriptive manner. The description includes the parameter identification
and its unit, decimal, and scaling specifications. The actual data are in binary code. The meta data
are stored in BUFR tables. These tables are therefore essential to read (write) and decode
(encode) the data.
BUFR tables are issued by the various meteorological centers. The largest part of the data
descriptors specified in the BUFR tables follows the official BUFR descriptor standards
maintained by the World Meteorological Organization (WMO, e.g., www.wmo.int). However, for
their different observational products meteorological centers do locally introduce additional
descriptors in their BUFR tables.
Appendix A contains a listing of the data descriptors of the BUFR data input and the BUFR data
output of the SDP program in the SWS\_met BUFR product format (NOAA format). For more
details on BUFR and the SWS\_met BUFR product, the reader is referred to [Dragosavac, 1994;
Leidner et al., 2000].
ECMWF maintains a library of routines reading (writing) and decoding (encoding) the binary
BUFR messages. This library forms the basis of the genscat BUFR module and hence the SDP
program BUFR interface, see Chapter 7.
3.5.2
Output resolution
An important feature of the SDP program is that it may produce a level 2 wind product on
different resolutions. Of course, there is a trade off between the output resolution and the
statistical error of the mean wind vectors. Therefore KNMI has developed a SeaWinds product
with 100 km resolution for assimilation in most NWP models. However, a different resolution
may be optimal for a specific NWP application. The statistical error of the wind vectors for the
higher resolutions is currently a topic of further testing.
3.5.3
Quality Control
The quality of every WVC is controlled. An import aspect is the contamination of the Ku-band
scatterometer signals by rain. The rain flag used in the SDP program is based on the value of the
normalized maximum likelihood estimator (MLE) [Portabella and Stoffelen, 2001, 2002].
Compared to the JPL flag, the KNMI flag accepts more non-rain winds between 10 and 20 m/s
that occur in meteorologically dynamic areas. It also yields less tropical rain contaminated winds
[Portabella and Stoffelen, 2001, 2002]. See appendices C1 and C2 for more information on how
34
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
to find these flags in the BUFR output.
3.5.4
Inversion
In the inversion step of wind retrieval, the radar backscatter observations in terms of the
Normalized Radar Cross Sections ( σ 0 ’s) are converted into a set of ambiguous wind vector
solutions. In fact, a Geophysical Model Function (GMF) is used to map a wind vector (specified
in term of wind speed and wind direction) to a σ 0 value. The GMF depends not only wind speed
and wind direction but also on the measurement geometry (relative azimuth and incidence angle)
and beam parameters (frequency and polarization).
For SeaWinds, a maximum likelihood estimator (MLE) is used to preselect a set of wind vector
solutions and associated probabilities that yields the best match with the observed σ 0 's. This
preselection depends on the number of independent σ 0 values available within the wind vector
cell.
The SDP program also includes the Multiple Solution Scheme (MSS). In MSS mode, a much
larger preselection of wind vector solutions is produced. The wind vector solutions are ranked
according to their probability based on the MLE and constitute the full wind vector probability
density function. Subsequently, the 2DVar Ambiguity Removal method, see e.g., section 3.5.5 is
applied with a much larger set of wind vector solutions. The output may be written in the so
called Generic Wind Section BUFR format, which allows up to 144 wind vector solutions but is
still to be approved by the WMO. Details on the KNMI SeaWinds inversion approach can be
found in [Portabella, 2002]. MSS compares better to an independent NWP model reference than
conventional four-solution schemes at 100 km resolution [Portabella and Stoffelen, 2004].
Technical information on the KNMI inversion approach can be found in Chapter 5. Details of the
original JPL SeaWinds wind retrieval can be found in [Draper and Long, 2002].
3.5.5
Ambiguity Removal
The Ambiguity Removal (AR) step of the wind retrieval is the selection of the most probable
surface wind vector among the available wind vector solutions, the so-called ambiguities. Various
methods have been developed for AR. More information on Ambiguity Removal is given in
Chapter 6. The default method implemented in the SDP program is the KNMI 2DVar AR
scheme. A description of its implementation can be found in section 6.4. The Multiple Solution
Scheme (MSS) offers the possibility to postpone AR to the NWP step in order to treat all
information from models and measurements in the same manner. Further details on the
algorithms and their validation can be found in the reports [de Vries and Stoffelen, 2000; de Vries
et al., 2004]. These documents may be downloaded from the EUMETSAT website,
www.eumetsat.int, or the KNMI website, www.knmi.nl/scatterometer.
The performance of the SDP 2DVar with meteorological balance constraints was tested and
optimized for ERS data. It was found to be superior to other schemes.
35
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Remarks:
−
The Fortran implementation of the 2DVar system strongly differs from that of QDP.
−
The recent genscat development on ambiguity removal allows the use of the PRESCAT
ambiguity removal scheme.
3.5.6
Monitoring
For the automatic ingestion of observations into their NWP systems meteorological centers
require quality checks on the NRT products. For the Seawinds BUFR products a monitor flag is
developed. This flag indicates that several measures on the level of corruption of the output
BUFR files are over a specified threshold. Onset of the flag indicates that the input should be
rejected for ingestion by the NWP system. Details on the monitor developed can be found in the
NWP SAF document [de Vries et al., 2004], downloadable from the EUMETSAT or KNMI
website, www.eumetsat.int or www.knmi.nl/scatterometer, respectively.
3.6
Details of performance
SDP is delivered with a BUFR input file named QS_D02001_S0006_E0120_B1320303,
which contains half an orbit of data. Table 3.3 gives the approximate times needed for processing
this file under various options on a personal workstation with a 2.66 GHz Pentium 4 processor
under LINUX using the GNU g95 Fortran compiler.
Script
sdp_025
sdp_025
sdp_qdp
Resolution
(m)
25
25
100
MSS?
No
Yes
No
Inversion
(seconds)
59
63
4
AR
(seconds)
6
44
2
BUFR IO
(seconds)
21
20
7
Total
(seconds)
88
129
13
Table 3.3 Approximate times needed by SDP to process BUFR file
QS_D02001_S0006_E0120_B1320303 under various input options.
As can be seen from table 3.3, choosing the MSS scheme results in slightly larger times needed
for inversion, and much more time needed for AR. The computation time, of course, increases
with decreasing resolution.
The processing times depend only little on the number of WVC’s in the orbit being processed.
The choice of platform, compiler, and optimization options will generate more variation.
36
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Chapter 4
Program Design
In this chapter, the design of the SDP program is described in detail. Readers to whom only a
summary will suffice are referred to the Top Level Design (TLD) in section 4.1. Readers who
really want to know the very detail should not only read the complete chapter, but also the
documentation within the code.
4.1
Top Level Design
4.1.1
Main program
The main program, SDP, (file sdp in the SDP/sdp directory) is a UNIX (LINUX) executable
which processes SeaWinds BUFR input files. The main output consists of BUFR files. The output
BUFR messages have the same descriptors as the input messages. The user may provide
arguments and parameters according to UNIX command line standards. The purpose of the
different options is described in the User Manual (chapter 2).
When executed, the SDP program logs information on the standard output. The detail of this
information may be set with the verbosity flag. The baseline of processing is described in Figure
4.1. A more detailed representation of the SDP structure is given in Appendices A and B.
The first step is to process the arguments given at the command line. Next, the SDP program
loops over the input files specified in the arguments. For every input file the BUFR messages are
read and mapped onto the SeaWinds data structure, see e.g., subsection 4.1.3. As part of the
preprocessing a similar SeaWinds data structure is created for the output. Subsequently, the
37
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
output data structure is filled with level 1 ( σ 0 -related) data. The next steps are the inversion and
the ambiguity removal. These steps are performed on the output data. The loop over the input
files ends with the post-processing step (which includes some conversions and the monitoring)
and the mapping of the output data structure onto BUFR messages of the BUFR output file. The
different stages in the processing correspond directly to specific modules of the code. These
modules form the process layer, see section 4.4.
Process arguments
Loop over input files
Read input BUFR messages
Pre-processing
Inversion
Ambiguity Removal
Post-processing
Write output BUFR message
Figure 4.1 Baseline of the Seawinds Data Processor
4.1.2
Layered model structure
SDP is a Fortran90 program consisting of several Fortran90 modules which are linked after their
individual compilation. The SPD program is set up from three layers of software modules, see
Figure 4.2. The purpose of the layer structure is to divide the code with respect to its genericity.
Details on the individual modules can be found in sections 4.2 to 4.4.
The first layer (the process layer) consists of five modules which serve the main steps of the
process. These steps are:
1) BUFR input and output;
2) pre- and post-processing;
3) inversion;
4) ambiguity removal;
5) support.
38
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
SDP
SdpPrePost
SdpInversion
SdpIO
SdpSupport
SdpAmbrem
PROCESS
LAYER
SwsSupport
SwsBufr
SwsData
SWS
inversion
BufrMod
LunManager
SortMod
DateTimeMod
LAYER
ambrem (→)
GENSCAT
LAYER
Figure 4.2 Module layer and top level module dependencies. The dependencies for module ambrem are
continued in figure 6.1
Module name
SdpIO
SdpPrePost
SdpInversion
SdpAmbrem
SdpSupport
Τasks
BUFR file handling
Command line processing
Spatial averaging
Quality control
Rain flagging
Scale conversion
Monitoring
Inversion
Ambiguity Removal
Support for processing
Comments
Averaging to 50 m or 100 m resolution
Usability of input data
Rain flag based on normalized MLE
Linear versus logarithmic
Monitoring
Interface to genscat/inversion
Interface to genscat/ambrem
Definition of data structures
Interface to genscat/support via SwsSupport
Table 4.1 SDP process modules.
39
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Each module contains code for performing one or more of the specific tasks. These tasks are
shortly described in Table 4.1. A more elaborate description is given in section 4.4. The last
module listed, SdpSupport is a general support module. This module is used by the other four
modules of the process layer for the inclusion of definitions of the data structures and the support
routines. (Note that the names of the process modules start with the prefix Sdp while the source
code is stored in the subdirectory with the name sdp).
The second layer (the SeaWinds layer) consists of SeaWinds Data Support modules. These
modules, see table 4.2, contain the SeaWinds data structure definitions and the interface between
these data structures and the (input/output) BUFR data format. The key module is SwsData. This
module contains all the important data types that are introduced for the processing. An overview
of these data structures is given in subsection 4.1.3. Details on the actual types and routines are
given in section 4.3. The names of these modules start with the prefix Sws. The Sws-modules are
stored in the subdirectory SDP/sws.
Finally, the third module layer is the genscat layer. The genscat module classes (i.e., groups of
modules) used in the SDP program are listed in table 4.3. genscat is a set of generic modules
which can be used to assemble processors as well as pre-, and post-processing tools for different
scatterometer instruments available for the user community. A short description of the main
(interface) modules is given in section 4.2. The most important classes of modules are related to
the inversion processing step (chapter 5), the Ambiguity Removal step (chapter 6), and the BUFR
file handling (chapter 7). The genscat modules are located in subdirectory genscat.
Module name
SwsBufr
SwsData
SwsSupport
Tasks
BUFR handling
Data definitions,
Data quality control
Processing support
Description
Mapping of BUFR messages on SeaWinds data structure
Composed type declarations
Checking and flagging
Interface to genscat/support routines
Table 4.2 SeaWinds data support modules.
Module class
Ambrem
Inversion
Support
Tasks
Ambiguity Removal
Wind retrieval
BUFR support
FFT, minimization
Error handling
File handling
Conversion
Sorting
Argument handling
Date and time
Description
2DVar and other schemes, see chapter 6
Inversion in one cell, see chapter 5
BufrMod, based on ECMWF library
Support for 2DVAR
Print error messages
Finding, opening and closing free file units
Conversion of meteorological quantities
Sorting of ambiguities to their probability
Compiler independent reading of command line arguments
General purpose
Table 4.3 genscat module classes.
40
SDP User Manual and
Reference Guide
NWP SAF
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
In addition, genscat contains a large support class to convert and transform meteorological,
geographical, and time data, to handle file access and error messages, sorting, and to perform
more complex numerical calculations on minimization and Fourier transformation. Many routines
are co-developed for ERS and ASCAT data processing.
The layer set-up facilitates a fast and comprehensive development of pre- and post-processing
functionality without interfering with the code of the processor itself. In fact, the SeaWinds
support layer does not contain any real processing functionality, but this layer provides the
required functionality to develop applications which only need the input or output of the process.
4.1.3
Data Structure
Along track, the SeaWinds swath is divided into rows. Within a row (across track) the SeaWinds
orbit is divided into cells, also called Wind Vector Cells (WVC) or nodes. This division in rows
and cells forms the basis of the main data structures within the SDP package. In fact, both the
input and the output structure are one dimensional arrays of the row data structure, SwsRowType.
These arrays represent just a part of the swath. Reading and writing (decoding and encoding)
SeaWinds BUFR files corresponds to the mapping of a BUFR message to an instance of the
SwsRowType and vice versa.
The main constituent of the SwsRowType is the cell data structure, CellType, see figure 4.3. Since
most of the processing is done on a cell-by-cell basis the CellType is the pivot data structure of
the processor. The level 1 data of a cell are stored in a data structure called BeamType.
SwsRowType
CellType
BeamType
Sigma0Type
KpType
Ambiguity Type
Figure 4.3 Schematic representation of the nested data definitions in the SwsRowType data structure.
41
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Every cell contains 4 instances of the BeamType, corresponding to the inner fore and aft beams
and the outer fore and aft beams. The BeamType is further subdivided in the Sigma0Type
containing σ 0 -related data and the KpType. The latter contains the σ 0 variance coefficients.
A cell may also contain an array of instances of the AmbiguityType data structure. This array
stores the results of a successful wind retrieval step, the wind ambiguities (level 2 data). Details
of all the data structures and methods working on them are described in chapter 6.
Remarks:
−
4.1.4
In QDP the input and output array structure are called obs and obs2, respectively. In SDP,
this naming convention is reused by giving the input instances of CellType the name cll
and the output instances of CellType the name cll2.
Quality flagging and error handling
Important aspects of the data processing are to check the validity of the data and to check the data
quality. In the SDP program two WVC flags are set for every WVC, see table 4.4, and three flags
are set for each of the four beams, see table 4.5. Therefore, 14 flags in total report on the quality
and other aspects of the data in each WVC. Furthermore, the flags themselves do not address a
single aspect of the data, but the flags are composed of several bits each addressing a specific
aspect of the data. A bit is set to 0 (1) in case the data is valid (not valid) with respect to the
corresponding aspect. In order to enhance the readability of the SDP code, each flag is translated
to a data type consisting of only booleans (false = valid, true = invalid). On input and output these
data types are converted to integer values by set and get routines.
Flag
Quality Flag
Process Flag
Tasks
Quality checking
Range checking
Description
In BUFR output
Not in BUFR output
Table 4.4 Flags for every WVC (attributes of CellType).
Flag
Surf Flag
Mode Flag
Qual Flag
Tasks
Check surface condition
Check mode
Check quality
Description
In BUFR output
In BUFR output
In BUFR output
Table 4.5 Flags for every beam (attributes of Sigma0Type).
4.1.5
Verbosity
Every routine in a module may produce some data and statements for the log of the processor. To
control the size the log, several modules contain parameters for the level of verbosity. The
verbosity of the SDP program may be controlled by the verbosity command line option verbosity.
42
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
In general, there are three levels of verbosity specified:
≤ -1:
0:
≥ 1:
be quiet as possible;
only report top level processing information;
report additional information.
Of course, errors are logged in any case. Table 4.6 gives a (incomplete) list of verbosity
parameters. They are not all set by the command line option as some of them serve testing and
debugging purposes.
Module
Ambrem2Dvar
AmbremBGclosest
BatchMod
Ambrem
SwsBufr
Verbosity parameter
TDVverbosity
BGverbosity
BatchVerbosity
AmbremVerbosity
BufrVerbosity
Table 4.6 Verbosity parameters.
4.2
Module Design for genscat layer
4.2.1
Module inversion
The module inversion contains the genscat inversion code. It is located in subdirectory
genscat/inversion. Details of this module are described in chapter 5. In the SDP program,
the inversion module is only used in the SdpInversion module, see subsection 4.4.4.
4.2.2
Module ambrem
The module ambrem is the main module of the genscat Ambiguity Removal code. It is located in
subdirectory genscat/ambrem. Details of this module are described in chapter 6. In the SDP
program, the ambrem module is only used in the SdpAmbRem module, see subsection 4.4.5.
4.2.3
Module Bufrmod
Genscat contains several support modules. In particular, the BufrMod module is the Fortran90
wrapper around the BUFR library used for BUFR input and output. It is located in subdirectory
SDP/genscat/support/bufr. Details of this module are described in chapter 7. In the SDP
program, the BufrMod module is only used in the SwsBufr module, see subsection 4.3.2.
4.2.4
Support modules
Subdirectory genscat/support contains more support modules besides Bufrmod. The KNMI
2DVar Ambiguity Removal method requires minimization of a cost function and numerical
Fourier transformation. These routines are located in subdirectories BFGS and multiFFT,
43
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
respectively, and are discussed in more detail in section 6.4.
Subdirectory Compiler_Features contains alternative routines for iargc and getarg .
Routines iargc and getarg are not part of the Fortran standard and therefore not supported by each
Fortran compiler.
Subdirectory convert contains module convert for the conversion of meteorological and
geographical quantities. So far, only routine uv_to_sd is used by module AmbremBGclosest, but
this may change in future updates of SDP.
Subdirectory datetime contains module DateTimeMod for date and time conversions. SDP
only uses routines GetElapsedSystemTime (for calculating the running time of the various
processing steps) and julian2ymd (for conversion of Julian day number to day, month and year).
Module DateTimeMod needs modules ErrorHandler and numerics.
Subdirectory ErrorHandler contains module ErrorHandler for error management. This
module is needed by module DateTimeMod.
Subdirectory file contains module LunManager for finding, opening and closing free logical
units in Fortran. SDP uses only routines get_lun and free_lun (for opening and closing,
respectively, of a logical unit) in the genscat routine calc_sigma0 (see figure B1.4).
Subdirectory num contains module numerics for handling missing values, for instance in the
BUFR library. This module is needed by module DateTimeMod and is used in the test program
test_modules.
Subdirectory sort, finally, contains module SortMod for sorting the wind vector solutions
according to their probability.
4.3
Module Design for SeaWinds layer
The SeaWinds layer consists of the modules SwsData, SwsBufr, and SwsSupport. Table 4.7 lists
the routines within these modules. A star indicates that the routine is not (yet) called in the
processing chain.
4.3.1
Module SwsData
The module SwsData contains all the important data types relevant for the processing.
Elementary data types are introduced for the most basic data structures of the processing. These
are, e.g. WindType, TimeType, and RainType. Using these data types (and of course the standard
types as integer, real etc.), more complex (composed) data types are derived. Examples are
BeamType, AmbiguityType, CellType, and SwsRowType. A complete description of all types is
given below. The attributes of all these types have intentionally self-documenting names.
Example: the KpType has been introduced for the σ 0 variance K p . The common three
coefficients of K p , i.e., α , β , and γ , are stored for every beam in the Sws\_met BUFR
messages. The values of these coefficients are copied into an instance of KpType (part of
44
SDP User Manual and
Reference Guide
NWP SAF
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
BeamType), respectively as the real attributes Alpha, Beta and Gamma (see table 4.12).
In the following the different data types are described in alphabetical order.
SwsBufr
OpenSwsBufrFile
CloseSwsBufrFile
SwsBufrInit
ReadSwsBufrData
WriteSwsBufrData
Values2CellNOAA
Values2CellGen
Cell2ValuesNOAA
Cell2ValuesGen
SwsData
MergeRow
CheckCell (*)
TestCell
InitCell
CopyCell
PrintCell
SetDummyCell (*)
InitBeam
PrintBeam
InitSigma0
TestSigma0
PrintSigma0
InitKp
TestKp
PrintKp
InitAmbi (*)
PrintAmbi
InitAntenna
PrintAntenna
InitRain
PrintRain
SetDummyWind (*)
InitWind (*)
PrintWind
TestWind
InitTime
PrintTime
TestTime
InitProcessFlag
getProcessFlag (*)
setProcessFlag (*)
PrnProcessFlag
getCellQualFlagNOAA
getCellQualFlagGen
setCellQualFlagNOAA
setCellQualFlagGen
PrnCellQualFlag
getSigma0QualFlag
setSigma0QualFlag
PrnSigma0QualFlag
getSigma0ModeFlag
setSigma0ModeFlag
PrnSigma0ModeFlag
getSigma0SurfFlag
setSigma0SurfFlag
PrnSigma0SurfFlag
Table 4.7 Routines in the genscat layer modules. Routines marked with (*) are not needed for SDP. Note
that module SwsSupport contains no routines..
Ambiguity data: The AmbiguityType data type contains information on an individual ambiguity
(wind vector solution). The attributes are listed in table 4.8. The routine InitAmbi() sets all
ambiguity data to missing. The routine PrintAmbi() may be used to print all ambiguity data.
Attribute
Wind
Error
Prob
Type
WindType
WindType
Real
Description
Wind vector solution
Error in wind vector solution
Probability of wind vector solution
Table 4.8 Ambiguity data structure.
Antenna data: The AntennaType data type contains additional information on the scatterometer
beams, see CellType. The attributes are listed in table 4.9. The routine InitAntenna() sets all
antenna data to missing. The routine PrintAntenna() may be used to print all antenna data.
Attribute
Num
Polarization
Tb_Mean
Tb_StdDev
Type
Integer
Real (integer)
Real (integer)
Real (integer)
Description
Beam number
Polarization (H or V)
Mean brightness temperature
Standard deviation of brightness temperature
Table 4.9 Antenna data structure.
45
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Beam data: Every WVC contains up to 4 beams. The information of every beam is stored in the
data type BeamType. The attributes are listed in table 4.10. The routine InitBeam() sets all beam
data to missing. The routine PrintBeam() may be used to print all beam data.
Attribute
Num
Sigma0
Kp
K_Polar
Type
Integer
Sigma0Type
KpType
Real (integer)
Description
Beam number: 1 = inner fore, 2 = outer fore, 3 = inner aft, and 4 = outer aft
σ0 data
Kp data
Kp data
Table 4.10 Beam data structure.
Cell Data: The CellType data type is a key data type in the SDP program, because many
processing steps are done on a cell by cell basis. The attributes are listed in table 4.11.
.
Attribute
RevNr
RowNr
NodeNr
Lat
Lon
Across_Track_Res
Along_Track_Res
Time_to_Edge
TimeDiff
Time
Satellite_ID
Sat_Motion
Instrument_ID
GMF_ID
Software_ID
Sigma0_In_cell
Rain
Antenna(2)
Beam(4)
Num_Ambigs
Selection
Ambi
Model
EC
JPL
TwoDV
Quality_Flag
ProcessFlag
Type
Integer
Integer
Integer
Real (integer)
Real (integer)
Real (integer)
Real (integer)
Real (integer)
Real (integer)
TimeType
Integer
Real (integer)
Integer
Integer
Integer
Integer
RainType
AntennaType
BeamType
Integer
Integer
AmbiguityType
WindType
WindType
WindType
WindType
CellQualFlagType
ProcessFlagType
Description
Revolution (orbit) number
Row number (along track)
Node number (across track)
Latitude of cell
Longitude of cell
Across track resolution
Along track resolution
Time to edge
Time difference
Date and time
Satellite identification
Satellite motion
Instrument identification
GMF identification
Processor identification
Number of beams for cell
Rain data
Brightness temperature
Beam data σ0 Kp
Number of ambiguities
Array of ambiguities
Model wind
ECMWF wind (KNMI)
JPL wind (KNMI)
2DVar analysis wind (KNMI)
Quality flag
Processing flag
Table 4.11 Cell data structure.
46
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
The routine InitCell() sets the cell data to missing values. Also the flags are set to missing. The
routine TestCell() tests the validity of data. This routine sets the cell process flag. The routine
PrintCell() may be used to print the cell data.
NB. The routine CheckCell() may be used to select cells with a specified quality. The selection is
controlled by a check flag which is an instance of the CellProcessFlagType.
Kp data: The error variance of the σ0 signals are specified in terms of Kp values. Kp values are
generally a quadratic approximation in terms of σ0. The coefficients of this approximation are
stored in instances of KpType, see table 4.12. The routine InitKp() sets the Kp coefficients to
missing values. The routine TestKp() tests the validity of coefficients specification (see also the
cell process flag). The routine PrintKp() may be used to print the coefficients.
Attribute
Alpha
Beta
Gamma
Type
Real (integer)
Real (integer)
Real (integer)
Description
Variance coefficient of quadratic term
Variance coefficient of linear term
Variance offset coefficient
Table 4.12 Variance (Kp) data structure.
Normalized Radar Cross Section (σ0) data: The Sigma0Type data type contains the σ0
(Normalized Radar Cross-Section) information of a specific beam. The attributes are listed in
table 4.13. The data types of the flags are discussed further on in this section. The routine
InitSigma0() sets the σ0 data to missing values. Also the flags are set to missing. The routine
TestSigma0() tests the validity of the σ0 data (see also the cell process flag). The routine
PrintSigma0() may be used to print the σ0 data.
Attribute
Lat
Lon
Atten_Value
Azimuth
Incidence
Value
Qual_Flag
Mode_Flag
Surf_Flag
Variance_QC
Type
Real (integer)
Real (integer)
Real (integer)
Real (integer)
Real (integer)
Real (integer)
Sigma0QualFlagType
Sigma0ModeFlagType
Sigma0SurfFlagType
Real (integer)
Description
Latitude
Longitude
Attenuation value
Azimuth angle
Incidence angle
σ0 value
σ0 quality flag
σ0 mode flag
σ0 surface flag
Variational quality control value
Table 4.13 Signal σ0 data structure.
Rain data: For every WVC, information on rain is stored in the data type RainType. The
attributes are listed in table 4.14. The routine InitRain() sets all rain data to missing. The routine
PrintRain() may be used to print all the rain data.
47
SDP User Manual and
Reference Guide
NWP SAF
Attribute
MP
NOF
Rate
Attenuation
Type
Integer
Integer
Real (integer)
Real (integer)
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Description
Rain rate
Attenuation correction
Table 4.14 Rain data structure.
Row data: The data of a complete row of the swath is stored in the data type SwsRowType, see
table 4.15. The routine InitRow() sets all row data to missing. A complete row corresponds to a
single BUFR message in the SDP input and output, see module SwsBufr in subsection 4.3.4. In
some cases two messages are stored for the same row. The routine MergeRow() is used to
combine the data.
Attribute
RevNr
RowNr
NrCells
FirstNode
Cell(76)
Type
Integer
Integer
Integer
Integer
CellType
Description
Revolution number
Along track row number
Actual number of WVC’s
Node number of first non-empty WVC cell
Array of cells
Table 4.15 SeaWinds row data structure.
Time data: The TimeType data type contains a tuple of 6 integers representing both the date and
the time, see table 4.16. The routine InitTime() sets the time tuple to missing values. The routine
TestTime() tests the validity of the date and time specification (see also the cell process flag). The
routine PrintTime() can be used to print the time tuple.
Attribute
Year
Month
Day
Hour
Min
Sec
Type
Integer
Integer
Integer
Integer
Integer
Integer
Description
19XX or 20XX
1 – 12
1 – 31
0 – 23
0 – 59
0 – 59
Table 4.16 Time data structure.
Wind Data: The WindType data type contains the wind speed and wind direction, see table 4.17.
The routine SetDummyWind() fills the wind data type with arbitrary values (remark: should use
randomization). The routine InitWind() sets the wind vector to missing. The routine PrintWind()
may be used to print the wind vector. The routine TestWind() tests the validity of the wind
specification, see also the cell process flag.
48
NWP SAF
SDP User Manual and
Reference Guide
Attribute
Speed
Dir
Type
Real (integer)
Real (integer)
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Description
Wind speed
Wind direction
Table 4.17 Wind data structure.
Some special data types are introduced for the data (quality) flags. These are discussed below.
Cell quality flag: Every WVC contains a flag for its quality. Therefore the CellType contains an
instance of the CellQualFlagType. Table 4.18 gives an overview of its attributes and the bit
number each flag occupies. Note that the bit position differs for the NOAA format and the KNMI
format (generic wind section, generated with the -genws option): that in the KNMI format is 8
larger than in the NOAA format.
The function getCellQualFlag() interprets an integer flag (BUFR input) to an instance of
CellQualFlagType. The function setCellQualFlag() transforms an instance of CellQualFlagType
to an integer flag.
Note that the MLE and AR flags have a different definition than the original NOAA product. The
MLE flag has been modified following the procedure described in section 4.3.3. The AR flag
indicates the quality of the solution found by KNMI’s 2D variational Ambiguity Removal
procedure (chapter 6).
Attribute
Missing
QualSigma0
Azimuth
Reserved3
MonFlag
MonValue
MLE
AR
Land
Ice
Retrieval
Large
Small
RainFall
RainDetect
FourBeam
NOAA
Bit
2Bit
KNMI
Bit
2Bit
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
22
21
20
19
18
17
16
15
14
13
12
11
10
9
--
32768
16384
8192
4096
2048
1024
512
256
128
64
32
16
8
4
2
4194304
2097152
1048576
524288
262144
131072
65536
32768
16384
8192
4096
2048
1024
512
Description
Flag not set (all bits on)
Inferior quality of σ0 data
Invalid azimuth angle
Monitoring flag not calculated
Monitor flag
KNMI + JPL MLE flag
KNMI VarQC flag
Land flag
Ice flag
No retrieval
σ0 too large
σ0 too small
Rain flag not calculated
Rain detected
Sigma0_in_Cell does not equal 4
Table 4.18 Cell quality flag bits (Fortran) in the NOAA and KNMI BUFR output formats.
49
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Cell process flag: Besides a cell quality flag, every WVC contains a process flag. The process
flag checks on aspects that are important for a proper processing, but are not available as a check
in the cell quality flag. The cell process flag is set by the routine TestCell.
Table 4.19 lists the attributes of the CellProcessFlagType. The function getCellProcessFlag()
interprets an integer flag (BUFR input) to an instance of CellProcessFlagType. The function
setCellProcessFlag() transforms an instance of CellProcessFlagType to an integer flag. The
routines PrnCellProcessFlag() and PrnCellQualityFlag() may be used to print the bit values of
the flags.
Attribute
Missing
RevNr
RowNr
NodeNr
Lat
Lon
MLEQC
Along_Track_Res
Across_Track_Res
ModelWind
Time2Edge
Year
Month
Day
Hour
Minute
Second
Beam(4)
Beam(3)
Beam(2)
Beam(1)
Sigma0_In_Cell
Ambiguity
Selection
Rain
Tb
Bit
30
29
28
27
26
25
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
2Bit
2147483647
1073741824
536870912
268435456
134217728
67108864
33554432
16777216
8388608
4194304
2097152
1048576
524288
262144
131072
65536
32768
16384
8192
4096
2048
1024
512
256
128
64
32
Description
Flag not set (all bits on)
Invalid revolution number
Invalid row number
Invalid node number
Invalid latitude
Invalid longitude
MLE quality control set
Invalid along track resolution
Invalid across track resolution
Invalid background wind
Invalid time to edge
Invalid year specification
Invalid moth specification
Invalid day specification
Invalid hour specification
Invalid minute specification
Invalid second specification
Invalid data of outer aft beam
Invalid data of inner aft beam
Invalid data of outer fore beam
Invalid data of inner fore beam
Invalid number of cells
Invalid ambiguities
Invalid selection
Invalid rain data
Invalid brightness temperature
Table 4.19 Cell process flag bits (Fortran).
Flags for σ0 data: Every beam contains an instance of Sigma0Type. This instance has three
attributes to flag the information on σ0. These attributes are of type Sigma0QualFlagType,
Sigma0ModeFlagType, and Sigma0SurfFlagType. Table 4.20 gives an overview of the (bit)
attributes of these flags.
The
functions
getSigma0QualFlag(),
setSigma0QualFlag(),
getSigma0ModeFlag(),
setSigma0ModeFlag(), getSigma0SurfFlag(), and setSigma0SurfFlag() are introduced for the
(backward) conversions to the corresponding integer flags. The routines PrnSigma0QualFlag(),
50
SDP User Manual and
Reference Guide
NWP SAF
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
PrnSigma0ModeFlag(), and PrnSigma0SurfFlag() may be used to print the bit values of the flags.
Attribute
Missing
Useability
NoiseRatio
Negative
Range
Pulse
Convergence
FreqShift
Temperature
Attitude
Ephemeresis
Bit
Attribute
Horizontal
Vertical
Right
Left
HoriVert
RightLeft
Bit
16
15
14
13
12
11
Attribute
Land
Ice
IceMap
AttenuationMap
Bit
15
14
5
4
15
14
13
12
11
10
9
8
7
6
Quality flag
2Bit
2147483647
32768
16384
8192
4096
2048
1024
512
256
128
64
Mode flag
2Bit
65536
32768
16384
8192
4096
2048
Surface flag
2Bit
32768
16384
32
16
Description
Flag not set (all bits on)
σ0 value
Azimuth diversity
Negativeσ0 value
2DVar
Pulse
Convergence
Frequency shift
Temperature
Attitude
Ephemereris
Description
Description
Table 4.20 σ0 flag bits for quality, mode, and surface (Fortran).
4.3.2
Module SwsBufr
The module SwsBufr maps the SeaWinds data structure on BUFR messages and vice versa. A list
of the BUFR data descriptors can be found in appendix A. Satellite and GMF identifiers are listed
in tables 4.21 and 4.22. The module uses the genscat module BufrMod, see subsection 4.2.3, for
the interface with the BUFR routine library. The SeaWinds data structure is defined in module
SwsData, see subsection 4.3.1.
Satellite
ADEOS-1
QuikSCAT
ADEOS-2
Parameter
Adeos1Id
QscatId
Adeos2Id
Value
280
281
282
Table 4.21 BUFR SeaWinds satellite identifiers.
51
NWP SAF
SDP User Manual and
Reference Guide
Instrument
Reserved
SASS
SASS2
NSCAT0
NSCAT1
NSCAT2
NSCAT2P
QSCAT1
Parameter
GmfReserved
GmfSass
GmfSass2
GmfNscat0
GmfNscat1
GmfNscat2
GmfNscat2P
GmfQscat1
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Value
0
1
2
3
4
5
6
7
Table 4.22 BUFR GMF identifiers.
Routine
SwsBufrInit
OpenSwsBufrInit
CloseSwsBufrInit
ReadSwsBufrInit
WriteSwsBufrInit
Values2CellNOAA
Values2CellGen
Cell2ValuesNOAA
Cell2ValuesGen
Call
WriteSwsBufrData
WriteSwsBufrData
ReadSwsBufrData
ReadSwsBufrData
Description
Initialize module SwsBufr
Open BUFR file
Close BUFR file
BUFR message to SwsRowType
SwsRowType to BUFR message
BUFR values array to CellType in NOAA format
BUFR values array to CellType in generic format
CellType to BUFR values array in NOAA format
CellType to BUFR values array in generic format
Table 4.23 Routines in module SwsBufr
Table 4.23 provides an overview of the different routines and their calls in this module. Ex
general, the SDP module SdpIO uses the SwsBufr module to set up its BUFR interface. The
genscat support routines GetCurrentDate() and GetCurrentTime() are used to tag the BUFR
messages with the date and time of creation.
Note that the routines Values2Cell and Cell2Values, which convert between BUFR and SDP
internal representation, have two variants: one for the official NOAA BUFR format that supports
up to four wind solutions, and one for the experimental generic format that supports up to 144
wind solutions. The latter format has not yet been approved by the WMO.
Remarks:
−
4.3.3
−
BUFR message subset indices are fixed for Sws\_Met BUFR. Therefore they are set once
during the initialization of SwsBufr (for example in ERS processing). These indices must
be computed from the BUFR data descriptors.
Module SwsSupport
Module SwsSupport is the interface between the SWS layer and the general purpose
routines in genscat/support. This module contains no routines or declarations, but
only some use-statements referring to genscat routines.
52
NWP SAF
4.4
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Module Design for process layer
The process (SDP) layer consists of the modules SdpAmbrem, SdpInversion, SdpIO, SdpPrePost,
a:nd SdpSupport. Module SdpSupport contains only declarations and initializations, no
subroutines. Table 4.24 lists the routines in the other modules. Routines indicated by a star are not
called in the SDP processing chain.
SdpAmbRem
SdpInversion
SdpIO
SdpPrePost
RemoveAmbiguity
GetBatch
SelectWind
InitProbGross
DummyAmbRem (*)
InitInversion
InitMeanMle
CalcSortProb
InversionInCell
InvertWVCs
DummyInversion (*)
ReadBufrInput
WriteBufrOutput
ProcessSwsFileName
GetNwpFileNames
GetOutputFileNames
ProcessArguments
usage
ProcessInit
Preprocess
CopyInputOutput
PrepareInput
SetInputMleQC
PrepareOutput
PostProcess
Monitoring
MonitoringCalculateData
MonitoringWriteStats
MonitoringSetMonitorBits
OutputConversion
DummyPreProcess (*)
Table 4.24 Routines in the process layer modules.
4.4.1
Module SdpSupport
Module SdpSupport contains many support routines for the processing steps of the SDP program.
The module inherits a lot of functionality (data structures and routines) from the Sws-modules,
see section 4.3. In addition, the module contains the global definitions of the SDP program. Table
4.25 provides an overview.
Name
AlongRes
AcrossRes
RowStride
NodeStride
NrInputRows
NrOutputRows
NrInputNodes
NrOutputNodes
VerbosityLevel
ResolutionIndex
Lnwp
Lqdp
Lmss
Lqc
Linvert
Lambrem
Lmonitor
InpRow()
Outrow()
Type
Real
Real
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Logical
Logical
Logical
Logical
Logical
Logical
Logical
SwsRowType
SwsRowType
Description
Output along track resolution
Output across track resolution
Along track Stride
Across track stride
Actual number of input rows
Actual number of output rows
Actual number of input WVC’s
Actual number of output WVC’s
Verbosity level
Index of resolution (0 – 15)
Switch NWP
Switch QDP mode
Switch MSS
Switch quality control
Switch inversion
Switch ambiguity removal
Switch monitoring
Input orbit rows
Output orbit rows
Remark
Nrows in QDP
Nrows2 in QDP
Default 0
Default 0
Default .false.
Default .false.
Default .false.
Default .true.
Default .true.
Default .true.
Default .false.
Obs in QDP
Obs2 in QDP
Table 4.25 Globals for the processing steps in the SDP program defined in module SdpSupport.
53
NWP SAF
4.4.2
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Module SdpIO
Module SdpIO has two tasks. The first task is to process the command line options and
parameters; the second task is to read (write) BUFR messages from (to) the input (output) BUFR
files. The data have to be converted from the BUFR data structures to the SeaWinds data
structures and vice versa. Table 4.26 provides an overview of the different routines and their calls
in this module.
Routine
ReadBufrInput
WriteBufrOutput
ProcessSwsFileName
GetNwpFileNames
GetOutputFileNames
ProcessArguments
usage
Call
SDP
SDP
ProcessArguments
ProcessArguments
SDP
SDP
ProcessArguments
Description
Read BUFR message from input file
Write BUFR message on output file
Process SDP command line options
Report on the use of SDP
Table 4.26 Routines of module SdpIO.
4.4.3
Module SdpPrePost
Module SdpPrePost contains the routines to do all the pre- and postprocessing. Preprocessing
consists of the procedures between the reading of the BUFR input and the wind retrieval for the
output product. This includes assessments of the quality of the input data, rain flagging, land and
ice flagging, and interpolation to the specified resolution.
Routine
ProcessInit
PreProcess
PrepareInput
SetInputMleQc
PrepareOutput
CopyInputOutput
PostProcess
OutputConversion
Monitoring
MonitoringCalculateData
MonitoringWriteStats
MonitoringSetMonitorBits
ProcessCleanUp
Call
SDP
SDP
PreProcess
PreProcess
PreProcess
PreProcess
SDP
PostProcess
PostProcess
Monitoring
Monitoring
Monitoring
SDP
Description
Initialization of the processing
Main routine of the preprocessing
Preparation of the input cells for averaging
Set normalized MLE quality control tag to input cells
Preparation of output cells (supercells) by averaging
Copy σ0 and Kp data from input to output
Main routine of the postprocessing
Convert output from internal data types to BUFR format
Monitoring
Memory management
Table 4.27 Routines of module SdpPrePost.
Table 4.27 lists the tasks of the individual routines. SDP first calls routine ProcessInit() to be sure
that essential dependencies are set and/or initialized. Next PrepareInput() is called to sort the row
with respect to the revolution, row and node numbers. It also checks on the appearance of double
54
SDP User Manual and
Reference Guide
NWP SAF
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
rows, that is, rows with the same revolution and row number. If PrepareInput() finds a double
row it merges it into one row. In that case the number of input rows will be reduced. Once the
input rows are initialized, SetInputMleQC() will set the quality flag using the normalized MLE,
Rn, defined as [Portabella, 2002; Portabella and Stoffelen, 2001]
Rn =
MLE
MLE
,
(4.1)
with MLE the maximum likelihood estimator and
actual measurements. The MLE is defined as
1
MLE =
N
N
∑
i =1
(σ
− σ 0simul,i
K p (σ 0simul,i )
meas,i
0
)
MLE
its average value, obtained from
2
.
(4.2)
In (4.2), σ 0meas,i stands for the measured value of the radar cross section in a WVC, and σ 0simul,i for
the simulated value which depends on wind speed and direction. The denominator K p (σ 0simul,i )
quantifies the noise in the simulation, i.e., the estimated uncertainty in the GMF. The summation
is over all beams of the scatterometer. For SeaWinds, N =4 in the sweet and central swath.
The MLE can be regarded upon as the distance between an actual scatterometer measurement and
the GMF in N-dimensional measurement space. The MLE is related to the probability P that the
GMF at a certain wind speed and direction represents the measurement by
P ∝ e − MLE
.
(4.3)
Therefore, wind vectors with low MLE have a high probability of being the correct solution. On
the other hand, wind vectors with high MLE are not likely represented by any point on the GMF,
probably because the measurements are contaminated by ice, rain, and/or confused sea state,
phenomena not included in he GMF. The ratio Rn further refines this notion by taking the
uncertainty of the GMF into account. Portabella [2002] and Portabella and Stoffelen [2001]
derive the following threshold values for Rn as a function of wind speed w
Rnthres
⎧4 − 0.05 ( w − 5) 2
⎪
=⎨
⎪
2
⎩
,
w < 15 m/s
.
,
(4.4)
w > 15 m/s
When Rn < Rnthres , the solution is considered close enough to the measurement and is accepted.
If, on the other hand, Rn > Rnthres , the solution lies too far away from the measurement, probably
because the measurement is not well described by the GMF. The solution is therefore rejected.
55
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
In addition, for the nadir swath (that is for node number between 28 and 49) the rain flag of the
NOAA product is adopted. That is, if NOAA sets the rain flag for this region also the MLE rain
flag is set. The main task of the PrepareOutput() is to average the input data and to produce an
output orbit with a lower resolution (less rows, less nodes). The wind vector cells of the output
orbit are sometimes called supercells. The averaging concerns all input data needed to define the
temporal and spatial location of the output cell and the beam data (σ0, Kp) of the output cell. In
addition, PrepareOutput() adjusts the quality flags of the output cells and the output σ0 data.
Postprocessing consists of the procedure between the ambiguity removal step and the BUFR
encoding of the output. Currently, postprocessing is confined to some simple conversions. It also
includes the monitoring.
4.4.4
Module SdpInversion
Module SdpInversion serves the inversion step in the wind retrieval. The inversion step is done
cell by cell. The actual inversion algorithm is implemented in the genscat module Inversion, see
subsection 4.2.1. Table 4.28 provides an overview of the different routines and their calls in this
module.
Routine
InitInversion
InitMeanMle
CalcSortProb
InversionInCell
InvertWVCs
Call
InvertWVCs
InitInversion
InversionInCell
InvertWVCs
SDP
Description
Initialization
Set the mean MLE
Calculate the probabilities and sort the ambiguities on probability
Call to the genscat inversion module
Loop over all output cells
Table 4.28 Routines of module SpdInversion.
4.4.5
Module SdpAmbrem
Module SdpAmbrem controls the ambiguity removal step of the SDP program. The actual
ambiguity removal schemes are implemented in the genscat module ambrem, see subsection
4.2.2. The default method is the KNMI 2DVar scheme. Table 4.29 lists the tasks of the individual
routines.
Routine
RemoveAmbiguity
GetBatch
SelectWind
InitProbGross
Call
SDP
RemoveAmbiguity
RemoveAmbiguity
RemoveAmbiguity
Description
Main routine of ambiguity removal
Obtain a batch of observations
Final selection
Set the gross probabilities
Table 4.29 Routines of module SpdAmbrem.
56
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
The ambiguity removal scheme works on a so-called batch. The batch is defined in the
GetBatch() routine. For the SDP program a batch is just a set of rows. The size of the batch is
determined by the resolution of the structure functions and the number of FFT. The genscat
routine DoAmbrem() performs the actual ambiguity removal scheme.
Finally SelectWind passes the selection to the output WVC's.
4.5
Flag use
The NOAA BUFR input files that serve as input for SDP are in themselves already level 2
products and contain flags for quality control. Some of these are used in SDP, and some are
redefined. Table 4.30 gives an overview.
Inversion is performed for all cells for which both the Cell Quality FourBeam and the Cell
Quality QualSigma0 flags are not set (see table 4.18).
Ambiguity removal is performed for all cells containing ambiguities and model winds, and that
have the Cell Quality MLE flag not set (see table 4.18).
More information on the structure of the BUFR output files can be found in Appendix C1 and
Appendix C2.
Flag
Where used
Cell Quality Flag (see table 4.18)
RainFail
Input MLE quality check
RainDetect
Input MLE quality check
Land
Input MLE quality check
Land
Preparation BUFR output
Ice
Input MLE quality check
Ice
Preparation BUFR output
Retrieval
Input MLE quality check
MLE
Quality control
Sigma0 Surface Flag (see table 4.20)
Land
Preparation BUFR output
Land
Preparation BUFR output
Ice
Preparation BUFR output
Ice
Preparation BUFR output
Sigma0 Quality Flag (see table 4.20)
Useability
Preparation BUFR output
Negative
Preparation BUFR output
Description
Must be false in order to use the Rain detect flag
If set for WVC 29-48, the cell is rejected
If set, the JPL MLE quality check is not performed
If set, the input beam information is not used
If set, the JPL MLE quality check is not performed
If set, the input beam information is not used
If set, the JPL MLE quality check is not performed
Redefined
If set, set Cell Quality land flag
If set, the input beam information is not used
If set, set Cell Quality ice flag
If set, the input beam information is not used
If set, the input beam information is not used
If set, set sign of σ 0
Table 4.30 Flag handling in SDP.
57
NWP SAF
SDP User Manual and
Reference Guide
58
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
SDP User Manual and
Reference Guide
NWP SAF
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Chapter 5
Inversion module class
5.1
Background
In the inversion step of the wind retrieval, the radar backscatter observations in terms of the
normalized radar cross-sections (σ0's) are converted into a set of ambiguous wind vector
solutions. In fact, a Geophysical Model Function (GMF) is used to map a wind vector (specified
in term of wind speed and wind direction) to a σ0 value. The GMF further depends not only wind
speed and wind direction, but also on the measurement geometry (relative azimuth and incidence
angle), and beam parameters (frequency, polarisation). For SeaWinds, a maximum likelihood
estimator (MLE) is used to select a set wind vector solutions that optimally match the observed
σ0's. The wind vector solutions correspond to local minima of the MLE function
(
1 N σ 0obs (i ) − σ 0GMF (i )
MLE = ∑
N i =1
Kp
)
2
,
(5.1)
With N the number of independent σ0 measurements available within the wind vector cell, and Kp
the covariance of the measurement error. This selection depends on the number of independent σ0
values available within the wind vector cell.
Details on the SeaWinds inversion problem can be found in [Portabella, 2002]. Details on the
original JPL inversion approach can be found in [Draper et al., 2002]. The SDP program includes
the Multiple Solution Scheme (MSS), see [Portabella and Stoffelen, 2001].
59
NWP SAF
5.2
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Routines
The inversion module class contains only one module named inversion. It is located in
subdirectory genscat/inversion. Table 5.1 lists all routines in this module. Appendix B.1
shows the calling tree for the inversion routines.
Routine
invert_one_wvc
fill_wind_quality_code
remove_one_solution
save_inv_input
read_inv_input
save_inv_output
do_parabolic_winddir_search
calc_normalisation
calc_sign_MLE
print_message
init_inv_input
init_inv_output
init_inv_settings_to_default
write_inv_settings_to_file
get_inv_settings
set_inv_settings
check_input_data
find_minimum_cone_dist
get_parabolic_minimum
calc_cone_distance
calc_dist_to_cone_center
convert_sigma_to_zspace
get_ers_node_formfactor
calc_var_s0_ers
get_wind_speed_first_guess
get_ers_noise_estimate
calc_var_s0
get_wind_speed_first_guess
Call
SDP
invert_one_wvc
fill_wind_quality_code
not used
not used
not used
invert_one_wvc
invert_one_wvc
invert_one_wvc
see B.1
SDP
invert_one_wvc
SDP
not used
SDP
SDP
invert_one_wvc
invert_one_wvc
do_parabolic_winddir_search
find_minimum_cone_dist
fill_wind_quality_code
invert_one_wvc
calc_var_s0
invert_one_wvc
find_minimum_cone_dist
calc_var_s0
calc_normalisation
find_minimum_cone_dist
Routine
set_wind_speed_first_guess
get_dynamic_range
get_GMF_version_used
calc_sigma0
INTERPOLATE
interpolate1d
interpolated2d
interpolate2dv
interpolate3d
read_LUT
create_LUT_C_VV
test_for_identical_LUTs
my_mod360
my_mod
my_min
my_max
my_average
get_indices_lowest_local_minimum
my_index_max
my_exit
print_wind_quality_code
print_input_data_of_inversion
print_output_data_of_inversion
print_inout_data_of_inversion
calc_sigma0_cmod4
f1
Get_Br_from_Look_Up_Table
calc_sigma0_cmod5
Call
see B.1
not used
not used
not used
generic
calc_sigma0
calc_sigma0
calc_sigma0
calc_sigma0
calc_sigma0
calc_sigma0
calc_sigma0
not used
not used
see B.1
see B.1
see B.1
invert_one_wvc
see B.1
see B.1
see B.1
check_input_data
see B.1
not used
create_LUT_C_VV
calc_sigma0_cmod4
calc_sigma0_cmod4
create_LUT_C_VV
Table 5.1 Routines in module inversion.
To establish the MLE function (1), the radar cross section according to the GMF, σ oGMF , must be
calculated. This is done in routine calc_sigma0. The GMF at Ku band for HH and VV
polarization needed for SeaWinds, is not known in analytical form. It is only available in the form
of Look Up Tables (in directory SDP/lut). The value for σ oGMF is obtained from interpolation
of these tables. The interpolation is done via symbolic routine INTERPOLATE which is set to
interpolate1d, interpolate2d, interpolate2dv, or interpolate3d, depending on the type of
interpolation needed.
60
SDP User Manual and
Reference Guide
NWP SAF
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
For C-band at VV polarization the GMF is given in analytical form (routines calc_sigma0_cmod4
and calc_sigma0_cmod5, respectively). In order to treat all scatterometer types in the same way,
the radar cross section at C-band is also calculated from interpolation of Look Up Tables (LUTs).
If a C-band LUT is not present it will be created by routine create_LUT_C_VV. This routine calls
one of the routines calc_sigma0_cmod4 or calc_sigma0_cmod5 that contain the analytical
expressions of the CMOD4 or CMOD5 algorithm. Routines get_lun and free_lun from module
LunManager in subdirectory genscat/support/file are needed when reading and creating
the LUTs.
5.3
Antenna direction
The output wind direction of inversion routines are generally given in the meteorological
convention, see table 5.2. The inversion routine uses a wind direction that is relative to the
antenna direction. The convention is that if the wind blows towards the antenna then this relative
wind direction equals to 0. Therefore, it is important to be certain about the convention of your
antenna (azimuth) angle.
For the SeaWinds Met product the radar look angle (antenna angle or simply azimuth) equals 0 if
the antenna is orientated towards the north. The SeaWinds radar look angle increases clockwise.
For ERS, however the antenna direction equals zero if the antenna directs towards the south.
Therefore the final output wind direction needs a correction of 180 degrees.
Meteorological
0
90
180
270
Mathematical
270
180
90
0
u
0
-1
0
1
v
-1
0
1
0
Description
Wind blowing from the north
Wind blowing from the east
Wind blowing from the south
Wind blowing from the west
Table 5.2 Meteorological conventions for the wind direction.
61
NWP SAF
SDP User Manual and
Reference Guide
62
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Chapter 6
Ambiguity Removal module class
6.1
Ambiguity Removal
Ambiguity Removal (AR) schemes select a surface wind vector among the different surface wind
vector solutions per cell for the set of wind vector cells in consideration. The goal is to set a
unique, meteorological consistent surface wind field. The surface wind vector solutions per cell,
simply called ambiguities, result from the wind retrieval process step.
Whenever the ambiguities are ranked, a naive scheme would be to select the ambiguity with the
first rank (e.g., the highest probability, the lowest distance to the wind cone). In general, such a
persistent first rank selection will not suffice to create a realistic surface wind vector field:
scatterometer measurements tend to generate ambiguous wind solutions with approximately equal
likelihood (mainly due to the 180° invariance of stand alone scatterometer measurements).
Therefore additional spatial constraints and/or additional (external) information are needed to
make sensible selections.
A common way to add external information to a WVC is to define a background surface wind
vector. The background wind acts as a first approximation for the expected mean wind over the
cell. In general, a NWP model wind is interpolated for this purpose. Whenever a background
wind is set for the WVC, a second naive Ambiguity Removal scheme is at hand: the Background
Closest (BC) scheme. The selected wind vector is just the minimizer of the distance (e.g., in the
least squares sense) to the background wind vector. This scheme may produce far more realistic
wind vector fields than the first rank selection, especially if the background surface wind field is
63
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
meteorologically consistent.
However, background surface winds have their own uncertainty. Therefore, sophisticated
schemes for Ambiguity Removal take both the likelihood of the ambiguities and the uncertainty
of the background surface wind into account. Examples are the KNMI Two-Dimensional
Variational (2DVar) scheme and the PreScat scheme.
The implementation of these schemes is described in sections 6.4 and 6.5.
6.2
Module Ambrem
Module Ambrem is the interface module between the various ambiguity removal methods and the
different scatterometer data processors. Table 6.1 provides an overview of the different routines
and their calls. A dummy method and the first rank selection method are implemented as part of
ambrem. More elaborate Ambiguity Removal methods have an interface module, see table 6.2.
Figure 6.1 shows schematically the interdependence of the various modules for Ambiguity
Removal.
Routine
InitAmbremModule
InitAmbremMethod
DoAmbrem
Ambrem1stRank
DoDummyMeth
SetDummyMeth
InitDummyMeth
InitDummyBatch
Call
SDP
SDP
SDP
DoAmbrem
DoAmbrem
DoAmbrem
DoAmbrem
not used
Description
Initialization of module Ambrem
Initialization of specified AR scheme
Execution of specified AR scheme
First rank selection method
Dummy AR scheme for testing
Batch definition of dummy method
Initialization of dummy method
Table 6.1 Routines of module Ambrem.
Routine
Ambrem2DVAR
AmbremBGClosest
AmbremPrescat
Description
Interface to KNMI 2DVar method
Interface to Background Closest method
Interface to Prescat method
Documentation
Section 6.4
Section 6.1
Section 6.5
Table 6.2 Interface modules for different Ambiguity Removal schemes.
6.3
Module BatchMod
After the wind retrieval step, the Ambiguity Removal step is performed on selections of the
available data. In general, these selections are just a compact part of the swath or a compact part
of the world ocean. The batch module BatchMod facilitates these selections of data. In fact, a
batch data structure is introduced to create an interface between the swath related data and the
data structures of the different AR methods. Consequently, the attributes of the batch data
64
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
ambrem
Ambrem2DVAR
AmbremPreScat
AmbremBGclosest
BatchMod
TwoDvar
TwoDvarData
convert
CostFunc
StrucFunc
BFGSMod
MultiFFT
Figure 6.1 Interdependence of the modules for Ambiguity Removal. The connections from module
ambrem to module BatchMod and from module Ambrem2DVAR to convert are not drawn.
BatchType
BatchRowType
BatchCellType
BatchQualFlagType
BatchAmbiType
Figure 6.2 Schematic representation of the batch data structure.
65
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
BatchType
Attribute
NrRows
Row
Type
Integer
BatchRowType
Description
Number of rows in batch
Array of rows
BatchRowType
Attribute
RowNr
NrCells
Cell
Type
Integer
Integer
BatchCellType
Description
Row number within orbit
Number of cells in batch (max 76)
Array of cells within row
BatchCellType
Attribute
NodeNr
lat
lon
ubg
vbg
NrAmbiguities
Ambi
Type
Integer
Real
Real
Real
Real
Integer
BatchAmbiType
Description
Node number within orbit row
Latitude
Longitude
u-component of background wind
v-component of background wind
Number of ambiguities
Array of ambiguities
BatchAmbiType
Attribute
selection
uana
vana
f
gu
gv
qualflag
Type
Integer
Real
Real
Real
Real
Real
BatchQualFlagType
Description
Index of selected ambiguity
u-component of analysis wind
v-component of analysis wind
Contribution of this cell to cost function
Derivative of f to u
Derivative of f to v
Quality control flag
Table 6.3 Batch data structures.
To check the quality of the batch a quality flag is introduced for instances of the BatchCellType.
The flag is set by routine TestBatchCell(). The attributes of this flag of type BatchQualFlagType
are listed in table 6.4.
Module BatchMod contains a number of routines to control the batch structure. The calls and
tasks of the various routines are listed in table 6.5. The batch structure is allocatable because it is
only active between the wind retrieval and the ambiguity removal step.
66
NWP SAF
SDP User Manual and
Reference Guide
Attribute
Missing
Node
Lat
Lon
Ambiguities
Selection
Background
Analysis
Threshold
Cost
Gradient
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Description
Quality flag not set
Incorrect node number specification
Incorrect latitude specification
Incorrect longitude specification
Invalid ambiguities
Invalid selection indicator
Incorrect background wind specification
Incorrect analysis
Threshold overflow
Invalid cost function value
Invalid gradient value
Table 6.4 Batch quality flag attributes.
Routine
AllocRowsAndCellsAndInitBatch
AllocAndInitBatchRow
AllocAndInitBatchCell
AllocRowsOnlyAndInitBatch
InitBatchModule
InitBatch
InitBatchRow
InitBatchCell
InitbatchAmbi
DeallocBatch
DeallocBatchRows
DeallocBatchCells
DeallocBatchAmbis
TestBatch
TestBatchRow
TestBatchCell
TestBatchQualFlag
getBatchQualFlag
setBatchQualFlag
PrnBatchQualFlag
Call
Processor
AllocRowsAndCellsAndInitBatch
AllocAndInitBatchRow
not used
Ambrem
AllocRowsAndCellsAndInitBatch
InitBatch
InitBatchRow
InitBatchCell
Processor
DeallocBatch
DeallocBatchRows
DeallocBatchCells
Processor
TestBatch
TestBatchRow
Processor
not used
not used
not used
Description
Allocation of batch
Allocation of batch rows
Allocation of batch cells
Initialization module
Initialization of batch
Initialization of batch rows
Initialization of batch cells
Initialization of batch ambiguities
Deallocation of batch
Deallocation of batch rows
Deallocation of batch cells
Deallocation of batch ambiguities
Test complete batch
Test complete batch row
Test batch cell
Print the quality flag
Table 6.5 Routines of module BatchMod.
6.4
The KNMI 2DVar scheme
6.4.1
Introduction
The purpose of the KNMI 2DVar scheme is to make an optimal selection provided the (modeled)
likelihood of the ambiguities and the (modeled) uncertainty of the background surface wind field.
First, an optimal estimated surface wind vector field (analysis) is determined based on variational
principles. This is a very common method originating from the broad discipline of Data
67
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Assimilation. The optimal surface wind vector field is called the analysis. Second, the selected
wind vector field (the result of the 2DVar scheme) consists of the wind vector solutions that are
closest to the analysis wind vector. For details of the KNMI 2DVar scheme formulation the
reader is referred to [Vogelzang, 2007]. Information on 2DVAR can also be found in [Stoffelen et
al., 2004; de Vries et al., 2004; de Vries and Stoffelen, 2000]. These three documents may be
downloaded from the EUMETSAT website, www.eumetsat.int.
The calculation of the cost function and its gradient is rather complex matter. The reader who is
only interested in how the 2DVar scheme is assembled into the genscat module class ambrem is
referred to subsection 6.4.2. Readers interested in the details of the cost function calculations and
the minimization should also read the subsequent subsections. Subsection 6.4.3 forms an
introduction to the cost function. It is recommended to first read this section, because it provides
necessary background information to understand the code. Subsection 6.4.8 on the actual
minimization and subsection 6.4.9 on Fast Fourier Transforms are in fact independent of the cost
function itself. The reader might skip these subsections.
Remarks:
−
6.4.2
The 2DVar scheme in SDP is in essence the same as that of QDP. However the
implementation largely differs.
Data structure, interface and initialisation
The main module of the 2DVar scheme is TwoDvar. Within the genscat ambiguity removal
module class, the interface with the 2DVar scheme is set by module Ambrem2DVAR. Table 6.6
lists its routines that serve the interface with TwoDvar.
Routine
Do2DVARonBatch
BatchInput2DVAR
BatchOutput2DVAR
SetAlpha
GetBatchSize2DVAR
latlon2xyz
rotuv
Call
DoAmbrem
Do2DVARonBatch
Do2DVARonBatch
BatchInput2DVAR
Description
Apply 2DVar scheme on batch
Fills the 2DVar data structure with input
Fills the batch data structure with output
Sets the observation orientation
Determine maximum size of batch
Coordinate transformation
Calculates the rotation of the (u,v) wind field
SetAlpha
BatchInput2DVAR
Table 6.6 Routines of module Ambrem2DVAR.
These routines are sufficient to couple the 2DVar scheme to the processor. The actual 2DVar
processing is done by the routines of module TwoDvar itself. These routines are listed in table
6.7. Figures B2.1-B2.7 show the complete calling tree of the AR routines.
The Obs2dvarType data type is the main data structure for the observed winds. Its attributes are
listed in table 6.8. The TDV_Type data type contains all parameters that have to do with the
2DVAR batch grid: dimensions, sizes, and derived parameters. These data structures are defined
in module TwoDvarData and the routines in this module are listed in table 6.10.
68
SDP User Manual and
Reference Guide
NWP SAF
Routine
InitTwodvarModule
Do2DVAR
Init2DVARmeth
ExitTwodvarModule
Call
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Description
Initialization of module TwoDvar
Cost function minimization
Initialize the 2DVar scheme
Deallocation of module TwoDvar
Do2DVARonBatch
Do2DVARonBatch
Table 6.7 Routines of module TwoDvar.
Attribute
alpha
cell
row
igrid
jgrid
lat
Wll
Wlr
Wul
Wur
ubg
vbg
NrAmbiguities
incr()
uAnaIncr
vAnaIncr
selection
QualFlag
f
gu
gv
Type
Real
Integer
Integer
Integer
Integer
Real
Real
Real
Real
Real
Real
Real
Integer
AmbiIncrType
Real
Real
Integer
TwoDvarQualFlagType
Real
Real
Real
Description
Rotation angle
Store batch cell number
Store batch row number
Row index (yxc in QDP)
Node index (yyc in QDP)
Latitude to determine structure function
Weight lower left
Weight lower right
Weight upper left
Weight upper right
Background EW wind component (yub in QDP)
Background NS wind component (yvb in QDP)
Number of ambiguities (ynsol in QDP)
Ambiguity increments
Analysis increment (yua in QDP)
Analysis increment (yva in QDP)
Selection flag
Quality control flag
Cost function at observation
df/du
df/dv
Table 6.8 The Obs2dvarType data structure.
Attribute
delta
delta_p
delta_q
N1
H1
K1
N2
H2
K2
Ncontrol
Type
Real
Real
Real
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Description
2DVAR grid size in position domain
2DVAR grid size in frequency domain
2DVAR grid size in frequency domain
Dimension 1 of 2DVAR grid
N1/2
H1+1;number of nonnegative frequencies
Dimension 2 of 2DVAR grid
N2/2
H2+1;number of nonnegative frequencies
Size of control vector
Table 6.9 The TDV_Type data structure.
69
NWP SAF
Routine
Init_TDV
Set_HelmholzCoefficients
Set_CFW
Exit_TDV
InitObs2dvar
DeallocObs2dvar
InitOneObs2dvar
TestObs2dvar
PrintObs2dvar
Prn2DVARQualFlag
set2DVARQualFlag
get2DVARQualFlag
SDP User Manual and
Reference Guide
Call
InitTwodvarModule
Init_TDV
Init_TDV
ExitTwodvarmodule
BatchInput2DVAR,
BatchOutput2DVAR
BatchOutput2DVAR
InitObs2dvar
Do2DVAR
BatchInput2DVAR
Do2DVAR
TestObs2DVAR
not used
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Description
Initialization of 2DVAR grid and preparations
Set Helmholz transformation coefficients
Set cost function weights
Deallocate memory
Allocation of observations array
Deallocation of observations array
Initialization of single observation
Test single observation
Print a single 2DVar observation
Print observation quality flag
Convert observation quality flag to integer
Convert integer to observation quality flag
Table 6.10 Routines in module TwoDvarData.
The quality status of an instance of Obs2dvarType is indicated by the attribute QualFlag which is
an instance of TwoDvarQualFlagType. The attributes of this flag are listed in table 6.11.
Attribute
missing
wrong
Lat
Background
Ambiguities
Selection
Analyse
Cost
gradient
weights
grid
Description
Flag values not set
Invalid 2DVar process
Invalid latitude
Invalid background wind increment
Invalid ambiguity increments
Invalid selection
Invalid analysis wind increment
Invalid cost function specification
Invalid gradient specification
Invalid interpolation weights
Invalid grid indices
Table 6.11 Attributes of 2DVar observation quality flag.
6.4.3
Reformulation and transformation
The minimization problem to find the analysis surface wind field (the 2D variational Data
Assimilation problem) may be formulated as
min J (v) , J (v) = J obs (v) + J bg (v) ,
(6.1)
v
where v is the surface wind field in consideration and J the total cost function consisting of the
70
SDP User Manual and
Reference Guide
NWP SAF
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
observational term Jobs and the background term Jbg. The solution, the analysis surface wind field,
may be denoted as va. Being just a weighted least squares term, the background term may be
further specified as
J bg (v) = [v − vbg ]T B −1 [v − vbg ] ,
(6.2)
where B is the background error covariance matrix. The Jobs term of the 2DVar scheme is not
simply a weighted least squares term.
Such a formulation does not closely match the code of the 2DVar scheme. In fact, for scientific
and technical reasons several transformations are applied to reformulate the minimization
problem. Description of these transformations is essential to understand the different procedures
within the code. The interested reader is referred to Vogelzang [2007].
6.4.4
Module CostFunc
Module CostFunc contains the main procedure for the calculation of the cost function and its
gradient. It also contains the minimization procedure. Table 6.12 provides an overview of the
routines.
Routine
Jt
Jb
Jo
JoScat
Unpack_ControlVector
Pack_ControlVector
Uncondition
Uncondition_adj
minimize
Call
minimise
Jt
Jt
Jo
Jo
Jo
Jo
Jo
Do2DVAR (TwoDvar)
Description
Total cost function and gradient
Background term of cost function
Observational term of cost function
Single observation contribution to the cost function
Unpack of control vector
Pack of control vector (or its gradient)
Several transformations of control vector
Adjoint of Uncondition.
Minimization
Table 6.12 Routines of module CostFunc.
6.4.5
Adjoint method
The minimization of cost function is done with a quasi-Newton method. Such a method requires
an accurate approximation of the gradient of the cost function. The adjoint method is just a very
economical manner to calculate this gradient. For introductory texts on the adjoint method and
adjoint coding, see, e.g., [Talagrand, 1991; Giering, 1997]. For detailed information on the
adjoint model in 2DVAR see Vogelzang [2007].
71
SDP User Manual and
Reference Guide
NWP SAF
6.4.6
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Structure Functions
Module StrucFunc contains the routines to calculate the covariance matrices for the stream
function, ψ , and the velocity potential, χ. Its routines are listed in table 6.13.
Routine
PrintStrcFuncPars
SetCovMat
InitStrucFunc
StrucFuncPsi
StrucFuncChi
Call
not used
Do2DVAR
SetCovMat
SetCovMat
SetCovMat
Description
Calculate the covariance matrices
Initialize the structure functions
Calculate ψ
Calculate χ
Table 6.13 Routines of module StrucFunc.
Routine InitStrucFunc reads the structure function parameters from a file specified by the –par
command line option, or sets them to a default value. The parameters are read in format free. The
parameter value input file contains three lines (records), for the Northern hemisphere (latitude
larger than 20°), the Tropics (latitude between -20° and +20°), and the Southern hemisphere
(latitude below -20°). Each record should contain the following five numbers:
R_psi
R_chi
E_psi
E_chi
nu_sq
with R_psi and R_chi the background error correlation lengths in the spatial domain, E_psi
end E_chi the background error standard deviations, and nu_sq the divergence/rotation ratio.
6.4.7
Minimisation
The minimization routine used is LBFGS. This is a quasi Newton method with a variable rank for
the approximation of the Hessian written by J. Nocedal. A detailed description of this method is
given by Liu and Nocedal [1989]. Routine LBFGS is freeware and can be obtained from web
page www.netlib.org/opt/index.html, file lbfgs_um.shar. The original Fortran77 code has
been adjusted to compile under Fortran90 compilers. Routine LBFGS and its dependencies are
located in module BFGSMod.F90 in directory genscat/support/BFGS. Table 6.14
provides an overview of the routines in this module.
Routine LBFGS uses reverse communication. This means that the routine returns to the calling
routine not only if the minimization process has converged or when an error has occurred, but
also when a new evaluation of the function and the gradient is needed. This has the advantage that
no restrictions are imposed on the form of routine Jt calculating the cost function and its gradient.
The formal parameters of LBFGS have been extended to include all work space arrays needed by
the routine. The work space is allocated in the calling routine minimise. The rank of LBFGS
affects the size of the work space. It has been fixed to 3 in routine minimise, because this value
gave the best results (lowest values for the cost function at the final solution).
72
NWP SAF
Routine
LBFGS
LB1
daxpy
ddot
MCSRCH
MCSTEP
Call
minimise
LBFGS
LBFGS
LBFGS
LBFGS
MCSRCH
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Description
Main routine
Printing of output (switched off)
Sum of a vector times a constant plus another vector with loop unrolling.
Dot product of two vectors using loop unrolling.
Line search routine.
Calculation of step size in line search.
Table 6.14 Routines in module BFGSMod.
Some of the error returns of the line search routine MCSRCH have been relaxed and are treated as
a normal return. Further details can be found in the comment in the code itself.
Routines daxpy and ddot were rewritten in Fortran90. These routines, originally written by J.
Dongarra for the Linpack library, perform simple operations but are highly optimized using loop
unrolling. Routine ddot, for instance, is faster than the equivalent Fortran90 intrinsic function
dot_product.
6.4.9
MultiFFT
Module MultFFT in directory genscat/support/multifft contains the multi-variate
complex Fourier routines needed in the 2DVar scheme. Actually there are two methods
(implementations) available. These are the simple method, which is straightforward
implementation of the 2D transform, and the fast method, which is an implementation of the
Cooley-Tukey algorithm. The fast method is default in SDP, but the simple method is useful for
testing purposes.
Routine
Fourier2DForward
Fourier2DForward_adj
Fourier2DBackward
Fourier2DBackward_adj
Simple2DFT
Simple2DFT_adj
Simple2DFT_test
Simple1DFT
Simple1DFT_adj
Simple1DFT_test
Fast2DFT
Fast2DFT_adj
Fast2DFT_test
FastFT
Fourier2DBackward_adj_test
Fourier2DForward_adj_test
PrintCompare2D
NormCompare2D
Call
SetCovMat
not used
frq2grd
frq2grd_adj
Fourier2DForward, Fourier2DBackward
Fourier2DForward_adj, Fourier2DBackward_adj
not used
not used
not used
not used
Fourier2DForward, Fourier2DBackward
Fourier2DForward_adj, Fourier2DBackward_adj
not used
Fast2DFT
not used
not used
not used
not used
Table 6.15 Fourier transform routines.
73
Description
Forward 2D Fourier transform
Adjoint of Fourier2DForward
Inverse 2D Fourier transform
Adjoint of Fourier2DBackward
Basic 2D Fourier routine
Adjoint of Simple2DFT
Test of Simple2DFT
Basic 1D FT routine
Adjoint of Simple1DFT
Test of Simple1DFT
Fast 2D Fourier routine
Adjoint of Fast2DFT
Test of Fast2DFT
Basic 1D FFT routine
Adjoint test
Adjoint test
Support routine for test
Support routine for test
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Table 6.15 gives an overview of the available routines. Figure B.2.11 shows the calling tree of the
FT routines relevant for 2DVar.
Remarks:
6.5
−
Reading in the 2DVAR structure function parameters from an external file with the –par
command line option, thereby changing their default values, is at your own risk!
−
The 2DVAR implementation can be made more efficient by using a real-to-real FFT
routine rather than a complex-to-complex one as implemented now. Since SDP satisfies
the requirements in terms of computational speed, this has low priority.
The PreScat scheme
The PreScat ambiguity removal scheme is not used within SDP. More information on this scheme
can be found in [Stoffelen et al., 2004].
74
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Chapter 7
Module BufrMod
Module BufrMod is part of the genscat support modules. The current version is a Fortran90
wrapper around the ECMWF BUFR library (see www.ecmwf.int). The goal of this support
module is to provide a comprehensive interface to BUFR data for every Fortran90 program using
it. In particular, BufrMod provides all the BUFR functionality required for the scatterometer
processor based on genscat. Special attention has been paid to testing and error handling.
7.1
Background
The acronym BUFR stands for Binary Universal Form for the Representation of data. BUFR is
maintained by the World Meteorological Organization WMO and other meteorological centers. In
brief, the WMO FM-94 BUFR definition is a binary code designed to represent, employing a
continuous binary stream, any meteorological data. It is a self defining, table driven and very
flexible data representation system. It is beyond the scope of this document to describe BUFR in
detail. Complete descriptions are distributed via the websites of WMO (www.wmo.ch) and of the
European Center for Medium-range Weather Forecasts ECMWF (www.ecmwf.int).
Module BufrMod is in fact an interface. On the one hand it contains (temporary) definitions to set
the arguments of the ECMWF library functions. On the other hand, it provides self explaining
routines to be incorporated in the wider Fortran90 program. Section 7.2 describes the routines in
module BufrMod. The public available data structures are described in section 7.3. BufrMod uses
two libraries: the BUFR software library of ECMWF and BUFRIO, a small library in C for file
handling at the lowest level. These libraries are discussed in some more detail in section 7.4.
7.2
Routines
Table 7.1 provides an overview of the routines in module BufrMod. The most important ones are
described below.
75
NWP SAF
Routine
InitAndSetNrOfSubsets
set_BUFR_fileattributes
open_BUFR_file
get_BUFR_nr_of_messages
get_BUFR_message
get_expected_BUFR_msg_size
ExpandBufrMessage
PrintBufrErrorCode
CheckBufrTables
get_file_size
get_bufrfile_size_c
encode_table_b
encode_table_d
FillBufrSecData
close_BUFR_file
BufrReal2Int
save_BUFR_message
EncodeBufrData
CheckBufrData
FillBufrData
bufr_msg_is_valid
set_bufr_msg_to_invalid
PrintBufrData
GetPosBufrData
GetRealBufrData
GetIntBufrData
GetRealBufrDataArr
GetIntBufrDataArr
GetRealAllBufrDataArr
CloseBufrHelpers
missing_real
missing_int
int2real
do_range_check_int
do_range_check_real
AddRealDataToBufrMsg
AddIntDataToBufrMsg
PrintBufrModErrorCode
BufrInt2Real
GetFreeUnit
SDP User Manual and
Reference Guide
Call
SDP
SDP
SDP
SDP
SDP
get_BUFR_message
get_BUFR_message
ExpandBufrMessage
ExpandBufrMessage
CheckBufrTables
get_file_size
CheckBufrTables
CheckBufrTables
get_BUFR_message
SDP
SDP
not used
save_BUFR_message
EncodeBufrData
save_BUFR_message
not used
not used
not used
not used
not used
not used
not used
not used
not used
not used
not used
not used
not used
not used
not used
not used
not used
not used
not used
not used
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Description
Initialization routine
Initialization routine
Opens a BUFR file
Inquiry of BUFR file
Reads instance of BufrDataType to file
Convert from BufrMessageType to BufrSectionsType
Control
Determine size of BUFR file
Support routine in C
Convert from BufrSectionsType to BufrDataType
Closes a BUFR file
Conversion
Saves instance of BufrDataType to file
Convert from BufrSectionsType to BufrMessageType
Control
Convert from BufrDataType to BufrSectionsType
There is also a copy in module SwsSupport
Table 7.1 Routines of module BufrMod.
Reading (decoding): Routine get_BUFR_message() reads a single BUFR message from the
BUFR file and creates an instance of BufrDataType.
Writing (encoding): Routine save_BUFR_message() saves a single BUFR message to the BUFR
file. The data should be provided as an instance of BufrDataType. This routine is not used in
SDP.
76
SDP User Manual and
Reference Guide
NWP SAF
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Checking and Printing: The integer parameter BufrVerbosity controls the extent of the log
statements while processing the BUFR file. The routines PrintBufrData() and CheckBufrData()
can be used to respectively print and check instances of BufrDataType.
Open and Close BUFR files: The routine open_BUFR_file() opens the BUFR file for both read
(writemode=.false.) and writing (writemode=.true.). Routine set_BUFR_fileattributes()
determines several aspects of the BUFR file and saves these data in an instance of
bufr_file_attr_data, see table 7.5. Routine get_BUFR_nr_of_messages() is used to determine the
number of BUFR messages in the file. Finally, routine close_BUFR_file() closes the BUFR file.
As said before, the underlying encoding and decoding routines originate from the ECMWF
BUFR library, with the BUFRIO library acting as an intermediate. Appendix B3 shows the
calling trees of the routines in module BufrMod that are used in SDP.
7.3
Data structures
The data type closest to the actual BUFR messages in the BUFR files is the BufrMessageType,
see table 7.2. These are still encoded data. Every BUFR message consists of 5 sections and one
supplementary section. After decoding (expanding) the BUFR messages, the data are transferred
into an instance of BufrSectionsType, see table 7.3, which contains the data and meta data in
integer values subdivided in these sections.
Attribute
buff
size
nr_of_words
Type
Integer (max_bufr_mess_size)
Integer
Integer
Description
BUFR message, all sections
Size in bytes of BUFR message
Idem, now size in words
Table 7.2 Attributes for the BufrMessageType data type.
Attribute
ksup(9)
ksec(3)
ksec1(40)
ksec2(64)
ksec3(4)
ksec4(2)
Type
Integer
Integer
Integer
Integer
Integer
Integer
Description
Supplementary info and items selected from the other sections
Expanded section 0 (indicator)
Expanded section 1 (identification)
Expanded section 2 (optional)
Expanded section 3 (data description)
Expanded section 4 (data)
Table 7.3 Attributes for the BufrSectionsType data type.
The next step is to bring the section data to actual dimensions, descriptions and values of data
which can be interpreted as physical parameters. Therefore, instances of BufrSectionsType are
transferred to instances of BufrDataType, see table 7.4. The actual data for input or output in a
BUFR message should be an instance of the BufrDataType data type. Some meta information on
the BUFR file is contained in the self explaining bufr_file_attr_data data type, see table 7.5.
77
NWP SAF
SDP User Manual and
Reference Guide
Attribute
Nsec0
nsec0size
nBufrLength
nBufrEditionNumber
Nsec1
nsec1size
kEditionNumber
Kcenter
kUpdateNumber
kOptional
ktype
ksubtype
kLocalVersion
kyear
kmonth
kday
khour
kminute
kMasterTableNumber
kMasterTableVersion
ksubcenter
klocalinfo()
Nsec2
nsec2size
key(46)
Nsec3
nsec3size
Kreserved3
ksubsets
kDataFlag
Nsec4
nsec4size
kReserved4
nelements
nsubsets
nvals
nbufrsize
ktdlen
ktdexl
ktdlst()
ktdexp()
values()
cvals()
cnames()
cunits()
Type
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer
Integer array
Integer array
Real array
Character array
Character array
Character array
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Description
ksup ( 9) dimension section 0
ksec0( 1) size section 0
ksec0( 2) length BUFR
ksec0( 3)
ksup ( 1) dimension section 1
ksec1( 1) size section 1
ksec1( 2)
ksec1( 3)
ksec1( 4)
ksec1( 5)
ksec1( 6)
ksec1( 7) local use
ksec1( 8)
ksec1( 9) century year
ksec1(10)
ksec1(11)
ksec1(12)
ksec1(13)
ksec1(14)
ksec1(15)
ksec1(16)
ksec1(17:40)
ksup ( 2) dimension section 2
ksec2( 1) size section 2
ksec2( 2: ) key
ksup ( 3) dimension section 3
ksec3( 1) size section 3
ksec3( 2) reserved
ksec3( 3) number of reserved subsets
ksec3( 4) compressed (0,1) observed (0,1)
ksup ( 4) dimension section 4
ksec4( 1) size section 4
ksec4( 2) reserved
ksup ( 5) actual number of elements
ksup ( 6) actual number of subsets
ksup ( 7) actual number of values
ksup ( 8) actual size of BUFR message
Actual number of data descriptors
Actual number of expanded data descriptors
List of data descriptors
List of expanded data descriptors
List of values
List of CCITT IA no. 5 elements
List of expanded element names
List of expanded element units
Table 7.4 Attributes of the BUFR message data type BufrDataType.
78
NWP SAF
SDP User Manual and
Reference Guide
Attribute
nr_of_BUFR_mesasges
bufr_filename
bufr_fileunit
file_size
file_open
writemode
is_cray_blocked
list_of_BUFR_startpointers()
message_is_valid()
Type
Integer
Character
Integer
Integer
Logical
Logical
Integer
Integer
Logical
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Description
Number of BUFR messages
BUFR file
Fortran unit of BUFR file
Size of BUFR file
Open status of BUFR file
Reading or writing mode of BUFR file
Cray system blocked?
Pointers to BUFR messages
Validity of BUFR messages
Table 7.5 Attributes of the bufr_file_attr_data data type for BUFR files.
7.4
Libraries
Module BufrMod uses two libraries: the BUFR software library of ECMWF and BUFRIO, a
small library in C for file handling at the lowest level.
The BUFR software library of ECMWF is used as a basis to encode and decode BUFR data. This
software library is explained in [Dragosavac, 1994].
Appendix D provides an overview of the different routines of this library. From the calling trees
in Appendix B3 it can be inferred that only a few routines of the BUFR software library are
actually used.
Library BUFRIO contains routines for BUFR file handling at the lowest level. Since this is quite
hard to achieve in Fortran, these routines are coded in C. The routines of BUFRIO are listed in
table 7.6. The source file (bufrio.c) is located in subdirectory genscat/support/bufr.
Routine
bufr_open
bufr_split
bufr_read_allsections
bufr_get_section_sizes
bufr_swap_allsections
bufr_write_allsections
bufr_close
bufr_error
Call
open_BUFR_file
open_BUFR_file
get_BUFR_message
get_BUFR_message
get_BUFR_message, save_BUFR_message
save_BUFR_message
close_BUFR_file
see appendix B.3
Description
Read BufrMessageType from BUFR file
Optional byte swapping
Write BufrMessageType to BUFR file
Error handling
Table 7.6 Routines in library BUFRIO.
7.5
BUFR table routines
BUFR tables are used to define the data descriptors. The presence of the proper BUFR tables is
checked before calling the reading and writing routines. If absent, it is tried to create the needed
79
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
BUFR tables from the text version, available in genscat.
7.6
Center specific modules
BUFR data descriptors are integers. These integers consist of class numbers and numbers for the
described parameter itself. These numbers are arbitrary. To establish self documenting names for
the BUFR data descriptors for a Fortran90 code several center specific modules are created.
These modules are listed in table 7.7. Note that these modules are just cosmetic and not essential
for the encoding or decoding of the BUFR data.
Module
WmoBufrMod
KnmiBufrMod
EcmwfBufrMod
Description
WMO standard BUFR data description
KNMI BUFR data description
ECMWF BUFR data description
Table 7.7 Fortran90 BUFR modules.
80
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
References
•
Dragosavac, M., 1994,
BUFR User Guide and Reference Manual. ECMWF. (Available via the ECMWF website
www.ecmwf.int)
•
Draper, D.W. and Long, D.G., 2002,
An assessment of SeaWinds on QuikSCAT wind retrieval. Journal of Geophysical
Research, 107, C12, p. 3212 (2002JC001330).
•
Freilich, M.H.,
SeaWinds algorithm theoretical basis document. Report atbd-sws-01.
•
Giering, R., 1997,
Tangent linear and Adjoint Model Compiler, Users manual. Max-Planck- Institut fuer
Meteorologie.
•
Leidner, S.M., Hoffman, R.N., and Augenbaum, J., 2000,
SeaWinds Scatterometer Real Time BUFR Geophysical Data Product User's guide.
NOAA NESDIS. (Available from the KNMI site www.knmi.nl/scatterometer/
publications).
•
Liu, D.C., and Nocedal, J., 1989
On the limited memory BFGS method for large scale optimization methods.
Mathematical Programming, 45, pp. 503-528.
•
Portabella, M., 2002,
Wind field retrieval from satellite radar systems. PhD thesis, University of Barcelona.
(Available via the KNMI site www.knmi.nl/scatterometer/publications).
•
Portabella, M. and Stoffelen, A., 2001,
Rain Detection and Quality Control of SeaWinds. Journal of Atmospheric and Oceanic
Technology, 18 , pp. 1171-1183.
•
Portabella, M. and Stoffelen, A., 2002,
A Comparison of KNMI Quality Control and JPL Rain Flag for SeaWinds. Canadian
Journal of Remote Sensing (special issue on Remote Sensing of Marine Winds), 28, 3.
81
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
•
Portabella, M. and Stoffelen, A., 2004,
A probabilistic approach for SeaWinds Data Assimilation. Quarterly Journal of the Royal
Meteorological Society, 130, pp. 1-26.
•
SCAT group, 2005,
SDP Test Report, version 1.3. Report NWPSAF-KN-TR-001, UKMO, UK. (Available
via the NWPSAF web site, http://www.metoffice.gov.uk/research/interproj/nwpsaf/
scatterometer/index.html.
Stoffelen, A., de Haan, S., Quilfen, Y., and Schyberg, H., 2004,
ERS scatterometer ambiguity removal scheme comparison. KNMI/ EUMETSAT Ocean
and Sea Ice report. (Available from the EUMETSAT website, www.eumetsat.int).
•
•
Stoffelen, A., de Vries, J., and Voorrips, A., 2000,
Towards the real-time use of QuikScat winds. Beleidscommissie Remote Sensing, report
nr. USP-2/00-26.
•
Stoffelen, A.C.M., 1998,
Scatterometry. PhD thesis, University of Utrecht, ISBN 90-393-1708-9. (Available via
the KNMI site www.knmi.nl/scatterometer/publications).
•
Talagrand, O., 1991,
The use of adjoint equations in numerical modeling of the atmospheric circulation. In:
Automatic Differentiation of Algorithms: Theory, Implementation and Application, A.
Griewank and G. Corliess Eds. pp. 169-180, Philadelphia, Penn: SIAM.
•
Vogelzang, J., 2007,
Two dimensional variational ambiguity removal (2DVAR). Report NWPSAF-KN-TR003, UKMO, UK. (Available via the NWPSAF web site, http://www.metoffice.gov.uk/
research/interproj/nwpsaf/scatterometer/index.html.
•
de Vries, J. and Stoffelen, A., 2000,
2D Variational Ambiguity Removal. KNMI, Feb 2000. (Available from the EUMETSAT
website, www.eumetsat.int).
•
de Vries, J., Stoffelen, A., and Beysens, J., 2004,
Ambiguity Removal and Product Monitoring for SeaWinds. KNMI. (Available from the
EUMETSAT website, www.eumetsat.int).
•
de Vries, J., et al., 2004,
QuikScat Data Processor User Manual, KNMI. (Available from the KNMI website,
www.knmi.nl/scatterometer/publications).
•
Wentz, and Smith, D.K., 1999,
A model function for the ocean normalized cross section at 14 GHz derived from NSCAT
observations. Journal of Geophysical Research, 104, C5, pp. 11499-11507.
82
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Appendix A
Calling tree for SDP
The figures in this appendix show the calling tree for the SDP program. Routines in white boxes
are part of the SDP process layer and the SeaWinds Support layer. Routines in black boxes are
part of genscat. An arrow (→) before a routine name indicates that this part of the calling tree is a
continuation of a branch in a previous figure. The same arrow after a routine name indicated that
this branch will be continued in a following figure.
83
SDP User Manual and
Reference Guide
NWP SAF
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
sdp
ProcessArguments
usage
GetOutputFileNames
GetNwpFileNames
ProcessSwsFileName
julian2ymd
ReadBufrInput (→)
ProcessInit
GetElapsedSystemTime
PreProcess (→)
InvertWVCs (→)
RemoveAmbiguity (→)
PostProcess (→)
WriteBufrOutput (→)
ProcessCleanUp
GetElapsedSystemTime
Figure A.1 Calling tree for program sdp (top level). Light grey boxes are cut here and will be continued in
one of the first level or second level calling trees in the next figures. Black boxes with light text indicate
genscat routines.
84
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
(→) ReadBufrInput
GetElapsedSystemTime
OpenSwsBufrFile
SwsBufrInit
Set_Bufr_FileAttributes
open_BUFR_file (→)
get_BUFR_nr_of_messages
ReadSwsBufrData
get_Bufr_message (→)
BufrReal2Int
Values2CellNOAA
InitCell (→)
BufrReal2Int
getCellQualFlagNOAA
getSigma0QualFlag
getSigma0ModeFlag
getSigma0SurfFlag
Values2CellGen
InitCell (→)
BufrReal2Int
getSigma0QualFlag
getSigma0ModeFlag
getSigma0ModeFlag
getSigma0SurfFlag
TestCell (→)
PrintCell (→)
InitCell (→)
CloseSwsBufrFile
close_BUFR_file (→)
Figure A.2 Calling tree for routine ReadBufrInput (first level).
85
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
(→) PreProcess
GetElapsedSystemTime
PrepareInput
GetSortIndex
MergeRow
CopyCell
TestCell (→)
SetInputMleQC
PrepareOutput
InitCell (→)
latlon2xyz
met2uv
xyz2latlon
uv2met
TestCell (→)
CopyInputOutput
InitCell (→)
CopyCell
Figure A.3 Calling tree for routine PreProcess (first level).
86
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
(→) InvertWVCs
GetElapsedSystemTime
InitInversion
InitMeanMle
get_lun
init_inv_settings_to_default
get_inv_settings
set_inv_settings
InversionInCell
init_inv_input
invert_one_wvc (→)
CalcProbabilities
GetSortIndex
SortWithIndex
TestCell (→)
Figure A.4 Calling tree for routine InvertWVCs (first level).
Figure A.5 (next page) Calling tree for routine RemoveAmbiguity (first level). The full name of the 12th
routine is AllocRowsAndCellsAndInitBatch.
87
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
SDP User Manual and
Reference Guide
NWP SAF
(→) RemoveAmbiguity
GetElapsedSystemTime
InitAmbremModule
InitBatchModule
InitAmbremMethod
InitAmbremBGclosest
InitTwodvarModule (→)
InitDummyMethod
GetBatch
GetBatchSize2DVAR
AllocRowsAndCellsAnd…
InitBatch
AllocAndInitBatchRow
InitBatchRow
InitBatchCell
TestCell (→)
AllocAndInitBatchCell
InitBatchCell
InitBatchAmbi
met2uv
TestBatch
TestBatchRow
TestBatchCell
DoAmbrem (→)
SelectWind
TestBatchCell
uv2met
TestCell (→)
DeAllocBatch
DeallocBatchRows
DeallocBatchCells
DeallocBatchAmbis
ExitAmbremMethod
ExitTwodvarModule (→)
88
NWP SAF
SDP User Manual and
Reference Guide
89
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
(→) WriteBufrOutput
GetElapsedSystemTime
OpenSwsBufrFile
WriteSwsBufrData
InitAndSetNrOfSubsets
Cell2ValuesNOAA
BufrInt2Real
setCellQualFlagNOAA
setSigma0QualFlag
setSigma0ModeFlag
setSigma0SurfFlag
MonitoringSetMonitorBits
BufrInt2Real
setSigma0QualFlag
setSigma0ModeFlag
setSigma0SurfFlag
setCellQualFlagNOAA
save_Bufr_message
CloseSwsBufrFile
close_Bufr_file (→)
Figure A.6 Calling tree for routine WriteBufrOutput (first level).
90
NWP SAF
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
SDP User Manual and
Reference Guide
(→) PostProcess
GetElapsedSystemTime
OutputConversion
Monitoring
MonitoringCalculateData
met2uv
MonitoringWriteStats
MonitoringSetMonitorBits
Figure A.7 Calling tree for routine PostProcess (first level).
(→) InitCell
InitProcessFlag
InitTime
InitRain
InitAntenna
InitBeam
InitSigma0
InitKp
Figure A.8 Calling tree for routine InitCell (second level).
(→) TestCell
TestTime
TestSigma0
TestKp
TestWind
Figure A.9 Calling tree for routine TestCell (second level).
91
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
(→) PrintCell
PrintTime
PrintWind
PrintBeam
PrintSigma0
PrnSigma0QualFlag
setSigma0QualFlag
PrnSigma0ModeFlag
setSigma0ModeFlag
PrnSigma0SurfFlag
setSigma0SurfFlag
PrintKp
PrintAmbi
PrintWind
PrintRain
PrnCellQualFlag
setCellQualFlag
PrintAntenna
PrnProcessFlag
Figure A.10 Calling tree for routine PrintCell (second level).
92
NWP SAF
SDP User Manual and
Reference Guide
93
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Appendix B1
Calling tree for inversion routines
The figures in this appendix show the calling tree for the inversion routines in genscat. All
routines are part of genscat, as indicated by the black boxes. An arrow (→) before a routine name
indicates that this part of the calling tree is a continuation of a branch in a previous figure. The
same arrow after a routine name indicated that this branch will be continued in a following figure.
94
SDP User Manual and
Reference Guide
NWP SAF
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
(→) invert_one_wvc
init_inv_settings_to_default
init_inv_output
print_message
check_input_data
print_input_data_of_inversion
my_exit
print_message
convert_sigma_to_zspace
calc_normalisation
calc_var_s0
set_wind_speed_first_guess
find_minimum_cone_dist (→)
my_min
my_average
my_max
get_indices_lowest_local_minimum
my_index_max
print_message
do_parabolic_winddir_search
get_parabolic_minimum
my_exit
calc_var_s0_ers
get_ers_noise_estimate
get_ers_node_formfactor
calc_sign_MLE
calc_sigma0 (→)
fill_wind_quality_code (→)
Figure B1.1 Calling tree for inversion routine invert_one_wvc.
95
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
(→) find_minimum_cone_dist
get_wind_speed_first_guess
calc_cone_distance
get_parabolic_minimum
calc_sigma0 (→)
my_exit
set_wind_speed_first_guess
Figure B1.2 Calling tree for inversion routine find_minimum_cone_dist
(→) fill wind_quality_code
remove_one_solution
print_output_data_of_inversion
print_wind_quality_code
my_min
my_max
calc_distance_to_cone_center
calc_sigma0 (→)
Figure B1.3 Calling tree for inversion routine fill_wind_quality_code.
96
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
(→) calc_sigma0
read_LUT
get_lun
free_lun
create_LUT_C_VV
get_lun
calc_sigma0_cmod4
Get_Br_from_Look_Up_Table
f1
calc_sigma0_cmod5
free_lun
test_for_identical_LUTs
my_exit
INTERPOLATE
Figure B1.4 Calling tree for inversion routine calc_sigma0. Routine INTERPOLATE is an interface that
can have the values interpolate1d, interpolate2d, interpolate2dv or interpolate3d.
97
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Appendix B2
Calling tree for AR routines
The figures in this appendix show the calling tree for the Ambiguity Removal routines in genscat.
All routines are part of genscat, as indicated by the black boxes. An arrow (→) before a routine
name indicates that this part of the calling tree is a continuation of a branch in a previous figure.
The same arrow after a routine name indicated that this branch will be continued in a following
figure.
98
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
SDP User Manual and
Reference Guide
NWP SAF
(→) DoAmbrem
TestBatch
TestBatchRow
TestBatchCell
AmbRem1stRank
DoAmbremBGclosestOnBatch
uv_to_dir
DoAmbremPreScatOnBatch
DoAmbremBGclosestOnBatch
uv_to_dir
Do2DVARonBatch
BatchInput2DVAR
TestBatchCell
InitObs2DVAR (→)
Set_WVC_Orientations
WVC_Orientation
rotuv
PrintObs2DVAR
Do2DVAR (→)
BatchOutput2DVAR
rotuv
InitObs2DVAR (→)
DeallocObs2DVAR
D
DoDummyMeth
Figure B2.1 Calling tree for AR routine DoAmbrem.
99
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
(→) InitObs2DVAR
InitOneObs2DVAR
TestObs2DVAR
Figure B2.2 Calling tree for AR routine InitObs2DVAR.
(→) Do2DVAR
TestObs2dvar
set2DVARQualFlag
Prn2DVARQualFlag
SetCovMat
StrucFuncPsi
StrucFuncChi
Fourier2DForward (→)
minimise
Jt (→)
LBFGS
daxpy
ddot
LB1
MCSRCH
MCSTEP
Figure B2.3 Calling tree for AR routine Do2DVAR.
100
SDP User Manual and
Reference Guide
NWP SAF
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
(→)Jt
Jb
Jo
Unpack_ControlVector
Uncondition
Fourier2DBackward
JoScat
Uncondition_adj
Fourier2DForward
Pack_ControlVector
Figure B2.4 Calling tree for AR routine Jt (calculation of cost function).
(→) Fourier2DForward
Simple2DFT
Fast2DFT
FastFT
Figure B2.5 Calling tree for AR routine Fourier2DForward. The calling tree for routine
Fourier2DBackward is identical.
(→) InitTwodvarModule
Init_TDV
Set_CFW
Set_HelmholzCoefficients
(→) ExitTwodvarModule
Exit_TDV
Figure B.2.6 Calling trees for AR routines InitTwodvarModule and ExitTwodvarModule.
101
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Appendix B3
Calling tree for BUFR routines
The figures in this appendix show the calling tree for the BUFR file handling routines in genscat.
Routines in black boxes are part of genscat. Routines in grey boxes with names completely in
capitals belong to the ECMWF BUFR library. Other routines in grey boxes belong to the
BUFRIO library. An arrow (→) before a routine name indicates that this part of the calling tree is
a continuation of a branch in a previous figure. The same arrow after a routine name indicated
that this branch will be continued in a following figure.
102
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
(→) open_BUFR_file
bufr_open
bufr_error
bufr_split
Figure B3.1 Calling tree for BUFR file handling routine open_BUFR_file.
(→) get_BUFR_message
get_expected_BUFR_msg_size
bufr_read_allsections
bufr_error
bufr_get_section_sizes
bufr_swap_allsections
ExpandBufrMessage
BUS012
PrintBufrErrorCode
ChechBufrTables
get_file_size
encode_table_b
encode_table_d
BUFREX
FillBufrSecData
BUSEL
Figure B3.2 Calling tree for BUFR handling routine get_BUFR_message.
103
NWP SAF
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
SDP User Manual and
Reference Guide
(→) save_BUFR_message
EncodeBufrData
FillBufrData
bufr_swap_allsections
bufr_write_allsections
bufr_error
Figure B3.3 Calling tree for BUFR file handling routine save_BUFR_file.
(→) close_BUFR_file
bufr_close
bufr_error
Figure B3.4 Calling tree for BUFR handling routine close_BUFR_file.
104
NWP SAF
SDP User Manual and
Reference Guide
105
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Appendix C1
NOAA BUFR output file
Number Parameter
001
Satellite Identifier
002
Direction of Flight
003
Satellite Instrument Identifier
004
Wind Scatterometer GMF
005
Software Identification
006
Cross Track Resolution
007
Along Track Resolution
008
Orbit Number
009
Year
010
Month
011
Day
012
Hour
013
Minute
014
Second
015
Latitude (Coarse Accuracy)
016
Longitude (Coarse Accuracy)
017
Time Difference Qualifier
018
Time to Edge
019
Along Track Row Number
020
Cross Track Cell Number
021
Seawinds Wind Vector Cell Quality Flag
022
Model Wind Direction At 10 M
023
Model Wind Speed At 10 M
024
Number of Vector Ambiguities
025
Index of Selected Wind Vector
Continued on next page
106
Unit
CODE TABLE
DEGREE TRUE
CODE TABLE
CODE TABLE
NUMERIC
M
M
NUMERIC
YEAR
MONTH
DAY
HOUR
MINUTE
SECOND
DEGREE
DEGREE
CODE TABLE
S
NUMERIC
NUMERIC
FLAG TABLE
DEGREE TRUE
M/S
NUMERIC
NUMERIC
Descriptor
(01007)
(01012)
(02048)
(21119)
(25060)
(02026)
(02027)
(05040)
(04001)
(04002)
(04003)
(04004)
(04005)
(04006)
(05002)
(06002)
(08025)
(04001)
(05034)
(06034)
(21109)
(11081)
(11082)
(21101)
(21102)
NWP SAF
SDP User Manual and
Reference Guide
Continued from previous page
Number Parameter
026
Total Number of Sigma0 Measurements
027
Seawinds Probability of Rain
028
Seawinds NOF Rain Index
029
Intensity Of Precipitation
030
Attenuation Correction On Sigma-0 (from Tb)
031
Wind Speed At 10 M
032
Formal Uncertainty In Wind Speed
033
Wind Direction At 10 M
034
Formal Uncertainty In Wind Direction
035
Likelihood Computed for Wind Solution
036
Wind Speed At 10 M
037
Formal Uncertainty In Wind Speed
038
Wind Direction At 10 M
039
Formal Uncertainty In Wind Direction
040
Likelihood Computed for Wind Solution
041
Wind Speed At 10 M
042
Formal Uncertainty In Wind Speed
043
Wind Direction At 10 M
044
Formal Uncertainty In Wind Direction
045
Likelihood Computed for Wind Solution
046
Wind Speed At 10 M
047
Formal Uncertainty In Wind Speed
048
Wind Direction At 10 M
049
Formal Uncertainty In Wind Direction
050
Likelihood Computed for Wind Solution
051
Antenna Polarisation
052
Total Number w.r.t. accumulation or average
053
Brightness Temperature
054
Standard Deviation Brightness Temperature
055
Antenna Polarisation
056
Total Number w.r.t. accumulation or average
057
Brightness Temperature
058
Standard Deviation Brightness Temperature
059
Number of Inner-Beam Sigma0 (fwd of sat.)
060
Latitude (Coarse Accuracy)
061
Longitude (Coarse Accuracy)
062
Attenuation Correction On Sigma-0
063
Radar Look (Azimuth) Angle
064
Radar Incidence Angle
065
Antenna Polarisation
066
Normalized Radar Cross Section
067
Kp Variance Coefficient (Alpha)
068
Kp Variance Coefficient (Beta)
069
Kp Variance Coefficient (Gamma)
070
Seawinds Sigma-0 Quality Flag
071
Seawinds Sigma-0 Mode Flag
072
Seawinds Land/Ice Surface Flag
073
Sigma-0 Variance Quality Control
Continued on next page
107
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Unit
NUMERIC
NUMERIC
NUMERIC
KG/M**2/SEC
dB
M/S
M/S
DEGREE TRUE
DEGREE TRUE
NUMERIC
M/S
M/S
DEGREE TRUE
DEGREE TRUE
NUMERIC
M/S
M/S
DEGREE TRUE
DEGREE TRUE
NUMERIC
M/S
M/S
DEGREE TRUE
DEGREE TRUE
NUMERIC
CODE TABLE
NUMERIC
K
K
CODE TABLE
NUMERIC
K
K
NUMERIC
DEGREE
DEGREE
dB
DEGREE
DEGREE
CODE TABLE
NUMERIC
NUMERIC
NUMERIC
dB
FLAG TABLE
FLAG TABLE
FLAG TABLE
NUMERIC
Descriptor
(21103)
(21120)
(21121)
(13055)
(21122)
(11012)
(11052)
(11011)
(11053)
(21104)
(11012)
(11052)
(11011)
(11053)
(21104)
(11012)
(11052)
(11011)
(11053)
(21104)
(11012)
(11052)
(11011)
(11053)
(21104)
(02104)
(08022)
(12063)
(12065)
(02104)
(08022)
(12063)
(12065)
(21110)
(05002)
(06002)
(21118)
(02112)
(02111)
(02104)
(21105)
(21106)
(21107)
(21114)
(21115)
(21116)
(08018)
(21117)
NWP SAF
SDP User Manual and
Reference Guide
Continued from previous page
Number Parameter
074
Number of Outer-Beam Sigma0 (fwd of sat.)
075
Latitude (Coarse Accuracy)
076
Longitude (Coarse Accuracy)
077
Attenuation Correction On Sigma-0
078
Radar Look (Azimuth) Angle
079
Radar Incidence Angle
080
Antenna Polarisation
081
Normalized Radar Cross Section
082
Kp Variance Coefficient (Alpha)
083
Kp Variance Coefficient (Beta)
084
Kp Variance Coefficient (Gamma)
085
Seawinds Sigma-0 Quality Flag
086
Seawinds Sigma-0 Mode Flag
087
Seawinds Land/Ice Surface Flag
088
Sigma-0 Variance Quality Control
089
Number of Inner-Beam Sigma0 (aft of sat.)
090
Latitude (Coarse Accuracy)
091
Longitude (Coarse Accuracy)
092
Attenuation Correction On Sigma-0
093
Radar Look (Azimuth) Angle
094
Radar Incidence Angle
095
Antenna Polarisation
096
Normalized Radar Cross Section
097
Kp Variance Coefficient (Alpha)
098
Kp Variance Coefficient (Beta)
099
Kp Variance Coefficient (Gamma)
100
Seawinds Sigma-0 Quality Flag
101
Seawinds Sigma-0 Mode Flag
102
Seawinds Land/Ice Surface Flag
103
Sigma-0 Variance Quality Control
104
Number of Outer-Beam Sigma0 (aft of sat.)
105
Latitude (Coarse Accuracy)
106
Longitude (Coarse Accuracy)
107
Attenuation Correction On Sigma-0
108
Radar Look (Azimuth) Angle
109
Radar Incidence Angle
110
Antenna Polarisation
111
Normalized Radar Cross Section
112
Kp Variance Coefficient (Alpha)
113
Kp Variance Coefficient (Beta)
114
Kp Variance Coefficient (Gamma)
115
Seawinds Sigma-0 Quality Flag
116
Seawinds Sigma-0 Mode Flag
117
Seawinds Land/Ice Surface Flag
118
Sigma-0 Variance Quality Control
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Unit
NUMERIC
DEGREE
DEGREE
dB
DEGREE
DEGREE
CODE TABLE
NUMERIC
NUMERIC
NUMERIC
dB
FLAG TABLE
FLAG TABLE
FLAG TABLE
NUMERIC
NUMERIC
DEGREE
DEGREE
dB
DEGREE
DEGREE
CODE TABLE
NUMERIC
NUMERIC
NUMERIC
dB
FLAG TABLE
FLAG TABLE
FLAG TABLE
NUMERIC
NUMERIC
DEGREE
DEGREE
dB
DEGREE
DEGREE
CODE TABLE
NUMERIC
NUMERIC
NUMERIC
dB
FLAG TABLE
FLAG TABLE
FLAG TABLE
NUMERIC
Table C1.1 List of data descriptors (NOAA BUFR format).
108
Descriptor
(21111)
(05002)
(06002)
(21118)
(02112)
(02111)
(02104)
(21105)
(21106)
(21107)
(21114)
(21115)
(21116)
(08018)
(21117)
(21112)
(05002)
(06002)
(21118)
(02112)
(02111)
(02104)
(21105)
(21106)
(21107)
(21114)
(21115)
(21116)
(08018)
(21117)
(21113)
(05002)
(06002)
(21118)
(02112)
(02111)
(02104)
(21105)
(21106)
(21107)
(21114)
(21115)
(21116)
(08018)
(21117)
NWP SAF
SDP User Manual and
Reference Guide
109
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Appendix C2
KNMI BUFR output file
Number Parameter
001
Satellite Identifier
002
Direction of motion of moving observation platform
003
Satellite sensor indicator
004
Wind Scatterometer GMF
005
Software Identification
006
Cross Track Resolution
007
Along Track Resolution
008
Orbit Number
009
Year
010
Month
011
Day
012
Hour
013
Minute
014
Second
015
Latitude (Coarse Accuracy)
016
Longitude (Coarse Accuracy)
017
Time Difference Qualifier
018
Second
019
Along Track Row Number
020
Cross Track Cell Number
Continued on next page
110
Unit
CODE TABLE
DEGREE TRUE
CODE TABLE
CODE TABLE
NUMERIC
M
M
NUMERIC
YEAR
MONTH
DAY
HOUR
MINUTE
SECOND
DEGREE
DEGREE
CODE TABLE
SECOND
NUMERIC
NUMERIC
Descriptor
(01007)
(01012)
(02048)
(21119)
(25060)
(02026)
(02027)
(05040)
(04001)
(04002)
(04003)
(04004)
(04005)
(04006)
(05002)
(06002)
(08025)
(04006)
(05034)
(06034)
NWP SAF
SDP User Manual and
Reference Guide
Continued from previous page
021
Model Wind Direction At 10 M
022
Model Wind Speed At 10 M
023
Total Number of Sigma0 Measurements
024
Probability of Rain
025
Seawinds NOF Rain Index
026
Intensity Of Precipitation
027
Attenuation Correction On Sigma-0 (from Tb)
028
Antenna Polarisation
029
Total Number w.r.t. accumulation or average
030
Brightness Temperature
031
Standard Deviation Brightness Temperature
032
Antenna Polarisation
033
Total Number w.r.t. accumulation or average
034
Brightness Temperature
035
Standard Deviation Brightness Temperature
036
Number of Inner-Beam Sigma0 (forward of satellite)
037
Latitude (Coarse Accuracy)
038
Longitude (Coarse Accuracy)
039
Attenuation Correction On Sigma-0
040
Radar Look (Azimuth) Angle
041
Radar Incidence Angle
042
Antenna Polarisation
043
SeaWinds Normalized Radar Cross Section
044
Kp Variance Coefficient (Alpha)
045
Kp Variance Coefficient (Beta)
046
Kp Variance Coefficient (Gamma)
047
Seawinds Sigma-0 Quality Flag
048
Seawinds Sigma-0 Mode Flag
049
Seawinds Land/Ice Surface Flag
050
Sigma-0 Variance Quality Control
051
Number of Outer-Beam Sigma0 (forward of satellite)
052
Latitude (Coarse Accuracy)
053
Longitude (Coarse Accuracy)
054
Attenuation Correction On Sigma-0
055
Radar Look Angle
056
Radar Incidence Angle
057
Antenna Polarisation
058
Normalized Radar Cross Section
059
Kp Variance Coefficient (Alpha)
060
Kp Variance Coefficient (Beta)
061
Kp Variance Coefficient (Gamma)
062
Seawinds Sigma-0 Quality
063
Seawinds Sigma-0 Mode
064
Seawinds Land/Ice Surface Type
065
Sigma-0 Variance Quality Control
066
Number of Inner-Beam Sigma0 (aft of satellite)
067
Latitude (Coarse Accuracy)
068
Longitude (Coarse Accuracy)
069
Attenuation Correction On Sigma-0
Continued on next page
111
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
DEGREE TRUE
M/S
NUMERIC
NUMERIC
NUMERIC
KG/M**2/SEC
dB
CODE TABLE
NUMERIC
K
K
CODE TABLE
NUMERIC
K
K
NUMERIC
DEGREE
DEGREE
dB
DEGREE
DEGREE
CODE TABLE
NUMERIC
NUMERIC
NUMERIC
dB
FLAG TABLE
FLAG TABLE
FLAG TABLE
NUMERIC
NUMERIC
DEGREE
DEGREE
dB
DEGREE
DEGREE
CODE TABLE
NUMERIC
NUMERIC
NUMERIC
dB
FLAG TABLE
FLAG TABLE
FLAG TABLE
NUMERIC
NUMERIC
DEGREE
DEGREE
dB
(11081)
(11082)
(21103)
(21120)
(21121)
(13055)
(21122)
(02104)
(08022)
(12063)
(12065)
(02104)
(08022)
(12063)
(12065)
(21110)
(05002)
(06002)
(21118)
(02112)
(02111)
(02104)
(21105)
(21106)
(21107)
(21114)
(21115)
(21116)
(08018)
(21117)
(21113)
(05002)
(06002)
(21118)
(02112)
(02111)
(02104)
(21105)
(21106)
(21107)
(21114)
(21115)
(21116)
(08018)
(21117)
(21113)
(05002)
(06002)
(21118)
NWP SAF
SDP User Manual and
Reference Guide
Continued from previous page
070
Radar Look Angle
071
Radar Incidence Angle
072
Antenna Polarisation
073
Normalized Radar Cross Section
074
Kp Variance Coefficient (Alpha)
075
Kp Variance Coefficient (Beta)
076
Kp Variance Coefficient (Gamma)
077
Seawinds Sigma-0 Quality
078
Seawinds Sigma-0 Mode
079
Seawinds Land/Ice Surface Type
080
Sigma-0 Variance Quality Control
081
Number of Outer-Beam Sigma0 (aft of satellite)
082
Latitude (Coarse Accuracy)
083
Longitude (Coarse Accuracy)
084
Attenuation Correction On Sigma-0
085
Radar Look Angle
086
Radar Incidence Angle
087
Antenna Polarisation
088
Normalized Radar Cross Section
089
Kp Variance Coefficient (Alpha)
090
Kp Variance Coefficient (Beta)
091
Kp Variance Coefficient (Gamma)
092
Seawinds Sigma-0 Quality
093
Seawinds Sigma-0 Mode
094
Seawinds Land/Ice Surface Type
095
Sigma-0 Variance Quality Control
096
Wind Vector Cell Quality
097
Number of Vector Ambiguities
098
Index of Selected Wind vector
099
Delayed Description Replication Factor
100
Wind Speed at 10 m
101
Wind Direction at 10 m
102
Backscatter Distance
103
Likelihood Computed for Solution
etc.
etc.
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
DEGREE
DEGREE
CODE TABLE
NUMERIC
NUMERIC
NUMERIC
dB
FLAG TABLE
FLAG TABLE
FLAG TABLE
NUMERIC
NUMERIC
DEGREE
DEGREE
dB
DEGREE
DEGREE
CODE TABLE
NUMERIC
NUMERIC
NUMERIC
dB
FLAG TABLE
FLAG TABLE
FLAG TABLE
NUMERIC
FLAG TABLE
NUMERIC
NUMERIC
NUMERIC
M/S
DEGREES TRUE
NUMERIC
NUMERIC
etc.
(02112)
(02111)
(02104)
(21105)
(21106)
(21107)
(21114)
(21115)
(21116)
(08018)
(21117)
(21113)
(05002)
(06002)
(21118)
(02112)
(02111)
(02104)
(21105)
(21106)
(21107)
(21114)
(21115)
(21116)
(08018)
(21117)
(21216)
(21101)
(21102)
(31001)
(11012)
(11011)
(21226)
(21104)
etc.
Table C2.1 List of data descriptors (KLNMI BUFR format). Numbers 100 to 103 (yellow background)
form the first element of the generalized wind section. They may be repeated up to 144 times.
112
NWP SAF
SDP User Manual and
Reference Guide
113
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Appendix D
ECMWF BUFR data routines
Function
Description
bbuprs0.F
Print BUFR section 0 (?)
bbuprs1.F
Print BUFR section 1 (?)
bbuprs2.F
Print BUFR section 2 (?)
bbuprs3.F
Print BUFR section 3 (?)
bbuprt.F
Print BUFR (?)
bbuprtbox.F Print BUFR box (?)\\
buaug.F
Update augmented BUFR table B
bubox.F
??
bucomp.F
Pack number of subsets in a compressed form
bucrkey.F
Extract elements needed for RDB key definition(update)
bucrekey.F
Extract elements needed for RDB key definition
buedd.F
Expand section 3 of BUFR message
buens0.F
Pack section 0 of BUFRsage
buens1.F
Pack section 1 of BUFR message
buens2.F
Pack section 2 of BUFR message
buens3.F
Pack section 3 of BUFR message
buens4.F
Pack preliminary items/data of sect.4 of BUFR message
buens5.F
Pack sect.5 of BUFR message
buepmrk.F
Process marker operator, replace with table B descriptor
buepmrkc.F
Process marker operator, replace with table B descriptor
buepwt.F
Updates working tables (name, unit, scale, ref, data width)
buepwtc.F
Updates working tables (name, unit, scale, ref, data width)
buerr.F
Print error code
buetab.F
Load BUFR table B, D and C according to BUFR code
buetd.F
Expand sect.3 of BUFR message
Continued on next page
114
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Continued from previous page
Function
Description
buetdr.F
Solve BUFR table D reference
buevar.F
Initialize constants and variables
buexs0.F
Expand section 0 of BUFR message
buexs1.F
Expand section 1 of BUFR message
buexs2.F
Expand section 2 of BUFR message
buexs3.F
Expand section 3 of BUFR message
buexs3p.F
Expand section 3 of BUFR message (preliminary items)
buexs4.F
Expand section 4 of BUFR message
buexs5.F
Expand section 5 of BUFR message
bufren.F
Encode BUFR message
bufrex.F
Decode BUFR message into fully expanded form
bugbts.F
Load BUFR table B, D and C according to BUFR code
bugetbm.F
Create bit map to resolve marker operators
buivar.F
Initialize constants and variables
bunexs.F
Sets word/bit pointers at the start of next BUFR sect
bunpck.F
Unpack bit string
bunpks.F
Unpack bit string of KSIZE bits
buoctn.F
Calculate number of octets from bit position
buoper.F
Process BUFR operator
buoperc.F
Process BUFR operator
bupck.F
Pack value *KS* in *KSI* bits
bupkey.F
Pack local ECMWF information (rdb key)
bupks.F
Pack bit string of KSIZE bits
bupmrk.F
Process marker operator, replace with table B descriptor
buprco.F
Process BUFR operator
buprq.F
Sets variable KPMISS,KPRUS into common block
buprs0.F
Print section 0 of BUFR message
buprs1.F
Print section 1 of BUFR message
buprs2.F
Print section 2 of BUFR message (expanded RDB key)
buprs3.F
Print section 3 of BUFR message
buprt.F
Print expanded BUFR message
buprtbox.F
Print boxed expanded BUFR message
burep.F
Resolve data descriptor replication problem
burepc.F
Resolve data descriptor replication problem
burqc.F
Create parameters needed for partial expansion of BUFR
burquc.F
Create parameters needed for partial expansion of BUFR
bus012.F
Expands sections 0, 1, and 2 of BUFR message
busel.F
Returns Data Descriptors as in Section 3 of BUFR
buset.F
Set flags in common block (?)
busrp.F
Resolve data descriptor replication problem
busrq.F
Set BUFR table B references for partial expansion
bustdr.F
Solve BUFR table D reference
buuatb.F
Update augmented BUFR table B
buukey.F
Expands local ECMWF information from sect.2
buunp.F
Unpack bit string of KSIZE bits
buunps.F
Unpack bit string of KSIZE bits
buupwt.F
Updates working tables (name, unit, scale, ref, data width)
buxdes.F
Expand data descriptors to show user's template
Continued on next page
115
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Continued from previous page
Function
Description
fmmh.F
Find max/min latitude/longitude
setlalo.F
Return indices for latitude and longitude
Table D.1 List of ECMWF BUFR routines.
116
NWP SAF
SDP User Manual and
Reference Guide
117
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
NWP SAF
SDP User Manual and
Reference Guide
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
Appendix E
Acronyms
Name
AR
BUFR
C-band
ECMWF
EUMETSAT
GMF
HIRLAM
KNMI
Ku-band
L1b
LUT
MLE
MSS
NRCS
NWP
QC
RFSCAT
RMS
SAF
WVC
Description
Ambiguity Removal
Binary Universal Form for the Representation of data
Radar wavelength at about 5 cm
European Center for Medium-range Weather Forecasts
European Organization for the Exploitation of Meteorological Satellites
Geophysical model function
High resolution Local Area Model
Koninklijk Nederlands Meteorologisch Instituut (Royal Dutch Meteorological Institute)
Radar wavelength at about 2 cm
Level 1b product
Look up table
Maximum Likelihood Estimator
Multiple Solution Scheme
Normalized radar cross-section (σ0)
Numerical Weather Prediction
Quality Control
Rotating fan beam scatterometer
Root mean square
Satellite Application Facility
Wind vector cell, also called node or cell
Table E.1 List of acronyms.
118
NWP SAF
SDP User Manual and
Reference Guide
119
Doc ID : NWPSAF-KN-UD-002
Version : 1.5
Date
: 24-07-2007
">