On This Page:
Welcome & Description
Welcome to the Unified Post Processor (UPP) page. The UPP is a software package designed to generate useful products from raw model output. It:
- Reads and processes GFS and Limited Area Model (LAM) data from the FV3 dynamical core
- Generates output in GRIB2 format
- Uses MPI parallel code
- Produces hundreds of products like those used operationally on the same operational grids
The UPP was developed at the National Centers for Environmental Prediction (NCEP) and is used operationally for several models maintained by NCEP. It is also used in the Unified Forecast System (UFS), including the Rapid Refresh Forecast System (RRFS), the Hurricane Analysis and Forecast System (HAFS), the Medium-Range Weather (MRW) Application, and the Short-Range Weather (SRW) Application. For more information regarding the UFS, please visit the UFS Portal.
Getting Started
Before running the UPP, users should determine which of the four levels of support is applicable to their system. Generally, Level 1 & 2 systems are restricted to those with access through NOAA and its affiliates. These systems are named (e.g., Hera, Hercules). The UPP should be able to run on any UNIX-based platform (e.g., Mac, Linux) that has the prerequisite libraries installed. However, this has not been tested.
The Building, Running, & Testing chapter of the UPP User’s Guide is an excellent place for new users to begin. It provides details on how to clone the UPP, build it, and run the post-processor using the latest code in the develop branch. Users may prefer to use the v11.0.0 User’s Guide and code, which provides similar instructions for the last stable standalone release of the UPP.
Documentation & User Support
The UPP User’s Guide has the most comprehensive information on the UPP, including links to more thorough technical documentation for its components. Users may need different versions of the User’s Guide depending on their goals:
Version
Description
Documentation for the head of the develop branch. This may have gaps and errors.
Documentation for the most recent standalone release (v11.0.0).
Documentation for the v10.1.0 release.
Developer Support
- The UPP Directory Structure page provides information on the structure of the UPP repository.
The Code Contribution Guidelines describe how to contribute code to the UPP repository.
The Testing the UPP chapter of the User’s Guide provides information on how to run regression tests on UPP code.
Releases
The latest standalone release of the UPP is v11.0.0. See the Releases page for more information on current and past releases.
Release Date: 6/15/2022
Release Description: This release represents a major effort to refactor the UPP code to implement a 2-dimensional decomposition capability. This development was led by UPP developers at NOAA’s Environmental Modeling Center (EMC) with funding through the Hurricane Supplemental funding line. Please see the User’s Guide below for details on how to use this capability.
- Capability added to decompose grids in both x- and y-directions.
- 1D (y-direction only) decomposition is still available.
- Identical results are expected for 1D vs 2D.
- Testing indicated no deterioration in run time and in many cases reduction in run time for existing configurations/applications.
- Further timing improvement expected especially for larger domains.
- Support for this feature has been implemented in offline standalone UPP and inline UPP.
- 2D decomposition is only supported for UFS-based model outputs.
Documentation:
Release Date: 5/26/2022
Release Description: This release is from the release/public-v3 branch and can be used in standalone mode or with the UFS Short-Range Weather (SRW) Application v2.0.0 release.
- Bug fix for SLLEVEL bound issue when not RUC LSM (#464 Commit f22a590)
- Updated Doxygen (#460, #464, #465, #467, #469, #470, #473, #474, #476, #477, #479, #480, #487)
- Unify global and regional FV3 read interfaces for offline post (#453 Commit 5bdb289)
- Add ability to compile script to use non-Intel compilers; add Cheyenne modulefiles for GNU and Intel (#468 Commit af62bd2)
- Bug fix in UPP build script on WCOSS2 Cactus (#481 Commit 95f8383)
- Updated README (#486 Commit b495345)
- Fix a bug in AOD calculation (#489 Commit 44edaf7)
- Update GFS itag files for inline post (#508 Commit fbd41a5)
- Updates to User’s Guide documentation (#475)
- Updates to run_upp script (#492)
Documentation:
Other Links
- The spack-stack repository contains the prerequisite software needed to run the UPP and other UFS software. This software is preinstalled on Level 1 systems (e.g., Hera, Orion). Users can view the spack-stack User’s Guide to install it on other systems.
- UPP README.md file
- GRIB2/WGRIB2 Information: UPP outputs data in GRIB2 format, and WGRIB2 is a utility for reading/writing GRIB2 files. Interested users/developers can find additional information at the following links:
Get Started with the UPP
Resources
Public Release Code
v11.0.0 (latest standalone release)
For Developers
Useful Links
Unified Post Processor (UPP)
Welcome to the Unified Post Processor (UPP) page. The UPP is a software package designed to generate useful products from raw model output. It:
- Reads and processes GFS and Limited Area Model (LAM) data from the FV3 dynamical core
- Generates output in GRIB2 format
- Uses MPI parallel code
- Produces hundreds of products like those used operationally on the same operational grids
The UPP was developed at the National Centers for Environmental Prediction (NCEP) and is used operationally for several models maintained by NCEP. It is also used in the Unified Forecast System (UFS), including the Rapid Refresh Forecast System (RRFS), Hurricane Analysis and Forecast System (HAFS), Medium-Range Weather (MRW) Application, and Short-Range Weather (SRW) Application. For more information regarding the UFS, please visit Unified Forecast System (UFS).
The Earth Prediction Innovation Center (EPIC) provides assistance with facilitating research-to-operations (R2O) code development and encourages community members to contribute to the UPP.
Links
- The UPP GitHub repository hosts UPP code.
- User’s Guides can be found here:
- The UPP wiki contains information and links related to releases, documentation, and code development.
- Code contributors should refer to the UPP Code Development Guidelines.
- Spack-stack: The required libraries for building the UPP are available via spack-stack. Spack-stack provides a Spack-based method for building the software stack required for numerical weather prediction (NWP) tools. Users working on unsupported platforms will need to install the spack-stack on their system following the instructions in the spack-stack User’s Guide.
- GRIB2/WGRIB2 Information: UPP outputs data in GRIB2 format, and WGRIB2 is a utility for reading/writing GRIB2 files. Interested users/developers can find additional information at the following links:
Releases
| Version | Release & Release Notes | Documentation | Date |
|---|---|---|---|
|
11.0.0 |
2022-06-15 |
||
|
10.1.0 |
2022-05-26 |
User Guide
This is the documentation for the develop branch of the UPP. It may be slightly behind the head of develop branch code but generally reflects the most recent additions to the UPP. For release documentation, see the “Releases” section above.
User Support
User support includes:
- Answering user questions regarding the UPP
- Maintaining the released UPP code and its documentation
- Assistance facilitating research-to-operations code development
For frequently asked questions, users can refer to the UPP section of EPIC’s Technical FAQs page.
Users should direct general UFS questions to the UFS Community GitHub Discussions page. For UPP questions, including build and runtime issues, users should post their questions to the UPP GitHub Discussions page. Begin a new discussion by selecting “New Discussion” using the Q&A category.
Users or developers wishing to contribute innovations to the UPP should reference the wiki available through the GitHub repository.
Acknowledgement:
If significant help was provided via the EPIC UPP support team for work resulting in a publication, please acknowledge EPIC’s UPP support team. For referencing the UPP User’s Guide, please use the following citation (changing version numbers if appropriate):
UPP User’s Guide v11.0.0, 29 pp. [available online at https://upp.readthedocs.io/en/upp_v11.0.0/]
Other
At this time, EPIC provides community support for UFS-based forecasts. Support for WRF-based applications has been retired; please see the archived UPP for WRF website for information. Users may also find support for WRF-related questions on the UPP section of the WRF/MPAS forum.
Frequently Asked Questions (FAQ)
The UPP is compatible with NetCDF4 when used on UFS model output.
We are not able to support all platform and compiler combinations out there but will try to help with specific issues when able. Users may request support on the UPP GitHub Discussions page. We always welcome and are grateful for user-contributed configurations.
Currently, the stand-alone release of the UPP can be utilized to output satellite fields if desired. The UPP documentation lists the grib2 fields, including satellite fields, produced by the UPP. After selecting which fields to output, the user must adjust the control file according to the instructions in the UPP documentation to output the desired fields. When outputting satellite products, users should note that not all physics options are supported for outputting satellite products. Additionally, for regional runs, users must ensure that the satellite field of view overlaps some part of their domain.
Most UFS application releases do not currently support this capability, although it is available in the Short-Range Weather (SRW) Application. This SRW App pull request (PR) added the option for users to output satellite fields using the SRW App. The capability is documented in the SRW App User’s Guide.
If the desired variable is already available in the UPP code, then the user can simply add that variable to the postcntrl.xml file and remake the postxconfig-NT.txt file that the UPP reads. Please note that some variables may be dependent on the model and/or physics used.
If the desired variable is not already available in the UPP code, it can be added following the instructions for adding a new variable in the UPP User’s Guide.
There are a few possible reasons why a requested variable might not appear in the UPP output:
- The variable may be dependent on the model.
- Certain variables are dependent on the model configuration. For example, if a variable depends on a particular physics suite, it may not appear in the output when a different physics suite is used.
- The requested variable may depend on output from a different field that was not included in the model output.
If the user suspects that the UPP failed (e.g., no UPP output was produced or console output includes an error message like mv: cannot stat `GFSPRS.GrbF00`: No such file or directory), the best way to diagnose the issue is to consult the UPP runtime log file for errors. When using the standalone UPP with the run_upp script, this log file will be located in the postprd directory under the name upp.fHHH.out, where HHH refers to the 3-digit forecast hour being processed. When the UPP is used with the SRW App, the UPP log files can be found in the experiment directory under log/run_post_fHHH.log.
UPP output is in standard grib2 format and can be interpolated to another grid using the third-party utility wgrib2. Some basic examples can also be found in the UPP User’s Guide.
This may be a memory issue; try increasing the number of CPUs or spreading them out across nodes (e.g., increase ptiles). We also know of one version of MPI (mpich v3.0.4) that does not work with UPP. A work-around was found by modifying the UPP/sorc/ncep_post.fd/WRFPOST.f routine to change all unit 5 references (which is standard I/O) to unit 4 instead.
For re-gridding grib2 unipost output, the wgrib2 utility can be used. See complete documentation on grid specification with examples of re-gridding for all available grid definitions. The Regridding section of the UPP User’s Guide also gives examples (including an example from operations) of using wgrib2 to interpolate to various common grids.
This warning appears for some platforms/compilers because a call in the nemsio library is never used or referenced for a serial build. This is just a warning and should not hinder a successful build of UPP or negatively impact your UPP run.
This error message is displayed when using more recent versions of the wgrib2 utility on files for forecast hour zero that contain accumulated or time-averaged fields. This is due to the newer versions of wgrib2 no longer allowing for the n parameter to be zero or empty.
Users should consider using a separate control file (e.g., postcntrl_gfs_f00.xml) for forecast hour zero that does not include accumulated or time-averaged fields, since they are zero anyway. Users can also continue to use an older version of wgrib2; v2.0.4 is the latest known version that does not result in this error.
