diff --git a/.codecov.yml b/.codecov.yml index 9bf04a5..775f7fb 100644 --- a/.codecov.yml +++ b/.codecov.yml @@ -3,6 +3,7 @@ coverage: - "R/sounding_wyoming.R" - "R/imgw_read.R" - "R/onAttach.R" + - "R/meteo_shortening_imgw.R" status: patch: default: diff --git a/NEWS.md b/NEWS.md index a6e735f..3fd7bc6 100644 --- a/NEWS.md +++ b/NEWS.md @@ -50,6 +50,7 @@ * `meteo_imgw_daily` fix for current year data retrieval from IMGW-PIB repository * `imgw_read` - fix for wrongly formatted input CSVs for hydrological data in the IMGW-PIB repository + # climate 1.2.4 * R version >= 4.1 is now required @@ -58,6 +59,7 @@ * Adding `hydro_imgw_datastore()` to retrieve data for hydrological stations from IMGW-PIB repository datastore (telemetry system) * Adding `stations_hydro_imgw_telemetry()` to retrieve coordinates metadata for hydrological telemetry stations from IMGW-PIB + # climate 1.2.3 * Fixes and modifications for `meteo_imgw_daily()` and `meteo_imgw_hourly()` due @@ -69,6 +71,7 @@ to changes in the IMGW-PIB meteorological datasets since 2024 * Logging errors in package environment and providing summary at the end of downloading instead of instant messaging * Minor changes in unit-tests to adjust for most recent code changes + # climate 1.2.2 * Fixes and modifications for `hydro_imgw()` set of functions due to changes in the IMGW-PIB hydrological datasets @@ -79,23 +82,27 @@ to changes in the IMGW-PIB meteorological datasets since 2024 * Resolving date formatting for hydrological data - the Date column represents calendar date * Corrected logic in downloading hourly OGIMET dataset + # climate 1.2.1 * Major fixes for adjusting code to stay in line with CRAN policies * Corrected duplicated column names for IMGW-PIB stations * Adjusted encoding changes and documentation updates in `meteo_imgw_telemetry_stations()` + # climate 1.2.0 * Corrected encoding problems for some of platforms where IMGW-PIB metadata were not parsed correctly * Added location of IMGW-PIB telemetry stations within `meteo_imgw_telemetry_stations()` * Minor bug fixes + # climate 1.1.1 * Fix problems with downloading `precip` dataset from IMGW-PIB repository after recent changes in metadata * Bug fix for `ogimet_daily` if data contains more than one year + # climate 1.1.0 * A new approach for handling CRAN policy for resolving problems if network issues are detected or some of the external services are temporarily down. @@ -112,12 +119,14 @@ to changes in the IMGW-PIB meteorological datasets since 2024 * units added to column labels for `nearest_stations_` set of functions instead of column names for easier processing (e.g. [km] are visible now only as attributes) * new functions follow lintr settings + # climate 1.0.4 * Function `spheroid_dist` added to improve accuracy of calculations between points, but also avoid installing GIS dependencies (thanks to @kadyb) * Function `nearest_stations_imgw()` now uses the Vincenty's formula in `spheroid_dist` to calculate the distance between points on a spheroid, not the Euclidean distance (previously results were inaccurate for some specific cases) * minor bugs fixes and improvements + # climate 1.0.3 * Adding possibility to download BUFR vertical sounding dataset from `http://weather.uwyo.edu/upperair/sounding.html`; extra information with supporting example added to the `sounding_wyoming`'s documentation @@ -125,29 +134,35 @@ to changes in the IMGW-PIB meteorological datasets since 2024 * `ogimet_daily` automatically detects column names to be used for extraction in final data.frame; extra debugging info when temperature or precipitation columns are missing * minor changes in documentation (e.g. updated links to NOAA website) + # climate 1.0.1 * Adding `data.table` package to read CP1250 on machines that do not support this encoding (translit used instead) + # climate 0.9.9 * Changing URL `danepubliczne.imgw.pl` to `dane.imgw.pl` where needed * Fixing minor ogimet and IMGW bugs * Do not stop downloading data from `ogimet.com`, instead check for all available data in given period of time + # climate 0.9.8 * Adding informative message if problems with NOAA hourly dataset occur * Informative message if problems with downloading detected for non-IMGW dataset + # climate 0.9.7 * stop working if no internet connection detected + # climate 0.9.6 * Adding nearest_stations_noaa for NOAA hourly dataset + # climate 0.9.5 * Following CRAN policies @@ -155,11 +170,13 @@ to changes in the IMGW-PIB meteorological datasets since 2024 * RCurl dependency removal * CO2 & Wyoming examplary data can be loaded offline + # climate 0.9.4 * New dataset: * Hourly NOAA ISH (Integrated Surface Hourly) data - global meteorological dataset dated back up to 1900 + # climate 0.9.3 * Bug fixes @@ -167,11 +184,13 @@ to changes in the IMGW-PIB meteorological datasets since 2024 * New datasets: * CO2 concentration from Mauna Loa observatory + # climate 0.9.2 * Bug fixes * #26 + # climate 0.9.1 * climate is independent of imgw package @@ -179,14 +198,17 @@ to changes in the IMGW-PIB meteorological datasets since 2024 * #24 * restored possibility of downloading single station from Polish (IMGW) repository + # climate 0.3 * improves API + # climate 0.2 * splits imgw into two packages: imgw and climate + # imgw 0.1.1 * New datasets @@ -201,6 +223,7 @@ to changes in the IMGW-PIB meteorological datasets since 2024 * Bug fixes * Fixes a bug in the `hydro_daily()` that prevented from merging more than 1 dataset + # imgw 0.1.0 * Deploying the package on CRAN! diff --git a/R/compute_relative_humidity.R b/R/compute_relative_humidity.R index d0cc414..7a2c150 100644 --- a/R/compute_relative_humidity.R +++ b/R/compute_relative_humidity.R @@ -7,9 +7,9 @@ #' @param dpt2m Numeric vector. Dew-point temperature (2 m) in degrees Celsius. #' Must be the same length as `t2m`. #' -#' @return Numeric vector of relative humidity values in percent (0–100). +#' @return Numeric vector of relative humidity values in percent (0-100). #' Returns `NA` where either input is `NA`. Values are not clamped, so -#' rounding errors may produce results marginally outside 0–100. +#' rounding errors may produce results marginally outside 0-100. #' #' @details #' The August-Roche-Magnus approximation is: diff --git a/R/nearest_stations_ogimet.R b/R/nearest_stations_ogimet.R index c29b27c..5637e8d 100644 --- a/R/nearest_stations_ogimet.R +++ b/R/nearest_stations_ogimet.R @@ -29,7 +29,7 @@ #' nearest_stations_ogimet = function(country = "United Kingdom", - date = Sys.Date(), + date = Sys.Date() - 1, add_map = FALSE, point = c(2, 50), no_of_stations = 10, @@ -113,6 +113,7 @@ nearest_stations_ogimet_bp = function(country = country, # run only if downloaded file is valid if (!is.na(body) & (object.size(body) > 500)) { + # nocov start a = paste(body, sep = "", collapse = "") b = strsplit(a, "Decoded synops since") b1 = lapply(b, function(x) substr(iconv(x, from = 'UTF-8', to = 'ASCII//TRANSLIT'), 1, 400)) @@ -159,6 +160,7 @@ nearest_stations_ogimet_bp = function(country = country, res = data.frame(wmo_id = res1[, 4], station_names = station_names, lon = lon, lat = lat, alt = as.numeric(res1[, 3])) result = rbind(result,res) + # nocov end } else { result = NULL message(paste("Wrong name of a country. Please check countries names at @@ -168,7 +170,7 @@ nearest_stations_ogimet_bp = function(country = country, } if (!is.null(result)) { - + # nocov start point = as.data.frame(t(point)) names(point) = c("lon", "lat") distmatrix = rbind(point, result[, 3:4]) @@ -187,7 +189,6 @@ nearest_stations_ogimet_bp = function(country = country, attr(result[["lat"]], "label") = "decimal degrees" attr(result[["alt"]], "label") = "metres AGL" - # nocov start if (add_map == TRUE) { if (!requireNamespace("maps", quietly = TRUE)) { stop("package maps required, please install it first") diff --git a/R/ogimet_daily.R b/R/ogimet_daily.R index 7811213..a17ba94 100644 --- a/R/ogimet_daily.R +++ b/R/ogimet_daily.R @@ -67,7 +67,7 @@ ogimet_daily_bp = function(date = date, "Wind_Int." = character(), "Wind_Gust" = character(), "Pres.s.lev" = character(), - "Precmm" = character(), + "Prec" = character(), "SunD-1" = character(), "SnowDep" = character(), "TotClOct" = character(), diff --git a/R/sounding_wyoming.R b/R/sounding_wyoming.R index de40316..3c2c3e2 100644 --- a/R/sounding_wyoming.R +++ b/R/sounding_wyoming.R @@ -1,14 +1,15 @@ #' Sounding data #' -#' Downloading the measurements of the vertical profile of atmosphere (also known as sounding data). Data can be retrieved using TEMP and BUFR sounding formatting. +#' Downloading the measurements of the vertical profile of atmosphere (also known as sounding data). +#' Data can be retrieved using TEMP and BUFR sounding formatting. By default automatic detection of input format is used. #' -#' @param wmo_id international WMO station code (World Meteorological Organization ID); For Polish stations: Leba - 12120, Legionowo - 12374, Wrocław- 12425 -#' @param yy year - single number -#' @param mm month - single number denoting month -#' @param dd day - single number denoting day -#' @param hh hour - single number denoting initial hour of sounding; for most stations this measurement is done twice a day (i.e. at 12 and 00 UTC), sporadically 4 times a day -#' @param min minute - single number denoting initial minute of sounding; applies only to BUFR soundings. -#' @param source - "BUFR", "TEMP" or "DETECT". By default "DETECT" is used. +#' @param wmo_id international WMO station code (World Meteorological Organization ID); For Polish stations: Leba - 12120, Legionowo - 12374, Wrocław - 12425 +#' @param yy year - calendar year +#' @param mm month - calendar month +#' @param dd day - calendar day +#' @param hh hour - hour of sounding; for most stations measurements are performed twice a day (i.e. at 12 and 00 UTC), sporadically 4 times a day +#' @param min minute - minute of sounding; Default 00; Other values applies only to BUFR soundings. +#' @param source - optional; `AUTODETECT`, `BUFR` (raw; most detailed) or `TEMP` (FM-35) input format. By default "AUTODETECT" is used which means that the function will try to detect the format automatically. If a station use only a specific format that is known you can specify it explicitly. #' @param allow_failure logical - whether to proceed or stop on failure. By default set to TRUE (i.e. don't stop on error). For debugging purposes change to FALSE #' @importFrom utils read.fwf #' @return Returns two lists with values described at: weather.uwyo.edu ; The first list contains: @@ -35,36 +36,27 @@ #' #' @examples #' \donttest{ -#' ############################################################################## #' # download data for Station 45004 starting 1120Z 11 Jul 2021; Kowloon, HONG KONG, CHINA -#' # using TEMP and BUFR sounding formats -#' ############################################################################## -#' TEMP = sounding_wyoming(wmo_id = 45004, yy = 2021, mm = 07, dd = 17, -#' hh = 12, min = 00, source = "TEMP") -#' head(TEMP[[1]]) -#' -#' BUFR = sounding_wyoming(wmo_id = 45004, yy = 2021, mm = 07, dd = 17, -#' hh = 12, min = 00, source = "BUFR") -#' head(BUFR[[1]]) +#' # using AUTODETECT, TEMP and BUFR sounding formats #' +#' # autodect input format: +#' sounding_auto = sounding_wyoming(wmo_id = 45004, +#' yy = 2021, mm = 07, dd = 17, hh = 12) #' -#' ############################################################################## -#' ### example with a random date to download sounding from LEBA, PL station: ### -#' ############################################################################## -#' -#' # no "source" argument is needed as the function will detect format automatically -#' profile = sounding_wyoming(wmo_id = 12120, -#' yy = sample(2000:2019,1), -#' mm = sample(1:12,1), -#' dd = sample(1:20,1), -#' hh = 0) -#' # plot(profile[[1]]$HGHT, profile[[1]]$PRES, type = 'l') +#' # temp (fm35) input format: +#' sounding_temp = sounding_wyoming(wmo_id = 45004, +#' yy = 2021, mm = 07, dd = 17, hh = 12, +#' source = "TEMP") +#' # bufr input format: +#' sounding_bufr = sounding_wyoming(wmo_id = 45004, +#' yy = 2021, mm = 07, dd = 17, hh = 12, min = 00, +#' source = "BUFR") #' } #' sounding_wyoming = function(wmo_id, yy, mm, dd, hh, min = 00, - source = "DETECT", + source = "AUTODETECT", allow_failure = TRUE) { if (allow_failure) { diff --git a/R/stations_ogimet.R b/R/stations_ogimet.R index d289b91..156d588 100644 --- a/R/stations_ogimet.R +++ b/R/stations_ogimet.R @@ -96,6 +96,7 @@ stations_ogimet_bp = function(country = country, date = date, add_map = add_map) res = suppressWarnings(do.call("rbind", strsplit(res, " "))) res1 = matrix(res[, c(1, 3, 5:7)], ncol = 5) + # nocov start lat = suppressWarnings( as.numeric(substr(res1[, 1], 1, 2)) + (as.numeric(substr(res1[, 1], 4, 5)) / 100) * 1.6667 @@ -144,6 +145,7 @@ if (!is.null(res)) { col = 'grey70', cex = 0.6) maps::map(add = TRUE) } + # nocov end } # end of checking if res is NULL return(res) diff --git a/README.md b/README.md index 5cafdf2..1497d32 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,7 @@ The stable release of the **climate** package from the [CRAN](https://CRAN.R-pro install.packages("climate") ``` -It is highly recommended to install the most up-to-date development version of **climate** from [GitHub](https://github.com/bczernecki/climate) with: +It is also possible to install the most up-to-date development version of **climate** from [GitHub](https://github.com/bczernecki/climate) with: ``` r library(remotes) @@ -42,42 +42,50 @@ install_github("bczernecki/climate") ### Meteorological data -- 🌍 **meteo_ogimet()** - Downloading hourly and daily meteorological data from the SYNOP stations available in the ogimet.com collection. +- 🌍 [**meteo_ogimet**](https://bczernecki.github.io/climate/reference/meteo_ogimet.html) - +Downloading hourly and daily meteorological data from the SYNOP stations available in the ogimet.com collection. Any meteorological (aka SYNOP) station working under the World Meteorological Organization framework after year 2000 should be accessible. Two backends are available and selected automatically: raw **SYNOP decoding** (`source = "synop"`, default for `interval = "hourly"`) and **HTML scraping** (`source = "html"`, default for `interval = "daily"`). Country-level bulk downloads are supported via the `country_name` argument (SYNOP backend only). -- 🌍 **meteo_noaa_hourly()** - Downloading hourly NCEI/NOAA Integrated Surface Hourly (ISH) meteorological data - Some stations have > 100 years long history of observations +- 🌍 [**meteo_noaa_hourly**](https://bczernecki.github.io/climate/reference/meteo_noaa_hourly.html) - +Downloading hourly NCEI/NOAA Integrated Surface Hourly (ISH) meteorological data - Some stations have > 100 years long history of observations -- 🌍 **meteo_noaa_co2()** - Downloading monthly CO2 measurements from Mauna Loa Observatory +- 🌍 [**meteo_noaa_co2**](https://bczernecki.github.io/climate/reference/meteo_noaa_co2.html)] - +Downloading monthly CO2 measurements from Mauna Loa Observatory -- 🌍 **sounding_wyoming()** - Downloading measurements of the vertical profile of atmosphere (aka rawinsonde data) +- 🌍 [**sounding_wyoming**](https://bczernecki.github.io/climate/reference/sounding_wyoming.html) - +Downloading measurements of the vertical profile of atmosphere (aka rawinsonde data) -- 🇵🇱 **meteo_imgw()** - Downloading hourly, daily, and monthly meteorological data from the Polish met service across +- 🇵🇱 [**meteo_imgw**](https://bczernecki.github.io/climate/reference/meteo_imgw.html) - +Downloading hourly, daily, and monthly meteorological data from the Polish met service across all types of stations (i.e. SYNOP/CLIMATE/PRECIP ) available in the danepubliczne.imgw.pl collection. It is a wrapper for `meteo_monthly()`, `meteo_daily()`, `meteo_hourly()` and `meteo_imgw_datastore()` which gives access from montly to even to 10-min dataset. ### (Polish) Hydrological data -- 🇵🇱 **hydro_imgw()** - Downloading hourly, daily, and monthly hydrological data from stations available in the +- 🇵🇱 [**hydro_imgw**](https://bczernecki.github.io/climate/reference/hydro_imgw.html) - +Downloading hourly, daily, and monthly hydrological data from stations available in the danepubliczne.imgw.pl collection. It is a wrapper for previously developed set of functions such as: `hydro_monthly()`, and `hydro_daily()` -- 🇵🇱 **hydro_imgw_datastore()** - Downloading hourly and subhourly hydrological data from the IMGW-PIB hydro telemetry stations. + +- 🇵🇱 [**hydro_imgw_datastore**](https://bczernecki.github.io/climate/reference/hydro_imgw_datastore.html) - +Downloading hourly and subhourly hydrological data from the IMGW-PIB hydro telemetry stations. ### Auxiliary functions and datasets -- 🌍 **stations_ogimet()** - Downloading information about all stations available in the selected +- 🌍 **stations_ogimet** - Downloading information about all stations available in the selected country in the Ogimet repository -- 🌍 **nearest_stations_ogimet()** - Downloading information about nearest stations to the selected point using Ogimet repository -- 🌍 **nearest_stations_noaa()** - Downloading information about nearest stations to the selected point available for the selected country in the NOAA ISH meteorological repository -- 🇵🇱 **nearest_stations_imgw()** - List of nearby meteorological or hydrological IMGW-PIB stations in Poland +- 🌍 **nearest_stations_ogimet** - Downloading information about nearest stations to the selected point using Ogimet repository +- 🌍 **nearest_stations_noaa** - Downloading information about nearest stations to the selected point available for the selected country in the NOAA ISH meteorological repository +- 🇵🇱 **nearest_stations_imgw** - List of nearby meteorological or hydrological IMGW-PIB stations in Poland - 🇵🇱 **imgw_meteo_stations** - Built-in metadata from the IMGW-PIB repository for meteorological stations, their geographical coordinates, and ID numbers - 🇵🇱 **imgw_hydro_stations** - Built-in metadata from the IMGW-PIB repository for hydrological stations, their geographical coordinates, and ID numbers - 🇵🇱 **stations_meteo_imgw_telemetry** - Downloading complete and up-to-date information about coordinates for IMGW-PIB telemetry meteorological stations - 🇵🇱 **stations_hydro_imgw_telemetry** - Downloading complete and up-to-date information about coordinates for IMGW-PIB telemetry hydrological stations -- 🌍 **parser()** - Decoding raw SYNOP meteorological messages into structured R lists or data frames. For a full walkthrough see the [SYNOP Messages vignette](https://bczernecki.github.io/climate/articles/synop_parser.html). +- 🌍 [**synop_parser**](https://bczernecki.github.io/climate/reference/synop_parse.html) - Decoding raw SYNOP meteorological messages into structured R lists or data frames. For a full walkthrough see the [SYNOP Messages vignette](https://bczernecki.github.io/climate/articles/synop_parser.html). ## Example 1 #### Download hourly dataset from NCEI/NOAA ISH meteorological repository: @@ -101,7 +109,7 @@ head(noaa) ``` r # find 100 nearest UK stations to longitude 1W and latitude 53N : -nearest_stations_ogimet(country = "United+Kingdom", +nearest_stations_ogimet(country = "United Kingdom", date = Sys.Date(), add_map = TRUE, point = c(-1, 53), diff --git a/man/compute_relative_humidity.Rd b/man/compute_relative_humidity.Rd index 1882581..2380a6b 100644 --- a/man/compute_relative_humidity.Rd +++ b/man/compute_relative_humidity.Rd @@ -13,9 +13,9 @@ compute_relative_humidity(t2m, dpt2m) Must be the same length as \code{t2m}.} } \value{ -Numeric vector of relative humidity values in percent (0–100). +Numeric vector of relative humidity values in percent (0-100). Returns \code{NA} where either input is \code{NA}. Values are not clamped, so -rounding errors may produce results marginally outside 0–100. +rounding errors may produce results marginally outside 0-100. } \description{ Uses the August-Roche-Magnus approximation to derive relative humidity from diff --git a/man/nearest_stations_ogimet.Rd b/man/nearest_stations_ogimet.Rd index e6c5797..415e867 100644 --- a/man/nearest_stations_ogimet.Rd +++ b/man/nearest_stations_ogimet.Rd @@ -6,7 +6,7 @@ \usage{ nearest_stations_ogimet( country = "United Kingdom", - date = Sys.Date(), + date = Sys.Date() - 1, add_map = FALSE, point = c(2, 50), no_of_stations = 10, diff --git a/man/sounding_wyoming.Rd b/man/sounding_wyoming.Rd index 81de49b..1dafc21 100644 --- a/man/sounding_wyoming.Rd +++ b/man/sounding_wyoming.Rd @@ -14,25 +14,25 @@ sounding_wyoming( dd, hh, min = 0, - source = "DETECT", + source = "AUTODETECT", allow_failure = TRUE ) } \arguments{ -\item{wmo_id}{international WMO station code (World Meteorological Organization ID); For Polish stations: Leba - 12120, Legionowo - 12374, Wrocław- 12425} +\item{wmo_id}{international WMO station code (World Meteorological Organization ID); For Polish stations: Leba - 12120, Legionowo - 12374, Wrocław - 12425} -\item{yy}{year - single number} +\item{yy}{year - calendar year} -\item{mm}{month - single number denoting month} +\item{mm}{month - calendar month} -\item{dd}{day - single number denoting day} +\item{dd}{day - calendar day} -\item{hh}{hour - single number denoting initial hour of sounding; for most stations this measurement is done twice a day (i.e. at 12 and 00 UTC), sporadically 4 times a day} +\item{hh}{hour - hour of sounding; for most stations measurements are performed twice a day (i.e. at 12 and 00 UTC), sporadically 4 times a day} -\item{min}{minute - single number denoting initial minute of sounding; applies only to BUFR soundings.} +\item{min}{minute - minute of sounding; Default 00; Other values applies only to BUFR soundings.} \item{source}{\itemize{ -\item "BUFR", "TEMP" or "DETECT". By default "DETECT" is used. +\item optional; \code{AUTODETECT}, \code{BUFR} (raw; most detailed) or \code{TEMP} (FM-35) input format. By default "AUTODETECT" is used which means that the function will try to detect the format automatically. If a station use only a specific format that is known you can specify it explicitly. }} \item{allow_failure}{logical - whether to proceed or stop on failure. By default set to TRUE (i.e. don't stop on error). For debugging purposes change to FALSE} @@ -58,34 +58,26 @@ A list of 2 data.frames where first data frame represents parameters of upper pa depending on the height. Second data.frame presents a description of the conditions under which the sounding was carried out. } \description{ -Downloading the measurements of the vertical profile of atmosphere (also known as sounding data). Data can be retrieved using TEMP and BUFR sounding formatting. +Downloading the measurements of the vertical profile of atmosphere (also known as sounding data). +Data can be retrieved using TEMP and BUFR sounding formatting. By default automatic detection of input format is used. } \examples{ \donttest{ -############################################################################## # download data for Station 45004 starting 1120Z 11 Jul 2021; Kowloon, HONG KONG, CHINA -# using TEMP and BUFR sounding formats -############################################################################## - TEMP = sounding_wyoming(wmo_id = 45004, yy = 2021, mm = 07, dd = 17, - hh = 12, min = 00, source = "TEMP") - head(TEMP[[1]]) - - BUFR = sounding_wyoming(wmo_id = 45004, yy = 2021, mm = 07, dd = 17, - hh = 12, min = 00, source = "BUFR") - head(BUFR[[1]]) +# using AUTODETECT, TEMP and BUFR sounding formats +# autodect input format: +sounding_auto = sounding_wyoming(wmo_id = 45004, + yy = 2021, mm = 07, dd = 17, hh = 12) -############################################################################## -### example with a random date to download sounding from LEBA, PL station: ### -############################################################################## - - # no "source" argument is needed as the function will detect format automatically - profile = sounding_wyoming(wmo_id = 12120, - yy = sample(2000:2019,1), - mm = sample(1:12,1), - dd = sample(1:20,1), - hh = 0) - # plot(profile[[1]]$HGHT, profile[[1]]$PRES, type = 'l') +# temp (fm35) input format: +sounding_temp = sounding_wyoming(wmo_id = 45004, + yy = 2021, mm = 07, dd = 17, hh = 12, + source = "TEMP") +# bufr input format: +sounding_bufr = sounding_wyoming(wmo_id = 45004, + yy = 2021, mm = 07, dd = 17, hh = 12, min = 00, + source = "BUFR") } } diff --git a/tests/testthat/test-meteo_ogimet_synop.R b/tests/testthat/test-meteo_ogimet_synop.R index f424d2c..8d5d9bc 100644 --- a/tests/testthat/test-meteo_ogimet_synop.R +++ b/tests/testthat/test-meteo_ogimet_synop.R @@ -1,9 +1,3 @@ -test_that("meteo_ogimet_synop stops when neither station nor country_name is given", { - expect_error( - meteo_ogimet_synop(allow_failure = FALSE), - "station.*country_name" - ) -}) test_that(".ogimet_synop_raw_lines splits and recurses when server limit is reached", { # Stub the HTTP layer so no network call is made. @@ -46,7 +40,7 @@ test_that("meteo_ogimet_synop warns when both station and country_name are given # only check the warning; the download itself may or may not succeed expect_warning( - meteo_ogimet_synop(station = 12330, + meteo_ogimet(station = 12330, source = "synop", country_name = "Poland", date = c("2009-12-15", "2009-12-15"), allow_failure = TRUE), @@ -58,7 +52,7 @@ test_that("meteo_ogimet_synop station mode returns a data.frame with expected co if (!curl::has_internet()) return(invisible(NULL)) skip_on_cran() - result = meteo_ogimet_synop(station = 12330, + result = meteo_ogimet(station = 12330, source = "synop", date = c("2009-12-01", "2009-12-04"), simplified = FALSE) @@ -76,7 +70,7 @@ test_that("meteo_ogimet_synop simplified station mode returns expected columns", if (!curl::has_internet()) return(invisible(NULL)) skip_on_cran() - result = meteo_ogimet_synop(station = 12330, + result = meteo_ogimet(station = 12330, source = "synop", date = c("2009-12-01", "2009-12-04")) if (is.null(result)) return(invisible(NULL)) @@ -84,32 +78,15 @@ test_that("meteo_ogimet_synop simplified station mode returns expected columns", expect_s3_class(result, "data.frame") expect_true(all(c("date", "station", "t2m", "ws", "Nt") %in% names(result))) expect_true(nrow(result) > 0) -}) - -test_that("meteo_ogimet_synop station mode Date column is POSIXct UTC", { - if (!curl::has_internet()) return(invisible(NULL)) - skip_on_cran() - - result = meteo_ogimet_synop(station = 12330, - date = c("2009-12-01", "2009-12-04")) - - if (is.null(result) || nrow(result) == 0) return(invisible(NULL)) expect_s3_class(result$date, "POSIXct") expect_equal(attr(result$date, "tzone"), "UTC") -}) - -test_that("meteo_ogimet_synop station mode clips to requested date range", { - if (!curl::has_internet()) return(invisible(NULL)) - skip_on_cran() - - result = meteo_ogimet_synop(station = 12330, - date = c("2009-12-01", "2009-12-04")) - - if (is.null(result) || nrow(result) == 0) return(invisible(NULL)) expect_true(all(as.Date(result$date) >= as.Date("2009-12-01"))) expect_true(all(as.Date(result$date) <= as.Date("2009-12-04"))) + + expect_true("source" %in% names(result)) + expect_true(all(nzchar(result$source[!is.na(result$source)]))) }) test_that("meteo_ogimet_synop station mode handles allow_failure gracefully", { @@ -117,30 +94,16 @@ test_that("meteo_ogimet_synop station mode handles allow_failure gracefully", { skip_on_cran() expect_no_error( - meteo_ogimet_synop(station = 9999999, date = c("2009-12-01", "2009-12-02"), + meteo_ogimet(station = 9999999, date = c("2009-12-01", "2009-12-02"), source = "synop", allow_failure = TRUE) ) }) -test_that("meteo_ogimet_synop station mode source column contains SYNOP strings", { - if (!curl::has_internet()) return(invisible(NULL)) - skip_on_cran() - - result = meteo_ogimet_synop(station = 12330, - date = c("2009-12-01", "2009-12-04"), - simplified = FALSE) - - if (is.null(result) || nrow(result) == 0) return(invisible(NULL)) - - expect_true("source" %in% names(result)) - expect_true(all(nzchar(result$source[!is.na(result$source)]))) -}) - test_that("meteo_ogimet_synop country mode returns a data.frame for one day", { if (!curl::has_internet()) return(invisible(NULL)) skip_on_cran() - result = meteo_ogimet_synop(country_name = "Poland", + result = meteo_ogimet(country_name = "Poland", source = "synop", date = c("2009-12-15", "2009-12-15"), simplified = FALSE) @@ -152,29 +115,9 @@ test_that("meteo_ogimet_synop country mode returns a data.frame for one day", { expect_true("air_temperature" %in% names(result)) # Poland has many SYNOP stations; expect multiple rows expect_true(nrow(result) > 1) -}) - -test_that("meteo_ogimet_synop country mode Date column is POSIXct UTC", { - if (!curl::has_internet()) return(invisible(NULL)) - skip_on_cran() - - result = meteo_ogimet_synop(country_name = "Poland", - date = c("2009-12-15", "2009-12-15")) - - if (is.null(result) || nrow(result) == 0) return(invisible(NULL)) - + expect_s3_class(result$date, "POSIXct") expect_equal(attr(result$date, "tzone"), "UTC") -}) - -test_that("meteo_ogimet_synop country mode clips to the requested date", { - if (!curl::has_internet()) return(invisible(NULL)) - skip_on_cran() - - result = meteo_ogimet_synop(country_name = "Poland", - date = c("2009-12-15", "2009-12-15")) - - if (is.null(result) || nrow(result) == 0) return(invisible(NULL)) expect_true(all(as.Date(result$date) == as.Date("2009-12-15"))) }) @@ -186,7 +129,7 @@ test_that("parser Nt=cloud_cover and Nh=low_cloud_amount are decoded correctly", # 71703 -> Nddff: N=7 (total cloud cover 7 oktas) # 85232 -> 8NhCLCMCH: Nh=5 (low cloud cover 5 oktas), CL=2 (Sc), CM=3, CH=2 msg = "AAXX 15151 12120 42461 71703 11013 21016 30184 40192 58006 85232=" - row = parser(msg, as_data_frame = TRUE) + row = synop_parser(msg, as_data_frame = TRUE) expect_equal(row$cloud_cover, 7) # Nt source expect_equal(row$low_cloud_amount, 5) # Nh source }) diff --git a/vignettes/articles/figures/ogimet_fig01.png b/vignettes/articles/figures/ogimet_fig01.png new file mode 100644 index 0000000..9b8f639 Binary files /dev/null and b/vignettes/articles/figures/ogimet_fig01.png differ diff --git a/vignettes/articles/figures/ogimet_fig02.png b/vignettes/articles/figures/ogimet_fig02.png new file mode 100644 index 0000000..3059aec Binary files /dev/null and b/vignettes/articles/figures/ogimet_fig02.png differ diff --git a/vignettes/articles/figures/ogimet_fig03.png b/vignettes/articles/figures/ogimet_fig03.png new file mode 100644 index 0000000..47d91b5 Binary files /dev/null and b/vignettes/articles/figures/ogimet_fig03.png differ diff --git a/vignettes/articles/figures/ogimet_fig04.png b/vignettes/articles/figures/ogimet_fig04.png new file mode 100644 index 0000000..66e1951 Binary files /dev/null and b/vignettes/articles/figures/ogimet_fig04.png differ diff --git a/vignettes/articles/figures/ogimet_fig05.png b/vignettes/articles/figures/ogimet_fig05.png new file mode 100644 index 0000000..4c276ad Binary files /dev/null and b/vignettes/articles/figures/ogimet_fig05.png differ diff --git a/vignettes/articles/pl.Rmd b/vignettes/articles/pl.Rmd index 78a2896..121543e 100644 --- a/vignettes/articles/pl.Rmd +++ b/vignettes/articles/pl.Rmd @@ -5,8 +5,11 @@ date: "`r Sys.Date()`" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Wstęp do pakietu 'climate' (PL)} - %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} + %\VignetteEngine{knitr::rmarkdown} +editor_options: + markdown: + wrap: 72 --- ```{r setup, include = FALSE} @@ -20,94 +23,118 @@ library(dplyr) options(scipen = 999) ``` -Głównym celem pakietu **climate** jest zapewnienie wygodnego i programowalnego dostępu do danych meteorologicznych -i hydrologicznych z publicznie dostępnych repozytoriów, w tym m.in.: +Głównym celem pakietu **climate** jest zapewnienie wygodnego i +programowalnego dostępu do danych meteorologicznych i hydrologicznych z +publicznie dostępnych repozytoriów, w tym m.in.: -- Instytut Meteorologii i Gospodarki Wodnej - Państwowy Instytut Badawczy (IMGW-PIB) +- Instytut Meteorologii i Gospodarki Wodnej - Państwowy Instytut + Badawczy (IMGW-PIB) - OGIMET (ogimet.com) -- University of Wyoming - dane atmosferyczne dla radiosondaży (http://weather.uwyo.edu/upperair/). +- University of Wyoming - dane atmosferyczne dla radiosondaży + (). ## Funkcje - Pakiet **climate** składa się z kilku głównych funkcji: -1. Dane meteorologiczne +1. Dane meteorologiczne -- **meteo_ogimet()** - Umożliwia pobieranie godzinowych i dobowych danych meteorologicznych ze stacji meteorologicznych nadających depesze SYNOP (FM-12) które są udostępnione w serwisie ogimet.com. Każda stacja meteorologiczna (synoptyczna) pracująca w ramach World Meteorological -framework udostępnia swoje dane w ramach serwisu od ok. 2000 roku +- [**meteo_ogimet**](https://bczernecki.github.io/climate/reference/meteo_ogimet.html) - + Umożliwia pobieranie godzinowych i dobowych danych meteorologicznych + ze stacji meteorologicznych nadających depesze SYNOP (FM-12) które są + udostępnione w serwisie ogimet.com. Każda stacja meteorologiczna + (synoptyczna) pracująca w ramach World Meteorological Organization + (WMO) udostępnia swoje dane w ramach serwisu od ok. 2000 roku - - **stations_ogimet()** - Umożliwia uzyskiwanie współrzędnych geograficznych, położenia stacji nad poziomem - morza oraz identyfikatora WMO i nazwy stacji dla kraju określonej przez użytkownika; opcjonalnie - wykreśl wyniki na mapie + - [**stations_ogimet**](https://bczernecki.github.io/climate/reference/stations_ogimet.html) - + Umożliwia uzyskiwanie współrzędnych geograficznych, położenia stacji + nad poziomem morza oraz identyfikatora WMO i nazwy stacji dla kraju + określonej przez użytkownika; opcjonalnie wykreśl wyniki na mapie -- **meteo_imgw()** - Pozwala pobrać dane meteorologiczne interwale godzinowym, dziennym lub -miesięcznym o danej randze stacji(SYNOP / CLIMATE / PRECIP) dostępnych w kolekcji dane.imgw.pl. +- [**meteo_imgw**](https://bczernecki.github.io/climate/reference/meteo_imgw.html) - + Pozwala pobrać dane meteorologiczne interwale godzinowym, dziennym lub + miesięcznym o danej randze stacji (SYNOP / CLIMATE / PRECIP) + dostępnych w kolekcji dane.imgw.pl. -2. Dane hydrologiczne: +2. Dane hydrologiczne: -- **hydro_imgw()** - Pobieranie danych hydrologicznych o interwale dziennym, miesięcznym lub rocznym ze -stacji dostępnych w repozytorium dane.imgw.pl. +- [**hydro_imgw**](https://bczernecki.github.io/climate/reference/hydro_imgw.html) - + Pobieranie danych hydrologicznych o rozdzielczości dobowej lub + miesięcznej ze stacji dostępnych w repozytorium danepubliczne.imgw.pl. -3. Dane radiosondażowe : +3. Dane radiosondażowe : -- **meteo\_sounding()** - Pobieranie pomiarów pionowych profil atmosfery (danych radiosondażowych) +- [**sounding_wyoming**](https://bczernecki.github.io/climate/reference/sounding_wyoming.html) - + Pobieranie pomiarów pionowych profil atmosfery (danych + radiosondażowych) ## Przykłady -Pokażemy, jak korzystać z naszego pakietu i przygotować dane do analizy przestrzennej z dodatkową pomocą paczek -[dplyr](https://cran.r-project.org/web/packages/dplyr/vignettes/dplyr.html) oraz +Pokażemy, jak korzystać z naszego pakietu i przygotować dane do analizy +przestrzennej z dodatkową pomocą paczek +[dplyr](https://cran.r-project.org/web/packages/dplyr/vignettes/dplyr.html) +oraz [tidyr](https://cran.r-project.org/web/packages/tidyr/vignettes/tidy-data.html). -Najpierw pobierzemy 10 lat (**2001-2010**) miesięcznych obserwacji hydrologicznych dla wszystkich dostępnych stacji i automatycznie dodają ich współrzędne przestrzenne. +Najpierw pobierzemy 10 lat (**2001-2010**) miesięcznych obserwacji +hydrologicznych dla wszystkich dostępnych stacji i automatycznie dodają +ich współrzędne przestrzenne. - - -```{r data, echo = FALSE} -h = hydro_imgw(interval = "monthly", year = 2001:2010, coords = TRUE) +```{r data, eval=TRUE, echo=TRUE, message=FALSE, warning=FALSE, include=TRUE} +h = hydro_imgw(interval = "monthly", year = 2001:2010) head(h) ``` -Zmienna `MCWSKEX` reprezentuje etykietę ekstremum, gdzie "1" oznacza minimum, "2" oznacza średnią, a "3" maksimum. ^[Więcej informacji na ten temat można znaleźć w zestawie danych `hydro_abbrev`.] -Analizy hydrologiczne często koncentrują się na jednej grupy zjawisk, np. związanych z przepływami maksymalnymi. W tym -celu pozostaną w ramce danych tylko wartości przepływów maksymalnych oraz kolumny zawierające interesujące nas -informacje, tj. identyfikator stacji - `id`, rok hydrologiczny (`MCROKH`), szerokość geograficzną `X` i długość -geograficzną` Y`. -Następnie obliczymy średnią maksymalną wartość przepływu na stacjach w każdym roku za pomocą ** dplyr ** `summarise()`, -oraz rozdzielimy dane według roku używając ** tidyr **' `spread ()` aby uzyskać roczne średnie maksymalne przepływy (`MCPRZP`) w -kolejnych kolumnach. - -```{r filtering, eval=TRUE, include=TRUE} +Zmienna `MCWSKEX` reprezentuje etykietę ekstremum, gdzie "1" oznacza +minimum, "2" oznacza średnią, a "3" maksimum. [^1] + +[^1]: Więcej informacji na ten temat można znaleźć w zestawie danych + `hydro_abbrev`. + +Analizy hydrologiczne często koncentrują się na jednej grupy zjawisk, +np. związanych z przepływami maksymalnymi. W tym celu pozostaną w ramce +danych tylko wartości przepływów maksymalnych oraz kolumny zawierające +interesujące nas informacje, tj. identyfikator stacji - `id`, rok +hydrologiczny (`MCROKH`), szerokość geograficzną `X` i długość +geograficzną`Y`. + +Następnie obliczymy średnią maksymalną wartość przepływu na stacjach w +każdym roku za pomocą `dplyr::summarise()`, oraz rozdzielimy dane według +roku używając `spread ()` aby uzyskać roczne średnie maksymalne +przepływy (`MCPRZP`) w kolejnych kolumnach. + +```{r filtering, eval=TRUE, echo=TRUE, message=FALSE, warning=FALSE, include=TRUE, paged.print=TRUE} h2 = h %>% dplyr::filter(MCWSKEX == 3) %>% dplyr::select(id, PSNZWP, X, Y, MCROKH, MCPRZP) %>% dplyr::group_by(MCROKH, id, PSNZWP, X, Y) %>% - dplyr::summarise(srednie_roczne_Q = round(mean(MCPRZP, na.rm = TRUE),1)) %>% + dplyr::summarise(srednie_roczne_Q = round(mean(MCPRZP, na.rm = TRUE), 1)) %>% spread(MCROKH, srednie_roczne_Q) ``` -```{r filtering2, echo=FALSE} +```{r filtering2, eval=TRUE, echo=TRUE, message=FALSE, warning=FALSE, include=TRUE, paged.print=TRUE} library(knitr) kable(head(h2), caption = "Examplary data frame of hydrological preprocesssing.") ``` -Wynik pokazuje jak zmienia się maksymalna roczna średnia prędkość przepływu wody w ciągu dekady dla wszystkich -dostępnych stacji w Polsce. -Możemy zapisać to do: +Wynik pokazuje jak zmienia się maksymalna roczna średnia prędkość +przepływu wody w ciągu dekady dla wszystkich dostępnych stacji w Polsce. -- **.csv** przy pomocą funckcji: `write.csv(result, file = "result.csv", sep = ";",dec = ".", col.names = T, row.names = F)`. -To polecenie zapisuje nasz wynik do pliku: `result.csv`, gdzie separatorem kolumn jest `;`, część dziesiętna liczby -następuje po `.`, zachowujemy nagłówki kolumn i usuwamy nazwy wierszy, które są po prostu liczbami obserwacji. +Możemy wynik zapisać do: -- **.xlsx** przy pomocą funckcji: `write.xlsx(result, file = "result.xlsx", sheetName = "Poland", append = FALSE)` -To polecenie zapisuje nasz wynik do pliku: `result.xlsx` o nazwie arkusza `Poland`. Argument `append=TRUE` dodaje arkusz -do istniejącego pliku `.xlsx`. -By zapisać dane do formatu`.xlsx` najpierw należy zaisntalować pakiet **writexl** przy pomocy -komendy`install.packages("writexl")`, oraz dodać go do naszego środowiska: `library(writexl)`. - -```{r, eval=FALSE, include=TRUE} +- **.csv** przy pomocą funkcji: `write.csv(result, file = "result.csv")` +- **.xlsx** przy pomocą funckcji: + `write.xlsx(result, file = "result.xlsx", sheetName = "Poland", append = FALSE)`\ + To polecenie zapisuje nasz wynik do pliku: `result.xlsx` o nazwie + arkusza `Poland`. Argument `append=TRUE` dodaje arkusz do istniejącego + pliku `.xlsx`. By zapisać dane do formatu`.xlsx` najpierw należy + zainstalować pakiet **writexl** przy pomocy komendy + `install.packages("writexl")`, oraz dodać go do naszego środowiska: + `library(writexl)`. + +```{r, eval=FALSE, echo=TRUE} library(sf) library(tmap) library(rnaturalearth) @@ -129,5 +156,6 @@ tm_shape(h3) + panel.labels = as.character(c(2001:2010))) ``` - - +[Mean maximum annual +flow](https://klimat.amu.edu.pl/climate_flow.png)![Mean maximum annual +flow](https://klimat.amu.edu.pl/climate_flow.png) diff --git a/vignettes/articles/synop_parser.Rmd b/vignettes/articles/synop_parser.Rmd index 780bc74..c0987a4 100644 --- a/vignettes/articles/synop_parser.Rmd +++ b/vignettes/articles/synop_parser.Rmd @@ -1,5 +1,8 @@ --- title: "SYNOP Messages" +editor_options: + markdown: + wrap: 72 --- ```{r setup, include=FALSE} @@ -8,42 +11,34 @@ knitr::opts_chunk$set(echo = TRUE, eval = FALSE) ## Overview -SYNOP is a standardized format for reporting weather observations usually every hour. -This vignette demonstrates how to work with SYNOP FM-12 meteorological messages using the -**climate** package. +SYNOP is a standardized format for reporting weather observations +usually every hour. This vignette demonstrates how to work with SYNOP +FM-12 meteorological messages using the **climate** package. -The package provides two complementary functions for handling SYNOP data: - -- **`synop_parser()`** — parse raw SYNOP message strings into structured data -- **`meteo_ogimet(source = "synop")`** — download raw SYNOP messages from Ogimet and decode them automatically +The package provides two complementary functions for handling SYNOP +data: +- **`synop_parser()`** — parse raw SYNOP message strings into structured + data +- **`meteo_ogimet(source = "synop")`** — download raw SYNOP messages + from Ogimet web service and decode them automatically ## The `synop_parser()` Function -The `synop_parser()` function decodes raw SYNOP messages into either a tidy data frame or a nested list structure. +The `synop_parser()` function decodes raw SYNOP messages into either a +tidy data frame or a nested list structure. ### Basic Usage - data frame output: - -```{r} +```{r, eval=TRUE, include=FALSE} suppressMessages(library(climate)) -``` +suppressMessages(library(knitr)) - -```{r ru, echo=TRUE} -# A simple SYNOP message -synop_code = "AAXX 01004 88889 12782 61506 10094 20047 30111 40197 53007 60001 81541" - -# Parse into a data frame (default for analysis) -df_single = synop_parser(synop_code, as_data_frame = TRUE) -head(df_single) ``` -### Multiple Messages +SYNOP messages should be provided as text vector of any length, e.g: -To parse multiple SYNOP messages at once: - -```{r echo=TRUE} +```{r simple_synop_multi, eval=TRUE, echo=TRUE, include=TRUE, paged.print=TRUE} # Parse multiple messages messages = c( "AAXX 01004 88889 12782 61506 10094 20047 30111 40197 53007 60001 81541", @@ -52,61 +47,58 @@ messages = c( # Multiple messages as a data frame df_multiple = synop_parser(messages, as_data_frame = TRUE) -head(df_multiple) +df_multiple$source = NULL # remove raw message from output +# Display the results in a table +kable(t(df_multiple), caption = "Examplary results of parsing multiple SYNOP message into data frame") ``` ### Available Columns -When `as_data_frame = TRUE`, the returned data frame includes the following columns: - -**Identification and metadata:** -- `station_type` — station classification code -- `station_id` — WMO station identifier -- `region` — geographic region code -- `obs_day` — observation day of month -- `obs_hour` — observation hour (UTC) - -**Wind:** -- `wind_unit` — either "m/s" or "knots" -- `wind_estimated` — whether wind is estimated -- `wind_direction` — direction in degrees (0–360°) -- `wind_speed` — speed in the reported unit - -**Temperature and moisture:** -- `air_temperature` — air temperature (°C) -- `dewpoint_temperature` — dew point (°C) - -**Pressure:** -- `station_pressure` — station-level pressure (hPa) -- `sea_level_pressure` — sea-level pressure (hPa) -- `pressure_tendency` — pressure change indicator -- `pressure_change` — 3-hour pressure change (hPa) - -**Clouds:** -- `cloud_cover` — total cloud cover (oktas, 0–8) -- `low_cloud_type` — type of low clouds (Sc, St, Cu, Cb) -- `middle_cloud_type` — type of middle clouds (Ac, As, Ns) -- `high_cloud_type` — type of high clouds (Ci, Cc, Cs) -- `low_cloud_amount` — cover of low clouds (oktas) -- `visibility` — horizontal visibility (m) -- `cloud_base_min`, `cloud_base_max` — cloud base height range (m) - -**Precipitation and other:** -- `precipitation_amount` — total precipitation (mm) -- `precipitation_time` — precipitation period (hours) -- `maximum_temperature` — daily maximum (°C, from Section 3) -- `minimum_temperature` — daily minimum (°C, from Section 3) -- `gust` — highest wind gust (in the message's wind unit) -- `sunshine_duration` — daily sunshine duration (hours) -- `snow_depth` — total snow depth (cm) -- `snow_depth_state` — descriptive snow/ground state +- When `as_data_frame = TRUE`, the returned data frame includes the + following columns: + + - **Identification and metadata:** - `station_type` — station + classification code - `station_id` — WMO station identifier - + `region` — geographic region code - `obs_day` — observation day of + month - `obs_hour` — observation hour (UTC) + + - **Wind:** - `wind_unit` — either "m/s" or "knots" - `wind_estimated` + — whether wind is estimated - `wind_direction` — direction in + degrees (0–360°) - `wind_speed` — speed in the reported unit + + - **Temperature and moisture:** - `air_temperature` — air temperature + (°C) - `dewpoint_temperature` — dew point (°C) + + - **Pressure:** - `station_pressure` — station-level pressure (hPa) - + `sea_level_pressure` — sea-level pressure (hPa) - + `pressure_tendency` — pressure change indicator - `pressure_change` + — 3-hour pressure change (hPa) + + - **Clouds:** - `cloud_cover` — total cloud cover (oktas, 0–8) - + `low_cloud_type` — type of low clouds (Sc, St, Cu, Cb) - + `middle_cloud_type` — type of middle clouds (Ac, As, Ns) - + `high_cloud_type` — type of high clouds (Ci, Cc, Cs) - + `low_cloud_amount` — cover of low clouds (oktas) - `visibility` — + horizontal visibility (m) - `cloud_base_min`, `cloud_base_max` — + cloud base height range (m) + + - **Precipitation and other:** - `precipitation_amount` — total + precipitation (mm) - `precipitation_time` — precipitation period + (hours) - `maximum_temperature` — daily maximum (°C, + from Section 3) - `minimum_temperature` — daily minimum (°C, from Section 3) - + `gust` — highest wind gust (in the message's wind unit) - + `sunshine_duration` — daily sunshine duration (hours) - `snow_depth` + — total snow depth (cm) - `snow_depth_state` — descriptive + snow/ground state ### Nested List Output -For more granular access most of decoded field, use the default list output: +For more granular access most of decoded field, use the default list +output: -```{r echo=TRUE} +```{r echo=TRUE, include = TRUE, eval = TRUE} # Single message as a nested list +synop_code = messages[1] decoded = synop_parser(synop_code) str(decoded) ``` @@ -123,7 +115,8 @@ decoded_list[[1]] ### Country-Specific Decoding -Some SYNOP groups (e.g., precipitation indicators) are country-specific. Pass the `country` parameter to adjust decoding: +Some SYNOP groups (e.g., precipitation indicators) are country-specific. +Pass the `country` parameter to adjust decoding: ```{r echo=TRUE} # Parse with Russia-specific precipitation decoding @@ -135,28 +128,31 @@ decoded_default = synop_parser(synop_code, as_data_frame = TRUE) ## The `meteo_ogimet()` Function — SYNOP Backend -The `meteo_ogimet()` function (with `source = "synop"`) downloads SYNOP messages from -[Ogimet](http://www.ogimet.com) and automatically decodes them using `parser()`. +The `meteo_ogimet()` function (with `source = "synop"`) downloads SYNOP +messages from [Ogimet](http://www.ogimet.com) and automatically decodes +them using `parser()`. ### Station Mode: Single or Multiple Stations -Fetch SYNOP data for one or more WMO station IDs: +Fetch SYNOP data from OGIMET service for one or more WMO station IDs: -```{r echo=TRUE} +```{r echo=TRUE, include = TRUE, eval = TRUE} # can be used with multiple stations: two_stations = meteo_ogimet( source = "synop", station = c(12330, 12375), date = c("2024-01-01", "2024-01-10") ) -head(two_stations) + +kable(head(two_stations)) ``` ### Country mode: all stations in a country -Fetch SYNOP data for all Ogimet stations in a country for a given time range: +Fetch SYNOP data for all Ogimet stations in a country for a given time +range: -```{r} +```{r, eval=TRUE, include=TRUE, include = TRUE} # All Polish stations for a single day poland = meteo_ogimet( source = "synop", @@ -168,10 +164,11 @@ head(poland) ### Simplified vs. full data frame output -By default, `meteo_ogimet()` returns a simplified data frame with the most commonly used variables. -To get all decoded SYNOP fields, use `simplified = FALSE`: +By default, `meteo_ogimet()` returns a simplified data frame with the +most commonly used variables. To get all decoded SYNOP fields, use +`simplified = FALSE`: -```{r echo=TRUE} +```{r echo=TRUE, eval = TRUE, include = TRUE} # Full synop_parser output full = meteo_ogimet( source = "synop", @@ -182,7 +179,8 @@ full = meteo_ogimet( head(full) ``` -To obtain both the simplified and full data frames in one call, use `return_list = TRUE`: +To obtain both the simplified and full data frames in one call, use +`return_list = TRUE`: ```{r echo=TRUE} result = meteo_ogimet( @@ -197,16 +195,22 @@ names(result$data) names(result$full) ``` - - ## Notes and Limitations -- **Ogimet server limits:** The Ogimet server caps responses at 200,000 rows. If a date range would exceed this, the function automatically splits the range and recursively fetches each half. -- **Station delays:** When fetching multiple stations in station mode, a 20-second delay is inserted between requests to avoid overloading the Ogimet server. -- **Missing observations:** Not all SYNOP variables are reported at all times. Missing fields are returned as `NA`. -- **Historical data:** Ogimet typically provides historical SYNOP data since approx. 1999/2000; availability varies by station and country. +- **Ogimet server limits:** The Ogimet server caps responses at 200,000 + rows. If a date range would exceed this, the function automatically + splits the range and recursively fetches each half. +- **Station delays:** When fetching multiple stations in station mode, a + 20-second delay is inserted between requests to avoid overloading the + Ogimet server. +- **Missing observations:** Not all SYNOP variables are reported at all + times. Missing fields are returned as `NA`. +- **Historical data:** Ogimet typically provides historical SYNOP data + since approx. 1999/2000; availability varies by station and country. ## See Also -- Daily and hourly OGIMET data via the HTML backend: `meteo_ogimet(source = "html")` -- WMO SYNOP documentation for FM-12: [FM 12-XIII Ext. SYNOP](https://library.wmo.int/records/item/35713-manual-on-codes-volume-i-1-international-codes) +- Daily and hourly OGIMET data via the HTML backend: + `meteo_ogimet(source = "html")` +- WMO SYNOP documentation for FM-12: [FM 12-XIII Ext. + SYNOP](https://library.wmo.int/records/item/35713-manual-on-codes-volume-i-1-international-codes) diff --git a/vignettes/articles/usecase_ogimet.Rmd b/vignettes/articles/usecase_ogimet.Rmd index d33472a..877dfb7 100644 --- a/vignettes/articles/usecase_ogimet.Rmd +++ b/vignettes/articles/usecase_ogimet.Rmd @@ -4,7 +4,7 @@ output: html_document --- ```{r setup, include=FALSE} -knitr::opts_chunk$set(echo = TRUE, eval = FALSE) +knitr::opts_chunk$set(echo = TRUE, eval = FALSE, include = TRUE) ``` Most of meteorological stations working under the umbrella of the World Meteorlogical Organization (WMO) report their observations in the SYNOP format. The **climate** package provides a convenient interface to download and decode SYNOP messages from the [Ogimet](https://www.ogimet.com/) database, which aggregates data from thousands of stations worldwide. Therefore, it is often the most preferable source of meteorological information being a valuable resource for researchers, students, and weather enthusiasts looking to access historical and real-time meteorological data for analysis, visualization, and educational purposes. This information is most often used as a complementary source to national meteorological services that is delivered free of charge and without any API key requirements. @@ -39,24 +39,37 @@ library(climate) # Download hourly data for Svalbard Lufthavn (WMO 01008) df = meteo_ogimet(interval = "hourly", - date = c("2018-01-01", "2018-12-31"), + date = c("2020-01-01", "2020-01-31"), station = "01008") head(df[, c("date", "station", "t2m", "ws", "wd")]) ``` -```{r svalbard-windrose, echo=TRUE, message=FALSE, warning=FALSE} +``` +## station: 01008 +## http://www.ogimet.com/cgi-bin/getsynop?block=01008&begin=202001010000&end=202001312359 +## Downloaded 712 SYNOP messages for: 01008 +## date station t2m ws wd +## 1 2020-01-01 00:00:00 01008 -17.2 4 100 +## 2 2020-01-01 01:00:00 01008 -16.2 4 110 +## 3 2020-01-01 02:00:00 01008 -15.9 2 100 +## 4 2020-01-01 03:00:00 01008 -16.3 3 120 +## 5 2020-01-01 04:00:00 01008 -16.0 3 100 +## 6 2020-01-01 05:00:00 01008 -17.5 5 100 +``` + +```{r svalbard-windrose, echo=TRUE} library(openair) # Wind rose by season — no unit conversion needed windRose(mydata = df, ws = "ws", wd = "wd", - type = "season", paddle = FALSE, - main = "Svalbard Lufthavn (2018)", - ws.int = 3, - dig.lab = 3, - layout = c(4, 1)) + title = "Svalbard Lufthavn") +``` + +```{r svalbard-windrose-img, echo=FALSE, eval=TRUE, out.width="100%"} +knitr::include_graphics("figures/ogimet_fig01.png") ``` ```{r svalbard-timeplot, echo=TRUE} @@ -64,6 +77,10 @@ windRose(mydata = df, timePlot(df, pollutant = c("t2m", "ws")) ``` +```{r svalbard-timeplot-img, echo=FALSE, eval=TRUE, out.width="100%"} +knitr::include_graphics("figures/ogimet_fig02.png") +``` + ```{r svalbard-polar, echo=TRUE} # Which wind sectors bring warm / cold air masses? polarPlot(df, @@ -72,12 +89,14 @@ polarPlot(df, wd = "wd", k = 50, force.positive = FALSE, - type = "season", - layout = c(4, 1), resolution = "fine", normalise = FALSE) ``` +```{r svalbard-polar-img, echo=FALSE, eval=TRUE, out.width="100%"} +knitr::include_graphics("figures/ogimet_fig03.png") +``` + --- ## 2. Daily summaries — multi-station temperature comparison @@ -90,26 +109,73 @@ library(climate) # Daily summaries for two Polish stations: Poznan (12330) and Warsaw (12375) daily = meteo_ogimet(interval = "daily", - date = c("2023-06-01", "2023-08-31"), - station = c(12330, 12375), - coords = TRUE) + date = c("2026-06-01", "2026-06-30"), + station = c(12330, 12375)) head(daily) ``` -```{r daily-plot, echo=TRUE, message=FALSE} +``` +## Daily raports will be generated for 6 UTC each day. Use the >>hour<< argument to change it +## station: 12330 +## INFO: Please note that the Ogimet has recently limited number of queries that are accepted +## by the server from a single IP address. Therefore, downloading more than 1 month of data +## for a single station requires 20 seconds pause between subsequent queries and +## may take a while. Thank you for your patience. +## station: 12375 +## +## Date Lon Lat Alt Temperature_Max Temperature_Min +## +## 1: 2026-06-30 16.8344 52.4167 88 32.7 18.9 +## 2: 2026-06-30 20.9608 52.1628 107 36 20.7 +## 3: 2026-06-29 16.8344 52.4167 88 38.9 21.6 +## 4: 2026-06-29 20.9608 52.1628 107 38.1 24.7 +## 5: 2026-06-28 16.8344 52.4167 88 35.8 21.5 +## 6: 2026-06-28 20.9608 52.1628 107 34.7 20.4 +## Temperature_Avg TdAvg Hr.Avg App.TAvg Wind_Dir. Wind_Int. Wind_Gust +## +## 1: 25.5 17.7 64.7 24.8 NNW 16.8 43.2 +## 2: 29.5 16.5 49.2 28.3 N 16.7 +## 3: 30.5 17.7 50.9 30.9 W 11.6 43.2 +## 4: 31.9 16.5 41.2 32.8 SW 6.9 36 +## 5: 29 15.1 46.8 29.3 SSE 8 +## 6: 28.4 13.6 42.7 28.4 SE 6.5 +## Pres.s.lev Precmm SunD.1 SnowDep TotClOct lowClOct VisKm station_ID Prec. +## +## 1: 1021.3 5.6 3.3 43.1 12330 0.2 +## 2: 1018.6 4.2 0.8 44.4 12375 0.9 +## 3: 1016.2 4.6 1.2 32.5 12330 0.0 +## 4: 1015.9 2.5 0.7 37 12375 0.0 +## 5: 1017.4 2.7 0.2 38.8 12330 0.0 +## 6: 1019 1.5 0 37.1 12375 0.0 +## SunD-1 +## +## 1: 10.1 +## 2: 14.7 +## 3: 13.4 +## 4: 15.0 +## 5: 15.5 +## 6: 15.5 +``` + +```{r daily-plot, echo=TRUE} library(ggplot2) +daily$Temperature_Avg = as.numeric(daily$Temperature_Avg) # Compare average temperatures between the two stations -ggplot(daily, aes(x = as.Date(Date), y = TemperatureCAvg, +ggplot(daily, aes(x = as.Date(Date), y = Temperature_Avg, colour = factor(station_ID), group = station_ID)) + geom_line() + - labs(title = "Daily average temperature — summer 2023", + labs(title = "Daily average temperature - June 2026", x = "Date", y = "Temperature (°C)", colour = "Station (WMO ID)") + theme_bw() ``` +```{r daily-plot-img, echo=FALSE, eval=TRUE, out.width="100%"} +knitr::include_graphics("figures/ogimet_fig04.png") +``` + --- ## 3. Country-level bulk download @@ -130,11 +196,26 @@ cat("Observations:", nrow(poland), "\n") head(poland[, c("date", "station", "t2m", "ws", "slp")]) ``` -```{r country-map, echo=TRUE, message=FALSE, warning=FALSE} +``` +## Downloading country: Poland +## http://www.ogimet.com/cgi-bin/getsynop?begin=202312150000&end=202312152359&state=Poland +## Downloaded 1438 SYNOP messages for: Poland +## Stations: 61 +## Observations: 1438 +## date station t2m ws slp +## 1 2023-12-15 00:00:00 12001 0.9 8 1022.0 +## 2 2023-12-15 06:00:00 12001 2.5 10 1022.3 +## 3 2023-12-15 12:00:00 12001 3.6 8 1024.5 +## 4 2023-12-15 18:00:00 12001 5.0 14 1025.2 +## 5 2023-12-15 00:00:00 12100 1.5 1 1023.8 +## 6 2023-12-15 01:00:00 12100 2.1 2 1024.7 +``` + +```{r country-map, echo=TRUE} # Quick spatial overview — requires coordinates from stations_ogimet() library(dplyr) -station_meta = stations_ogimet(country = "Poland", date = Sys.Date()) +station_meta = stations_ogimet(country = "Poland") # Join decoded observations with coordinates poland_geo = poland %>% @@ -146,11 +227,15 @@ ggplot(poland_geo, aes(x = lon, y = lat, colour = t2m_mean, size = t2m_mean)) + geom_point(alpha = 0.7) + scale_colour_distiller(palette = "RdYlBu", direction = -1, name = "T2m (°C)") + coord_quickmap() + - labs(title = "Mean temperature — Poland, 2023-12-15") + + labs(title = "Mean temperature - Poland, 2023-12-15") + theme_bw() + theme(legend.position = "right") ``` +```{r country-map-img, echo=FALSE, eval=TRUE, out.width="100%"} +knitr::include_graphics("figures/ogimet_fig05.png") +``` + --- ## 4. Full SYNOP output with `return_list` @@ -174,10 +259,50 @@ names(result$data) names(result$full) # Variables only available in the full output -result$full[1, c("low_cloud_type", "middle_cloud_type", "high_cloud_type", +result$full[1:5, c("low_cloud_type", "middle_cloud_type", "high_cloud_type", "precipitation_time", "snow_depth_state", "source")] ``` +``` +## station: 12330 +## http://www.ogimet.com/cgi-bin/getsynop?block=12330&begin=202306010000&end=202306032359 +## Downloaded 72 SYNOP messages for: 12330 +## [1] "date" "station" "t2m" "dpt2m" "rel_hum" +## [6] "tmax" "tmin" "wd" "ws" "gust" +## [11] "press" "slp" "press_tend" "precip" "Nt" +## [16] "Nh" "N_base" "insol" "visibility" "snow" +## [1] "station_id" "Date" "station_type" +## [4] "region" "obs_day" "obs_hour" +## [7] "wind_unit" "wind_estimated" "visibility" +## [10] "cloud_cover" "wind_direction" "wind_speed" +## [13] "air_temperature" "dewpoint_temperature" "station_pressure" +## [16] "sea_level_pressure" "pressure_tendency" "pressure_change" +## [19] "precipitation_amount" "precipitation_time" "cloud_base_min" +## [22] "cloud_base_max" "low_cloud_type" "middle_cloud_type" +## [25] "high_cloud_type" "low_cloud_amount" "maximum_temperature" +## [28] "minimum_temperature" "gust" "cloudiness_height" +## [31] "sunshine_duration" "snow_depth" "snow_depth_state" +## [34] "source" +## low_cloud_type middle_cloud_type high_cloud_type precipitation_time +## 1 0 0 1 12 +## 2 0 0 1 NA +## 3 0 0 1 NA +## 4 0 0 4 NA +## 5 0 0 1 NA +## snow_depth_state +## 1 NA +## 2 NA +## 3 NA +## 4 NA +## 5 NA +## source +## 1 AAXX 01001 12330 12982 12101 10119 20060 30074 40182 57008 60001 80001== +## 2 AAXX 01011 12330 42982 22201 10124 20064 30072 40179 58006 80001== +## 3 AAXX 01021 12330 42982 22101 10113 20064 30067 40175 57011 80001== +## 4 AAXX 01031 12330 42981 62601 10101 20059 30066 40174 57008 80004== +## 5 AAXX 01041 12330 42981 50000 10133 20069 30065 40172 56007 80001== +``` + --- ## 5. Selecting the backend explicitly @@ -187,21 +312,51 @@ Override the automatic backend selection with the `source` argument: ```{r backend-override, echo=TRUE} library(climate) -# Force SYNOP backend for daily data -poznan_synop = meteo_ogimet(interval = "daily", - station = 12330, - date = c("2023-06-01", "2023-06-07"), - source = "synop") -head(poznan_synop[, c("date", "station", "t2m", "tmax", "tmin", "precip")]) - -# Force HTML backend for hourly data (returns km/h wind speed, character wind direction, etc.) +# Force HTML backend for hourly data poznan_html = meteo_ogimet(interval = "hourly", station = 12330, date = c("2023-06-01", "2023-06-03"), - source = "html", - coords = TRUE, + source = "html", # or synop precip_split = TRUE) -head(poznan_html[, c("station_ID", "Date", "TC", "ffkmh", "ddd", "pr6", "pr12", "pr24")]) +head(poznan_html) ``` -> **Note:** `meteo_ogimet_synop()` is deprecated. Use `meteo_ogimet(source = "synop")` instead. +``` +## station: 12330 +## INFO: Please note that the Ogimet has recently limited number of queries that are accepted +## by the server from a single IP address. Therefore, downloading more than approx. 1 day of data +## for a single station requires 20 seconds pause between subsequent queries and +## may take a while. Thank you for your patience. +## station_ID Lon Lat Date T Td Tmax Tmin +## +## 1: 12330 16.8344 52.4167 2023-06-03 06:00:00 10.3 1.9 16.1 5.4 +## 2: 12330 16.8344 52.4167 2023-06-03 05:00:00 8.9 2.7 +## 3: 12330 16.8344 52.4167 2023-06-03 04:00:00 7.1 2.6 +## 4: 12330 16.8344 52.4167 2023-06-03 03:00:00 5.7 1.6 +## 5: 12330 16.8344 52.4167 2023-06-03 02:00:00 5.6 1.1 +## 6: 12330 16.8344 52.4167 2023-06-03 01:00:00 6.9 1 +## ddd ffkmh Gustkmh P0hPa P.seahPa PTnd Prec Nt Nh HKm +## +## 1: E 18 1014.9 0.5 0.0/12h 0 - +## 2: E 14.4 1014.8 0.8 0 - +## 3: ENE 14.4 1014.7 1.1 0 - +## 4: ENE 7.2 1014.4 1 0 - +## 5: ENE 7.2 1014 1 0 - +## 6: E 14.4 1013.6 0.9 0 - +## InsoD.1 Viskm Snowcm WW W1 W2 Hr Ta Gustmax P seahPa +## +## 1: 50 56 5.1 1025.8 +## 2: 50 65 4.5 1025.7 +## 3: 50 73 2.7 1025.7 +## 4: 50 75 2.6 1025.5 +## 5: 50 73 2.4 1025.1 +## 6: 50 66 2.3 1024.6 +## InsoD-1 pr6 pr12 pr24 +## +## 1: 10.6 NA 0 NA +## 2: NA NA NA +## 3: NA NA NA +## 4: NA NA NA +## 5: NA NA NA +## 6: NA NA NA +```