Introduction to s2dv_cube objects

Loading multi-dimensional data with c4h_load

The loading function in clim4health is c4h_load(), and takes the following input parameters:

• path A string to the folder containing the files to be loaded, or a vector of specific file paths.

• variable The variable that you want to load.

• year A vector of years, e.g. 1994:2016. Default is all years found in the specified path.

• month A vector of months, e.g. c(1,2) for January and February. Default is all months (1:12).

• day A vector of days in the month, e.g. c(1, 2, 3) for the first three days of each month. Default is all days.

• time A vector of hours (0:23). Only needed when datasets are hourly. Default is all hours.

• leadtime_month A vector of months for the forecast leadtime. e.g. c(1, 2) for February and March for data initialised in February. Default is "all". If "all" for hindcast and forecast data, all leadtime months will be loaded. If "all" for reanalysis or station data, data will be loaded as a time series in the time dimension, and the sdate dimension will be set to 1.

• ext File extension. Options include "nc", ".nc", "csv", and ".csv".

• bbox A vector of coordinates to subset the data spatially. In order: c(lat_max, lon_min, lat_min, lon_max).

We use the parameters year, month, day, time, and leadtime_month to specify the time period and time dimensions of the data we want to load. To load data in the sdate or time dimension:

• sdate: year, month, day, and time correspond to all the start dates that you want to load. To load data for January and February for the years 2010 to 2016, you would select year = 2010:2016 and month = 1:2.

• time: leadtime_month corresponds to the forecast lead time and tells clim4health how many months to load in the time dimension. To load our hindcast data issued only in January for the forecast months of January, February and March, for the years 1994 to 2016, we would select year = 1994:2016, month = 1, and leadtime_month = 1:3. Note that this exact specification would result in a 2D time structure as given in the table above (Section 1).

Loading example data

Consider our seasonal forecast example. We will load the forecast data for January, February, and March 2025, and the corresponding hindcast data for January, February, and March of previous years (2010 to 2012). We will also load the reanalysis data for the same time period. In a full analysis, the reference period should be much longer (typically 1994 to 2016). All of these datasets will be stored as s2dv_cube objects.

fcst_path <- system.file("extdata/forecast/", package = "clim4health")
fcst_path <- paste0(fcst_path, "/")

# Load the forecast data
forecast <- c4h_load(fcst_path,
                     variable = "t2m",
                     year = 2025,
                     month = 1,
                     leadtime_month = "all",
                     ext = "nc")

forecast <- c4h_convert_units(forecast, to = "celsius")

hindcast_path <- system.file("extdata/hindcast/", package = "clim4health")
hindcast_path <- paste0(hindcast_path, "/")

# Load the hindcast data
hindcast <- c4h_load(hindcast_path,
                     variable = "t2m",
                     year = 2010:2012,
                     month = 1,
                     leadtime_month = 1:3,
                     ext = "nc")

hindcast <- c4h_convert_units(hindcast, to = "celsius")

reanalysis_path <- system.file("extdata/reanalysis/", package = "clim4health")
reanalysis_path <- paste0(reanalysis_path, "/")

# Load the reanalysis data
reanalysis <- c4h_load(reanalysis_path,
                       variable = "t2m",
                       year = 2010:2012,
                       month = 1,
                       leadtime_month = 1:3,
                       ext = "nc")

reanalysis <- c4h_convert_units(reanalysis, to = "celsius")

Exploring the loaded data

c4h_load() will return an s2dv_cube object. We can explore the structure of this object to understand how the data is stored and how to manipulate it.

An s2dv_cube is a named list of the data, its dimensions, coordinates, and any additional attributes. Given our s2dv_cube object called forecast containing forecast data, the list elements contained within forecast are:

1. forecast$data: a multi-dimensional array containing one or more variables, such as temperature and pressure.
2. forecast$dims: a named vector of the dimensions of forecast$data containing the names of the dimensions and their lengths.
3. forecast$coords: a named list of the coordinates of the array. For example, if a variable is stored on dimensions of latitude, longitude, and time, then forecast$coords will contain the values of each of these coordinates. In some cases, only indices will be stored in the coordinates, such as in the case of the time dimension, where the actual dates are stored in the attributes (see below).
4. forecast$attrs: a named list of all available metadata. It includes metadata for each variable, and common shared attributes between variables.

message("print forecast class")
class(forecast)
message("print dimensions of the stored data")
dim(forecast$data)
message("print the names of the list elements in forecast")
names(forecast)
message("print a summary of the data stored in forecast")
summary(forecast$data)
message("print extended information about the list elements in forecast")
str(forecast)

Data dimensions in clim4health

clim4health enforces the following dimensions for its s2dv_cube objects:

• dataset - the datasets that are loaded.

• var - the variables that are loaded, e.g. temperature, precipitation, etc.

• sdate - the forecast initialisation or start dates.

• time - the forecast lead time.

• ensemble - the model ensemble members.

• one or two spatial dimensions, either:

  • latitude and longitude for gridded data

  • area for polygon data, e.g. administrative areas such as states or provinces

  • location for point data, e.g. weather station data

💡 Tip: the dimension dataset will always have length 1 in clim4health cases. It only exists because it is required for package dependencies in the downscaling and skill functions.

To better understand how dates are handled in clim4health, take a look at the attributes of the s2dv_cube. For example, all s2dv_cubes loaded via clim4health will contain a Dates attribute.

print(forecast$attrs$Dates)

print(dim(forecast$attrs$Dates))

print(forecast$attrs$Dates[, 1]) # to obtain the forecast start dates
print(forecast$coords$sdate) # start dates are also stored as a coordinate

We can plot the s2dv_cube objects using the function c4h_plot. The plotting function will search the available dimensions in the data cube and plot maps for each of them. This is problematic for the forecast object, which has three time points and 51 ensemble members, which would mean a total of 153 maps! We can do several things, first, simply select the time points and ensemble members we are interested in:

c4h_plot(forecast, time = 1:3, ensemble = 1:3)

1. Load the data

Let’s load the example csv file stored in clim4health using another method.

csv_path <- system.file("extdata/stations/", package = "clim4health")
data_in <- read.csv(paste0(csv_path, "/temp_vallecauca.csv"))

Take a look at the structure of the data to understand how it is stored and what information is available.

print(head(data_in))

We see that this data contains columns called station_code, lat, lon, date, temp_mean, temp_min, and temp_max.

💡 Tip: At this point, you could also filter your data by requirements such as relevant dates, latitudes, longitudes, and variables.

2. Reshape the data

First, let’s extract the coordinates at which our data will be stored.

# these will be used to match values in the data_in object
locations_long <- data_in$station_code
lats_long <- data_in$lat
lons_long <- data_in$lon

# these will be our unique coordinates for the s2dv_cube
locations <- unique(locations_long)
lats <- unique(lats_long[match(locations, locations_long)])
lons <- unique(lons_long[match(locations, locations_long)])

# extract dates in a time series format (sdate = 1, time = length of dates)
dates_long <- data_in$date
dates <- unique(dates_long)
# we can choose specific date ranges if desired
dates <- dates[which(lubridate::month(dates) %in% 1:2)]

# set up final coordinates
ensemble <- 1
var <- c("temp_mean", "temp_min", "temp_max")

Now, we need to match the values in the data frame to the coordinates we have just extracted, and reshape the data into a multi-dimensional array with the correct dimensions.

# create an empty array with the correct dimensions
data_array <- array(NA, dim = c(dataset = 1,
                                var = length(var),
                                sdate = 1,
                                time = length(dates),
                                ensemble = length(ensemble),
                                location = length(locations)))

# fill the array with the values from the data frame, matching the coordinates
for (i in seq_along(var)) {
  var_name <- var[i]
  for (j in seq_along(locations)) {
    location_name <- locations[j]
    for (k in seq_along(dates)) {
      date_value <- dates[k]
      value <- data_in[locations_long == location_name &
                       dates_long == date_value, var_name]
      # only fill the array if there is a value in the data frame
      if (length(value) != 0) {
        data_array[1, i, 1, k, 1, j] <- value
      }
    }
  }
}

💡 Tip: Here we loop over dimensions to explicitly show how the new array is filled, but it can be more efficiently filled in case there are more locations or dates. Similarly, we set sdate to have length 1 here, but you can also add a loop over this too.

3. Add dimensions, coordinates, and attributes

Now that we have reshaped the data, we need to add the dimensions, coordinates, and attributes to create the full s2dv_cube object.

new_dims <- dim(data_array)

new_coords <- list(dataset = 1,                     # index
                   var = var,                       # variable names
                   sdate = 1,                       # index
                   time = seq_along(dates),         # index
                   ensemble = ensemble,             # index
                   location = seq_along(locations)) # index

new_attrs <- list(
  Dates = array(dates, dim = c(sdate = 1, time = length(dates))),
  Variable = list(varName = var,
                  metadata = list(variable = var,
                                  units = c(temp_mean = "degrees C",
                                            temp_min = "degrees C",
                                            temp_max = "degrees C"),
                                  source_files = "inst/extdata/stations/temp_vallecauca.csv")),
  source_files = "inst/extdata/stations/temp_vallecauca.csv")

# in the case of station data, we also need to add the latitude and longitude coordinates to the attributes
new_attrs$location <- list(longitude = lons,
                           latitude = lats)

# we also need to ensure the Dates attribute is a POSIXct object
new_attrs$Dates <- as.POSIXct(new_attrs$Dates)
dim(new_attrs$Dates) <- c(sdate = 1, time = length(dates))

💡 Warning: Converting to POSIXct objects can flatten your dates array, so make sure you double check both the values of the dates array and its dimension!

4. Create the s2dv_cube object

Finally, we can create the s2dv_cube object. There are two ways to do this. The first is simply to manually create the list and set its class, and the second is to use the function s2dv_cube() from the package CSTools.

# method 1: manually create the list and set its class
s2dv_cube_manual <- list(data = data_array,
                         dims = new_dims,
                         coords = new_coords,
                         attrs = new_attrs)
class(s2dv_cube_manual) <- "s2dv_cube"

# method 2: use the function s2dv_cube() from CSTools
s2dv_cube_function <- CSTools::s2dv_cube(data_array,
                                         coords = new_coords,
                                         varName = var,
                                         metadata = new_attrs$Variable$metadata,
                                         Dates = new_attrs$Dates,
                                         source_files = new_attrs$source_files)

We have successfully created our own s2dv_cube object from a csv file, and we can now use this object with the functions in clim4health.