,
Arnon Karnieli [ctb, fnd] ,
Ron Beiden [ctb],
Natalya Panov [ctb],
Lauren O'Brien [rev],
Harry Eslick [rev]| Title: | Derive Soil Moisture Using the OPTRAM Algorithm |
|---|---|
| Description: | The OPtical TRapezoid Model (OPTRAM) derives soil moisture based on the linear relation between a vegetation index and Land Surface Temperature (LST). The Short Wave Infra-red (SWIR) band is used as a proxy for LST. See: Sadeghi, M. et al., 2017. <https://doi.org/10.1016/j.rse.2017.05.041> . |
| Authors: | Micha Silver [aut, cre] ,
Arnon Karnieli [ctb, fnd] ,
Ron Beiden [ctb],
Natalya Panov [ctb],
Lauren O'Brien [rev],
Harry Eslick [rev] |
| Maintainer: | Micha Silver <[email protected]> |
| License: | GPL (>= 3) |
| Version: | 0.3.0.000 |
| Built: | 2024-07-04 17:26:23 UTC |
| Source: | https://github.com/ropensci/rOPTRAM |
This non-exported function uses the openeo package
to send a request to Copernicus DataSpace, and prepare the products.
Called by optram_acquire_s2
acquire_openeo( aoi, from_date, to_date, output_dir = tempdir(), scale_factor = 10000 )acquire_openeo( aoi, from_date, to_date, output_dir = tempdir(), scale_factor = 10000 )
aoi |
sf object, POLYGON or MULTIPOLYGON of area of interest |
from_date |
string, represents start of date range, formatted as "YYYY-MM-DD" |
to_date |
string, end of date range, formatted as "YYYY-MM-DD" |
output_dir |
string, path to save downloaded, and processed imagery |
scale_factor |
integer, scaling factor for EO data source default 10000 , to scale Sentinel-2 15 bit DN to range (0, 1) |
list of BOA files
This function utilizes the openeo package.
Instructions for the login process:
First of all, to authenticate your account on the backend of the Copernicus
Data Space Ecosystem, it is necessary for you to complete the registration
process. Follow these instructions for registration:
https://documentation.dataspace.copernicus.eu/Registration.html
After you have registered and installed the openeo package, you can run the
acquire_openeo function.
During the process of connecting to the server and logging in, you need to
follow these steps:
When the message "Press
When prompted with 'Copy CTGB-UGFU and paste when requested by the browser' in the console, it may appear but can be ignored, as it is related to an older version of the openeo package.
Calling this method opens your system web browser, with which you can authenticate yourself on the back-end authentication system.
After that, the website will give you instructions to go back to the R client, where your connection has logged your account in. This means that every call that comes after that via the connection variable is executed by your user account.
You will be redirected to https://identity.dataspace.copernicus.eu/. Ensure you have an account and are logged in. You will be required to grant access - press "yes".
## Not run: from_date <- "2018-12-01" to_date <- "2019-04-30" aoi <- sf::st_read(system.file("extdata", "lachish.gpkg", package = 'rOPTRAM')) Important! Assign the output to an object for later retrieval acq <- acquire_openeo(aoi, from_date, to_date, veg_index = "SAVI", SWIR_band = 11) ## End(Not run)## Not run: from_date <- "2018-12-01" to_date <- "2019-04-30" aoi <- sf::st_read(system.file("extdata", "lachish.gpkg", package = 'rOPTRAM')) Important! Assign the output to an object for later retrieval acq <- acquire_openeo(aoi, from_date, to_date, veg_index = "SAVI", SWIR_band = 11) ## End(Not run)
CDSE PackageThis function uses the CDSE package
to send a request to Copernicus Dataspace, and prepare the products.
Called by optram_acquire_s2
acquire_scihub( aoi, from_date, to_date, output_dir = tempdir(), save_creds = TRUE, clientid = NULL, secret = NULL )acquire_scihub( aoi, from_date, to_date, output_dir = tempdir(), save_creds = TRUE, clientid = NULL, secret = NULL )
aoi |
sf object, POLYGON or MULTIPOLYGON of area of interest |
from_date |
string, represents start of date range, formatted as "YYYY-MM-DD" |
to_date |
string, end of date range, formatted as "YYYY-MM-DD" |
output_dir |
string, path to save downloaded, and processed imagery |
save_creds |
logical, whether to save CDSE credentials. Default TRUE. |
clientid |
string, user's OAuth client id. Required if |
secret |
string, user's OAuth secret. Required if |
list of BOA files
#' This function utilizes the CDSE package.
Make sure to install the CDSE and jsonlite packages.
Create OAuth account and token:
Creating an Account:
Navigate to the https://dataspace.copernicus.eu/.
Click the "Register" button to access the account creation page.
If already registered, enter your username and password, and click "Login."
Once logged in, go to the User dashboard and click "User Settings" to access the Settings page.
Creating OAuth Client:
On the Settings page, click the green "Create New" button located on the right.
Enter a suitable "Client Name" and click the green "Create Client" button.
A Client secret is generated.
The user must save her secret and clientid somewhere.
These credentials will be saved automatically to a standard filesystem
location if the user calls store_cdse_credentials()
with the clientid and secret parameters.
If the user chooses not to save credentials to the standard filesystem
location, then she will need to add both clientid and secret to each
acquire_scihub() function call.
Using Credentials with aquire_scihub:
If the credentials were stored using store_cdse_credentials(), the credentials are retrieved automatically.
Otherwise, you can utilize the generated clientid and secret from https://dataspace.copernicus.eu/ within the aquire_scihub() function.
If you want to store your credentials on your computer, ensure that when running aquire_scihub(), the save_creds parameter is set to 'TRUE'.
During the first run of aquire_scihub(), manually input your clientid and secret in the function signature. Subsequent runs will use the stored credentials.
Subject Area Constraint: The downloadable images are restricted to a maximum size of 2500 pixels on each side. This limitation is established due to the final resolution set to 10 meters using JavaScript. Consequently, the subject area available for download is limited to 25 kilometers in both directions. Please be aware of this restriction when selecting your desired area for download. Area of Interest (AOI) Specification: When defining your Area of Interest (AOI), please ensure that it is represented as a polygonal layer with only one feature. This feature can either be a single POLYGON or a MULTIPOLYGON, which may consist of non-contiguous areas, but only one feature is permissible.
## Not run: from_date <- "2018-12-01" to_date <- "2019-04-30" aoi <- sf::st_read(system.file("extdata", "lachish.gpkg", package = 'rOPTRAM')) Make sure the AOI is a MULTIPOLYGON aoi <- sf::st_as_sf(sf::st_union(aoi)) optram_options("veg_index", "NDVI", show_opts = FALSE) optram_options("SWIR_band", 11) acquire_scihub(aoi, from_date, to_date, clientid = '...enter clientid here...', secret = '...enter secret here...') ## End(Not run)## Not run: from_date <- "2018-12-01" to_date <- "2019-04-30" aoi <- sf::st_read(system.file("extdata", "lachish.gpkg", package = 'rOPTRAM')) Make sure the AOI is a MULTIPOLYGON aoi <- sf::st_as_sf(sf::st_union(aoi)) optram_options("veg_index", "NDVI", show_opts = FALSE) optram_options("SWIR_band", 11) acquire_scihub(aoi, from_date, to_date, clientid = '...enter clientid here...', secret = '...enter secret here...') ## End(Not run)
Use this function to prepare STR from SAFE imagery when you have already downloaded Sentinel 2 image files in advance
calculate_str(img_stk, scale_factor = 10000)calculate_str(img_stk, scale_factor = 10000)
img_stk |
terra SpatRaster, multiband stack of images, already clipped to aoi |
scale_factor |
integer, scaling factor for EO data source default 10000, to scale Sentinel-2 15 bit DN to range (0, 1) |
STR, SpatRaster of STR band
For Landsat images, scale_factor should be 1, since Landsat metadata contains gain and offset for scaling image bands.
img_stk <- terra::rast(system.file("extdata", "BOA", "BOA_2022-12-11_T36RXV.tif", package = "rOPTRAM")) STR_dir = tempdir() str <- calculate_str(img_stk)img_stk <- terra::rast(system.file("extdata", "BOA", "BOA_2022-12-11_T36RXV.tif", package = "rOPTRAM")) STR_dir = tempdir() str <- calculate_str(img_stk)
Use this function to prepare vegetation index from SAFE imagery when you have already downloaded Sentinel 2 image files in advance
calculate_vi( img_stk, redband = 4, greenband = 3, blueband = 2, nirband = 5, scale_factor = 2^15 )calculate_vi( img_stk, redband = 4, greenband = 3, blueband = 2, nirband = 5, scale_factor = 2^15 )
img_stk |
terra SpatRaster, multiband stack of images, already clipped to aoi |
redband |
integer, number of red band |
greenband |
integer, number of green band |
blueband |
integer, number of blue band |
nirband |
integer, number of NIR band |
scale_factor |
numeric, scales Sentinel-2 15 bit values down to range (0,255) as expected for vegetation indices default 32768 (2^15) |
vi_rast, SpatRaster of vegetation index
The scale_factor parameter reduces numeric range "digital number", (DN) of Sentinel-2 images from 15 bit integer to 8 bit. Vegetation indices such as SAVI expect values in the range (0, 255). This scale_factor produces values in that range. When using Landsat 8 images (DN already in range of (0, 255)) then set scale_factor to 255.
img_stk <- terra::rast(system.file("extdata", "BOA", "BOA_2023-01-20_T36RXV.tif", package = "rOPTRAM")) vi <- calculate_vi(img_stk)img_stk <- terra::rast(system.file("extdata", "BOA", "BOA_2023-01-20_T36RXV.tif", package = "rOPTRAM")) vi <- calculate_vi(img_stk)
Utility function to prepare BOA bands cropped to Area of Interest.
crop_landsat_list(landsat_dir)crop_landsat_list(landsat_dir)
landsat_dir |
string, directory containing the original downloaded Landsat imagery folders. |
cropped_list, list, paths to derived BOA stacks, cropped to study area
## Not run: cropped_landsat_list <- crop_landsat_list(landsat_dir) ## End(Not run)## Not run: cropped_landsat_list <- crop_landsat_list(landsat_dir) ## End(Not run)
Called by optram_wetdry_coefficients
to prepare exponential curve along trapezoid edges.
Calculates the intercept and slope of both wet and dry edges
and updates the edges data.frame with these exp fitted values.
Not exported.
exponential_coefficients(df, output_dir)exponential_coefficients(df, output_dir)
df |
data.frame, values of VI and STR along edges of trapezoid |
output_dir |
string, path to save coefficients CSV file |
df, data.frame, the trapezoid line edge points with fitted wet/dry values added
Three CSV files are saved:
the regressions coefficients,
the trapezoid edge points
RMSE of the fitted curve
## Not run: df <- read.csv(system.file("extdata", "trapezoid_edges.csv", package = "rOPTRAM")) output_dir <- tempdir() coeffs <- rOPTRAM::exponential_coefficients(df, output_dir) coeffs ## End(Not run)## Not run: df <- read.csv(system.file("extdata", "trapezoid_edges.csv", package = "rOPTRAM")) output_dir <- tempdir() coeffs <- rOPTRAM::exponential_coefficients(df, output_dir) coeffs ## End(Not run)
Prepare soil moisture grid from STR and VI images for a single date, based on exponential function to derive trapezoid edges, using the derived slope and intercept coefficients
exponential_soil_moisture(coeffs, VI, STR)exponential_soil_moisture(coeffs, VI, STR)
coeffs |
list, 4 trapezoid coefficients |
VI |
terra rast, the vegetation index raster |
STR |
terra rast, the STR raster |
rast, soil moisture grid
This function is used after preparing the OPTRAM model coefficients with:
optram_wetdry_coefficients. Typically a new image date, (that was not used for preparing the model),
will be referenced in the img_date parameter.
The resulting soil moisture raster is saved to output_dir.
This function implements an exponential trapezoid, following:
Ambrosone, Mariapaola, et al. 2020.
“Retrieving Soil Moisture in Rainfed and Irrigated Fields
Using Sentinel-2 Observations and a Modified OPTRAM Approach.”
International Journal of Applied Earth Observation and Geoinformation 89 (July):
102113. doi:10.1016/j.jag.2020.102113
img_date <- "2023-03-11" VI_dir <- system.file("extdata", "NDVI", package = "rOPTRAM") STR_dir <- system.file("extdata", "STR", package = "rOPTRAM") SM <- optram_calculate_soil_moisture(img_date, VI_dir, STR_dir, data_dir = tempdir())img_date <- "2023-03-11" VI_dir <- system.file("extdata", "NDVI", package = "rOPTRAM") STR_dir <- system.file("extdata", "STR", package = "rOPTRAM") SM <- optram_calculate_soil_moisture(img_date, VI_dir, STR_dir, data_dir = tempdir())
Called by optram_wetdry_coefficients
to prepare linear regression line along trapezoid edges
Calculates the intercept and slope of both wet and dry edges
Not exported
linear_coefficients(df, output_dir)linear_coefficients(df, output_dir)
df |
data.frame, values of VI and STR along edges of trapezoid |
output_dir |
string, path to save coefficients CSV file |
df, data.frame, the trapezoid line edge points with fitted wet/dry values added
Three CSV files are saved:
the regressions coefficients,
the trapezoid edge points
RMSE of the fitted curve
## Not run: edges_file <- system.file("extdata/trapezoid_edges.csv", package = "rOPTRAM") df <- read.csv(edges_file) output_dir <- tempdir() coeffs <- linear_coefficients(df, output_dir) coeffs ## End(Not run)## Not run: edges_file <- system.file("extdata/trapezoid_edges.csv", package = "rOPTRAM") df <- read.csv(edges_file) output_dir <- tempdir() coeffs <- linear_coefficients(df, output_dir) coeffs ## End(Not run)
Prepare soil moisture grid from STR and VI images for a single date, using linear trapezoid method, and using the derived slope and intercept coefficients
linear_soil_moisture(coeffs, VI, STR)linear_soil_moisture(coeffs, VI, STR)
coeffs |
list, 4 trapezoid coefficients |
VI |
SpatRaster, the vegetation index raster |
STR |
SpatRaster, the STR raster |
rast, soil moisture grid
This function is used after preparing the OPTRAM model coefficients with:
optram_wetdry_coefficients. Typically a new image date,
(that was not used for preparing the model),
will be referenced in the img_date parameter.
The resulting soil moisture raster is saved to output_dir.
img_date <- "2023-03-11" VI_dir <- system.file("extdata", "NDVI", package = "rOPTRAM") STR_dir <- system.file("extdata", "STR", package = "rOPTRAM") SM <- optram_calculate_soil_moisture(img_date, VI_dir, STR_dir, data_dir = tempdir())img_date <- "2023-03-11" VI_dir <- system.file("extdata", "NDVI", package = "rOPTRAM") STR_dir <- system.file("extdata", "STR", package = "rOPTRAM") SM <- optram_calculate_soil_moisture(img_date, VI_dir, STR_dir, data_dir = tempdir())
The main wrapper function to download, and preprocess Sentinel 2 imagery then prepare the OPTRAM trapezoid to derive slope and intercept for wet and dry trapezoid lines. See: Sadeghi, M., Babaeian, E., Tuller, M., Jones, S.B., 2017. The optical trapezoid model: A novel approach to remote sensing of soil moisture applied to Sentinel-2 and Landsat-8 observations. Remote Sensing of Environment 198, 52–68, doi:10.1016/j.rse.2017.05.041
optram( aoi, from_date, to_date, S2_output_dir = tempdir(), data_output_dir = tempdir() )optram( aoi, from_date, to_date, S2_output_dir = tempdir(), data_output_dir = tempdir() )
aoi |
sf object, a POLYGON or MULTIPOLYGON, boundary of area of interest |
from_date |
string, the start of the date range, Formatted as "YYYY-MM-DD" |
to_date |
the end of the date range. |
S2_output_dir |
string, directory to save downloaded S2 and the derived products, defaults to tempdir() |
data_output_dir |
string, path to save coeffs_file and STR-VI data.frame, default is tempdir() |
rmse_df, data.frame, RMSE values of fitted trapezoid lines
the coefficients are also saved to a csv file in data_output_dir.
Sentinel downloaded products are saved to S2_output_dir. Data files (Trapezoid coefficients and STR-VI data) to data_output_dir
Products can be downloaded covering two "period" options: either "full" - all available images from from_date to to_date or "seasonal" - available images for all years but only for the months between
day/month of from_date to the day/month of to_date. Use optram_options to set this option.
Three trapezoid fitting methods are implemented: "linear", "exponential" and "polynomial". See the optram_options function for details
Two SWIR wavelength bands are available in Sentinel-2: 1610 nanometer (nm) and 2190 nm. The option SWIR_bands can be set in optram_options to choose which band is used in this model.
Several vegetation indices are implemented: "NDVI", "SAVI", etc. The optram_optionsfunction also sets this option.
## Not run: from_date <- "2018-12-01" to_date <- "2020-04-30" aoi <- sf::st_read(system.file("extdata", "lachish.gpkg", package = "rOPTRAM")) rmse <- optram(aoi, from_date, to_date) ## End(Not run)## Not run: from_date <- "2018-12-01" to_date <- "2020-04-30" aoi <- sf::st_read(system.file("extdata", "lachish.gpkg", package = "rOPTRAM")) rmse <- optram(aoi, from_date, to_date) ## End(Not run)
Use the CDSE package to acquire, preprocess and crop
Sentinel 2 satellite imagery.
optram_acquire_s2( aoi, from_date, to_date, output_dir = tempdir(), scale_factor = 10000, save_creds = TRUE, clientid = NULL, secret = NULL )optram_acquire_s2( aoi, from_date, to_date, output_dir = tempdir(), scale_factor = 10000, save_creds = TRUE, clientid = NULL, secret = NULL )
aoi |
sf POLYGON or MULTIPOLYGON, of boundary of area of interest |
from_date |
string, represents start of date range, formatted as "YYYY-MM-DD" |
to_date |
string, end of date range, formatted as "YYYY-MM-DD" |
output_dir |
string, path to save processed imagery. |
scale_factor |
numeric, scale factor for reflectance values. Default 10000. |
save_creds |
logical, whether to save CDSE credentials. Default TRUE. |
clientid |
string, user's OAuth client id. Required if |
secret |
string, user's OAuth secret. Required if |
output_path, string, path to downloaded files
This wrapper function calls one of multiple download functions, each accessing a different cloud-based resource. The cloud based resource can be one of "scihub" or "openeo":
"scihub",...
If "scihub" then:
This function utilizes the CDSE package.
Make sure to install the CDSE and jsonlite packages.
Create OAuth account and token:
Creating an Account:
Navigate to the Copernicus portal.
Click the "Register" button to access the account creation page.
If already registered, enter your username and password, and click "Login."
Once logged in, go to the User dashboard and click "User Settings" to access the Settings page.
Creating OAuth Client:
On the Settings page, click the green "Create New" button located on the right.
Enter a suitable "Client Name" and click the green "Create Client" button.
A Client secret is generated.
the user must save her secret and clientid somewhere.
these credentials will be saved automatically to a standard filesystem
location if the user calls check_scihub() with the argument save_creds
set to TRUE (recommended).
if the user chooses not to save credentials to the standard filesystem
location, then she will need to add both clientid and secret to each
acquire_scihub() function call.
Using Credentials with aquire_scihub():
Now, you can utilize the generated clientid and secret in
the aquire_scihub() function.
If you want to store your credentials on your computer, ensure that when running aquire_scihub, the save_creds parameter is set to TRUE.
During the first run of aquire_scihub(), manually input your clientid and secret in the function's signature.
Subsequent runs will use the stored credentials.
Using Credentials with aquire_scihub():
Now, you can utilize the generated clientid and secret in
the aquire_scihub() function.
If you want to store your credentials on your computer, ensure that when running aquire_scihub, the save_creds parameter is set to TRUE.
During the first run of aquire_scihub(), manually input your clientid and secret in the function's signature.
Subsequent runs will use the stored credentials.
Subject Area Constraint: The downloadable images are restricted to a maximum size of 2500 pixels on each side. This limitation is established due to the final resolution set to 10 meters using JavaScript. Consequently, the subject area available for download is limited to 25 kilometers in both directions. Please be aware of this restriction when selecting your desired area for download. Area of Interest (AOI) Specification: When defining your Area of Interest (AOI), please ensure that it is represented as a POLYGON or MULTIPOLYGON layer.
"openeo",...
If "openeo" then:
This function utilizes the openeo package.
Instructions for the login process:
First of all, to authenticate your account on the backend of the Copernicus
Data Space Ecosystem, it is necessary for you to complete the registration
process. Follow these instructions for registration:
https://documentation.dataspace.copernicus.eu/Registration.html
After you have registered and installed the openeo package, you can run the
acquire_openeo() function.
During the process of connecting to the server and logging in, you need to
follow these steps:
When the message "Press
Calling this method opens your system web browser, with which you can authenticate yourself on the back-end authentication system. After that, the website will give you instructions to go back to the R client, where your connection has logged your account in. This means that every call that comes after that via the connection variable is executed by your user account.
You will be redirected to https://identity.dataspace.copernicus.eu/. Ensure you have an account and are logged in. You will be required to grant access - press "yes".
Two SWIR bands are available in Sentinel-2: 1610 nanometer (nm) and 2190 nm.
Setting the option SWIR_bands with optram_options allows to choose which band is used in this model.
## Not run: from_date <- "2018-12-01" to_date <- "2019-04-30" aoi <- sf::st_read(system.file("extdata", "lachish.gpkg", package = 'rOPTRAM')) aoi <- sf::st_as_sf(sf::st_union(aoi)) s2_file_list <- optram_acquire_s2(aoi, from_date, to_date) ## End(Not run)## Not run: from_date <- "2018-12-01" to_date <- "2019-04-30" aoi <- sf::st_read(system.file("extdata", "lachish.gpkg", package = 'rOPTRAM')) aoi <- sf::st_as_sf(sf::st_union(aoi)) s2_file_list <- optram_acquire_s2(aoi, from_date, to_date) ## End(Not run)
Prepare soil moisture grid from STR and VI images for a single date using the derived slope and intercept coefficients
optram_calculate_soil_moisture( img_date, VI_dir, STR_dir, data_dir, output_dir = tempdir() )optram_calculate_soil_moisture( img_date, VI_dir, STR_dir, data_dir, output_dir = tempdir() )
img_date |
string, image date of single Sentinel 2 acquisition formatted as "YYYY-MM-DD" |
VI_dir |
string, full path to directory holding the VI rasters |
STR_dir |
string, full path to directory holding the STR rasters |
data_dir |
string, the directory where coefficients file was saved
(the |
output_dir |
string, full path to output directory for saving soil moisture raster |
SpatRaster, raster of soil moisture, file(s) saved in output_dir
This function is used after preparing the OPTRAM model coefficients with:
optram_wetdry_coefficients. Typically a new image date,
(possibly an image that was not used for preparing the model),
will be referenced in the img_date parameter.
The resulting soil moisture raster is saved to output_dir.
Three trapezoid models are offered through the trapezoid_method option:
either "linear", "exponential", or "polynomial". (set using optram_options)
The data_dir parameter is a directory name.
The coefficients CSV file that matches trapezoid_method
should be in that directory
For further details see: Ambrosone, Mariapaola, et al. 2020. “Retrieving Soil Moisture in Rainfed and Irrigated Fields Using Sentinel-2 Observations and a Modified OPTRAM Approach.” International Journal of Applied Earth Observation and Geoinformation 89 (July): 102113. doi:10.1016/j.jag.2020.102113.
The data_dir directory must contain the coefficients CSV file i.e. for "linear" method the file was saved as 'coefficients_lin.csv' for "exponential" it was saved as 'coefficients_exp.csv' for "polynomial" it was saved as 'coefficients_poly.csv'
img_date <- "2023-03-11" VI_dir <- system.file("extdata", "NDVI", package = "rOPTRAM") STR_dir <- system.file("extdata", "STR", package = "rOPTRAM") data_dir <- system.file("extdata") SM <- optram_calculate_soil_moisture(img_date, VI_dir, STR_dir, data_dir)img_date <- "2023-03-11" VI_dir <- system.file("extdata", "NDVI", package = "rOPTRAM") STR_dir <- system.file("extdata", "STR", package = "rOPTRAM") data_dir <- system.file("extdata") SM <- optram_calculate_soil_moisture(img_date, VI_dir, STR_dir, data_dir)
Create SWIR Transformed Reflectance
optram_calculate_str(BOA_dir, STR_dir = NULL)optram_calculate_str(BOA_dir, STR_dir = NULL)
BOA_dir |
string, the path to the Bottom of Atmosphere bands |
STR_dir |
string, output directory for STR rasters, Default is NULL, in which case, the STR_dir will be created alongside BOA_dir |
list of string, the path to transformed raster
This function follows: Sadeghi, M., Babaeian, E., Tuller, M., Jones, S.B., 2017. The optical trapezoid model: A novel approach to remote sensing of soil moisture applied to Sentinel-2 and Landsat-8 observations. Remote Sensing of Environment 198, 52–68. doi:10.1016/j.rse.2017.05.041
SWIR Transformed Reflectance is calculated as STR = (1-SWIR)^2 / 2*SWIR SWIR is band 12 (2190 nm) or 11 (1610 nm)
BOA_dir <- system.file("extdata", "BOA", package = "rOPTRAM") STR_dir = tempdir() STR <- optram_calculate_str(BOA_dir, STR_dir)BOA_dir <- system.file("extdata", "BOA", package = "rOPTRAM") STR_dir = tempdir() STR <- optram_calculate_str(BOA_dir, STR_dir)
Use this function to prepare vegetation index and SWIR Transformed Reflectance (STR) rasters when you have already downloaded Landsat image files in advance. This function assumes that atmospheric correction has been applied.
optram_landsat( landsat_dir, aoi, LC_output_dir = tempdir(), data_output_dir = tempdir() )optram_landsat( landsat_dir, aoi, LC_output_dir = tempdir(), data_output_dir = tempdir() )
landsat_dir |
string, full path to containing folder of downloaded (unzipped) Landsat data in original landsat format, after atmospheric correction (L2A) |
aoi |
sf object, POLYGON or MULTIPOLYGON of AOI boundary of area of interest |
LC_output_dir |
string, directory to save the derived products, defaults to tempdir() |
data_output_dir |
string, path to save coeffs_file and STR-VI data.frame, default is tempdir() |
rmse_df, data.frame, RMSE values of fitted trapezoid edges
Unlike the optram_acquire_s2 function, there is no implementation for automatic download of Landsat images.
This function requires a directory, set in the landsat_dir parameter,
which contains the set of Landsat tiles downloaded manually by the user, in advance.
This directory should contain folders of Landsat images, where each folder
consists of the individual Landsat bands as Geotiff files, as well as the metadata files as downloaded from,
i.e. the USGS EarthExplorer https://earthexplorer.usgs.gov/ website.
## Not run: aoi <- sf::st_read(system.file("extdata", "lachish.gpkg", package = "rOPTRAM")) # landsat_dir is a directory containing the original downloaded Landsat images. landsat_dir <- "...enter full path here..." optram_landsat(landsat_dir, aoi, veg_index = 'SAVI', LC_output_dir = tempdir(), data_output_dir = tempdir()) ## End(Not run)## Not run: aoi <- sf::st_read(system.file("extdata", "lachish.gpkg", package = "rOPTRAM")) # landsat_dir is a directory containing the original downloaded Landsat images. landsat_dir <- "...enter full path here..." optram_landsat(landsat_dir, aoi, veg_index = 'SAVI', LC_output_dir = tempdir(), data_output_dir = tempdir()) ## End(Not run)
Collect all pixel values of both vegetation index,
and Swir Transformed Reflectance, for a time series of images.
Prepare data.frame of all pairs of values
(as input for the optram_wetdry_coefficients function)
optram_ndvi_str(STR_list, VI_list, output_dir = tempdir(), aoi = NULL)optram_ndvi_str(STR_list, VI_list, output_dir = tempdir(), aoi = NULL)
STR_list |
list of paths to STR raster files |
VI_list |
list of paths to NDVI raster files |
output_dir |
string, path to save data.frames (in RDS format) |
aoi |
sf POLYGON or MULTIPOLYGON, must have a numeric column named "ID" for coloring trapezoid points by features Default NULL, (no coloring) |
full_df, data.frame with 7 columns: X,Y,Date,Month,NDVI,STR,Density
and optionally a 7th column with feature ID values from the AOI polygon.
The columns Month, Density, Feature_ID can be used in
plot_vi_str_cloud() function to color the points in the scatter plot
in various ways.
Use the option max_tbl_size (see optram_options)
to limit size of the NDVI-STR data.frame.
With a large area of interest, and long time frame,
the number of data points can overrun the computation resources.
This parameter sets a total size of data.frame from the max_tbl_size
parameter, together with the number of image time slots in the time range.
In some cases (i.e. water surfaces) NDVI can have values
below zero. These pixels can be removed from the trapezoid
by setting rm.low.vi
option to TRUE.
The vegetation index column is named "NDVI" even though it can represent other vegetation indices, such as SAVI, or MSAVI.
VI_list <- list.files(system.file("extdata", "NDVI", package = "rOPTRAM"), pattern = ".tif$", full.names = TRUE) STR_list <- list.files(system.file("extdata", "STR", package = "rOPTRAM"), pattern = ".tif$", full.names = TRUE) full_df <- optram_ndvi_str(STR_list, VI_list) # Show structure of output data.frame str(full_df)VI_list <- list.files(system.file("extdata", "NDVI", package = "rOPTRAM"), pattern = ".tif$", full.names = TRUE) STR_list <- list.files(system.file("extdata", "STR", package = "rOPTRAM"), pattern = ".tif$", full.names = TRUE) full_df <- optram_ndvi_str(STR_list, VI_list) # Show structure of output data.frame str(full_df)
rOPTRAM uses several package options. This function displays the current defined options, (showing the default options when the package is first loaded), and allows users to set new values for each option.
optram_options(opt_name = NULL, opt_value = NULL, show_opts = TRUE)optram_options(opt_name = NULL, opt_value = NULL, show_opts = TRUE)
opt_name |
string, one of the package options. Default NULL |
opt_value |
string, numeric, or boolean, depending on opt_name. |
show_opts |
boolean, default TRUE, prints a list of all optram options Default NULL |
When no new option name or value is specified, a list of currently defined options is printed.
rOPTRAM defines the following options at startup
| opt_name | default | other possible values |
| ---------------- | --------------- | ------------------- |
| veg_index | "NDVI" | "SAVI", "MSAVI" |
| remote | "scihub" | "openeo" |
| period | "full" | "seasonal" |
| max_cloud | 12 | between 0 and 100 |
| vi_step | 0.005 | usually between 0.01 and 0.001 |
| trapezoid_method | "linear" | "polynomial" or "exponential" |
| SWIR_band | 11 | 12 |
| max_tbl_size | 1e+6 | depends on computer resources |
| rm.low.vi | FALSE | TRUE |
| rm.hi.str | FALSE | TRUE |
| plot_colors | "no" | "no" = uniform green color for all points |
| "features" = points colored by aoi features | ||
| "density" = points colored by point density | ||
| "contours" = plots density contour lines | ||
| "months" = points colored by month of image date | ||
| feature_col | "ID" | string, name of numeric column |
| that contains feature ID's for coloring plot | ||
| edge_points | TRUE | FALSE, whether to add |
| the trapezoid edge points to the plot | ||
opts <- options() optram_options() # prints out list of current options optram_options("SWIR_band", 12) optram_options("veg_index", "SAVI") optram_options("trapezoid_list", "exp") # fails options(opts)opts <- options() optram_options() # prints out list of current options optram_options("SWIR_band", 12) optram_options("veg_index", "SAVI") optram_options("trapezoid_list", "exp") # fails options(opts)
This function prepares vegetation index and STR rasters from other (non-Sentinel) image files downloaded in advance.
optram_prepare_other_vi_str( img_dir, aoi, viname = "NDVI", output_dir = tempdir() )optram_prepare_other_vi_str( img_dir, aoi, viname = "NDVI", output_dir = tempdir() )
img_dir |
string, full path directory of downloaded images (assumes Geotiff) |
aoi |
string, full path to polygon spatial file of area of interest |
viname |
string, which VI to prepare, 'NDVI', 'SAVI', etc. |
output_dir |
string, where to save Geotiff, default is tempdir() |
output_files, list, full paths to saved Geotiff files (not exported yet)
Use this function to prepares vegetation index and SWIR Transformed Reflectance (STR) rasters when you have already downloaded Sentinel 2 image files in advance. Unzip the downloaded Sentinel 2 files and do not change the folder structure. This function assumes that atmospheric correction has been applied. i.e. by downloading Level L2A product or using the SNAP L2A_Process,
optram_safe( safe_dir, aoi, S2_output_dir = tempdir(), overwrite = TRUE, data_output_dir = tempdir() )optram_safe( safe_dir, aoi, S2_output_dir = tempdir(), overwrite = TRUE, data_output_dir = tempdir() )
safe_dir |
string, full path to containing folder of downloaded (unzipped) Sentinel 2 data in original SAFE format, after atmospheric correction (L2A) |
aoi |
sf object, a POLYGON or MULTIPOLYGON of the AOI boundary |
S2_output_dir |
string, directory to save the derived products, defaults to tempdir() |
overwrite |
boolean, overwrite derived products that already were created, defaults to TRUE |
data_output_dir |
string, path to save coeffs_file and STR-VI data.frame, default is tempdir() |
rmse_df, data.frame, RMSE values of fitted trapezoid lines
Use the max_tbl_size parameter to limit the total number of rows
in the VI-STR data.frame.
When the area of interest is large, or the time range of datasets is long,
the total size of the data.frame can grow
beyond the capacity of computation resources.
This parameter limits the size of the table
by sampling a number of data points from each time slot.
The sample size is determined based on max_tbl_size
and the total number of time slots in the full time range.
Two SWIR bands are available in Sentinel-2: 1610 nanometer (nm) and 2190 nm.
Set which band to use with optram_options.
## Not run: aoi <- sf::st_read(system.file("extdata", "lachish.gpkg", package = "rOPTRAM")) safe_dir <- "...enter directory containing downloaded SAFE folders..." rmse <- optram_safe(safe_dir, aoi_file) ## End(Not run)## Not run: aoi <- sf::st_read(system.file("extdata", "lachish.gpkg", package = "rOPTRAM")) safe_dir <- "...enter directory containing downloaded SAFE folders..." rmse <- optram_safe(safe_dir, aoi_file) ## End(Not run)
Derive slope and intercept coefficients
for both wet and dry trapezoid lines.
Write coefficients to a CSV file
(as input to optram_calculate_soil_moisture function)
optram_wetdry_coefficients(full_df, output_dir = tempdir())optram_wetdry_coefficients(full_df, output_dir = tempdir())
full_df |
data.frame of STR and NDVI values |
output_dir |
string, directory to save coefficients CSV file |
rmse_df, data.frame, RMSE values of fitted trapezoid edges
The vegetation index column is named "VI" though it can represent several vegetation indices, such as SAVI, or MSAVI.
The trapezoid method option (see optram_options) allows to choose one of three models for creating the edge coefficients of the trapezoid.
"linear" prepares a simple OLS regression line along the wet and dry edges of the trapezoid. Four coefficients are returned: intercept and slope for both edges.
"exponential" creates an exponential curve fitted to the intercept and slope, following: Ambrosone, Mariapaola, Alessandro Matese, et al. 2020. “Retrieving Soil Moisture in Rainfed and Irrigated Fields Using Sentinel-2 Observations and a Modified OPTRAM Approach.” International Journal of Applied Earth Observation and Geoinformation doi:10.1016/j.jag.2020.102113.
"polynomial" fits a second order polynomial curve to the wet and dry edges of the trapezoid, following: Ma, Chunfeng, Kasper Johansen, and Matthew F. McCabe. 2022. “Combining Sentinel-2 Data with an Optical-Trapezoid Approach to Infer within-Field Soil Moisture Variability and Monitor Agricultural Production Stages.” Agricultural Water Management 274 (December): 107942. doi:10.1016/j.agwat.2022.107942 This curve fitting function returns six coefficients: alpha, beta_1, and beta_2 for both wet and dry edges
full_df <- readRDS(system.file("extdata", "VI_STR_data.rds", package = "rOPTRAM")) rmse_df <- optram_wetdry_coefficients(full_df, tempdir()) print(rmse_df) optram_options("trapezoid_method", "polynomial") rmse_df <- optram_wetdry_coefficients(full_df, tempdir()) print(rmse_df)full_df <- readRDS(system.file("extdata", "VI_STR_data.rds", package = "rOPTRAM")) rmse_df <- optram_wetdry_coefficients(full_df, tempdir()) print(rmse_df) optram_options("trapezoid_method", "polynomial") rmse_df <- optram_wetdry_coefficients(full_df, tempdir()) print(rmse_df)
Plot STR-NDVI scatterplot to show dry and wet trapezoid lines over scatterplot of multi-temporal STR and NDVI pixel values
plot_vi_str_cloud(full_df, edges_df, edge_points = FALSE)plot_vi_str_cloud(full_df, edges_df, edge_points = FALSE)
full_df |
data.frame of NDVI and STR pixel values |
edges_df |
data.frame, points along the wet/dry edges for trapezoid |
edge_points |
boolean, whether to add to the plot the linear regression points that were used to derive coefficients. default FALSE |
ggplot object
Points in scatter plot can be colored in various ways, depending on the
plot_colors option, as set in optram_options()
If "plot_colors = none" all points are colored green.
If "plot_colors = density" points are colored by point density on the plot.
If "plot_colors = contour" points are colored green and density contour lines
are overlayed
If "plot_colors = feature" points are colored by some feature ID
in the original AOI polygon.
If "plot_colors = month" points are colored by the month of image acquisition.
aoi_name <- "Soil Moisture AOI" optram_options("trapezoid_method", "polynomial") full_df <- readRDS(system.file("extdata", "VI_STR_data.rds", package = "rOPTRAM")) edges_df <- read.csv(system.file("extdata", "trapezoid_edges_lin.csv", package = "rOPTRAM")) pl <- plot_vi_str_cloud(full_df, edges_df) pl + ggplot2::ggtitle(paste("Trapezoid plot for:", aoi_name)) plaoi_name <- "Soil Moisture AOI" optram_options("trapezoid_method", "polynomial") full_df <- readRDS(system.file("extdata", "VI_STR_data.rds", package = "rOPTRAM")) edges_df <- read.csv(system.file("extdata", "trapezoid_edges_lin.csv", package = "rOPTRAM")) pl <- plot_vi_str_cloud(full_df, edges_df) pl + ggplot2::ggtitle(paste("Trapezoid plot for:", aoi_name)) pl
Called by optram_wetdry_coefficients to prepare second order polynomial curve along trapezoid edges
Calculates six coefficients: intercept (alpha) of both wet and dry edges
and first and second order coefficients (beta), as in
and updates the edges data.frame with these polynomila values fitted values
polynomial_coefficients(df, output_dir)polynomial_coefficients(df, output_dir)
df |
data.frame, values of VI and STR along edges of trapezoid |
output_dir |
string, path to save coefficients CSV file |
df, data.frame, the trapezoid line edge points with fitted values added for both wet and dry edges
Three CSV files are saved:
the regressions coefficients,
the trapezoid edge points
RMSE of the fitted curve
## Not run: df <- read.csv(system.file("extdata", "trapezoid_edges.csv", package = "rOPTRAM")) output_dir <- tempdir() coeffs <- rOPTRAM::polynomial_coefficients(df, output_dir) coeffs ## End(Not run)## Not run: df <- read.csv(system.file("extdata", "trapezoid_edges.csv", package = "rOPTRAM")) output_dir <- tempdir() coeffs <- rOPTRAM::polynomial_coefficients(df, output_dir) coeffs ## End(Not run)
Prepare soil moisture grid from STR and VI images for a single date, based on polynomial function fitted to trapezoid edges.
polynomial_soil_moisture(coeffs, VI, STR)polynomial_soil_moisture(coeffs, VI, STR)
coeffs |
list, 6 trapezoid coefficients |
VI |
terra rast, the vegetation index raster |
STR |
terra rast, the STR raster |
rast, soil moisture grid
This function is used after preparing the OPTRAM model coefficients with:
optram_wetdry_coefficients. Typically a new image date, (that was not used for preparing the model),
will be referenced in the img_date parameter.
The resulting soil moisture raster is saved to output_dir
This function implements an polynomial fitted curve, following:
Ma, Chunfeng, Kasper Johansen, and Matthew F. McCabe. 2022.
“Combining Sentinel-2 Data with an Optical-Trapezoid Approach to Infer within-Field Soil Moisture Variability
and Monitor Agricultural Production Stages.”
Agricultural Water Management 274 (December): 107942.
doi:10.1016/j.agwat.2022.107942.
img_date <- "2023-03-11" VI_dir <- system.file("extdata", "NDVI", package = "rOPTRAM") STR_dir <- system.file("extdata", "STR", package = "rOPTRAM") optram_options("trapezoid_method", "polynomial") SM <- optram_calculate_soil_moisture(img_date, VI_dir, STR_dir, data_dir = tempdir())img_date <- "2023-03-11" VI_dir <- system.file("extdata", "NDVI", package = "rOPTRAM") STR_dir <- system.file("extdata", "STR", package = "rOPTRAM") optram_options("trapezoid_method", "polynomial") SM <- optram_calculate_soil_moisture(img_date, VI_dir, STR_dir, data_dir = tempdir())
Retrieve CDSE clientid and secret from file
The file location is system specific. It would have been setup
in advance using the store_cdse_credentials function
retrieve_cdse_credentials()retrieve_cdse_credentials()
A data frame containing the retrieved CDSE clientid and secret, or NULL if credentials are not available.
Store CDSE clientid and secret into a file
The file location is system specific.
Users who chose to save CDSE credentials can use this function
(and then the retrieve_cdse_credentials afterwards)
The clientid and secret are obtained from:
store_cdse_credentials(clientid = NULL, secret = NULL)store_cdse_credentials(clientid = NULL, secret = NULL)
clientid |
string, user's OAuth client id |
secret |
string, user's OAuth secret |
Both clientid and secret can alternatively be supplied as
environment variables: OAUTH_CLIENTID and OAUTH_SECRET.
If these env variables are available (and no values are entered
as function arguments) they will be used to store credentials.
## Not run: store_cdse_credentials(clientid="<...enter your client id...>", secret = "<...enter your secret...>") ## End(Not run)## Not run: store_cdse_credentials(clientid="<...enter your client id...>", secret = "<...enter your secret...>") ## End(Not run)