---
title: "Import & Processing"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Import & Processing}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r options, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  out.width = "90%",
  fig.width = 6,
  fig.asp = 0.618,
  fig.align = "center",
  dpi = 300
)

library(ggplot2)
ar <- arrow(length = unit(0.1, "inches"), type = "closed")
dsblue <- "#0053A4"
```

Different measuring devices and analysis software lead to opaque results in
measuring gas exchange parameters. To make exercise science more transparent and
reproducible, the `spiro` package offers a standardized workflow for data from
cardiopulmonary exercise testing.

This vignette provides information on how the `spiro` package imports and processes raw data from metabolic carts.

## Import and processing with `spiro()` {#spiro-function}

The `spiro` package makes import and processing of cardiopulmonary data easy: The `spiro()` function does all that work for you. You just need to paste the path of a file with raw data from cardiopulmonary exercise testing to the function. This will return a `data.frame` containing all relevant data for each second of testing, ready for [summarizing or plotting](https://docs.ropensci.org/spiro/articles/summarizing_plotting.html).

```{r setup}
library(spiro)

# Get example data
file <- spiro_example("zan_gxt")

spiro(file)

```

`spiro()` will return interpolated data for the following parameters:

* **load**: velocity or power, either retrieved from the raw data file or manually supplied by [setting a protocol](#protocol-set)
* **step**: coded variable for the number of the load step in the test protocol
* **time**: time (s)
* **VO2**: oxygen uptake (ml/min)
* **VCO2**: carbon dioxide output (ml/min)
* **RR**: respiratory rate (1/min)
* **VT**: tidal volume (l)
* **VE**: minute ventilation (l/min)
* **HR**: heart rate (bpm), if available
* **PetO2**: end-tidal partial pressure of oxygen (mmHg)
* **PetCO2**: end-tidal partial pressure of carbon dioxide (mmHg)
* **VO2_rel**: relative oxygen uptake (ml/min/kg)
* **VCO2_rel**: relative carbon dioxide output (ml/min/kg)
* **RE**: running economy (ml/kg/km), if applicable
* **RER**: respiratory quotient
* **CHO**: rate of carbohydrate oxidation (g/min)
* **FO**: rate of fat oxidation (g/min)

### Options for data processing {#options}

You can control the exercise protocol, the calculation of parameters related to body weight and the adding of heart rate data with the arguments of `spiro()` or with the helper functions `add_protocol()`, `add_weight()` and `add_hr()` within a piping syntax:

```{r piping, eval = FALSE}
# Note: The Base R pipe requires R version 4.1 or greater

protocol <- set_protocol(
  pt_wu(duration = 120, load = 50),
  pt_steps(duration = 30, load = 100, increment = 20, count = 24)
)

spiro(file = spiro_example("zan_ramp")) |>
  add_bodymass(bodymass = 63.4) |>
  add_protocol(protocol) |>
  add_hr(hr_file = spiro_example("hr_ramp.tcx"), hr_offset = 0)
```

### Use breath-by-breath data!

We highly recommended to import only raw breath-by-breath data for several reasons:

 - Prior averaging or interpolation happens outside of R and is therefore a non-reproducible data processing step.
 - If no data for single breaths is available, some functionality of the package is lost (e.g. breath-based averaging for VO_2max_ determination).
 - Prior processing usually leads to data containing less data points, which hinders the automated guessing of exercise protocols based on the available load data.
 
If you use a metabolic cart that measures, but does not output data on a breath-by-breath basis, read the manufacturer's instructions on how to export the raw data in such a way. Data from other systems (e.g., most mixing chamber metabolic carts) can still be processed with the `spiro`-package, but protocol guesses and summary calculations have to be treated with caution.

## Supported metabolic carts {#metabolic-carts}

The `spiro` package supports different metabolic carts. The metabolic cart a data file is produced by is usually determined automatically, but can also be set manually in the `spiro()` function. Currently this package supports the following devices:

* **CORTEX** (`"cortex"`): .xlsx, .xls or .xml files in English or German language
* **COSMED** (`"cosmed"`): .xlsx or .xls files in English or German language
* **VYNTUS** (`"vyntus"`): .txt files (tab-separated) in French, German or Norwegian language
* **ZAN** (`"zan"`): .dat files, usually with the name "EXED*" in German language

To only import the raw data without further processing (such as interpolation, exercise protocol guessing,...) use the function `spiro_raw()`:

```{r raw}
spiro_raw(file, device = NULL, anonymize = TRUE)
```

Alternatively you can also access the raw data after a `spiro()` call.

```{r raw2, eval=FALSE}
s <- spiro(file)
spiro_raw(s)
```

## Exercise protocols {#protocols}

To achieve the full functionality of data summary and plotting with the `spiro`
package, an exercise protocol needs to be attached to the data.

### Protocol guessing {#protocol-get}

By default, `spiro()` guesses the exercise protocol using `get_protocol()`,
looking for velocity or load data in the imported raw data. To return the
protocol guess after a `spiro()` call, access the `"protocol"` attribute.

```{r protocol}
s <- spiro(file)
attr(s,"protocol")
```

### Protocol setting {#protocol-set}

In cases where no load data is saved in the metabolic cart's file or
`get_protocol()` turns wrong, a protocol can be manually set.

There are two ways to initially generate a protocol: by providing all
load-duration combinations with `set_protocol_manual()` or by using the helper
functions within `set_protocol()`. Once a protocol has been set, it can be used
as the `protocol` argument in a `spiro()` call or attached to a `spiro` data
frame with `add_protocol()`.

```{r set_protocol_manual}
# manually setting a test protocol
pt <- set_protocol_manual(
  duration = c(60,300,30,300,30,300,30,300,30,300,30,300,30,300,30,300,30,300),
  load = c(0,3,0,3.2,0,3.4,0,3.6,0,3.8,0,4,0,4.2,0,4.4,0,4.6)
)

# attach protocol within spiro call
s <- spiro(file, protocol = pt)

# attach protocol with `add_protocol`
t <- spiro(file)
add_protocol(t, pt)
```

With `set_protocol()` a protocol can be defined without specifying every single load step. You can paste the pre-defined segment types `pt_pre()`, `pt_wu()`, `pt_const()` and `pt_steps()` into `set_protocol()` in the desired order. The following graph illustrates an example of this practice:

```{r set_protocol-variables, echo=FALSE}
ar2 <- arrow(length = unit(0.1, "inches"), type = "closed", ends = "both")
path <- data.frame(
  x = c(0,0.1,0.1,0.3,0.3,0.32,0.32,0.42,0.42,0.44,0.44,0.54,0.54,0.56,0.56,
        0.66,0.66,0.68,0.68,0.78,0.78,0.8,0.8,0.9,0.9,0.95),
  y = c(0,0,0.2,0.2,0,0,0.4,0.4,0,0,0.5,0.5,0,0,0.6,0.6,0,0,
        0.7,0.7,0,0,0.8,0.8,0,0),
  type = factor(
    c("pre","wu","wu","wu","wu",rep("load",21)),
    levels = c("pre","wu","load")
  )
)
ggplot() +
  (if (utils::packageVersion("ggplot2") >= "3.4") {
    list(
      geom_segment(
        aes(x = 0, xend = 0.95, y = 0, yend = 0),
        colour = "grey", 
        linewidth = 1
      ),
      geom_path(
        aes(x = x, y = y, colour = type), 
        data = path, 
        linewidth = 1, 
        group = TRUE, 
        show.legend = FALSE
      )
    )
  } else {
    list(
      geom_segment(
        aes(x = 0, xend = 0.95, y = 0, yend = 0),
        colour = "grey", 
        size = 1
      ),
      geom_path(
        aes(x = x, y = y, colour = type), 
        data = path, 
        size = 1, 
        group = TRUE, 
        show.legend = FALSE
      )
    )
  }) +
  annotate(
    "text", 
    x = c(0.03,0.05,0.05,0.05,0.03), y = c(0.98,0.91,0.84,0.77,0.70), 
    label = c(
      "set_protocol(",
      "pt_pre(duration),",
      "pt_wu(duration, load, rest),",
      "pt_steps(duration, load, increment, count, rest)",
      ")"
    ), 
    hjust = "left", vjust = "top",
    colour = c("black", "#d55e00", "#009e73", dsblue, "black"),
    size = 4.5
  ) +
  scale_x_continuous(
    limits = c(-0.02,1), 
    expand = expansion(0,0), 
    breaks = NULL
  ) +
  scale_y_continuous(
    limits = c(-0.15,1), 
    expand = expansion(0,0), 
    breaks = NULL
  ) +
  scale_colour_manual(values = c("#d55e00","#009e73",dsblue)) +
  labs(x = "time", y = "load") +
  theme_minimal()
```

```{r set_protocol}
set_protocol(pt_pre(60), pt_wu(300,80), pt_steps(180,100,25,6,30))
```

## Modify body mass {#bodymass}

The `spiro` package calculates parameters relative to body mass. Per default `spiro()` will look for information on body mass in the meta data of the original data file. If for some reason no or the wrong body mass is present in the raw data file, `bodymass` can be manually given as an argument in `spiro()` or with `add_bodymass()`.

```{r add_weight}
# set body mass as an argument in `spiro()`
s <- spiro(file, bodymass = 68.3)

# set body mass using `add_weight()`
t <- spiro(file) 
u <- add_bodymass(t, 68.3)

```

## Work with external heart rate data {#heart-rate}

Some metabolic carts only offer complicated options for connecting them to heart rate monitors. If heart rate data was recorded by another kind of device (e.g. wrist watch), this data can be added within the `spiro()` call or by using `add_hr()`.

```{r add_hr}
# get example data file path
hpath <- spiro_example("hr_ramp.tcx")

# add heart rate data within `spiro()`
h <- spiro(file, hr_file = hpath, hr_offset = 0)

# add heart rate data with `add_hr()`
i <- spiro(file)
j <- add_hr(i, hr_file = hpath, hr_offset = 0)
```

`add_hr()` will import the heart rate data from a .tcx file and attach it to the existing data set. You can manually set the starting point of the heart rate recording relative to the start of the gas exchange measures recording with the `hr_offset` argument.