Why and how to edit your vcr cassettes?

library(vcr)

Why edit cassettes?

By design vcr is very good at recording HTTP interactions that actually took place. Now sometimes when testing/demo-ing your package you will want to use fake HTTP interactions. For instance:

  • What happens if the web API returns a 503 code? Is there an informative error?
  • What happens if it returns a 503 and then a 200 code? Does the retry work?
  • What if the API returns too much data for even simple queries and you want to make your cassettes smaller?

In all these cases, you can edit your cassettes as long as you are aware of the risks!

Example 1: test using an edited cassette with a 503

First, write your test e.g.

vcr::use_cassette("api-error", {
  test_that("Errors are handled well", {
    vcr::skip_if_vcr_off()
    expect_error(call_my_api()), "error message")
  })
})

Then run your tests the first time.

  1. It will fail
  2. It will have created a cassette under tests/fixtures/api-error.yml that looks something like
http_interactions:
- request:
    method: get
    uri: https://eu.httpbin.org/get
    body:
      encoding: ''
      string: ''
    headers:
      User-Agent: libcurl/7.54.0 r-curl/3.2 crul/0.5.2
  response:
    status:
      status_code: '200'
      message: OK
      explanation: Request fulfilled, document follows
    headers:
      status: HTTP/1.1 200 OK
      connection: keep-alive
    body:
      encoding: UTF-8
      string: "{\n  \"args\": {}, \n  \"headers\": {\n    \"Accept\": \"application/json,
        text/xml, application/xml, */*\", \n    \"Accept-Encoding\": \"gzip, deflate\",
        \n    \"Connection\": \"close\", \n    \"Host\": \"httpbin.org\", \n    \"User-Agent\":
        \"libcurl/7.54.0 r-curl/3.2 crul/0.5.2\"\n  }, \n  \"origin\": \"111.222.333.444\",
        \n  \"url\": \"https://eu.httpbin.org/get\"\n}\n"
  recorded_at: 2018-04-03 22:55:02 GMT
  recorded_with: vcr/0.1.0, webmockr/0.2.4, crul/0.5.2

You can edit to (new status code)

http_interactions:
- request:
    method: get
    uri: https://eu.httpbin.org/get
    body:
      encoding: ''
      string: ''
    headers:
      User-Agent: libcurl/7.54.0 r-curl/3.2 crul/0.5.2
  response:
    status:
      status_code: '503'

And run your test again, it should pass! Note the use of vcr::skip_if_vcr_off(): if vcr is turned off, there is a real API request and most probably this request won’t get a 503 as a status code.

The same thing with webmockr

The advantage of the approach involving editing cassettes is that you only learn one thing, which is vcr. Now, by using the webmockr directly in your tests, you can also test for the behavior of your package in case of errors. Below we assume api_url() returns the URL call_my_api() calls.

test_that("Errors are handled well", {
  webmockr::enable()
  stub <- webmockr::stub_request("get", api_url())
  webmockr::to_return(stub, status = 503)
  expect_error(call_my_api()), "error message")
  webmockr::disable()

})

A big pro of this approach is that it works even when vcr is turned off. A con is that it’s quite different from the vcr syntax.

Example 2: test using an edited cassette with a 503 then a 200

Here we assume your package contains some sort of retry.

First, write your test e.g.

vcr::use_cassette("api-error", {
  test_that("Errors are handled well", {
    vcr::skip_if_vcr_off()
    expect_message(thing <- call_my_api()), "retry message")
    expect_s4_class(thing, "data.frame")
  })
})

Then run your tests the first time.

  1. It will fail
  2. It will have created a cassette under tests/fixtures/api-error.yml that looks something like
http_interactions:
- request:
    method: get
    uri: https://eu.httpbin.org/get
    body:
      encoding: ''
      string: ''
    headers:
      User-Agent: libcurl/7.54.0 r-curl/3.2 crul/0.5.2
  response:
    status:
      status_code: '200'
      message: OK
      explanation: Request fulfilled, document follows
    headers:
      status: HTTP/1.1 200 OK
      connection: keep-alive
    body:
      encoding: UTF-8
      string: "{\n  \"args\": {}, \n  \"headers\": {\n    \"Accept\": \"application/json,
        text/xml, application/xml, */*\", \n    \"Accept-Encoding\": \"gzip, deflate\",
        \n    \"Connection\": \"close\", \n    \"Host\": \"httpbin.org\", \n    \"User-Agent\":
        \"libcurl/7.54.0 r-curl/3.2 crul/0.5.2\"\n  }, \n  \"origin\": \"111.222.333.444\",
        \n  \"url\": \"https://eu.httpbin.org/get\"\n}\n"
  recorded_at: 2018-04-03 22:55:02 GMT
  recorded_with: vcr/0.1.0, webmockr/0.2.4, crul/0.5.2

You can duplicate the HTTP interaction, and make the first one return a 503 status code. vcr will first use the first interaction, then the second one, when making the same request.

http_interactions:
- request:
    method: get
    uri: https://eu.httpbin.org/get
    body:
      encoding: ''
      string: ''
    headers:
      User-Agent: libcurl/7.54.0 r-curl/3.2 crul/0.5.2
  response:
    status:
      status_code: '503'
- request:
    method: get
    uri: https://eu.httpbin.org/get
    body:
      encoding: ''
      string: ''
    headers:
      User-Agent: libcurl/7.54.0 r-curl/3.2 crul/0.5.2
  response:
    status:
      status_code: '200'
      message: OK
      explanation: Request fulfilled, document follows
    headers:
      status: HTTP/1.1 200 OK
      connection: keep-alive
    body:
      encoding: UTF-8
      string: "{\n  \"args\": {}, \n  \"headers\": {\n    \"Accept\": \"application/json,
        text/xml, application/xml, */*\", \n    \"Accept-Encoding\": \"gzip, deflate\",
        \n    \"Connection\": \"close\", \n    \"Host\": \"httpbin.org\", \n    \"User-Agent\":
        \"libcurl/7.54.0 r-curl/3.2 crul/0.5.2\"\n  }, \n  \"origin\": \"111.222.333.444\",
        \n  \"url\": \"https://eu.httpbin.org/get\"\n}\n"
  recorded_at: 2018-04-03 22:55:02 GMT
  recorded_with: vcr/0.1.0, webmockr/0.2.4, crul/0.5.2

And run your test again, it should pass! Note the use of vcr::skip_if_vcr_off(): if vcr is turned off, there is a real API request and most probably this request won’t get a 503 as a status code.

The same thing with webmockr

The advantage of the approach involving editing cassettes is that you only learn one thing, which is vcr. Now, by using the webmockr directly in your tests, you can also test for the behavior of your package in case of errors. Below we assume api_url() returns the URL call_my_api() calls.

test_that("Errors are handled well", {
  webmockr::enable()
  stub <- webmockr::stub_request("get", api_url())
  stub %>%
  to_return(status = 503)  %>%
  to_return(status = 200, body = "{\n  \"args\": {}, \n  \"headers\": {\n    \"Accept\": \"application/json,
        text/xml, application/xml, */*\", \n    \"Accept-Encoding\": \"gzip, deflate\",
        \n    \"Connection\": \"close\", \n    \"Host\": \"httpbin.org\", \n    \"User-Agent\":
        \"libcurl/7.54.0 r-curl/3.2 crul/0.5.2\"\n  }, \n  \"origin\": \"111.222.333.444\",
        \n  \"url\": \"https://eu.httpbin.org/get\"\n}\n", headers = list(b = 6))
  expect_message(thing <- call_my_api()), "retry message")
    expect_s4_class(thing, "data.frame")
  webmockr::disable()

})

The pro of this approach is the elegance of the stubbing, with the two different responses. Each webmockr function like to_return() even has an argument times indicating the number of times the given response should be returned.

The con is that on top of being different from vcr, in this case where we also needed a good response in the end (the one with a 200 code, and an actual body), writing the mock is much more cumbersome than just recording a vcr cassette.

Conclusion

In this vignette we saw why and how to edit your vcr cassettes. We also presented approaches that use webmockr instead of vcr for mocking API responses. We mentioned editing cassettes by hand but you could also write a script using the yaml or jsonlite package to edit your YAML/JSON cassettes programmatically.