jst_get_article

The basic usage of the jst_get_* functions is very simple. They take only one argument, the path to the file to import:

meta_data <- jst_get_article(file_path = jst_example("article_with_references.xml"))

The resulting object is a tibble with one row and 17 columns. The columns correspond to most of the elements documented here: https://www.jstor.org/dfr/about/technical-specifications.

The columns are:

• file_name (chr): The file name of the original .xml-file. Can be used for joining with other parts (authors, references, footnotes, full-texts).
• journal_doi (chr): A registered identifier for the journal.
• journal_jcode (chr): A identifier for the journal like “amerjsoci” for the “American Journal of Sociology”.
• journal_pub_id (chr): Similar to journal_jcode. Most of the time either one is present.
• article_doi (chr): A registered unique identifier for the article.
• article_jcode (chr): A unique identifier for the article (not a DOI).
• article_pub_id (chr): Infrequent, either part of the DOI or the article_jcode.
• article_type (chr): The type of article (research-article, book-review, etc.).
• article_title (chr): The title of the article.
• volume (chr): The volume the article was published in.
• issue (chr): The issue the article was published in.
• language (chr): The language of the article.
• pub_day (chr): Publication day, if specified.
• pub_month (chr): Publication month, if specified.
• pub_year (int): Year of publication.
• first_page (int): Page number for the first page of the article.
• last_page (int): Page number for the last page of the article.

Since the output from all functions are tibbles, the result is nicely formatted:

meta_data %>% kable()

jst_get_authors

Extracting the authors works in similar fashion:

authors <- jst_get_authors(jst_example("article_with_references.xml"))
kable(authors)

Here we have the following columns:

• file_name: The same as above, used for matching articles.
• prefix: A prefix to the name.
• given_name: The given name of the author (i.e. Albert or A.).
• surname: The surname of the author (i.e. Einstein).
• string_name: Sometimes instead of given_name and surname, only a full string is supplied, i.e.: Albert Einstein, or Einstein, Albert.
• suffix: A suffix to the name, as in Albert Einstein, II..
• author_number: An integer representing the order of how the authors appeared in the data.

The number of rows matches the number of authors – each author get its’ own row.

jst_get_references

references <- jst_get_references(jst_example("article_with_references.xml"))

# # we need to remove line breaks for knitr::kable() to work properly for printing
references <- references %>%
  mutate(ref_unparsed = stringr::str_remove_all(ref_unparsed, "\\\n"))

We have two columns:

• file_name: Identifier, can be used for matching.
• ref_title: The title of the references sections.
• ref_authors: A string of authors. Several authors are separated with ;.
• ref_editors: A string of editors, if present.
• ref_collab: A field that may contain information on the authors, if authors are not available.
• ref_item_title: The title of the cited entry.
• ref_year: A year, often the article’s publication year, but not always.
• ref_source: The source of the cited entry. For books often the title of the book, for articles the publisher of the journal.
• ref_volume: The volume of the journal article.
• ref_first_page: The first page of the article/chapter.
• ref_last_page: The last page of the article/chapter.
• ref_publisher: For books the publisher, for articles often missing.
• ref_publication_type: Known types: book, journal, web, other.
• ref_unparsed: The full references entry in unparsed form.

Here I display 5 random entries:

references %>% 
  sample_n(5) %>% 
  kable()

This example shows several things: file_name is identical among rows, since it identifies the article and all references came from one article. The the sample file doesn’t follow a typical convention (it was published in 1922), therefore there are several different headings (ref_title). Usually, this is only “Bibliography” or “References”.

Since the references were not parsed by JSTOR, we only get an unparsed version. In general, the content of references (unparsed_refs) is in quite a raw state, quite often the result of digitising scans via OCR. For example, the last entry reads like this: MACHADO, A.1911 Zytologische Untersuchungen fiber Trypanosoma rotatorium .... There is an error here: fiber should be über. The language of the source is German, but the OCR-software assumed English. Therefore, it didn’t recognize the Umlaut. Similar errors are common for text read via OCR.

For other files, we can set parse_refs = TRUE, so references will be imported in their parsed form, whenever they are available.

jst_get_references(
  jst_example("parsed_references.xml"),
  parse_refs = TRUE
) %>% 
  kable()

Note, that there might be other content present like endnotes, in case the article used endnotes rather than footnotes.

jst_get_footnotes

jst_get_footnotes(jst_example("article_with_references.xml")) %>% 
  kable()

Very commonly, articles either have footnotes or references. The sample file used here does not have footnotes, therefore a simple tibble with missing footnotes is returned.

I will use another file to demonstrate footnotes.

footnotes <- jst_get_footnotes(jst_example("article_with_footnotes.xml"))

footnotes %>% 
  mutate(footnotes = stringr::str_remove_all(footnotes, "\\\n")) %>% 
  kable()

In general, you might need to combine jst_get_footnotes() with jst_get_references() to get all available information on citation data.

jst_get_full_text

The function to extract full texts can’t be demonstrated with proper data, since the full texts are only supplied upon special request with DfR. The function guesses the encoding of the specified file via readr::guess_encoding(), reads the whole file and returns a tibble with file_name, full_text and encoding.

I created a file that looks similar to files supplied by DfR with sample text:

full_text <- jst_get_full_text(jst_example("full_text.txt"))
full_text %>% 
  mutate(full_text = stringr::str_remove_all(full_text, "\\\n")) %>% 
  kable()

Combining results

Different parts of meta-data can be combined by using dplyr::left_join().

meta_data %>% 
  left_join(authors) %>%
  select(file_name, article_title, pub_year, given_name, surname) %>% 
  kable()
#> Joining with `by = join_by(file_name)`

meta_data %>% 
  left_join(references) %>% 
  select(file_name, article_title, volume, pub_year, ref_unparsed) %>%
  head(5) %>% 
  kable()
#> Joining with `by = join_by(file_name)`