Data science notebooks

A substantial debate

• I don't like notebooks
• The first notebook war

• R vs. Python
• Data analysis vs. software engineering

Jupyter Notebooks

Source: Dataquest

Critiques about notebooks

• Hidden state and out-of-order execution
• Notebooks discourage modularity and testing
• Hard to copy/paste into Slack/GitHub issues
• Notebooks hinder reproducible and extensible science
• Notebooks cannot easily be tracked under version control

Source: The first notebook war

Data science is not the same as software engineering

R Markdown basics

---
title: "Gun deaths"
date: "`r lubridate::today()`"
output: html_document
---
```{r setup, include = FALSE}
knitr::opts_chunk$set(echo = FALSE)
```
```{r packages}
library(tidyverse)
library(rcfss)
theme_set(theme_minimal())
```
```{r youths}
youth <- gun_deaths %>%
  filter(age <= 65)
```
# Gun deaths by age
We have data about `r nrow(gun_deaths)` individuals killed by guns. Only `r nrow(gun_deaths) - nrow(youth)` are older than 65. The distribution of the remainder is shown below:
```{r youth-dist, echo = FALSE}
youth %>% 
  ggplot(mapping = aes(age)) + 
  geom_freqpoly(binwidth = 1)
```
# Gun deaths by race
```{r race-dist}
youth %>%
  ggplot(mapping = aes(fct_infreq(race) %>% fct_rev())) +
  geom_bar() +
  coord_flip() +
  labs(x = "Victim race")
```

Major components

1. A YAML header surrounded by ---s
2. Chunks of R code surounded by ```
3. Text mixed with simple text formatting using the Markdown syntax

Knitting process

Source: R for Data Science

Text formatting with Markdown

• Lightweight set of conventions for formatting plain text files
• $\LaTeX$ simplified

• Demonstration Markdown file

Write a simple Markdown file

• Edit my-cv.md to create a brief CV
• Title should be your name
• Create sections for education and employment (use headers)
• Include bulleted list of jobs/degrees
• Highlight the year in bold

Render and edit an R Markdown document

• Render gun-deaths.Rmd as an HTML document
• Add text describing the frequency polygon

Code chunks

```{r youth-dist, echo = FALSE, message = FALSE, warning = FALSE}
# code goes here
```

• Naming code chunks
• Code chunk options
• echo = FALSE
• message = FALSE or warning = FALSE
• eval = FALSE
• include = FALSE
• error = TRUE
• cache = TRUE

Global options

knitr::opts_chunk$set(
  echo = FALSE
)

Inline code

We have data about `r nrow(gun_deaths)` individuals killed by guns. Only `r
nrow(gun_deaths) - nrow(youth)` are older than 65.

We have data about 100798 individuals killed by guns. Only 15687 are older than 65.

Customize code chunk options

• Set echo = FALSE as a global option
• Enable caching as a global option and render the document. Look at the file structure for the cache. Now render the document again. Does it run faster?

YAML header

---
author: Benjamin Soltoff
date: '2021-11-12'
title: Gun deaths
output: github_document
---

• Yet Another Markup Language
• Standardized format for storing hierarchical data in a human-readable syntax
• Defines how rmarkdown renders your .Rmd file

HTML document

---
author: Benjamin Soltoff
date: '2021-11-12'
title: Gun deaths
output: html_document
---

Table of contents

---
author: Benjamin Soltoff
date: '2021-11-12'
title: Gun deaths
output:
  html_document:
    toc: true
    toc_depth: 2
---

Appearance and style

---
author: Benjamin Soltoff
date: '2021-11-12'
title: Gun deaths
output:
  html_document:
    theme: readable
    highlight: pygments
---

Customize YAML header

• Add a table of contents
• Use the "cerulean" theme
• Modify the figures so they are 8x6

PDF document

---
author: Benjamin Soltoff
date: '2021-11-12'
title: Gun deaths
output: pdf_document
---

• Requires installation of $\LaTeX$
• tinytex::install_tinytex()

Core R Markdown formats

Extensions of R Markdown

• flexdashboard
• bookdown
• distill
• blogdown

Slide presentations

• ioslides
• reveal.js
• Slidy
• Beamer
• xaringan

render()

rmarkdown::render("my-document.Rmd", output_format = "html_document")
rmarkdown::render("my-document.Rmd", output_format = "all")

Rendering different document formats

• Render gun-deaths.Rmd as
  • HTML document
  • PDF document
  • Word document

R Markdown notebooks

• Interactive version of an R Markdown document
• *.nb.html
• More similar to Jupyter Notebook
• Still plain-text files for version control

R scripts

# gun-deaths.R
# 2017-02-01
# Examine the distribution of age of victims in gun_deaths
# load packages
library(tidyverse)
library(rcfss)
# filter data for under 65
youth <- gun_deaths %>%
  filter(age <= 65)
# number of individuals under 65 killed
nrow(gun_deaths) - nrow(youth)
# graph the distribution of youth
youth %>% 
  ggplot(aes(age)) + 
  geom_freqpoly(binwidth = 1)
# graph the distribution of youth, by race
youth %>%
  ggplot(aes(fct_infreq(race) %>% fct_rev())) +
  geom_bar() +
  coord_flip() +
  labs(x = "Victim race")

When to use a script

• Software development
• For troubleshooting
• Initial stages of project
• Building a reproducible pipeline

• It depends

Running scripts

• Interactively
• Programmatically using source()