3 Data Management
In this chapter, we will learn how to replace values, switch values, import data, combine data, subset data, and split data.
3.1 Replacing Values
We can replace values with the replace() function.
x <- 1:10 # 1 through 10.
# If x equals 2, 5, or 7, replace with 0.
## replace(vector, condition, replacement value).
replace(x, x %in% c(2, 5, 7), 0)## [1] 1 0 3 4 0 6 0 8 9 103.2 Switching Values
We can switch–or recode–values with the switch() function. By default, switch() is a “scalar” function in that it only produces a single value. To produce a vector of values, we combine it with sapply()–see the Functionals chapter for more details on sapply().
3.2.1 Scalar Case
In the case of a single value, all we need to pass into switch() are (1) the data object and (2) an expression stating what the old value should become (i.e, provide the old value and the replacement value).
# SYNTAX OF switch():
## switch(x, old_value = new_value)
x <- "a"
x## [1] "a"xs <- switch(x, a = 1)
xs## [1] 13.2.2 Vector Case
For the case of applying switch() to vectors, we make use of sapply().
To take a case in point, let’s first generate some random data of racial groups.
set.seed(1) # Remember our random sampling
# Generate vector of unique values.
my_vector <- c('Asian', 'African American', 'White', 'Other')
my_vector## [1] "Asian" "African American" "White" "Other"# Conduct repeat sampling of my_vector
## See the Probability Functions chapter for more details on sample().
my_vector2 <- sample(my_vector, 20, replace = TRUE)
# Print the new vector.
my_vector2## [1] "Asian" "Other" "White" "Asian"
## [5] "African American" "Asian" "White" "White"
## [9] "African American" "African American" "White" "White"
## [13] "Asian" "Asian" "Asian" "African American"
## [17] "African American" "African American" "African American" "White"Let’s say that we want to recode these values: 0 for White, 1 for African American, 2 for Asian, and 3 for Other. To do so, we first define a function and pass it through sapply()–see the Function Writing and Functionals chapters respectively for more information.
# First, define a function that recodes the races into integers.
my_switch <- function(v) {
switch(v, White = 0, `African American` = 1, Asian = 2, Other = 3)
# We use back quotes for "African American" because of the space.
}
# Now we can pass my_switch to sapply() to execute the recoding.
sapply(my_vector2, my_switch)## Asian Other White Asian
## 2 3 0 2
## African American Asian White White
## 1 2 0 0
## African American African American White White
## 1 1 0 0
## Asian Asian Asian African American
## 2 2 2 1
## African American African American African American White
## 1 1 1 03.3 Importing Data
We can import datasets with read.table()–this method is the most general.
# Set path to dataset
# For this example, our data is in the data folder
# and our data are separated by commas.
my_data <- read.table('data/mtcars.csv', sep = ',', stringsAsFactors = FALSE)
# Setting stringsAsFactors = FALSE maintains strings as strings.
## See the Basics chapter for more detail on classes and types.In the case of files with comma-separated values, we can use read.csv() to import them more easily.
# Set path to dataset
my_data <- read.csv('data/mtcars.csv', stringsAsFactors = FALSE)3.4 Combining Data
There are three main ways to combine data: (1) cbind(), (2) rbind(), and (3) merge().
3.4.1 cbind()/rbind()
The function cbind() combines vectors or datasets column-wise, while rbind() does so row-wise.
# Creating x and y
x <- 1:5
y <- 6:10
# Seeing x and y separately
x## [1] 1 2 3 4 5y## [1] 6 7 8 9 10# Combining them
cbind(x, y)## x y
## [1,] 1 6
## [2,] 2 7
## [3,] 3 8
## [4,] 4 9
## [5,] 5 10rbind(x, y)## [,1] [,2] [,3] [,4] [,5]
## x 1 2 3 4 5
## y 6 7 8 9 10If we have a list of values we want to combine, we can use do.call() and cbind()/rbind() together. The former iteratively calls a function on a list, which can be useful for combining multiple datasets together. do.call() is a special case of a function called a functional, which is a function that takes other functions as inputs–this concept is discussed more in the Functionals chapter.
my_list <- list(x = 1:5, y = 6:10, z = 11:15)
do.call(cbind, my_list)## x y z
## [1,] 1 6 11
## [2,] 2 7 12
## [3,] 3 8 13
## [4,] 4 9 14
## [5,] 5 10 15do.call(rbind, my_list)## [,1] [,2] [,3] [,4] [,5]
## x 1 2 3 4 5
## y 6 7 8 9 10
## z 11 12 13 14 153.4.2 merge()
Merging data with merge() (AKA “joining data”) is powerful, as we can combine disparate datasets that have a common linking variable between them.
set.seed(1) # remember our random numbers from rnorm().
data1 <- data.frame(survey_id = 1:5,
wage = rnorm(5, mean = 15, sd = 5))
data1## survey_id wage
## 1 1 11.86773
## 2 2 15.91822
## 3 3 10.82186
## 4 4 22.97640
## 5 5 16.64754data2 <- data.frame(survey_id = 5:1,
experience = rnorm(5, mean = 5, sd = 3))
data2## survey_id experience
## 1 5 2.538595
## 2 4 6.462287
## 3 3 7.214974
## 4 2 6.727344
## 5 1 4.083835# merge(first data, second data, by = 'a common variable').
data_merge <- merge(data1, data2, by = 'survey_id')
data_merge # An "inner-join" of datasets## survey_id wage experience
## 1 1 11.86773 4.083835
## 2 2 15.91822 6.727344
## 3 3 10.82186 7.214974
## 4 4 22.97640 6.462287
## 5 5 16.64754 2.538595What we accomplished here is an inner join: a join in which two datasets overlap. See the documentation file for merge() for more information on different types of joins (i.e., type ?merge into the R console).
3.5 Subsetting Data
To subset data, we can pass data and relational/logic operators1 into the subset() function, or we can use the bracket syntax and use the operators there.
The relational operators are the following:
<,>,<=(less than or equal to),>=(greater than or equal to)==(equal to),!=(not equal to)
The main logic operators are the following:
&(and)|(or)
3.5.1 Vector Case
Suppose we have the following vector:
x <- -10:10 # integers from -10 to 10.
x## [1] -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
## [20] 9 10Then we can subset like the following:
x[x < 0] # same as subset(x, x < 0)## [1] -10 -9 -8 -7 -6 -5 -4 -3 -2 -1x[x > 2 & x < 5]## [1] 3 4# We can use functions inside the brackets.
## For example, %in% is a matching function:
## let's use it to subset for only 1 through 5.
x[x %in% 1:5] ## [1] 1 2 3 4 53.5.2 Data Frame Case
Suppose the dataset mtcars. Then we can subset like the following:
subset(mtcars, mpg > 30) # Same as mtcars[mtcars$mpg > 30, ]## mpg cyl disp hp drat wt qsec vs am gear carb
## Fiat 128 32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
## Honda Civic 30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2
## Toyota Corolla 33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1
## Lotus Europa 30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2subset(mtcars, mpg > 30 & wt > 1.7)## mpg cyl disp hp drat wt qsec vs am gear carb
## Fiat 128 32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
## Toyota Corolla 33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 13.6 Splitting Data
To split data, we pass a data frame and a variable into the split() function.
split(mtcars, mtcars$gear) # Splits into 3 subsets.## $`3`
## mpg cyl disp hp drat wt qsec vs am gear carb
## Hornet 4 Drive 21.4 6 258.0 110 3.08 3.215 19.44 1 0 3 1
## Hornet Sportabout 18.7 8 360.0 175 3.15 3.440 17.02 0 0 3 2
## Valiant 18.1 6 225.0 105 2.76 3.460 20.22 1 0 3 1
## Duster 360 14.3 8 360.0 245 3.21 3.570 15.84 0 0 3 4
## Merc 450SE 16.4 8 275.8 180 3.07 4.070 17.40 0 0 3 3
## Merc 450SL 17.3 8 275.8 180 3.07 3.730 17.60 0 0 3 3
## Merc 450SLC 15.2 8 275.8 180 3.07 3.780 18.00 0 0 3 3
## Cadillac Fleetwood 10.4 8 472.0 205 2.93 5.250 17.98 0 0 3 4
## Lincoln Continental 10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4
## Chrysler Imperial 14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4
## Toyota Corona 21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1
## Dodge Challenger 15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2
## AMC Javelin 15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2
## Camaro Z28 13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4
## Pontiac Firebird 19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
##
## $`4`
## mpg cyl disp hp drat wt qsec vs am gear carb
## Mazda RX4 21.0 6 160.0 110 3.90 2.620 16.46 0 1 4 4
## Mazda RX4 Wag 21.0 6 160.0 110 3.90 2.875 17.02 0 1 4 4
## Datsun 710 22.8 4 108.0 93 3.85 2.320 18.61 1 1 4 1
## Merc 240D 24.4 4 146.7 62 3.69 3.190 20.00 1 0 4 2
## Merc 230 22.8 4 140.8 95 3.92 3.150 22.90 1 0 4 2
## Merc 280 19.2 6 167.6 123 3.92 3.440 18.30 1 0 4 4
## Merc 280C 17.8 6 167.6 123 3.92 3.440 18.90 1 0 4 4
## Fiat 128 32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
## Honda Civic 30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2
## Toyota Corolla 33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1
## Fiat X1-9 27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1
## Volvo 142E 21.4 4 121.0 109 4.11 2.780 18.60 1 1 4 2
##
## $`5`
## mpg cyl disp hp drat wt qsec vs am gear carb
## Porsche 914-2 26.0 4 120.3 91 4.43 2.140 16.7 0 1 5 2
## Lotus Europa 30.4 4 95.1 113 3.77 1.513 16.9 1 1 5 2
## Ford Pantera L 15.8 8 351.0 264 4.22 3.170 14.5 0 1 5 4
## Ferrari Dino 19.7 6 145.0 175 3.62 2.770 15.5 0 1 5 6
## Maserati Bora 15.0 8 301.0 335 3.54 3.570 14.6 0 1 5 8Splitting can be useful when you want to apply a function that’s contingent on subsets of data. For example, we can split the data and perform a regression model on each of them.
# 1. Split dataset by a splitting variable.
my_split <- split(mtcars, mtcars$gear)
# 2. Estimate a regression model based on each subset.
my_models <- lapply(my_split, function(data) lm(mpg ~ wt, data))
# 3. Print the coefficients in a matrix form.
sapply(my_models, coef)## 3 4 5
## (Intercept) 28.395036 42.492769 42.562784
## wt -3.156854 -6.863478 -8.046336For more information about lapply() and sapply(), see the Functionals chapter; for more information about lm(), see the Linear Modeling chapter.
3.7 Summary
| Function | Description | Example |
|---|---|---|
| replace(x, condition, replacement) | Replace a value in a vector based on a condition. | x <- 1:10; replace(x, x %in% c(2, 5, 7), 0) |
| switch(x, expression) | Switch (recode) values. | x <- ‘a’; switch(x, a = 1) |
| read.table(‘path/to/file.csv’, sep = ‘,’) | Import a dataset. | my_data <- read.table(‘data/mtcars.csv’, sep = ‘,’) |
| cbind(x,y)/rbind(x,y) | Combine data column- or row-wise. | x <- 1:5; y <- 6:10; cbind(x, y); rbind(x,y) |
| do.call(function, list) | Iteratively call a function on a list | my_list <- list(x = 1:5, y = 6:10, z = 11:15); do.call(cbind, my_list) |
| merge(x, y, by = ‘linking_var’) | Join data by a linking variable. |
data1 <- data.frame(survey_id = 1:5, wage = rnorm(5, mean = 15, sd = 5)) data2 <- data.frame(survey_id = 5:1, experience = rnorm(5, mean = 5, sd = 3)) data_merge <- merge(data1, data2, by = ‘survey_id’) |
| subset(data, condition); x[condition] | Subset data via relational and logic operators. | subset(mtcars, mpg > 30 & wt > 1.7) |
| split(data, grouping_variable) | Split data by a grouping variable | split(mtcars, mtcars$gear) |