Applied Demonstration of the tipr R Package
Lucy D’Agostino McGowan
@LucyStats�
Department of Mathematics & Statistics, Wake Forest University
lucymcgowan.com/talk
Based on work completed jointly with Robert Greevy
@ Vanderbilt University
TABLE OF CONTENTS
THE DATA
TIPPING POINT ANALYSES
02
GETTING TO KNOW tipr
03
USING tipr WITH A KNOWN CONFOUNDER
04
USING tipr WITH AN UNKNOWN CONFOUNDER
05
tipr + broom
06
01
VA Health Record data on diabetes drug and cancer incidence
A bit on tipping point analyses
Some basic usage of the tipr package
Using tipr when we know we were missing something
Using tipr when we don’t know what we may be missing
Using tipr with the broom package
01
THE DATA
QUESTION
Metformin
Cancer
VERSUS SULFONYLUREAS
ANALYSIS
RESULTS
Lung Cancer | 0.87 (0.79, 0.96) |
Liver Cancer | 0.46 (0.37, 0.57) |
Colorectal Cancer | 0.86 (0.75, 0.99) |
OUTCOME
METFORMIN� (ADJ HAZARD RATIO)
RESULTS
Lung Cancer | 0.87 (0.79, 0.96) |
Liver Cancer | 0.46 (0.37, 0.57) |
Colorectal Cancer | 0.86 (0.75, 0.99) |
OUTCOME
METFORMIN� (ADJ HAZARD RATIO)
RESULTS
Lung Cancer | 0.87 (0.79, 0.96) |
Liver Cancer | 0.46 (0.37, 0.57) |
Colorectal Cancer | 0.86 (0.75, 0.99) |
OUTCOME
METFORMIN� (ADJ HAZARD RATIO)
02
TIPPING POINT ANALYSES
Quantifying unmeasured confounding
WHAT YOU NEED
EXPOSURE
OUTCOME
EFFECT
OUTCOME
UNMEASURED
CONFOUNDER
EFFECT
EXPOSURE
UNMEASURED
CONFOUNDER
EFFECT
TIPPING POINT
What will tip our confidence bound to cross 1?
WHAT YOU NEED
EXPOSURE
OUTCOME
EFFECT
OUTCOME
UNMEASURED
CONFOUNDER
EFFECT
EXPOSURE
UNMEASURED
CONFOUNDER
EFFECT
WHAT YOU NEED
EXPOSURE
OUTCOME
EFFECT
OUTCOME
UNMEASURED
CONFOUNDER
EFFECT
EXPOSURE
UNMEASURED
CONFOUNDER
EFFECT
TIPPING POINT
TIPPING POINT
relative risk - the unmeasured confounder-outcome effect
TIPPING POINT
relative risk* - the unmeasured confounder-outcome effect
*built for RRs, but can work for ORs and HRs
WHAT YOU NEED
EXPOSURE
OUTCOME
EFFECT
OUTCOME
UNMEASURED
CONFOUNDER
EFFECT
EXPOSURE
UNMEASURED
CONFOUNDER
EFFECT
TIPPING POINT
limiting bound (bound closest to the null)
WHAT YOU NEED
EXPOSURE
OUTCOME
EFFECT
OUTCOME
UNMEASURED
CONFOUNDER
EFFECT
EXPOSURE
UNMEASURED
CONFOUNDER
EFFECT
TIPPING POINT
standardized mean difference of the unmeasured confounder between the exposed and unexposed groups
WHAT YOU NEED
EXPOSURE
OUTCOME
EFFECT
OUTCOME
UNMEASURED
CONFOUNDER
EFFECT
EXPOSURE
UNMEASURED
CONFOUNDER
EFFECT
TIPPING POINT
TIPPING POINT
TIPPING POINT
TIPPING POINT
TIPPING POINT
03
GETTING TO KNOW tipr
install.packages("tipr")
devtools::install_github("LucyMcGowan/tipr")
Main function
tip()
Main function
tip()
WHAT YOU NEED
EXPOSURE
OUTCOME
EFFECT
OUTCOME
UNMEASURED
CONFOUNDER
EFFECT
EXPOSURE
UNMEASURED
CONFOUNDER
EFFECT
Main function
tip()
WHAT YOU NEED
EXPOSURE
OUTCOME
EFFECT
OUTCOME
UNMEASURED
CONFOUNDER
EFFECT
EXPOSURE
UNMEASURED
CONFOUNDER
EFFECT
Main function
tip()
WHAT YOU NEED
EXPOSURE
OUTCOME
EFFECT
OUTCOME
UNMEASURED
CONFOUNDER
EFFECT
EXPOSURE
UNMEASURED
CONFOUNDER
EFFECT
Main function
tip()
Main function
tip()
specify one, it will estimate the other
Main function
tip()
specify two, it will estimate the number of unmeasured confounders needed to tip
Main function
tip_with_binary()
Main function
tip_with_binary()
WHAT YOU NEED
EXPOSURE
OUTCOME
EFFECT
OUTCOME
UNMEASURED
CONFOUNDER
EFFECT
EXPOSURE
UNMEASURED
CONFOUNDER
EFFECT
Main function
tip_with_binary()
specify two, it will estimate the other
Main function
tip_with_binary()
specify three, it will estimate the # of unmeasured confounders needed
04
USING tipr, WITH A KNOWN CONFOUNDER
RESULTS
Lung Cancer | 0.87 (0.79, 0.96) |
Liver Cancer | 0.46 (0.37, 0.57) |
Colorectal Cancer | 0.86 (0.75, 0.99) |
OUTCOME
METFORMIN� (ADJ HAZARD RATIO)
What if there is residual confounding for smoking?
Murff, HJ, Roumie, CL, Greevy, RA, Hackstadt, AJ, D’Agostino McGowan, LE., Hung, AM, ... & Griffin, MR. (2018). Metformin use and incidence cancer risk: evidence for a selective protective effect against liver cancer. Cancer Causes & Control, 29(9), 823-832.
This is likely an undercount for a veteran population
Murff, HJ, Roumie, CL, Greevy, RA, Hackstadt, AJ, D’Agostino McGowan, LE., Hung, AM, ... & Griffin, MR. (2018). Metformin use and incidence cancer risk: evidence for a selective protective effect against liver cancer. Cancer Causes & Control, 29(9), 823-832.
What if smoking is differentially undercounted with 8% unidentified smokers among Metformin users and 9% among Sulfonylurea users?
library(tipr)
tip_with_binary(data.frame(conf.low = 0.79,
conf.high = 0.96),
exposed_p = .08,
unexposed_p = .10)
library(tipr)
tip_with_binary(data.frame(conf.low = 0.79,
conf.high = 0.96),
exposed_p = .08,
unexposed_p = .10)
RESULTS
Lung Cancer | 0.87 (0.79, 0.96) |
Liver Cancer | 0.46 (0.37, 0.57) |
Colorectal Cancer | 0.86 (0.75, 0.99) |
OUTCOME
METFORMIN� (ADJ HAZARD RATIO)
library(tipr)
tip_with_binary(data.frame(conf.low = 0.79,
conf.high = 0.96),
exposed_p = .08,
unexposed_p = .10)
What if smoking is differentially undercounted with 8% unidentified smokers among Metformin users and 10% among Sulfonylurea users?
library(tipr)
tip_with_binary(data.frame(conf.low = 0.79,
conf.high = 0.96),
exposed_p = .08,
unexposed_p = .10)
#> The observed effect (0.79, 0.96) WOULD be tipped by 1 unmeasured confounder
#> with the following specifications:
#> * estimated prevalence of the unmeasured confounder in the exposed population: 0.08
#> * estimated prevalence of the unmeasured confounder in the unexposed population: 0.1
#> * estimated association between the unmeasured confounder and the outcome: 3.5
#> # A tibble: 1 x 6
#> observed_lb observed_ub exposed_p unexposed_p outcome_association n_unmeasured_confounders
#> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 0.79 0.96 0.08 0.1 3.5 1
#> The observed effect (0.79, 0.96) WOULD be tipped by 1 unmeasured confounder
#> with the following specifications:
#> * estimated prevalence of the unmeasured confounder in the exposed population: 0.08
#> * estimated prevalence of the unmeasured confounder in the unexposed population: 0.1
#> * estimated association between the unmeasured confounder and the outcome: 3.5
#> # A tibble: 1 x 6
#> observed_lb observed_ub exposed_p unexposed_p outcome_association n_unmeasured_confounders
#> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 0.79 0.96 0.08 0.1 3.5 1
“If smoking were differentially undercounted, with 8% of unidentified smokers among metformin users and 10% among sulfonylureas users, this residual smoking would need to have an association with lung cancer indicence of 3.5 to tip this analysis at the 5% level, rendering it inconclusive. Given that associations between lung cancer and smoking have been reported to be quite large, we cannot rule out unmeasured confounding.
”
“If smoking were differentially undercounted, with 8% of unidentified smokers among metformin users and 10% among sulfonylureas users, this residual smoking would need to have an association with lung cancer indicence of 3.5 to tip this analysis at the 5% level, rendering it inconclusive. Given that associations between lung cancer and smoking have been reported to be quite large, we cannot rule out unmeasured confounding.
”
*Note in this particular paper, we adjusted for COPD and numerous other cardiovascular conditions which are highly associated with smoking and would potentially pick up much of that residual confounding.
05
USING tipr, WITH AN UNKNOWN CONFOUNDER
RESULTS
Lung Cancer | 0.87 (0.79, 0.96) |
Liver Cancer | 0.46 (0.37, 0.57) |
Colorectal Cancer | 0.86 (0.75, 0.99) |
OUTCOME
METFORMIN� (ADJ HAZARD RATIO)
What if there is an unmeasured confounder that increased one’s liver cancer risk with an association between liver cancer and the unmeasured confounder of 2?
tip(data.frame(conf.low = 0.37,
conf.high = 0.57),
outcome_association = 2)
#> The observed effect (0.37, 0.57) WOULD be tipped by 1 unmeasured confounder
#> with the following specifications:
#> * estimated standardized mean difference between the unmeasured confounder
#> in the exposed population and unexposed population: -0.81
#> * estimated association between the unmeasured confounder and the outcome: 2
#> # A tibble: 1 x 5
#> observed_lb observed_ub smd outcome_association n_unmeasured_confounders
#> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 0.37 0.570 -0.811 2 1
#> The observed effect (0.37, 0.57) WOULD be tipped by 1 unmeasured confounder
#> with the following specifications:
#> * estimated standardized mean difference between the unmeasured confounder
#> in the exposed population and unexposed population: -0.81
#> * estimated association between the unmeasured confounder and the outcome: 2
#> # A tibble: 1 x 5
#> observed_lb observed_ub smd outcome_association n_unmeasured_confounders
#> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 0.37 0.570 -0.811 2 1
Love Plot
Murff, HJ, Roumie, CL, Greevy, RA, Hackstadt, AJ, D’Agostino McGowan, LE., Hung, AM, ... & Griffin, MR. (2018). Metformin use and incidence cancer risk: evidence for a selective protective effect against liver cancer. Cancer Causes & Control, 29(9), 823-832.
Murff, HJ, Roumie, CL, Greevy, RA, Hackstadt, AJ, D’Agostino McGowan, LE., Hung, AM, ... & Griffin, MR. (2018). Metformin use and incidence cancer risk: evidence for a selective protective effect against liver cancer. Cancer Causes & Control, 29(9), 823-832.
#> The observed effect (0.37, 0.57) WOULD be tipped by 1 unmeasured confounder
#> with the following specifications:
#> * estimated standardized mean difference between the unmeasured confounder
#> in the exposed population and unexposed population: -0.81
#> * estimated association between the unmeasured confounder and the outcome: 2
#> # A tibble: 1 x 5
#> observed_lb observed_ub smd outcome_association n_unmeasured_confounders
#> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 0.37 0.570 -0.811 2 1
Love Plot
0.4
0.5
0.6
0.7
0.8
Murff, HJ, Roumie, CL, Greevy, RA, Hackstadt, AJ, D’Agostino McGowan, LE., Hung, AM, ... & Griffin, MR. (2018). Metformin use and incidence cancer risk: evidence for a selective protective effect against liver cancer. Cancer Causes & Control, 29(9), 823-832.
What if there were multiple independent unmeasured confounders that increased one’s cancer risk with an association of 2 and had a SMD of -0.1?
tip(data.frame(conf.low = 0.37,
conf.high = 0.57),
outcome_association = 2,
smd = -0.1)
#> The observed effect (0.37, 0.57) WOULD be tipped by 8 unmeasured confounders
#> with the following specifications:
#> * estimated standardized mean difference between the unmeasured confounder
#> in the exposed population and unexposed population: -0.1
#> * estimated association between the unmeasured confounder and the outcome: 2
#> # A tibble: 1 x 5
#> observed_lb observed_ub smd outcome_association n_unmeasured_confounders
#> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 0.37 0.570 -0.1 2 8.11
#> The observed effect (0.37, 0.57) WOULD be tipped by 8 unmeasured confounders
#> with the following specifications:
#> * estimated standardized mean difference between the unmeasured confounder
#> in the exposed population and unexposed population: -0.1
#> * estimated association between the unmeasured confounder and the outcome: 2
#> # A tibble: 1 x 5
#> observed_lb observed_ub smd outcome_association n_unmeasured_confounders
#> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 0.37 0.570 -0.1 2 8.11
“An unmeasured confounder with an association with liver cancer incidence of 2 and a standardized mean difference between the metformin users and sulfonylurea users of -0.81 would tip this analysis at the 5% level, rendering it inconclusive. Alternatively, you would need 8 independent unmeasured confounders, each with with an association with liver cancer incidence of 2 and a standardized mean difference -0.1 to tip this analysis.
”
RESULTS
Lung Cancer | 0.87 (0.79, 0.96) |
Liver Cancer | 0.46 (0.37, 0.57) |
Colorectal Cancer | 0.86 (0.75, 0.99) |
OUTCOME
METFORMIN� (ADJ HAZARD RATIO)
What if there is an unmeasured confounder that increased one’s colorectal cancer risk with an assoication of 2?
tip(data.frame(conf.low = 0.75,
conf.high = 0.99),
outcome_association = 2)
#> The observed effect (0.75, 0.99) WOULD be tipped by 1 unmeasured confounder
#> with the following specifications:
#> * estimated standardized mean difference between the unmeasured confounder
#> in the exposed population and unexposed population: -0.01
#> * estimated association between the unmeasured confounder and the outcome: 2
#> # A tibble: 1 x 5
#> observed_lb observed_ub smd outcome_association n_unmeasured_confounders
#> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 0.75 0.99 -0.0145 2 1
#> The observed effect (0.75, 0.99) WOULD be tipped by 1 unmeasured confounder
#> with the following specifications:
#> * estimated standardized mean difference between the unmeasured confounder
#> in the exposed population and unexposed population: -0.01
#> * estimated association between the unmeasured confounder and the outcome: 2
#> # A tibble: 1 x 5
#> observed_lb observed_ub smd outcome_association n_unmeasured_confounders
#> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 0.75 0.99 -0.0145 2 1
“An unmeasured confounder with an association with colorectal cancer indicence of 2 and a standardized mean difference between the metformin users and sulfonylurea users of -0.1 would tip this analysis at the 5% level, rendering it inconclusive.
”
06
tipr + broom
library(tipr)
library(broom)
glm(vs ~ am + wt, data = mtcars, family = "binomial") %>%
broom::tidy(conf.int = TRUE, exponentiate = TRUE) %>%
dplyr::filter(term == "am") %>%
tip(outcome_association = 2.5)
library(tipr)
library(broom)
glm(vs ~ am + wt, data = mtcars, family = "binomial") %>%
broom::tidy(conf.int = TRUE, exponentiate = TRUE) %>%
dplyr::filter(term == "am") %>%
tip(outcome_association = 2.5)
library(tipr)
library(broom)
glm(vs ~ am + wt, data = mtcars, family = "binomial") %>%
broom::tidy(conf.int = TRUE, exponentiate = TRUE) %>%
dplyr::filter(term == "am") %>%
tip(outcome_association = 2.5)
library(tipr)
library(broom)
glm(vs ~ am + wt, data = mtcars, family = "binomial") %>%
broom::tidy(conf.int = TRUE, exponentiate = TRUE) %>%
dplyr::filter(term == "am") %>%
tip(outcome_association = 2.5)
library(tipr)
library(broom)
glm(vs ~ am + wt, data = mtcars, family = "binomial") %>%
broom::tidy(conf.int = TRUE, exponentiate = TRUE) %>%
dplyr::filter(term == "am") %>%
tip(outcome_association = 2.5)
#> The observed effect (0, 0.16) WOULD be tipped by 1 unmeasured confounder
#> with the following specifications:
#> * estimated standardized mean difference between the unmeasured confounder
#> in the exposed population and unexposed population: -2.03
#> * estimated association between the unmeasured confounder and the outcome: 2.5
#> # A tibble: 1 x 5
#> observed_lb observed_ub smd outcome_association n_unmeasured_confounders
#> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 0.00000226 0.156 -2.03 2.5 1
library(tipr)
library(broom)
glm(vs ~ am + wt, data = mtcars, family = "binomial") %>%
broom::tidy(conf.int = TRUE, exponentiate = TRUE) %>%
dplyr::filter(term == "am") %>%
tip(outcome_association = 2.5, verbose = FALSE)
#> # A tibble: 1 x 5
#> observed_lb observed_ub smd outcome_association n_unmeasured_confounders
#> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 0.00000226 0.156 -2.03 2.5 1