1 of 37

Generalized Basket Trial and Applications in Mental Health

Sahil Patel, Nicole Ledwos, David Castle, and Clement Ma

JSM – August 8^th, 2023

2 of 37

1.

Basket Trials and Improvements

2.

Generalization

Results

Discussion

3.

4.

AGENDA

3 of 37

Confirmatory Basket Trials and Improvements

1.

3

4 of 37

Confirmatory Basket Trials

Basket trials evaluate a single targeted intervention across multiple diseases/disorders with a common disease modifiers

Confirmatory basket trials randomize participants within baskets

Results are either pooled or evaluated separately for each basket.

Examples of randomized or confirmatory basket trials:

SHIVA [2] (oncology)
NCI-MPACT [3] (oncology)

4

5 of 37

Improving the Confirmatory Basket Trial

Chen et al. 2016 [4] proposed procedures to improve power of confirmatory basket trials

Pooling (as done in other basket trials; combining remaining baskets to determine overall efficacy)

5

Cong Chen, Xiaoyun (Nicole) Li, Shuai Yuan, Zoran Antonijevic, Rasika Kalamegham & Robert A. Beckman (2016) Statistical Design and Considerations of a Phase 3 Basket Trial for Simultaneous Investigation of Multiple Tumor Types in One Study, Statistics in Biopharmaceutical Research, 8:3, 248-257, DOI: 10.1080/19466315.2016.1193044

6 of 37

Improving the Confirmatory Basket Trial

Chen et al. 2016 [4] proposed procedures to improve power of confirmatory basket trials

Pooling (as done in other basket trials; combining remaining baskets to determine overall efficacy)
Pruning (removing a basket that is inactive at an interim point)

Final pooled test statistic must be evaluated at a new 𝛼 (𝛼*) maintain an overall Type 1 error rate

6

Cong Chen, Xiaoyun (Nicole) Li, Shuai Yuan, Zoran Antonijevic, Rasika Kalamegham & Robert A. Beckman (2016) Statistical Design and Considerations of a Phase 3 Basket Trial for Simultaneous Investigation of Multiple Tumor Types in One Study, Statistics in Biopharmaceutical Research, 8:3, 248-257, DOI: 10.1080/19466315.2016.1193044

7 of 37

Improving the Confirmatory Basket Trial

Chen et al. 2016 [4] proposed procedures to improve power of confirmatory basket trials

Pooling (as done in other basket trials; combining remaining baskets to determine overall efficacy)
Pruning (removing a basket that is inactive at an interim point)

Final pooled test statistic must be evaluated at a new 𝛼 (𝛼*) maintain an overall Type 1 error rate

Resampling to maintain initial proposed sample size (if a basket is removed, reallocate that number of individuals to the remaining baskets)

7

Cong Chen, Xiaoyun (Nicole) Li, Shuai Yuan, Zoran Antonijevic, Rasika Kalamegham & Robert A. Beckman (2016) Statistical Design and Considerations of a Phase 3 Basket Trial for Simultaneous Investigation of Multiple Tumor Types in One Study, Statistics in Biopharmaceutical Research, 8:3, 248-257, DOI: 10.1080/19466315.2016.1193044

8 of 37

Chen et al. design

Overview of the “D2” basket trial

Sample size of 300 (100 per basket)
Individuals are randomized

8

9 of 37

Chen et al. design

Conduct Interim Analysis

In this example, it happens when each basket accrues 50% of its total (50 per basket)
Interim analysis is conduct at a type 1 error rate 𝛂_t
In this example, basket A and B show some activity

9

10 of 37

Chen et al. design

Resample the Baskets

Because basket C was pruned, it’s sample get reallocated
Basket C was planned to have 100 people, so 50 go to basket A, and 50 go to basket B

10

11 of 37

Chen et al. design

Conduct Pooled Analysis

Pool the remaining baskets (A and B) and conduct the final analysis at a calculated a type 1 error rate of 𝛂* to maintain an overall 𝛂 type 1 error rate

11

12 of 37

Motivating Trial: efficacy of psilocybin for patients with OCD, BDD, and AN

Study objective: To evaluate psilocybin (a psychedelic) as a treatment for patients with obsessive-compulsive disorder (OCD), body dysmorphic disorder (BDD), and anorexia nervosa (AN)

Current open-label studies at CAMH are looking at the safety, clinical effects, and feasibility of psilocybin

prompts the need to investigate psilocybin in a confirmatory setting

Investigators believed the disorders had a common pathway [5], thus the basket design would be efficient to evaluate psilocybin across all 3 disorders

Image from https://www.goodrx.com/well-being/diet-nutrition/psylocybin-magic-mushrooms

12

13 of 37

Motivating Further Developments

After the design was proposed to clinicians, though there was concerns in enrollment

Clinicians believed one disorder would accrue half as fast as the others

Limitation of current method: all baskets must have the same N and effect size

Objective: To generalize the Chen et al. design to account for different sized baskets and adapt formula to calculate power and sample size

13

14 of 37

Generalization

2.

14

15 of 37

Generalization - Sampling

Baskets aren’t sampled equally (we use a proportion vector p to allocate a certain proportion of the total sample to each basket)

In this example, the first basket gets 20% of the total sample, and the other two get 40%

Chen et al. 2016 “D2”

Our Generalization

16 of 37

Generalization - Sampling

Pruned sample size now proportionally returns to the remaining baskets (denoted as a weight, w)

Chen et al. 2016 “D2”

Our Generalization

17 of 37

Generalization – Pooled Test Statistic

Pooled test statistic is now a weighted sum using w from before

m is the number of baskets after pruning
Both are summed over the indices of the remaining baskets, i
Y_i2 is the final standardized test static of the i-th basket

Chen et al. 2016 “D2”

Our Generalization

18 of 37

Power Calculation (Chen et al. 2016 “D2”)

Number of baskets that make it past the interim analysis

18

Hmm, this may be too detailed. Not sure if you will have enough time. Perhaps your prior version was better?

You can present this on Monday. Let’s see how long it takes and we can cut accordingly.

Number of baskets that are truly active (non-zero effect size)

19 of 37

Power Calculation (Chen et al. 2016 “D2”)

Probability final pooled test statistic is significant given m baskets make it past the interim analysis and j out of m are truly active (non-zero effect size)

19

Hmm, this may be too detailed. Not sure if you will have enough time. Perhaps your prior version was better?

You can present this on Monday. Let’s see how long it takes and we can cut accordingly.

20 of 37

Power Calculation (Chen et al. 2016 “D2”)

Combinatorics to make use of equivalent situations that arise as a result of the baskets having equal effect and sample sizes

Every choice of which active and inactive baskets make it past interim at a fixed count of each of those baskets

20

Hmm, this may be too detailed. Not sure if you will have enough time. Perhaps your prior version was better?

You can present this on Monday. Let’s see how long it takes and we can cut accordingly.

21 of 37

Power Calculation (Our Generalization)

Vector of predicted standardized effect sizes (ex. (0.5,0.3,0.7))

Vector of which baskets make it past the interim analysis (ex. (0,0,1))

22 of 37

Power Calculation (Our Generalization)

Probability the final pooled test statistic is significant given a fixed choice of overall active baskets and which baskets make it past interim

Vector of which baskets are truly active (ex. (1,1,1))

23 of 37

Power Calculation (Our Generalization)

Sum over every possible choice of baskets that make it past interim

ex. (1,0,0), (0,1,0), (0,0,1), (1,1,0), (1,0,1), (0,1,1), (1,1,1)

Can’t use combinatorics because baskets are different sizes and have different effect sizes

24 of 37

Evaluating the Generalization

Code is built off the code provided by Chen et al. 2016 for power calculation

We take 4 steps to analyze this generalization

Validate with Chen et al. 2016

ensure that our results are consistent

Different Proportions vs. Final Pooled 𝛼*
Different Proportions vs. Power
Different Effect Sizes vs. Power

Points 2-4 are under the assumption that all baskets are truly active

We use the Gini Impurity to denote how even the baskets are split (higher = more even split)

25 of 37

Results

3.

25

26 of 37

1) Validation

Programming all of the formula into R, we test power for a couple scenarios to ensure that our formula calculate power correctly
Testing with no active baskets also yields the correct type 1 error rate

All calculations done at an information time of 0.5, and trying to maintain an overall one-sided 0.025 type 1 error rate

26

				Power for Final Pooled Statistic
# of Baskets	Sample Size	Sample Size Per Basket	Effect Size	Chen “D2”	Our Generalization
2	150	75	0.5	0.8682	0.8682
3	150	50	0.5	0.8786	0.8786
6	150	25	0.5	0.8915	0.8915

27 of 37

2) Proportions vs. Final Pooled 𝛂*

𝛂* represents the final pooled test statistic threshold to maintain an overall 𝛂 type 1 error

All calculations done with a final sample size of 150, effect size of 0.5, at an information time of 0.5, and trying to maintain an overall one-sided 0.025 type 1 error rate

2 Basket Design

3 Basket Design

Proportions (p)	𝛂*
(0.5, 0.5)	0.0143
(0.666, 0.333)	0.0152
(0.8, 0.2)	0.0168

Proportions (p)	𝛂*
(0.333, 0.333, 0.333)	0.0100
(0.4, 0.4, 0.2)	0.0106
(0.8, 0.1, 0.1)	0.0142

28 of 37

3) Proportions vs. Power for Final Pooled Analysis

2 Baskets; Equal Effect Sizes

3 Baskets; Equal Effect Sizes

All calculations done with a final sample size of 150, effect size of 0.5, at an information time of 0.5, and trying to maintain an overall one-sided 0.025 type 1 error rate

29 of 37

4) Different Effect Size vs. Power

All calculations done with a final sample size of 150 (equally sized baskets), effect size of 0.5, at an information time of 0.5, and trying to maintain an overall one-sided 0.025 type 1 error rate

29

Average Effect Size	Effect Size Per Basket	Power
0.2	(0.2, 0.2, 0.2)	0.2455
0.5	(0.5, 0.5, 0.5)	0.8786
0.5	(0.7, 0.5, 0.2)	0.8862
0.5	(0.8, 0.6, 0.1)	0.9428
0.5	(0.9, 0.5, 0.1)	0.9494
0.5	(1.1, 0.2, 0.2)	0.9692

30 of 37

Discussion

4.

30

31 of 37

Conclusion

Trends

More unequal proportions ⇒ slightly lower power
One strong basket drives the power of the whole trial

Feasibility

To test psilocybin against OCD, BDD, and AN, a basket trial seems reasonable despite the fact the accrual rate isn’t equal across all 3 disorders

31

32 of 37

Final Proposed Design

Based on the results, we proposed a basket trial with OCD, BDD, and AN.

OCD and AN would have 40% of the total sample
BDD would have 20% of the total sample

Based on a predicted effect size of 0.5, a total sample size of 150 individuals powers the study

however pilot studies are still ongoing and effect sizes will be updated once they conclude

33 of 37

Limitations and Future Work

Our work only looks at a generalization of the “D2” method from Chen et al. 2016

There are other methods we should investigate

Calculating expected number of baskets after the interim analysis
Providing a framework on choosing proportions based on anticipated effect size
The main limitation is that 𝛂* is not calculated directly

Calculated using a root solver which explains the slight variance in the charts
Could run more samples to reduce variance
Running 10,000 samples (3 baskets, 150 people, equal proportions, 0.5 effect size) yields:

Range: 0.0007
Standard Error: ~0.0001

34 of 37

Acknowledgements and Disclosures

Thank you to Nicole Ledwos, MSc (McGill University) and Dr. David Castle (University of Tasmania) for their clinical input
Thank you to Dr. Clement Ma whose knowledge, direction, and support made this presentation possible
No relevant disclosures

35 of 37

Works Cited

Park, J.J.H., Siden, E., Zoratti, M.J. et al. Systematic review of basket trials, umbrella trials, and platform trials: a landscape analysis of master protocols. Trials 20, 572 (2019). https://doi.org/10.1186/s13063-019-3664-1
Le Tourneau, Christophe, et al. “Molecularly Targeted Therapy Based on Tumour Molecular Profiling versus Conventional Therapy for Advanced Cancer (Shiva): A Multicentre, Open-Label, Proof-of-Concept, Randomised, Controlled Phase 2 Trial.” The Lancet Oncology, vol. 16, no. 13, Oct. 2015, pp. 1324–1334, https://doi.org/10.1016/s1470-2045(15)00188-6.
Chen, Alice P et al. “Molecular Profiling-Based Assignment of Cancer Therapy (NCI-MPACT): A Randomized Multicenter Phase II Trial.” JCO precision oncology vol. 5 PO.20.00372. 12 Jan. 2021, doi:10.1200/PO.20.00372
Cong Chen, Xiaoyun (Nicole) Li, Shuai Yuan, Zoran Antonijevic, Rasika Kalamegham & Robert A. Beckman (2016) Statistical Design and Considerations of a Phase 3 Basket Trial for Simultaneous Investigation of Multiple Tumor Types in One Study, Statistics in Biopharmaceutical Research, 8:3, 248-257, DOI: 10.1080/19466315.2016.1193044
Malcolm A, Labuschagne I, Castle D, Terrett G, Rendell PG, Rossell SL. The relationship between body dysmorphic disorder and obsessive-compulsive disorder: A systematic review of direct comparative studies. Australian & New Zealand Journal of Psychiatry. 2018;52(11):1030-1049. doi:10.1177/0004867418799925

36 of 37

Thank You

Feel free to reach out at sspate27@ncsu.edu

Thank You

37 of 37

Final Pooled Analysis

Calculation of ⍺* were done using by evaluating the root of the following equations

this caused a slight variance in power as seen in the previous charts

37