This app enables you to specify your prior feelings about the effectiveness of two competing treatments (standard and novel), and subsequently find the sample size you would need to test whether the two are significantly different.

First, click the “Prior elicitation tab” and select values that generate prior distributions that reflect your current knowledge (which may result from past data). Once you are satisfied with these distributions, click the “Design” tab to specify the study’s characteristics, which includes the sample size in each group, the minimum clinically significant difference between the two response rates you desire, and the probability threshold that defines treatment significance. The app will then simulate clinical trials under these conditions, and report the trial’s average Type I error (false positive) rate and power (probability of success assuming the treatment benefit is clinically significant). The app will also suggest changes you might make to your design in order to increase power or reduce type I error. Click on the buttons "start tour" in each tab to be guided through the application.

We offer the following flow chart of the basic approach, followed by the technical details underlying the app in the next box.

Technical details

Suppose we let $\theta_1$ be the complete response (CR) rate under a reference standard cancer treatment, while $\theta_2$ is the CR rate under a novel therapy. We wish to design a Bayesian clinical trial to test whether the response rate for the novel therapy is significantly better than that on the standard therapy. Define $\delta$ to be the difference between the two rates, i.e., $\delta$ = $\theta_2$ - $\theta_1$.

The Bayesian paradigm allows us to do this using direct probability statements about whether or not the difference between the two rates, $\delta$, exceeds some “clinically significant difference”, which we can denote by $\eta$. For instance, if we set $\eta$=0.2, we’d be saying that a clinician would not be impressed with the new treatment unless its CR rate was at least 20% higher than standard. This means we are indifferent between the use of the new and standard treatments if $\delta \in (0,\eta)$; this interval is sometimes called the indifference zone.

A Bayesian would operationalize the decision by computing the probability of success (PoS),

PoS= P($\delta > \eta/2 \mid data$),
which is the chance of a treatment difference greater than the midpoint of the indifference zone. If this PoS exceeds some threshold $\gamma$ (say, 0.90 or 0.95), we conclude the novel treatment is effective. Note that we could use other cutpoints (say, $\eta$, the upper bound of the indifference zone), but then we’d need to dramatically reduce the confidence threshold (say, to 0.50, since a posterior centered at $\eta$ would by definition not have most of its mass above $\eta$).

Data: Suppose we randomize $n_1$ patients to standard therapy and $n_2$ patients to the novel therapy. Assuming no missing data for the moment, each patient then delivers a success-failure (0-1) outcome. Let $X_1$ and $X_2$ be the total number of CRs in the standard and novel treatment groups, respectively. Assuming the patients are independent, $X_1$ and $X_2$ are both distributed as binomial random variables, i.e., $X_1 \sim Bin(n_1,\theta_1)$ and $X_2 \sim Bin(n_2,\theta_1)$.

Priors: It is likely that the user has some idea of what $\theta_1$ should be, based on published data or perhaps their own past experience with the standard treatment. Reliable information about $\theta_2$ is perhaps less likely to exist (since this treatment is novel), but the user may still be able to rule out certain values (e.g., they might be fairly certain that $\theta_2$ is not less than 0.10 nor bigger than 0.50). A convenient functional form for representing prior opinions is the beta distribution, i.e., $\theta_1 \sim Beta(\alpha_1,\beta_1)$ and $\theta_2 \sim Beta(\alpha_2,\beta_2)$. This distribution can be bell-shaped ($\alpha$ and $\beta$ > 1), one-tailed ($\alpha$ or $\beta$=1), or uniform ($\alpha$=$\beta$=1), the so-called “flat prior” often used when the user has absolutely no prior knowledge about a treatment’s success rate. The two parameters $\alpha$ and $\beta$ can be determined by asking the user to specify two quantities: the most likely values for the two CR rates, $\theta_1^*$ and $\theta_2^*$, and the “the prior effective sample size” for these two CR rates, $\theta_1^{ESS}$ and $\theta_2^{ESS}$. These ESS quantities capture the number of patients the prior is “worth”; if $\theta_1^{ESS} = 20$ and we also expect to enroll $n_1=20$ standard treatment patients in the trial, we would be saying that our prior and the new data would contribute equally to our final answer. A regulator might prefer that $\theta_1^{ESS}$ be much less than $n_1$. The resulting beta distributions can then be plotted to the screen, and the user can then be asked to judge their appropriateness (say, in terms of relative location and overlap) and perhaps revise them as necessary.

Berry, S.M., Carlin, B.P., Lee, J.J. and Muller, P., 2010. Bayesian adaptive methods for clinical trials. CRC press.
Carlin, B.P. and Louis, T.A., 2008. Bayesian methods for data analysis. CRC Press.
Chen, N., Carlin, B.P. and Hobbs, B.P., 2018. Web-based statistical tools for the analysis and design of clinical trials that incorporate historical controls. Computational Statistics & Data Analysis, 127, pp.50-68.
Fouarge, E., Monseur, A., Boulanger, B., Annoussamy, M., Seferian, A.M., De Lucia, S., Lilien, C., Thielemans, L., Paradis, K., Cowling, B.S. and Freitag, C., 2021. Hierarchical Bayesian modelling of disease progression to inform clinical trial design in centronuclear myopathy. Orphanet journal of rare diseases, 16(1), pp.1-11.
Lewis, C.J., Sarkar, S., Zhu, J. and Carlin, B.P., 2019. Borrowing from historical control data in cancer drug development: a cautionary tale and practical guidelines. Statistics in biopharmaceutical research, 11(1), pp.67-78.
Monseur, A., Carlin, B.P., Boulanger, B., Seferian, A., Servais, L., Freitag, C. and Thielemans, L., 2021. Leveraging Natural History Data in One-and Two-Arm Hierarchical Bayesian Studies of Rare Disease Progression. Statistics in Biosciences, pp.1-22.
Lesaffre, E., Baio, G. and Boulanger, B. eds., 2020. Bayesian Methods in Pharmaceutical Research. CRC Press.

Elicited prior distribution

STEP 1: Most likely (mode) values for $\; \theta_1 \;$ (reference treatment) and $\; \theta_2 \;$ (novel therapy) best guesses, i.e. typical value

STEP 2: Prior effective sample size, i.e., the “worth” of your prior feelings about the CR rate expressed as a number of hypothetical study participants in each group

Sample sizes and other trial design parameters

User input feasible sample size n1 and n2, clinical significant difference eta and user-defined threshold

Simulations z

Number of simulations

Sample size summary


Need more?
Release: 0.1 (August 2021)

System Information