Choosing Orthogonal Arrays that are �Less Susceptible to Bias

​

Robert Mee and Mengmeng Liu

​

DAE 2024 @ Virginia Tech

Outline

• Example – Motivated by Poorna and Kulkarni’s 2^15-11 Fraction
• Analysis of their data
• Conjecture regarding nature of response to nutrients
• The impact of different isomorphic fractions of main effect estimates
• Proposed Strategy
• Begin with an informative prior
• Specify objective function and a family of isomorphic designs
• Search for optimum design from the family
• Alias Matrices and Bias
• Minimum (row-sum) Range Designs

​

​

​

Example: 2^15-11 Fractional Factorial

• Poorna and Kulkarni (1995) �“A Study of Inulinase Production… Using Fractional Factorial Design”

​

• The authors used a �resolution III 2^15-11 fraction

​

• Responses:
• y_A – Activity @ 60 hours
• y_B – Biomass @ 96 hours

​

• Note how the last row has�all low levels and
• No activity
• No biomass

​

​

​

​

Example of a Problem

What were the low levels that �produced no activity?

​

• The first 12 factors were carbon and�nitrogen sources, with low level 0.

​

• It is no surprise that nothing grew!

​

Attempted Analysis for Biomass – Lenth’s Method

​

​

• Only A’S estimate has p-value < 10%

​

• All 15 estimates are positive, �due to the 0 response at (-1,-1,-1,…,-1).

What if we treat the last row as a missing value?

• Now 4 main effects have p < 0.05.
• A and D are carbon sources
• K and M are inorganic nitrogen sources

​

• The model with 4 main effects �{A, D, K, M} has R² = 86%.

​

• However, this simple model predicts �y_B = 166 for (-1, -1, -1, -1), far from 0.

​

• The all-low treatment combination is “far” from the other t.c.

Questions Raised by This Example

1. Should the authors have anticipated this poor result at the “all-low” treatment combination?�
2. Would other isomorphic fractions have provided better estimates?�
3. What resolution III fraction would lead to the least bias to main effect estimates?

​

​

What Prior Information Seems Reasonable?

1. We can anticipate that each nutrient would provide some benefit – though we don’t know which nutrients will be the most beneficial.
2. Whatever benefit a nutrient brings, it will eventually level off. Hence, the expected responses will be increasing concave functions of the combined nutrients.
3. If Assumption 2 is correct, this will produce many negative two-factor-interaction effects.

Prior for Expected Responses

Given these two E(Y), what are the main effects?

What is the meaning of the true main effect?

1

18.36

1

7.68

Given these priors, what will Poorna’s fraction show?

True β_i’s from full 2¹⁵ Estimates and Bias from 2^15-11

​

Y1’s true main effects are smaller (since Y1 plateaus faster that does Y2) and so have more bias.

There are 2¹¹ different 2^15-11 fractions �(where we change the signs of the 11 generators)

3.84

​

0

There are 2¹¹ different 2^15-11 fractions �(where we change the signs of the 11 generators)

9.18

​

​

​

​

​

0

Sum of Squared Bias for Y1 and Y2

• For Y1, the max sum(Bias²) is 2582, the minimum = 8.97.
• For Y2, �max sum(Bias²) = 874�min sum(Bias²) = 13

​

• Fractions with all “-1’s” for�X1-X12 are the worst.

​

• The same fractions minimize�sum(Bias²) for both Y1 and Y2

Which 2^15-11 fractions minimize sum(Bias²) ?

28 cases

Which 2^15-11 fractions minimize sum(Bias²) ?

Worst………………..…..

The worst cases have a tc with 1 or fewer +1’s

Which 2^15-11 fractions minimize sum(Bias²) ?

Worst……………….…..Best

The best cases have none @ extremes

Which 2^15-11 fractions minimize sum(Bias²) ?

All low

​

​

​

​

​

​

​

​

​

​

All high

Total 2048

Min range = 8

Row sum: -15 -13 -11 -9 -7 -5 -3 -1 1 3 5 7 9 11 13 15

Which 2^15-11 fractions minimize sum(Bias²) ?

• The 96 fractions that minimize sum(Bias²) come from rows 5,6,7,9,10

​

​

​

​

​

​

​

• We will focus on the 12 fractions from row 7 that also minimize the row sum range.

​

​

​

​

12 of

36 of

** 12 of

​

24 of

12 of

​

​

​

Total 2048

What interaction effects do these priors create?

For Y1 and Y2:

• All coefficients = 0 for effects involving �X₁₃, X₁₄, and/or X₁₅
• All coefficients involving only X₁,…, X₁₂ �depend only on the order of the interaction
• Odd effects are positive
• Even effects are negative
• Coefficients decline in magnitude �as the order of interaction increases,�with Y2’s interactions declining faster

​

​

Strategy for Selecting an Isomorphic Fraction

• Determine factors and their levels
• Propose a design

Then, do more:

1. Conjecture a model for the response, given these factors
2. Specify an objective function to be optimized
3. Create the family of all (isomorphic) fractions
4. Find the optimum fraction(s) – i.e., determine which fractions yield the best objective function value.

​

These we already do

Strategy for Selecting an Isomorphic Fraction – Our Example

Recommended Design

This is one of 12 designs with minimum range that also minimizes sum(Bias²).

Alias Matrices and Bias

• Let X = [X₁, X₂] denote the full factorial model matrix, partitioned, with X₁ denoting the model matrix for the fitted model.
• The alias matrix is A = (X₁’X₁)^-1 X₁’X₂
• For the full factorial design, all factorial effect columns are orthogonal, so A = 0.
• For a regular fractional factorial, A consists of 0’s, 1’s, and/or -1’s.
• For regular fraction with “all high” t.c., there are no -1’s
• For regular fraction with “all low” t.c.:
• Even interactions have 0’s and -1’s
• Odd interactions have 0’s and +1’s
• For Y1 and Y2:
• Bias depends on the sum of columns of A of a given order

Sum of Alias Matrix Coefficients �for Three Different 2^15-11 Fractions

This plot shows the row sums�of the alias matrix for interaction�orders 2, 3, 4, 5:

1. 2^15-11 with all-low t.c.: �-7, 28, -84, 189
2. 2^15-11 with all-high t.c.:�7, 28, 84, 189
3. Recommended fraction:� 3- 4 = -1, �12-16 = -4, �44-40 = 4, �97-92 = 5

​

Best!!

Summary of the 2048 Isomorphic 2^15-11

• Designs with minimum range for row sums also minimize �Q(D) = sum(|row sums of alias matrix from all two-factor interactions|)

​

​

​

​

​

​

​

​

• Some of the designs with minimum Range also minimize Sum(Bias²)

Sum of Bias² for Y1

14 Row Sum Distributions among the 2^15-11 Fractions

All row sum distributions have:

• mean 0
• sum of squares = kN = 15*16

​

10 cases have range = 16

2 cases have range = 12

2 cases have range = 8

Minimum Row Sum Range Design: N=16 Examples�

• k = 12 factors, resolution III 2^12-8
• Maximum range = 16 (44 fractions, one being the default fraction in software)
• All-high tc: Row sum = +12
• 3 tc’s with 4 +1’s: Row sums = -4
• Minimum range = 8 (12 such fractions)
• 6 tc’s with 8 +1’s: Row sums = +4
• 6 tc’s with 4 +1’s: Row sums = -4

​

• k = 8 factors, resolution IV 2^8-4
• Maximum range = 16 (1 such fraction, the default fraction in software)
• All-high tc: Row sum = +8
• All-low tc: Row sum = -8
• Minimum range = 8 (7 such fractions)
• 6 tc’s with 8 +1’s: Row sums = +4
• 6 tc’s with 4 +1’s: Row sums = -4

​

​

What is the minimum rows sum for 16-run designs for k = 6, …, 15

• k =15, min range = 8 (56 of 2¹¹ fractions)
• k =14, min range = 10 (56 of 2¹⁰ fractions)
• k =13, min range = 10 (24 of 2⁹ fractions)
• k =12, min range = 8 (12 of 2⁸ fractions)
• k =11, min range = 10 (12 of 2⁷ fractions)
• k =10, min range = 8 (2 of 64 fractions)
• k = 9, min range = 8 (2 of 32 fractions)
• k = 8, min range = 8 (7 of 16 fractions)
• k = 7, min range = 10 (7 of 8 fractions)
• k = 6, min range = 8 (3 of 4 fractions)

​

Res. IV Res. III

Minimum Row Sum Range Design: N=32 Example�

• N=32, k = 16 factors, 2^16-11 resolution IV
• Maximum range = 32 (1 such fraction, the default fraction in software)
• All-high tc: Row sum = +16
• All-low tc: Row sum = -16
• Row Sum Range = 32
• Minimum range = 8 (24 such fractions)
• 16 tc’s with 10 +1’s: Row sums = +4
• 16 tc’s with 6 +1’s: Row sums = -4
• Row sum Range = 8

​

Since main effects correspond to the effect of adding a factor, averaging over the levels of the other factors, designs spanning a narrower range of factors at the anticipated preferred level might be more robust.

​

​

​

Minimum and Maximum Row Sum Ranges �for 32-run designs: k = 7, …, 31

• The resolution IV �example at k=16 �with min range = 8 �is exceptional!
• Resolution III designs�have min range of half�(or less) than the max.
• Even resolution IV�designs (k>10) show �more gain than the�even/odd res. IV designs

Min and max rows sum for 64-run minimum aberration designs: k = 8, …, 32 (res. IV) and k=33 (res. III)

• These are all exhaustive�searches for the minimum�aberration designs, �even for 2^33-27
• Max range = 64
• Min range = 16
• Only 0.01% of fractions �have the min range)

Questions I’d Like to Answer

• What other situations have prior information that would inform a preference for certain fractions?

​

• How can we argue more forcefully that minimum range designs are more robust?

​

• What about nonregular fractions?