By way of example, this article lays out a strategy for design of experiments (DOE) that provides maximum efficiency and effectiveness for development of a robust system. It broadens the scope of a prior article (Anderson and Whitcomb 2014) that spelled out how to right-size multifactor tests via statistical power-calculations—a prerequisite for DOE success.
Engineers at a major medical device manufacturer used RSM to successfully model a key process for their flagship product. The RSM model then became the foundation for development of robust specifications to ensure quality at six-sigma levels.
An industrial equipment supplier made great improvements to their corn-ethanol measurement process using DOE.
Because interactions abound in the coatings industry, the multifactor and multicomponent test matrices provided by the design of experiments (DOE) approach is very appealing. However, carrying out DOE correctly requires that runs be randomized whenever possible to counteract the bias that may be introduced by time-related trends, such as aging of materials, increasing humidity, and the like. But what if complete randomization proves to be inconvenient or impossible? In this case, a specialized form of design called “split plot” becomes attractive because of its ability to effectively group hard-to-change (HTC) factors. A split plot accommodates both HTC factors and those factors that are easy to change (ETC).
Carrying out a DOE correctly requires that runs be randomized whenever possible to counteract the bias that may be introduced by time-related trends. If complete randomization proves to be impossible, however, a specialized form of design—called a split plot—is useful because of its ability to effectively group hard-to-change (HTC) factors. It accommodates both HTC and easy-to-change (ETC) factors in the design.
By sizing experiment designs properly, test and evaluation (T&E) engineers can assure they specify a sufficient number of runs to reveal any important effects on the system. For factorial designs laid out in an orthogonal matrix this can be done by calculating statistical power. However, when a defense system behaves in a nonlinear fashion, then response surface method experiment (RSM) designs must be employed. The test matrices for RSM generally do not exhibit orthogonality, thus the effect calculations become correlated and degrade the statistical power. This in turn leads to inflation in the number of test runs needed to detect important performance differences that may be generated by the experiment. A generally acceptable alternative to sizing designs makes use of fraction of design space (FDS) plots. This article details the FDS approach and explains why it works best to serve the purpose of RSM experiments done for T&E.
Due to operational or physical considerations, standard factorial and response surface method (RSM) design of experiments (DOE) often prove to be unsuitable. In such cases a computer-generated statistically-optimal design fills the breech. This article explores vital mathematical properties for evaluating alternative designs with a focus on what is really important for industrial experimenters. To assess “goodness of design” such evaluations must consider the model choice, specific optimality criteria (in particular D and I), precision of estimation based on the fraction of design space (FDS), the number of runs to achieve required precision, lack-of-fit testing, and so forth. With a focus on RSM, all these issues are considered at a practical level, keeping engineers and scientists in mind. This brings to the forefront such considerations as subject-matter knowledge from first principles and experience, factor choice and the feasibility of the experiment design.
This article provides insights on how many runs are required to make it very likely that a test will reveal any important effects. Due to the mathematical complexities of multifactor design of experiments (DOE) matrices, the calculations for adequate power and precision are not practical to do by 'hand' so the focus is kept at a high level--scoping out the forest rather than detailing all the trees. By example, reader will learn the price that must be paid for an adequately-sized experiment and the penalty incurred by conveniently grouping hard-to-change factors.
BOOK REVIEW: This book provides guidance on the construction of experiments, including sample size calculations, hypothesis testing, and confidence estimation.