Issue: Volume 2, Number 3
Date: March 2002
From: Mark J. Anderson, Stat-Ease, Inc.
Here's another set of frequently asked questions (FAQs) about doing design of experiments (DOE), plus alerts to timely information and free software updates. If you missed previous DOE FAQ Alerts, go to the links below. Feel free to forward this newsletter to your colleagues. They can subscribe by going to http://www.statease.com/doealertreg.html.
I offer the following link as an appetizer: http://www.design-engineering.com/index.cgi?pcode=ROCKETGUY which tells the story of Brian Walker - a man following his dream to build a rocket ship that will propel him into outer space. For more details on Brian's visions of blue sky and beyond, see his web site at http://www.rocketguy.com.
Here's what I cover in the body text
of this DOE FAQ Alert (topics that delve into statistical detail are rated "X"):
1. FAQ: DOE for non-manufacturing areas such as marketing
2. X-FAQ: Setting up and evaluating a Plackett-Burman (P-B) design as specified by standards for ruggedness testing
a. How Design-Expert handles P-B designs
b. Problems with aliasing in P-B are not resolved via replication
3. Info alert: Ruggedness DOE done on machine-made bread (link to an article published in "Today's Chemist at Work")
4. Workshop alert: "Robust Design, DOE Tools for Reducing Variation" and other upcoming classes. "Statistics for Technical Professionals workshop now computer-intensive"
5. Book alert: Free copies (seven only!) of Cornell's "How to Apply Response Surface Methodology" pamphlet
PS. Quote for the month - math versus creativity
1 - FAQ: DOE for non-manufacturing
areas such as marketing
"I was recently asked to assist our sales and marketing staff with implementing DOE principles within their department. I have only been working with DOE for about a year and it's been with chemical formulations so I don't have a really good grasp of business practices in relation to sales and marketing. Would you have any information, reports or recommendations as to how I might approach this experiment? Ideally I thought about using customer satisfaction (qualitative) or market share (quantitative) as a response variable, am I on the right track? Has this type of experiment, in this environment, been performed before?"
DOE requires controllable inputs
and measurable outputs. That's all! Therefore it can be applied to non-manufacturing
processes given the right circumstances. For example, it's common in marketing
to test variables such as price, promotion, place and product configuration.
The response can be sales, consumer preference, etc. These issues are near and
dear to me, because long ago, after completing my MBA, I branched out from my
chemical engineering career into product management, marketing and other business-related
functions. Unfortunately, as a practical matter, it's not all that easy to apply
DOE in these fields.
Several years ago I researched this issue and came up with a list of case studies that you can view at http://www.statease.com/pubs/doe-non-mfg.pdf. There's a story behind this. In March of 1996 I suddenly got a spate of calls asking if we did "MVT" because according to "Forbes" magazine issue 3/11/96* "A Minneapolis software firm, Stat-Ease, sells most of the software these MVT types use." Not having seen this issue of Forbes, my first question was "What's MVT?" It turns out that this meant "multivariable testing," which is an apt description of what DOE does. The article went on to say "If you haven't yet applied multivariable testing to your business, get moving. Whether you run a factory, a mail-order house or a hospital, it will probably improve your performance." I agree!
I see that you're from Kansas - the
great state from which Dorothy embarked on her wonderful adventure to the land
of Oz. Remember that classic scene when Toto the dog pulls back the curtain
and we all see the machinations of the Wizard of Oz? I hope DOE can also pull
back the curtain and help you see what really affects your marketing process.
*("The new mantra: MVT" by Rita Koselka, pages 114-118. For a synopsis and discussion on this article see
archives only go back to 2005 )
PS. Stat-Ease will be leading a session on "The Use of DOE in Non-Manufacturing Environments" at the Spring Research Conference of the Quality and Productivity (Q&P) section of the American Statistical Association (ASA) on June 5-7 in Tempe, Arizona (near Phoenix). It will be a great opportunity to hear how DOE and other statistical tools can be used to enhance productivity and improve the quality of products and services. For more details, see http://ceaspub.eas.asu.edu/ie/2002qprc/ (this link is no longer available).
2 - X-FAQ: Setting up and evaluating
a Plackett-Burman (P-B) as design specified by standards for ruggedness testing
a. How Design-Expert handles P-B designs
b. Problems with aliasing in P-B not resolved via replication
From: Connecticut (paraphrased from face-to-face consult)
a. "I don't see the eight-run option for Plackett-Burman design in Design-Expert software. Why not? How do I enter this?
b. "The design was fully replicated per protocol dictated by the standards I must meet for validation of my process. Does this improve its resolution?"
a. Design-Ease® and Design-Expert®
software offer Plackett-Burman (P-B) designs only for 12, 20, 24, 28 and 32
runs, which fill in gaps in options offered by our standard two-level catalog
(2^(k-p). For designs with eight runs, choose the seven-factor 2^(k-p) option
rather than the P-B. Then on the next screen click the "Make generators
editable" option and enter:
[D] = -AC
[E] = ABC
[F] = -AB
[G] = -BC
These generators reproduce the design in Plackett and Burman's original 1946 paper. Be aware that many standards and other references list alternative versions. If you find that the patterns don't match your version of the P-B, simply type over the design layout of factor levels and save it for future use as a template.
(My partner Pat Whitcomb provided the generators shown above. Check out the option to input design generators on your copy of Stat-Ease software. If you don't own a license, download a fully-functional trial version of Design-Expert at http://www.statease.com/dx6trial.html.)
b. The alias structure for the P-B
noted above is:
[A] = A - BF - CD - EG + BCE + BDG + CFG + DEF
[B] = B - AF - CG - DE + ACE + ADG + CDF + EFG
[C] = C - AD - BG - EF + ABE + AFG + BDF + DEG
[D] = D - AC - BE - FG + ABG + AEF + BCF + CEG
[E] = E - AG - BD - CF + ABC + ADF + BFG + CDG
[F] = F - AB - CE - DG + ACG + ADE + BCD + BEG
[G] = G - AE - BC - DF + ABD + ACF + BEF + CDE
Notice that main effects are confounded
with two-factor interactions, making this design Resolution III. Equivalent
P-B's or 2^(k-p) options for seven factors in eight runs will also be Resolution
III. Replicating such a design will improve the power, which obviously is the
intention of ASTM, but it will not increase the resolution. If analysis of variance
(ANOVA) shows significant effect(s), thus indicating ruggedness failure, you
will need resolve the troublesome aliases of main effects by performing further
runs that involve new combinations of factors. Typically
for P-B's this is done via a procedure called "foldover," which, in this case, would require eight more runs with levels opposite those performed in the first block.
If you're not constrained by protocol dictated by any particular standard, consider running a higher Resolution IV design for your initial ruggedness test. See the next Alert (#3) for an example and explanation on why I recommend this.
(Learn more about Plackett-Burman
designs and aliasing of two-level factorial designs by attending the 3-day computer-intensive
workshop "Experiment Design Made Easy" (EDME). For a description,
see http://www.statease.com/clasedme.html. Link
from this page to the course outline and schedule. We've got classes coming
- Minneapolis, April 9-11 and June 4-6, and
- San Jose on May 7-9.)
3 - Info alert: Ruggedness DOE
done on machine-made bread (link to an article published in "Today's Chemist
Go to http://pubs.acs.org/journals/tcwoe7/index.html, page down and click the link labeled "The Knead for Speed" to see how I did a ruggedness DOE to evaluate alternative raw materials for making machine-made bread. I encourage all of you kitchen chemists and engineers to try this at home!
Here's a 'sidebar' on the terminology
of "Ruggedness" versus "Robustness" that I uncovered while
doing the research for my bread DOE. Ruggedness is a term that's been used for
several decades, primarily for application to analytical method development,
for testing possible sources of variation. I discovered that more recent (within
last 10 years) handbooks and articles on assay validation assign the term "ruggedness"
to "external" conditions such as when and where the assay gets done,
ambient temperature and humidity, alternative sources of raw materials and lot-by-lot
changes. On the other hand, the term "robustness" now seems to be
in favor for those variables that are "internal" to the assay (or
process or product). For example, the USP (U.S. Pharmacopeia) and ICH (International
Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals
for Human Use) define robustness as "a measure of its [an analytical method]
capacity to remain unaffected by small but deliberate variations in method parameters."
Doing this, typically with the aid of DOE, "provides an indication of its
reliability during normal use." Method parameters include changes in input
factors, such as processing time and temperature,
composition of reagents, etc. Both types of variables, rugged versus robust, must be included in a proper validation test, so it really becomes just a matter of semantics.
4 - Workshop alert: "Robust
Design, DOE Tools for Reducing Variation" and other classes coming up.
"Statistics for Technical Professionals workshop now computer-intensive"
If you are proficient with the tools
of response surface methods (RSM) and desire knowledge on advanced tools for
Six Sigma, attend "Robust Design, DOE Tools for Reducing Variability"
(RDRV). See http://www.statease.com/clasrdrv.html
for course content. We've got plenty of room in our class next week in Minneapolis.
Call now to sign up for this session or the next one, also in Minneapolis, on
June 11-13. Do not enroll in these classes unless you know RSM. If you need
this prerequisite, attend our "Response Surface Methods for Process Optimization"
workshop in Minneapolis on April 16-18 (see http://www.statease.com/clas_rsm.html
Do you seek more elementary tools for quality improvement? If so, come to our "Statistics for Technical Professionals" (STP) workshop on March 3/26-27 in Minneapolis. We've tossed out the calculators and now solve all of the problems with the aid of Microsoft Excel, which will be made available in class for use by students on classroom computers. This is a great opportunity to gain a working knowledge of confidence versus tolerance intervals and much, much more (for details, see http://www.statease.com/clas_stp.html ).
See http://www.statease.com/clas_pub.html for schedule and site information on all Stat-Ease workshops open to the public. To enroll, click the "register online" link at our web site or call Stat-Ease at 1.612.378.9449. If spots remain available, bring along several colleagues and take advantage of quantity discounts in tuition, or consider bringing in an expert from Stat-Ease to teach a private class at your site. Call us to get a quote.
5 - Book alert: Free copies (seven only!) of "How to Apply Response Surface Methodology" pamphlet
A few months ago we cleaned house
at Stat-Ease and found a number of old, but still relevant, statistical texts
that we no longer use in our workshops. This month I am giving away seven paperback
copies of John Cornell's "How to Apply Response Surface Methodology."
The pamphlet is Volume 8 in a series published by the Statistics Division of
what was then called the American Society of Quality Control (they lost "Control"
a few years ago!). I will send these RSM pamphlets (four first editions from
1984 plus three revised in 1990) to the first seven people who e-mail me. Based
on my experience giving away other books the last few months, this will happen
within 1 hour of my DOE FAQ Alert
broadcast, so don't bother asking after that. I want to spread the wealth of information, so those of you who already received a book won't be eligible this time. Please note the shipping address in your e-mail request. If you want the Cornell pamphlet, but don't reply soon enough, go to the ASQ publications web site at http://qualitypress.asq.org/perl/catalog.cgi?item=T3508 where you can buy the revised edition for $32.00 (less if you're a member).
Last month I mentioned that I had just received new editions of two books which I highly recommend, but are not free: - "Response Surface Methodology," 2nd Edition, by Myers and Montgomery, John Wiley & Sons, 2002 - "Experiments with Mixtures," 3rd Edition, by Cornell, John Wiley & Sons, 2002. Purchase these books at http://www.statease.com/prodbook.html.
In addition to the new editions noted above, we recently added two new books to our e-commerce site:
- "Engineering Statistics," 2nd Edition, by Montgomery, Runger and Hubele, John Wiley & Sons, 2001
- "Statistical Intervals: A Guide for Practitioners," by Hahn and Meeker, John Wiley & Sons, 1991
We recommend these texts for our new "Statistics for Technical Professionals" workshop (http://www.statease.com/clas_stp.html).
I hope you learned something from this issue. Address your questions and comments to me at:
Mark J. Anderson, PE, CQE
Principal, Stat-Ease, Inc. (http://www.statease.com)
Minneapolis, Minnesota USA
PS. Quote for the month - math versus
"A lot of engineers are so into the math that they can't be creative. After four or five years of equations you start to wonder if you can think creatively any more."
- Robert Ripley, mechanical engineer and friend of Rocket Guy (see the "appetizer" at the beginning of this Alert)
Trademarks: Design-Ease, Design-Expert and Stat-Ease are registered trademarks of Stat-Ease, Inc.
Acknowledgements to contributors:
- Students of Stat-Ease training and users of Stat-Ease software
- Fellow Stat-Ease consultants Pat Whitcomb and Shari Kraber (see http://www.statease.com/consult.html for resumes)
- Statistical advisor to Stat-Ease: Dr. Gary Oehlert (http://www.statease.com/garyoehl.html)
- Stat-Ease programmers, especially Tryg Helseth (http://www.statease.com/pgmstaff.html)
- Heidi Hansel, Stat-Ease marketing director, and all the remaining staff.
Interested in previous FAQ DOE Alert e-mail newsletters? To view a past issue, choose it below.
#1 - Mar 01, #2 - Apr 01, #3 - May 01, #4 - Jun 01, #5 - Jul 01 , #6 - Aug 01, #7 - Sep 01, #8 - Oct 01, #9 - Nov 01, #10 - Dec 01, #2-1 Jan 02, #2-2 Feb 02, #2-3 Mar 02 (see above)