Issue: Volume 1, Number 10
Date: December 2001
From: Mark J. Anderson, Stat-Ease, Inc.
Here's another set of frequently asked questions (FAQs) about doing design of experiments (DOE), plus alerts to timely information and free software updates. If you missed previous DOE FAQ Alerts, go to http://www.statease.com/doealert.html. To look for specific topics, click the Search button and type in what you're looking for. Feel free to forward this newsletter to your colleagues. They can subscribe by going to: http://www.statease.com/doealertreg.html.
Happy holidays to you all! Some spots
in Minnesota got over two feet of snow last Tuesday, so it really looks festive.
Last year at this time we hosted an exchange student from Mexico. It was a treat
when this student saw snow for the first time out the car window here in Minnesota.
She said it looked just like the view from the space ship in "Star Wars"
when they went into hyperdrive: (link no longer available) .
Those of us who live in colder climates may not get quite so excited about the
advent of winter, but snow crystals can be a source of great fascination. Check
out this website for some neat pictures: http://www.its.caltech.edu/~atomic/snowcrystals.
Here's what I cover in the body text
of this DOE FAQ Alert:
1. FAQ: Reasons for not finding significant factors
2. FAQ: Confidence vs prediction intervals
3. "Stat-Teaser" alert: See the latest issue of our newsletter featuring the "Popcorn Shootout"
4. Info alert: Technical articles published on applying Six Sigma DOE tools and how to compute power
5. Events alert: Aerospace Sciences Meeting and Exhibit
6. Workshop alert: "Experiment Design Made Easy" coming to Charlotte, North Carolina
7. Reader feedback: How to cook a turkey 18 different ways
PS. Statistics quotes for the month: Subject - Experience (the best teacher!)
1 - FAQ: Reasons for not finding significant factors
"I carried out a 2-level, 4-factor full factorial design for wave soldering optimization. But the model is not significant. I would like to know what are the main reasons for an insignificant model."
Pat Whitcomb, consultant and principal
of Stat-Ease, replies:
"Some reasons for not finding significant effects in a DOE are:
- Not choosing the correct factors for the DOE. (Subject matter knowledge and screening can help.)
- Not varying the factors over a wide enough range. (Screening or range-finding trials can help.)
- Factors that are not part of the study are not held constant, or blocked. (Careful planning and keeping all the people involved with the DOE informed and on the same page are critical.)
- Poor measurement capability. (Improve the test method, use a different test method or replicate the measurements.)
- Sampling errors. (Samples must be homogenous, representative and from a steady state process.)
- Improper statistical analysis. (Data may not be entered correctly, may contain outliers, or exhibit a need for a
I'd like to add a few comments:
1. Too often, experimenters do only 8 run, or even just 4 run, DOE's when it may take 16 or 32 runs to reveal effects. This is an issue of power (refer to FAQ 4 for a technical article on this topic). You get what you pay for in runs invested in your design of experiments.
2. Don't give up if you see nothing significant on your normal plot of effects. Pick the largest ones, perform the ANOVA and check the outlier-T diagnostic plot. I've seen cases where true outliers (ones that can be attributed to a special cause) make it hard to see what turn out to be highly-significant effects. (Mark)
(Learn more about analyzing results
from two-level factorial designs by attending the 3-day computer-intensive workshop
"Experiment Design Made Easy." For a description, see http://www.statease.com/clasedme.html.
Link from this page to the course outline and schedule.)
2 - FAQ: Confidence vs prediction intervals
"In your paper "Designing experiments that combine mixture components with process factors," published in PCI, Nov. 2000, [see http://www.statease.com/pubs/chem-3.pdf] the 95% confidence interval prediction of thickness seems too narrow based on the standard deviation for the model that's derived from the ANOVA. (I used a 2-sigma value for the 95% CI). I assume you might have used some other techniques."
Yes, simply go to the Point Prediction node in Design-Expert® software and use the Factors Tool for setting the input levels. Then see the confidence interval displayed by the software. You calculated a rough estimate of the "prediction interval" (PI) which tells you what to expect for individual outcomes, whereas the "confidence interval" (CI) relates to the mean outcome. The CI is always narrower than the PI based on the Central Limit Theorem (CLT). (For a nice writeup on the CLT, see: http://www.statisticalengineering.com/central_limit_theorem.htm.)
Here's what you find in the Design-Expert User Guide on this topic (the outputs of Point Prediction): "The SE Mean is the standard deviation associated with the prediction of an average value at these settings. The 95% CI is the confidence interval that is calculated to contain the true mean 95% of the time. The SE Pred is the standard deviation associated with the prediction of an individual observation. Finally, the 95% PI is the prediction interval calculated to contain the true value of an individual observation 95% of the time. All of these values can be used to manage expectations of the process. Note that the 95% confidence interval on a mean will be a narrower spread than the 95% prediction interval for a single observation." (See page 21 of Section 3, available on-line at: http://www.statease.com/x6ug/DX03-Factorial-Levels-Two.pdf.)
(Learn more about intervals like this, plus many other very useful tools, in our new 2-day "Statistics for Technical Professionals" (STP) workshop. It shows engineers, scientists and quality professionals how to gear up their stats knowledge to achieve Six Sigma or other quality objectives. For all the details, see http://www.statease.com/clas_stp.html. Call Stat-Ease at 1.612.378.9449 to arrange for a private STP class at your site.)
3 - "Stat-Teaser" Alert: See the latest issue of our newsletter featuring the "Popcorn Shootout"
If you did not get a printed copy, see the latest Stat-Teaser newsletter by clicking http://www.statease.com/news/news0112.pdf. The feature article, "Popcorn Shootout," details a simple comparative taste test that turned out to be more complicated then I expected. I think you will enjoy reading about my travails. The newsletter also offers a very informative FAQ response by Stat-Ease consultant Shari Kraber. It explains why you sometimes see Christmas trees on effects plots from Design-Expert (or Design-Ease). Check it out if you've not done so already.
4 - Info Alert: Technical articles published on applying Six Sigma DOE tools and how to compute power
"Paint and Coatings Industry" magazine features as their cover story an article by Pat Whitcomb and me entitled "Achieving Six Sigma Objectives for Variability Reduction in Coating Formulation and Processing." It details how design of experiments (DOE) tools can make systems more robust to variations in component levels and processing factors. View this article at http://www.pcimag.com/ .
Pat and our statistical advisor, Professor Gary Oehlert, published a research article, entitled "Sizing fixed effects for computing power in experimental designs," in the technical journal "Quality and Reliability Engineering International," Volume 17, Issue 4, 2001, pages 291-306. The article provides details on an extremely important issue that affects all experimenters: How many runs they need to include in their design to see the effects of practical importance. To see the abstract go to: http://www3.interscience.wiley.com/cgi-bin/abstract/85007160/START. For the full text of their submission, see: http://www.statease.com/pubs/power.pdf.
The same journal and issue cited
above includes a review of "DOE Simplified: Practical Tools for Effective
Experimentation," by Pat and I. Productivity, Inc. (Portland, Oregon) published
this 236-page softcover book last year. It comes with a CD-ROM that includes
a copy of Design-Ease software and problem datasets. See http://www.statease.com/doe_simp.html
for details on this book, which we wrote for technical professionals who want
to get a start on applying DOE. You can purchase the book by clicking http://www.statease.com/prodbook.html
for our e-commerce site.
5 - Events alert: Aerospace Sciences Meeting and Exhibit
DOES Institute will be representing Stat-Ease at the American Institute of Aeronautics and Astronautics (http://www.aiaa.org) Aerospace Sciences Meeting and Exhibit on January 14-17 in Reno. Stop by the DOES booth (#500B) to see our Design-Expert software.
6 - Workshop alert: "Experiment Design Made Easy" coming to Charlotte, North Carolina
The computer-intensive "Experiment
Design Made Easy" (EDME) workshop will be presented for the first time
in Charlotte, North Carolina on January 8-10, 2002. For a description and links
to the course outline and online registration via our e-commerce site, see http://www.statease.com/clasedme.html.
You can also call Stat-Ease at 1.612.378.9449 to register for this or any other
workshop. If spots remain available, bring along several colleagues and take
advantage of quantity discounts in tuition.
Our next EDME classes will be presented on February 5-7, in San Jose, California, and April 9-11 in our home city of Minneapolis, Minnesota. We hope to see you and your colleagues at one of these classes.
for schedule and site information for all Stat-Ease workshops open to the public,
- "DOE Simplified"
- "Experiment Design Made Easy"
- "Response Surface Methods for Process Optimization"
- "Mixture Design for Optimal Formulations"
- "Robust Design: DOE Tools for Reducing Variability" (Six Sigma)
- "Statistics for Technical Professionals"
Consider bringing in an expert from Stat-Ease to teach a private class at your site. We offer some workshops, such as "Real-Life DOE: Tricks of the Trade" (http://www.statease.com/rldoe.html) only on a private basis. We can provide out-takes from any of our workshops, or combine bits and pieces to meet your needs and time budget. Call us to get a quote.
7 - Reader feedback: How to cook a turkey 18 different ways
From: Tom Gray (a long-time user of Stat-Ease software) "I saw this [article in USA Weekend] and immediately thought of Mark! Have a good holiday."
Tom's referring to an article called "The perfect holiday turkey": http://www.usaweekend.com/01_issues/011111/011111cooksmart1.html. [Link has expired.] Give the author an "A" for effort, but one wonders how much more she could've learned by applying a proper design of experiments approach. Thanks, Tom, for the heads-up on this, but I wonder why seeing something about a turkey made you think of me.
Here's a macabre sidebar on turkeys.
It's become a tradition for the U.S. President to pardon the turkey chosen for
the White House Thanksgiving dinner. They are spared the cook's cleaver for
what's supposed to be a life of leisure at a local petting zoo near Washington,
D.C. However, they're so pumped full of growth hormones that they rarelyonly goes back to 2005) live
until the next Thanksgiving. (See http://sand.loper.org/~george/trends/2000/Nov/84.html - archives only go back to 2005).
Some turkeys last only a few weeks beyond the date they're slated to be served
for dinner. Scary!
I hope you learned something from this issue. Address your questions and comments to me at:
Mark J. Anderson, PE, CQE
Principal, Stat-Ease, Inc. (http://www.statease.com)
Minneapolis, Minnesota USA
PS. Statistics quotes for the month: Experience (the best teacher!)
judgment comes from experience and a lot of that comes from bad judgment."
- Will Rogers
is the marvelous thing that enables you to recognize a mistake when you do it
- F. P. Jones
never going to fail unless you try."
- Scott Adams (from one of his "Dilbert" newsletter subscribers)
brings experience and experience brings wisdom."
- My fortune cookie this past weekend.
Trademarks: Design-Ease, Design-Expert and Stat-Ease are registered trademarks of Stat-Ease, Inc.
Acknowledgements to contributors:
- Students of Stat-Ease training and users of Stat-Ease software
- Fellow Stat-Ease consultants Pat Whitcomb and Shari Kraber (see http://www.statease.com/consult.html for resumes)
- Statistical advisor to Stat-Ease: Dr. Gary Oehlert (http://www.statease.com/garyoehl.html)
- Stat-Ease programmers, especially Tryg Helseth (http://www.statease.com/pgmstaff.html)
- Heidi Hansel, Stat-Ease marketing director, and all the remaining staff
Interested in previous FAQ DOE Alert e-mail newsletters? To view a past issue, choose it below.
#1 - Mar 01, #2 - Apr 01, #3 - May 01, #4 - Jun 01, #5 - Jul 01 , #6 - Aug 01, #7 - Sep 01, #8 - Oct 01, #9 - Nov 01, #10 - Dec 01 (See above)