by Rich Burnham
There is a simple explanation why design of experiments (DOE), which was invented in the decade after WW1, sat nearly dormant until the arrival of PCs -- DOE is extremely complex. Without modern, off-the-shelf DOE software programs now selling for less than $1,000, lengthy calculations are the norm.
Nevertheless, when an experimenter does not use DOE and relies on one-factor-at-a-time (OFAT) testing, the results are usually untrustworthy. OFAT experimentation ignores interactions among factors, an extreme omission. Lacking the ability to detect interactions would be like testing red ink and blue ink -- and never learning that the two combine to produce purple. Testing one factor at a time also requires a number of trial runs, which is cost-prohibitive in today's business environment.
Conscious of the need for easier DOE, in the mid-1980s a Midwest company was formed to begin offering to statisticians and researchers a PC-based DOE software program. The company refined the algorithms that were being used in mainframe computers and eventually came up with Design-Expert® DOE software (Stat-Ease Inc., Minneapolis). It was the right time for this to happen, as room-sized central processing units were becoming desktop computers. The company's prime mission since the beginning has been to make DOE easier and more understandable to engineers and researchers yet powerful enough for career statisticians. Today the company operates in an office within the industrial section of Minneapolis. It is an appropriate location, because like industrial sections in all cities, this is where the results of software-driven DOE are most welcome.
In the summer of 1998, I was asked to review Stat-Ease's latest DOE software version of Design-Expert -- not as an experienced DOE user, but as a professional trade publication writer with only a fragmentary DOE background. Why ask a non-DOE practitioner like me to review a design-of-experiments software package? Because many engineers and technicians within the coatings industry, and all industries, believe that DOE is too complex to be worth the bother. By some estimates, fewer than 10% of engineers in the work force use DOE. It is rumored that the remaining 90% are intimidated by anything related to statistics.
Like most newcomers to designed experiments, I attended a 3 1/2-day workshop in Minneapolis, conducted by the creators of the software. In all, there were 20 of us in the sold-out class who wanted to learn how to discover breakthroughs, optimize formulations and save their company potentially hundreds of thousands of dollars each year. In retrospect, I would have been helpless without the workshop.
The workshop, Experiment Design Made Easy, is accurately named. It begins with a half-day, basic review of high-school-level statistics that is clear and understandable. Although I wondered if I was adequately prepared for the remainder of the class, by the end of day two, I was using the software to actually set up, run and interpret experiments -- with minimal help from the instructors. The workshop taught me DOE methodology, and how to find solutions to real-world process and product problems.
The final one-and-a-half days of the workshop were spent working scenarios using real data from intriguing in-class experiments.
I was in awe of the power of DOE. It's an amazing, systematic approach to experimentation that considers all factors simultaneously. It provides information about interactions among factors and the way a total system works, not just a few factors. It does this by fitting response data to mathematical equations. These equations become models that accurately predict the outcome of any given combination of settings. Using models like these, formulators are finding and optimizing key responses that produce, for example, coatings with precise adhesion properties1 or paint that does not blister in curing ovens.2
In one of the first steps of conducting a DOE using the software, I'm prompted to enter the factors I am testing -- along with the high and low operating parameters of each factor. Design-Expert DOE software randomizes the order in which to conduct the experiments, ensuring that "noise" in a process is spread randomly across all control factors. The software produces a simple and handy worksheet, or "recipe," setting the order in which the user tests and records the results of different factor combinations. After the results are entered, the program draws a half-normal plot that indicates which factors or factor interactions are significant. These are the elements from which to produce dramatic improvements.
The program runs easily on any version of Windows® from 3.1 on up. A 486 as a minimum is recommended, as is 8 Mbyte RAM. Well-crafted, right-click options access useful features. Nicely written online help screens contain many internal links and understandable definitions. Moreover, technical support is impressive. After the workshop, while trying to hone my techniques, I called the software manufacturer three times in one week to ask a few questions. The calls netted a trio of knowledgeable programmers who sounded comfortable on the telephone -- and answered my novice-type questions in a professional and helpful manner.
My day-to-day activities are not as an experimenter. Yet I believe that DOE, with the guidance of Design-Expert software, free consulting and technical support help from Stat-Ease, is nothing to be feared. The workshop and software taught me that there are many categories of DOE, the hardiest and most common being two-level factorials. (The two levels being the high- and low-operating parameters of each factor being tested.) Up to 15 factors can be tested, requiring anywhere from four to 128 runs, or experiments.
One particular DOE design called Plackett-Burman extends the factor range to 31. But as is normal in any trade off, the greater factor number is not as discriminating as are leaner designs. While trying to set up such a design, a courteous warning screen devoid of intimidation alerted me that the Plackett-Burman results were aliased, meaning that some factors were being masked over by other factors -- possibly leading to ambiguous results. (A more advanced workshop, described within Stat-Ease's website at www.statease.com thoroughly addresses situations like these.)
After building a design (the "recipe"), actual factor levels or coded levels (+ for high, - for low), can be displayed interchangeably by way of column header manipulation. Coded factors are frequently used to protect proprietary information and to allow all results to be evaluated on a relative scale.
Design-Expert graphics are a gem. The clean and crisp images showed me the story of my experiments, simplifying my understanding of what was occurring. This is how results are meant to be displayed -- straightforward and simple. For example, a cube plot is a clear way to show how three factors combine to form ideal and repeatable responses. Even nicer to view are 3D response surface maps. (Response surface maps are the graphical results of Response Surface Methodology.
Stat-Ease programmers have given users the ability to rotate entire response maps for easier viewing. Two-dimensional contour plots and 3-D diagrams show precisely how factors interact, and the effect that those interactions have on factor responses.
Visualization is easy when experimenting to find optimum and robust settings. (Robustness means that the product or process withstands external "noise" such as temperature or vibration.)
So what did my DOE foray as a nonexperimenter reveal? Are the rumored fears of DOE being too complex justified? My attendance at a DOE workshop was essential to grasp proper experimentation knowledge. And I learned that because DOE computations are so complex, nearly all commercial workshops claim allegiance to one or more software programs within their syllabus. Design-Expert garnered unreserved respect from my fellow workshop attendees. The software encourages good habits, explains itself and DOE in ways that educate in a noncondescending manner, yet avoids oversimplification. In interviewing practiced users, I heard story after story of dramatic money savings produced with this software.
My job now is to write my next case study. However, if I were to return to a paint lab, I would do so willingly with this software under my arm.
References
1 Mills, Wayne. DOE Software Saves John Deere
Money & Time, PCI, September 1995
2 Hazelwood, Steve. Using DOE to Prevent Solvent
Pop, PCI, August 1998.
Rich Burnham is the founder of Written Solutions, a company in Minnesota that writes case studies and articles for trade publications. Rich can be contacted at (651) 773-0612 or by e-mail at published@worldnet.att.net.