All blocks and forced terms are put into the model.

The remaining terms with an “” next to them are considered.

Each term is checked, adding them one at a time to the model. The term that improves the criterion the most is added to the model.

Terms not yet in the model are checked until no further improvements can be found given the terms already in the model.

Forward works well when the best model is a small subset of the available terms (with an M next to them).

All blocks and forced terms are put into the model.

The remaining terms with an “” next to them are considered.

All remaining terms are added to the model simultaneously.

Terms are removed one at a time.

The best one-term-smaller model for the selected criterion is kept.

If no one-term-smaller model improves the criterion score the elimination process stops, otherwise use the model found in step 5 and return to step 4.

Backward works well when the best model is a large subset of the available terms (with an “” next to them) and there are extra degrees of freedom in the design.

Step-wise regression is a combination of forward and backward regressions, starting with two forward steps.

All blocks and forced terms are put into the model.

The remaining terms with an “” next to them are considered.

The term that improves the criterion the most is added to the model.

Terms not yet in the model are checked.

If a term is found that further improves the criterion it is added. If not the algorithm stops.

When two or more terms are in the model, backward steps are added.

Terms are removed one at a time.

The best one-term-smaller model that also improves the criterion from step 5 is kept. Return to step 4.

If no one-term-smaller model improves the criterion score return to step 4.

Note

The coefficient shown is the coefficient at the time the term was added or deleted from the model. This is likely to be different than the coefficient that is generated with the completed model. For designs with categorical factors, the coefficients are not shown.

The all hierarchical models procedure differs from the directional (forward and backward) selection methods. Rather than considering terms one at a time, all hierarchical models are compared simultaneously and the best one is selected. The procedure works as follows:

All blocks and forced terms are put into the model.

The remaining terms with an “” next to them are considered.

The best model of each size (with equal numbers of coefficients) is determined using R-squared and put into a candidate set.

The model from the candidate set with the best criterion score is selected.

One Backward + p-value step is done to clean up a lingering, insignificant, non-hierarchical term.

All Hierarchical Model Selection can take a long amount of time to complete for designs with 8 or more factors. The search can be manually stopped (Canceled) early and the best model found so far will be presented.

References

Brusco, Cradit, and Steinly. An exact algorithm for hierarchically well-formulated subsets in second-order polynomial regression.

*Technometrics*, 51(3):306–315, 2009.