Fractional Factorial Approaches to Emmaqua Experiments

Henry K. Hardcastle III
Atlas Weathering Services Group


One may consider the recent history of the weathering discipline (the past 100 years or so) as an evolution of weathering experiments. This evolution trends from simpler, single variable experiments towards more stochastic and broader experimental approaches. All the different major types of weathering experiments represent important tools engineers can use for answering different types of questions regarding product development. Reviewing some of the levels of sophistication in this evolution provides an understanding of experimental tools available. This presentation reviews tools in the context of weathering experimentation. These tools also apply well to most aspects of experimental science and one observes their use in agriculture, health sciences, industrial engineering, service industries, etc. Some of the major levels of experimental sophistication can be organized as follows:

Single Variable - Interested in the Presence or Absence

A single variable or "on-off" trial probably represents the earliest levels of experiment. This type represents "Experiment" in it's simplest application, forms the foundation for all other levels of experimentation, enjoys wide use in the weathering industry today, and represents a good starting point for developing a lexicon describing weathering experimentation evolution. In this level, the experimenter wants to understand the effect of a single variable on a system. The system is tested with and without the variable applied. Other conditions are kept constant or "blocked". Figure 1 shows a graphic representation of this simple design.

Application of this experimental design is simple. The experimenter prepares two sets of samples for trial: sample set "A" without the variable applied, and sample set "B" with the variable applied (often referred to as "The Input Variable" or the "Independent Variable") and then exposes the two sets side by side to the weather. After a period of exposure, the experimenter measures the two sets and analyzes the measured values (often referred to as the "Output Variables" or "Dependent Variable"). The analysis looks for differences between the two sets.

Spin-off of the Single Variable Experiment: Which is better?

Formulators often use an important spin-off of the single variable experiment in weathering experiments, especially for alternative vendor evaluations or "Drop In" formulation components. In this design, one includes multiple pairs instead of a single pair of experimental samples to determine "which is better." Figure 2 shows a graphic representation of this simple design.

Application of this design is similar to the single variable. The experimenter prepares a separate sample set for each alternative candidate (usually including a standard set to compare performance against as a control). The experimenter measures the output variable(s) of all sets after exposure and analyzes for significance and importance.

Interested in "How Much" (Ramp).

The previous two approaches were interested in "does it" or "does it not"- presence or absence of a variable. The next level of sophistication involves a different research question. Process engineers often use a "ramp" experimental design to determine "how much" of an input variable effects performance. This design involves amounts of an input variable and represents a higher level of context than simple presence or absence. Currently, this design is the most widely used tool for optimizing formulation component levels today. Figure 3 shows a graphical representation of a single variable design with multiple levels of the variable.

The on-off trial and single variable ramp experimental designs with variations and combinations represent the overwhelming majority of weathering experiments performed today. Their application is easy and analysis is clearly communicated to technical and non-technical audiences. The application of these designs to weathering characterization is so widespread that dependence on these designs may represent a limiting paradigm constraining growth in understanding of weathering phenomena today. Exclusive use of these narrow focus designs represents important limitations for product designers and process engineers struggling with weathering research as well as manufacturing capability and control issues. Experimentation at these levels of context alone does not provide an efficient approach for comprehensive improvement efforts faced with the plethora of variables effecting product weathering performance. Supplementing these approaches with experimentation at higher levels of context represents a more stochastic approach. Performing multi-variable screening experiments prior to application of these simple, fundamental designs offers efficiencies for understanding weathering through experimentation.

Two Variables:

A jump in the sophistication of weathering experiments occurs when designs utilize more than one input variable. Rarely, if ever, does a product's manufacturing or in-service environment involve only a single variable. Two variable experiments involve different research questions than those of simpler designs including investigation of interactions between input variables. Often these experiments are represented as "Square" designs.

One of the most popular applications of these "Square" designs involves identifying a Low and High setting for each independent variable. Four trials are then conducted according to the following array:

Trial 1 Both Variables At Low Setting
Trial 2 Variable "A" at Low Setting Variable "B" at High Setting
Trial 3 Variable "A" at High Setting Variable "B" at Low Setting
Trial 4 Both Variables at High Setting


Multiple "identical" samples or replicates are often included in each trial to characterize the background variance with in each trial. Output from each trial is often graphed on the same coordinate system for easy comparison with two low settings and two high settings compared for each variable. Figure 4 shows a graphical representation of this design. These graphs are read differently than traditional Cartesian coordinate systems. Often, intermediate input variable settings between the high and low settings are added to this design as additional trials and reveal intermediate topography. Sometimes this analysis is referred to as a "response surface."

These designs are also referred to as "Full Factorial" designs if each factor or input variable is tested at each setting. These designs are also said to be orthogonal or balanced; two trials with "A" set low, two trials with "A" set high, two trials with "B" set low, two trials with "B" set high. The othogonality is sometimes best understood by visual examination of the geometric layout of the "Square" design. The orthogonal characteristics of these designs result in experimental efficiency and power, and advance these designs to a higher level of sophistication than the traditional weathering experiments widespread throughout industry today.

Three Variables

A simple enhancement to the two variable "Square" concept is to add a third variable resulting in a "Cube" experimental design. This simple advancement drives experimental sophistication to a new level of context. These designs are more appropriate for weathering experiments since very rarely are only two weathering variables acting simultaneously on a product in end-use. In this three-dimensional experimental volume resides the classic weathering interaction of temperature, irradiance, and moisture variables. A two-dimensional experimental design does not have the inherent level of context necessary to characterize temperature-irradiance-moisture interaction phenomenon. It is befuddling to this author why the vast majority of weathering research is conducted at less sophisticated levels of experimentation to gain understanding of systems that operate at higher levels of context.

Application of this "cube" design follows similar procedures as for square designs. Eight Trials (23) are performed with individual variables set to high and low settings independently of other variable settings. The design is full factorial (each variable at each setting of high and low) and othogonally balanced. Intermediate input variable settings are often added to reveal intermediate topography. Figure 5 shows a graphical representation of a three variable design.

At the end of the experiment, the experimenter has obtained data from:

  • 4 trials with "A: set low,
  • 4 trials with "A: set high,
  • 4 trials with "B" set low,
  • 4 trials with "B" set high,
  • 4 trials with "C" set low and
  • 4 trials with "C" set high

by performing a total of only eight trials! This experimental efficiency constitutes the advantages of full factorial orthogonal arrays over less sophisticated experimental designs typically used in weathering studies today.

Figure 6 shows summary data for a three variable "Cube" weathering experiment. The purpose of this experiment was to understand the effect of three input variables on the weathering of a commercially available red automotive paint. The three input variables were exposure angle, presence of an hourly daytime spray, and exposure location. The response variable studied was Delta E color change and is noted in the graphic. The experiment indicated a comparatively large effect of daytime wetting cycles. The experiment also indicated a strong interaction between daytime wetting and exposure location.

Figure 6. Summary of a Three Variable Weathering Experiment.

Fractional Factorials

The orthogonality of previously discussed designs allows a jump to the next level of sophistication in weathering experimental designs. As we increase the number of variables in a full factorial design, the number of trials required dramatically increases: for a two variable orthogonal, four trials are required - two at high, two at low for each variable. In a three variable orthogonal; eight trials are required - four at high, four at low for each variable. In a four variable orthogonal; 16 trials - eight high, eight low. For any number of variables in a full factorial performed at two setting levels (high and low), the number of trials required equals the number of setting levels raised to the power of the number of variables investigated (for example; four variables at high and low settings = 24 = 32 trials = 16 at high setting, 16 at low setting). The experimenter may only need four trials at the high setting and four trials at the low setting for the required level of confidence needed to determine if a specific variable has a major effect on the system. Fractional Factorial designs allow the experimenter to delete some of the trials required by the full factorial design as long as the orthogonality of the design is maintained. The power of the orthogonality can be virtually exchanged for efficiency in the number of trials. The resulting design is termed a Fractional Factorial Screening Experiment.

Manufacturing Process Engineers widely use fractional factorial screening experiments to "screen out" the trivial many variables from the "important few" variables. A small confirmation experiment (typically two to four additional trials) follows a screening experiment to confirm the results. Once the manufacturing process engineer identifies the important variables, the engineer can focus process improvement efforts in an efficient manner on those variables that the process indicates are important. This analogy described for characterizing and improving manufacturing processes applies perfectly to improving weathering performance. Weathering processes are multi-variable complex processes that are highly material dependent. Weathering investigators can use fractional factorial screening experiments to "screen out" the trivial many variables from the "important few " variables. Once a screening experiment indicates the most important variables for a particular material, a small confirmation experiment always confirms the results. Once the weathering investigator identifies the important variables, he can effect the material formulation, processing variables, and in-service environments to improve weatherability. Equally important, the investigator can optimize variables identified as trivial by the process to reduce manufacturing costs! This advantage may be especially useful in raw material cost-cutting projects.

The materials weathering research efforts can then be focused in an efficient manner on the variables that the process indicates are important.

The fractional factorial experimental design answers questions such as:

  • Of the nine components in this vinyl formulation, which have the biggest effect on yellowing after five years Florida exposure, what is each component's order and magnitude of importance on yellowing, and which components can be optimized for cost without sacrificing weathering performance?
  • Of the ten major production line variables the line operator can control, on which should I establish control charting to improve quality of weatherability, and approximately what mean and tolerances should I begin with?
  • Of the major weathering agents this product will be exposed to (temperature, moisture, irradiance, pollution, abrasion, solvents, biologicals, cycling, etc.), which require research efforts to improve customer satisfaction for weathering performance?
  • For the major weathering failure modes I have identified in the FMEA, what are the risks associated with each?
  • For my material, which of these many weathering variables can be increased in order to accelerate weathering for test development?
  • For this vendor's candidate material, which environmental variables have the biggest effect on the system's weatherability?

Clearly, the types of research questions addressed with fractional factorial screening approaches represent a different level of context than single variable and ramp level experiments which make up the majority of weathering experiments performed today. Application of fractional factorial screening and confirmation designs is relatively simple to implement but relatively complicated to describe thoroughly. This presentation describes a straightforward ten-step procedure leaving the "why" to more esoteric and involved statistical publications. The reader should become familiar with theoretical underpinnings of these designs. The reader should also practice application of these designs - beginning with inexpensive, simple, non-critical investigations - to gain experience with these techniques. The reader should not become dissuaded by complex, restrictive, theoretical considerations that often become barriers to beginning these types of empirical applications in experimentation.

"The purpose of Mathematics is insight, not numbers." Sam Saunders, Ph.D., 02/08/99

Ten Step Procedure

Step 1 Write a simple, concise research question.
Step 2 List the variables to be investigated. Check that variables are truly independent.
Step 3 Select high and low settings (if a two-level experiment) or high, middle, low (if three-level experiment), etc. Check to make sure settings are not so far apart as to cause catastrophic failure if all variables are set high or low simultaneously. Check to make sure variable settings are far enough apart that they can be described as "significantly different."
Step 4 Select an appropriate fractional factorial orthogonal array for the number of variables under investigation, L16, L8, L64, etc. Use published arrays. Assign specific variables to specific columns in the design with regards to known interactions and alias concerns of the fractional factorial. Table 1 shows an L16 Fractional Factorial Array.
Step 5 Perform trials according to the fractional factorial array schedule. Include multiple replicates within each trial as necessary, given within trial variability
Step 6 Measure the desired output variables from each trial.
Step 7 Quality Control check all work. Recheck that all trials were performed in accordance with the array. Recheck all measurement values. Recheck all data entry. A simple error in variable settings, data entry, measurements, etc., may void the orthogonal basis resulting in erroneous decisions.
Step 8 Analyze the output using samples collected, effects graphs, and ANOVA techniques.
Step 9 Determine the significance and importance of each variable's effect on the output.
Step 10 Confirm the conclusions. Run a minimum of two confirmation trials: one trial with the variables indicated as significant and important set to high levels, one trial with significant and important variables set to low levels. Set insignificant variables to cost effective settings for both of these trials. Check that the results of these two confirmation trials confirm predictions of the screening experiment in both direction and magnitude of differences. Additional confirmation trials can be added for additional understanding of main effects, interactions, and aliases.

1 = Variable at Low Setting

2 =Variable at High Setting
Temperature   DayTime Spray Pretreat

NightTime Soak Pretreat




High UV

Variables 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Trial No.
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
2 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2
3 1 1 1 2 2 2 2 1 1 1 1 2 2 2 2
4 1 1 1 2 2 2 2 2 2 2 2 1 1 1 1
5 1 2 2 1 1 2 2 1 1 2 2 1 1 2 2
6 1 2 2 1 1 2 2 2 2 1 1 2 2 1 1
7 1 2 2 2 2 1 1 1 1 2 2 2 2 1 1
8 1 2 2 2 2 1 1 2 2 1 1 1 1 2 2
9 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2
10 2 1 2 1 2 1 2 2 1 2 1 2 1 2 1
11 2 1 2 2 1 2 1 1 2 1 2 2 1 2 1
12 2 1 2 2 1 2 1 2 1 2 1 1 2 1 2
13 2 2 1 1 2 2 1 1 2 2 1 1 2 2 1
14 2 2 1 1 2 2 1 2 1 1 2 2 1 1 2
15 2 2 1 2 1 1 2 1 2 2 1 2 1 1 2
16 2 2 1 2 1 1 2 2 1 1 2 1 2 2 1


















Table 1 An L16 Fractional Factorial Array

Fig. 7 Low and High Black Panel Temperature Distributions Achieved on 16 EMMAs


During the summer of 1999, Atlas Weathering Services Group performed a fractional factorial weathering experiment on commercially available materials.

Step 1:

The general purpose of this investigation was to better understand the effect of specific pretreatment and weathering variables on appearance properties using the EMMA natural accelerated weathering device. Understanding how selected variables effect EMMA weathering may indicate: 1) which variables are most important for weathering different material types, 2) which variables are important for focused AWSG R&D efforts, and ultimately 3) which variables to control with advancements in the EMMA weathering process. Additionally, it was hoped this experiment would provide a check of "conventional wisdom" regarding weathering phenomena of these materials as described in the literature, by AWSG customers, and in previous parametric studies performed at DSET Laboratories.


The materials for this experiment included current automotive paint systems, current coil coated building materials, colored in polyethylene sheet extrusion, clear polycarbonate sheet with a UV protected surface and injection molded clear polystyrene reference material used as indicators in Weather-O-Meters. All materials are commercially available at the time of this writing and were purchased from retail sources by AWSG. Although all the listed materials were subject to this experiment, this presentation includes results only for the Polystyrene reference material and the Brown Coil Coated construction material.

Clear polystyrene injection molded plaques were obtained from Test Fabrics, Inc. , West Pittson, Pennsylvania. All specimens were from the same production lot No. 4. The specimens are properly denoted as "Polystyrene plastic lightfastness standard - Plastic chips for use in monitoring controlled irradiance water cooled xenon arc apparatus - specified in SAE test methods J1960 and J1885, 6/89".

Brown coil coated aluminum was obtained from a commercial retail source. All specimens were cut from within a single twelve foot length of coil. Individual specimens were randomized before assignment of codes and labels.

Step 2:

Nine independently controllable variables were selected for this experiment from two types: pretreatment variables and exposure variables. 29 or 512 trials would have been required to perform this analysis using full factorial approaches. Both the pretreatment and exposure variables were included as follows;

  1. The temperature of the EMMA exposure under irradiance as indicated by metal black panel temperature instruments mounted on the EMMA target next to the specimens (TEM). This variable was controlled by changing the amount of air flow available for cooling the specimens.
  2. The strength of the radiant flux striking the surface and total radiant exposure from the solar spectral distribution. This variable will be referred to as "irradiance" for the rest of this presentation (IRR). This variable was controlled by changing the number of mirrors focused on the target area.
  3. The application of a daytime spray cycle which wet the specimens surfaces during periods of irradiation (SPR). This variable was controlled by turning on or turning off the water source available for the daytime spray cycle.
  4. The application of warm liquid water during periods when specimens were not being irradiated. This variable will be referred to henceforth as nighttime soak (NTS). Nighttime soak was controlled by removing the specimens from the target boards and placing them in a 40° C water bath each night and placing the specimens back on the target boards the next morning for daytime exposure.
  5. Abrasion of the specimen's surface prior to exposure (hencefourth referred to as abrasion pretreatment). An AATCC Crockmeter device was modified to move the abrasive cloth back and fourth on the specimen's surface under highly repeatable conditions. The Polystyrene was subjected to five reciprocal strokes using an 8000 grit abrasive. The brown coil was subjected to seven reciprocal strokes using a 3600 grit abrasive. The abrasion pretreatment resulted in a light haze or scratching on the specimen's surface.
  6. Thermo-mechanical cyclic stressing of the specimens using a soak - freeze - thaw pretreatment (SFT). Specimens were soaked for 16 hours in a 49° C de-ionized, re-circulating water bath followed by immediate placement into a -10° C freezer for two hours followed by a six hour thaw/dry in ambient office conditions. The cycle was repeated for five times. The soak - freeze - thaw pretreatment resulted in a slightly hazy or milky translucent appearance and the formation of micro cracks within the bulk of the Polystyrene injection molded plaques.
  7. Chemical pretreatment of the specimens was accomplished by immersing the specimens in a 40° C bath of dilute hydrogen peroxide (CHM). Polystyrene chips were soaked for 24 hours. Brown coil specimens were soaked for six hours and developed very minor blisters during the soak. After immersion, the specimens were immediately flushed with de-ionized water for 15 minutes.
  8. A high UV irradiance pretreatment by exposing specimens to a proprietary process reported to improve weathering resistance (ARC).
  9. Oven pretreatment was accomplished by placing the specimens in an air circulating oven set to 70° C for 84 hours (OVN).

Pretreatments were performed in a specific sequence: abrasion preceded soak - freeze - thaw which preceded chemical pretreatment which preceded UV arc which preceded oven, all of which preceded EMMA exposure. Prior to initial measurements, all specimens were thoroughly washed using a 5% mild detergent solution, gentle hand-wipe and thorough rinsing with de-ionized water after being pretreated.

Step 3

High and low settings for each input variable were selected according to the schedule shown in Table 2. This experiment utilized only two levels.

Variable Low setting High Setting
Temperature (TEM) Nominal -7° C Nominal +7° C
Irradiance (IRR) 8 Mirrors (-10 %) 10 Mirrors (+ 10 %)
Daytime Spray (SPR) No Daytime Spray With Daytime Spray
Nighttime Soak (NTS) No Nighttime Soak With Nighttime Soak
Abrasion Pretreatment (ABR) Not Abraded Abraded
Soak-Freeze-Thaw Pretreat <(SFT) No Soak-Freeze-Thaw Pretreat Soak-Freeze-Thaw Pretreatment
Chemical Pretreatment (CHM) No Chemical Pretreatment Chemical Pretreatment
High UV Pretreatment (ARC) No High UV Pretreatment High UV Pretreatment
Oven Pretreatment (OVN) No Oven Pretreatment Oven Pretreatment

Table 2 High and Low Variable Settings

Step 4

An L16 fractional factorial array was selected for this experiment. Although the L16 can theoretically handle up to 15 independent variables, it is not appropriate to fully saturate the array with variables. The nine variables identified for this investigation fit into the L16 while leaving six columns blank. The blank columns were used in the analysis for estimating the background variance, to check for significance of results, and to check for some interactions.

Assigning each of the input variables to specific columns in the array requires some judgment and understanding of aliases in the fractional design. For instance, we believe there is a reasonable likelihood of interaction between temperature and irradiance in this experiment. We want to understand the independent effects of these two variables as well as the synergy between them. The temperature variable is assigned to the first column since the first column will reveal temperature effects alone. The irradiance variable is assigned to the second column since the second column will reveal irradiance effects alone. The third column is left blank since the third column's settings will reveal any effects due to interaction between the temperature and irradiance variables. If we did assign a variable to the third column in this design, we would create a confound. We would not be able to tell if the effects tested by the third column were due to a synergy between temperature and irradiance or were due to the variable we assigned to the third column. This confound is often referred to as an "alias." The remaining variables are assigned to columns with similar justification; daytime spray is assigned to column 4. Column 5 will reveal a significant interaction between daytime spray and temperature settings. Interaction between daytime spray and irradiance will be revealed by column 6 and column 7 will reveal a three-way interaction between temperature, irradiance and daytime spray. We consider it highly unlikely that a three-way interaction would show up from treatments in column 7 without showing significant differences from columns 3, 5 and 6 (the two-way interaction indicators). We chose to assign a variable and alias column 7 with the soak - freeze - thaw pretreatment variable. The remaining variables for this experiment were assigned using similar justification. The final array with variable assignments is shown in Table 1. This schedule details 16 trials with unique settings of nine different variables.

Step 5

The 16 trials prescribed by the experimental array were performed simultaneously on 16 different EMMA (Equatorial Mount with Mirrors for Acceleration) machines at DSET Laboratories from May 21, 1999 to July 29, 1999. The machines utilized were quality checked throughout the exposure. Proper variable settings were maintained for each trial. Great care was utilized to insure all variables outside the scope of the experimental design were blocked across all 16 trials. At several intervals throughout the exposure, black panel temperatures were measured in the target area. A graph representative of the temperature differences is shown in Fig. 7. Note that the target black panel temperature variable was controlled independently of the irradiance variable for the 16 trials.

Step 6

After the exposure period, specimens were removed from exposure, measured for appearance properties and compared to their initial values before exposure. Polystyrene chips were measured for transmittance yellowness index according to ASTM E 313-96. Brown coil material was measured for color ( CIE L*, D65, Spherical, Specular included) using ASTM D2244-93 and ASTM E 308-95. Brown coil was also measured for 60 degree gloss using ASTM D523-89. Polystyrene specimens were measured three times across the exposed surface and the mean reported for each specimen. Brown coil specimens were measured five times across the exposed surface and the mean reported for each specimen. Two specimen replicates were included in each trial. The values reported are calculated deltas between the initial measurements before exposure and final measurements after exposure. The delta values for the two specimens included in each trial are shown in Table 3. The trial number for the output values corresponds to the trial numbers used in the fractional factorial array schedule in Table 1.

Trial Number Delta Yellowness Index of Polystyrene Replicate "A" Delta Yellowness Index of Polystyrene Replicate "B" Delta L* of Brown Coil Replicate "A" Delta L* of Brown Coil Replicate "B" Delta 60 degree Gloss of Brown Coil Replicate "A" Delta 60 degree Gloss of Brown Coil Replicate "B"
T01 12.92 13.26 0.90 0.89 11 11
T02 23.19 22.79 2.44 2.29 14 11
T03 11.95 12.34 0.61 0.60 22 19
T04 24.80 26.00 1.60 1.72 18 15
T05 18.62 17.99 0.59 0.64 23 23
T06 25.54 26.38 3.20 3.19 16 17
T07 20.05 19.18 1.01 1.00 13 15
T08 22.45 20.56 2.03 2.19 18 18
T09 15.53 14.78 0.48 0.47 18 22
T10 24.58 24.63 2.43 2.23 19 20
T11 16.48 17.73 1.12 1.09 17 19
T12 16.43 17.48 1.31 1.28 17 17
T13 17.93 17.63 1.30 1.23 22 18
T14 26.6 26.57 1.99 2.02 20 22
T15 13.13 12.86 0.68 0.74 28 27
T16 29.04 29.71 2.63 2.33 23 20

Table 3 Delta Yellowness Index, Color and Gloss for Exposed Replicates

Step 7

A thorough check of exposure conditions was performed weekly during the exposure. The scheduled pretreatments for specific trials were confirmed. All work was recorded as performed in engineering logs and written checklists were utilized throughout the work effort. Measurements were confirmed based on comparison of replicate values.

Step 8

Analysis of the output data was performed at three levels: 1) Visual review of exposed specimen collection, 2) review of experimental grouping using graphical techniques, and 3) ANOVA.

  1. Visual Review

    A visual review was performed by laying out exposed specimens in groups. Groups were formed using the orthogonal array for each variable. For the first layout, all specimens that were exposed at higher temperatures were grouped together. All specimens that were exposed to lower temperatures were grouped together. The two groups were compared for overall appearance. Next, the eight sets of specimens that were exposed to high irradiance were grouped together and compared to a group composed of the eight sets of specimens exposed to low irradiance. This grouping procedure was continued for each variable in the experimental design. The groupings that revealed the most apparent differences between high and low settings were identified.

  2. Graphical Technique

    A similar analysis to the visual grouping was performed using the Delta Yellowness Index values. A mean was calculated for the eight sets of specimens exposed to high temperature. A mean was calculated for the eight sets of specimens exposed to the low temperature condition. These two mean values were plotted on a graph and connected with a line. Next, a mean was calculated for all specimens exposed to high irradiance. A mean was calculated for all specimens exposed to low irradiance and these two means were plotted next to the temperature variable means. This procedure was continued for all the variables included in this design. This graphing technique allowed the effects of each variable to be compared with the effects of all other variables with a single graph. Using this analysis, it was quite simple to determine which variables had the largest effect on yellowing of the polystyrene and the magnitude of the effect compared to that of the other variables. Figures 8, 9, and 10 show the graphs obtained.
Fig. 8 Main Effects Graph for Polystyrene Yellowness Index Values
Fig. 9 Main Effects Graph for Brown Coil Coated Material L* Values
Fig. 10 Main Effects Graph for Brown Coil Coated Material 60° Gloss Values

3. ANOVA Because the two techniques so far do not sufficiently account for within and between trial variance, an ANOVA analysis was performed using a software package from ASD, Inc. ANOVA Analysis of Fractional Factorial Screening experiments allows experimenters to complete the following table:

Varibales Tested That Are
Variables Tested That Are
Siginificant But Unimportant
Variables Tested That Are
Significant And Important
1. 1. 1.
2. 2. 2.
3. 3. 3.
4. 4. 4.

The ANOVA performs an analysis using the F test. In an F test, an F ratio is calculated comparing variance due to treatment variables to variance due to experimental noise. The F ratio is often described as having the between column variance in the numerator and the within column variance in the denominator.

For this analysis, we simply compare the effects caused by input variables to the background variation of the experiment. As this ratio approaches one, we say that the effects due to the input variables are not significantly different than the background variation of the experiment or that the effect of input variables is not significant. However, as the F ratio becomes larger and larger, the effects due to the input variables become more different than background experimental noise; a large F ratio indicates the effect of the input variables is significant.

The variation due to input variables is easily traced back through the orthogonal array. The ANOVA uses outputs from the columns to which the input variables were assigned. An estimate of the variation due to experimental noise (experimental error, background variation) comes from a combination of three sources for this analysis. First, if replicate samples are used in the experiment (n>1), the sample to sample variation in the results can be a good estimate of within treatment variation. Second, the columns that were left blank in the orthogonal design represent a rich source for estimating experimental error once interaction effects are ruled out. By incorporating these blank columns and using unsaturated designs during the design steps of the experiment, we are now rewarded with a very robust and statistically valid estimate of experimental error. Third, any of the input variables that are not significant can also be pooled into the estimate of experimental variance. By using all three sources to develop a pooled estimate of experimental error, a very robust appropriate experimental error term can be developed for the denominator of the calculated F ratio. It is with this experimental error term that the effects of treatments are compared to determine significance.

Once the F ratio is calculated from the experimental data, it may be compared to values found in the standard F distributions given the degrees of freedom for the numerator and denominator and the desired confidence level. If the data generated F ratio value exceeds the critical F ratio value (from the table of standard F distribution values) the effect of the input variable can be said to be significant (the output due to that variable exceeds what would normally be expected due to random experimental noise).

For this analysis, the ANOVA table, treatment variables, pooled sources of experimental variance, and calculated F ratios are shown in Table 4, Table 5, and Table 6. Using the F column and r column in the ANOVA table, we can begin to assign a rank order to the significant input variables and understand the magnitude of effects compared to each other for polystyrene as follows:

Variables Tested That Are
Variables Tested That Are
Significant But Unimportant
Variables Tested That Are
Siginificant And Important
1. Temperature 1. Daytime Spray 1.
2. Oven Pretreatment 2. Soak-Freeze-Thaw Pretreat 2.
3. 3. Chemical Pretreat 3.
4. 4. 4.

For polystyrene yellowness index

It is important to compare the ANOVA table with the goals from Step 1. ANOVA only indicates significance, not direction. For instance, the F ratio for ARC pretreatment shows the effect of high UV pretreatment on Delta Yellowness Index after 68 days EMMA exposure is significant. Only the graphs, however, indicate that no pretreatment causes greater degradation while pretreating with high UV retards degradation of yellowness.

The ANOVA table should also be inspected for effects of interactions between input variables. Recall that these interactions should appear in the columns that were left blank. The interaction between temperature and irradiance for this experiment should show up in column 3, the interaction between temperature and daytime spray should show up in column 5, and so forth. In this experiment, the F ratio values for these interactions are small indicating no significant interactions in this experiment. After inspecting the F ratios of these interaction columns for significant effects on the output, if no significance is found, we can contribute these blank columns to our estimate of experimental variation.

Source Pool DF S V F S' r
TEM 1 0.3692 0.5883 0.5883 -0.2583 -0.03
IRR 1 74.9608 74.9608 119.4663 74.3334 8.49
Y 1 0.1172 0.1772
SPR 1 10.5536 10.5536 16.8194 9.9261 1.13
Y 1 4.2464 4.2464
Y 1 0.1454 0.1454
SFT 1 8.2855 8.2855 13.2047 7.6580 0.87
NTS 1 567.1459 567.1459 903.8700 566.5185 64.70
Y 1 0.4795 0.04795
Y 1 0.5452 0.5452
ABR 1 107.3735 107.3735 171.1230 106.7461 12.19
Y 1 2.2562 2.2562
OVN 1 0.0408 0.0408 0.0651 -0.5866 -0.07
CHM 1 7.7648 7.7648 12.3748 7.1373 0.82
ARC 1 85.2916 85.2916 135.9307 84.6641 9.67
e1 1
e2 Y 16 6.0142 0.3759
(e) 22 13.8042 0.6275 19.4514 2.22
Total 31 875.5899 28.2488

Table 4 ANOVA Table for Polystyrene Delta Yellowness Index

Source Pool DF S V F S' r
TEM 1 0.0770 0.0770 0.9385 -0.0051 -0.03
IRR 1 0.881 0.8811 10.7352 0.7991 3.96
Y 1 0.0026 0.0026
SPR 1 0.5913 0.5913 7.2045 0.5093 2.52
Y 1 0.1815 0.1815
Y 1 0.0488 0.0488
SFT 1 0.0001 0.0001 0.0010 -0.0820 -0.41
NTS 1 14.4857 14.4857 176.4862 14.4036 71.32
Y 1 0.3424 0.3424
Y 1 0.3301 0.3301
ABR 1 0.0604 0.0604 0.7356 -0.0217 -0.11
Y 1 0.7970 0.7970
OVN 1 0.1969 0.1969 2.3987 0.1148 0.57
CHM 1 0.3342 0.3342 4.0712 0.2521 1.25
ARC 1 63.2813 63.2813 24.6678 60.7159 11.24
e1 1
e2 Y 16 41.500 2.5938
(e) 22 56.4375 2.5653 79.5256 14.73
Total 31 539.9688 17.4183

Table 6 ANOVA Table for Brown Coil 60° Gloss

Step 9

Once the significance of the individual treatments is understood, it is appropriate to decide which input variables are important. Significance involves a statistical exercise. Importance is a human judgment exercise and often depends on other sources of data beyond the scope the experiment. For instance, in this experiment, a 20% difference in irradiance resulted in a difference in mean Yellowness Index of about three units (21.5 at -10% to 24.5 at +10%). In some type of end-uses, this magnitude of difference may cause great user consequences, including formulation and/or process changes. In other end-uses, this magnitude of difference might not be important to the end-use at all. Prior to beginning this study, the end-user of these polystyrene chips was interviewed regarding criteria for importance. The customer identified differences exceeding approximately two Delta Yellowness Index units at this experimental level to be important. Similar justification was used for determining the importance for Brown Coil color and gloss. Based on this information, the significant and important information can be completed as follows:

Variables Tested That Are
Variables Tested That Are
Siginificant But Unimportant
Variables Tested That Are
Significant And Important
1. Temperature 1. Daytime Spray 1. Nighttime Soak
2. Oven Pretreatment 2. Soak-Freeze-Thaw Pretreat 2. Abrasion
  3. Chemical Pretreatment 3. High UV Pretreatment
    4. Irradiance

Table 7 Summary Results for Polystyrene Delta Yellowness Index

Variables Tested That Are
Variables Tested That Are
Significant But Unimportant
Variables Tested That Are
Significant And Important
1. Temperature 1. Irridance 1. Nighttime Soak
2. Oven Pretreatment   2. High UV Pretreatment
3. Soak-Freeze-Thaw Pretreat    
4. Chemical Pretreatment    
5. Daytime Spray    
6. Abrasion Pretreatment    

Table 8 Summary Results for Brown Coil Delta L*

Variables Tested That Are
Variables Tested That Are
Significant But Unimportant
Variables Tested That Are
Significant And Important
1. Daytime Spray 1. High UV Pretreatment 1. Soak-Freeze-Thaw Pretreat
2. Oven Pretreatment   2. Temperature
3. Abrasion   3. Irradiance
4. Chemical Pretreatment    
5. Soak-Freeze-Thaw Pretreat    

Table 9 Summary Results for Brown Coil Delta 60° Gloss

Step 10

Confirmation trials should always be conducted in conjunction with fractional factorial screening experiments. Confirmation trials should also be considered as a critical part of the screening experiment. This is especially important if high levels of input variable saturation are designed into the orthogonal array and where significant interactions are identified between several input variables. Only confirmation trials can decode alias characteristic of the fractional array. Two confirmation trials were conducted with input variables set as shown in Table 10.

Confirmation Trials:
#1-Least Degradation Predicted #2-Most Degradation Predicted
• No Nighttime Soak • With Nighttime Soak
• No Abrasion Pretreatment • With Abrasion Pretreatment
• No Soak Freeze Thaw Pretreatment • With Soak Freeze Thaw Pretreatment
• Low Irradiance • High Irradiance
• With High UV Pretreatment • No High UV Pretreatment

Table 10 Conformation Trials

The remaining variables investigated in the screening experiment that were identified as not important or not significant were set to optimal levels for cost for both trials as shown in Table 11.

Confirmation Trials:
Variables optimized for cost
• Low Temperature Exposure
• No Oven Pretreatment
• No Chemical Pretreatment
• No Daytime Spray

Table 11 Conformation Trials Optimized for Cost

These confirmation trials were then conducted for polystyrene. The data obtained was compared to the screening experiment results. The confirmation trials at 220 MJ/m^2 UV dose compared directly with the predictions made in the original screening experiment for polystyrene. From this confirmation trial it was concluded that no hidden interactions, confounds or aliases were operating in the original screening experiment. Follow-on efforts can now be focused on those variables the weathering process has indicated are significant and important. Confirmation trials for the brown coil will be analyzed after publication of this paper.


There has been an evolution in the sophistication of experimental designs for weathering tests. The vast majority of current weathering exposures utilize more fundamental designs effecting few variables. This type of experimental design requires far more trials and thus more cost, less information, and poorer quality than more sophisticated approaches using screening fractional factorial experiments. Preceding fundamental level, few variable weathering trials with fractional factorial screening and confirmation experiments represent an efficient, stochastic, powerful approach for improving knowledge regarding weathering's n-dimensional hyper-volume of environmental effects on materials degradation.