Too much information, too few insights-that is a typical problem facing manufacturers who manage highly automated and well-instrumented processes. Over the past decade, a number of data acquisition, storage and related technologies have become less expensive. As a result, their implementation to support and monitor complex discrete, batch or continuous process manufacturing is now common in virtually all industries.
Too much information, too few insights-that is a typical problem facing manufacturers who manage highly automated and well-instrumented processes. Over the past decade, a number of data acquisition, storage and related technologies have become less expensive. As a result, their implementation to support and monitor complex discrete, batch or continuous process manufacturing is now common in virtually all industries. These technologies can provide a wealth of information describing a process. However, extracting useful insights from that information or leveraging that information to implement effective process monitoring and control systems often requires the application of lesser known multivariate data analysis and data mining techniques.
Multivariate Control Charting
Standard quality, or Shewhart, control charting methods for variables, such as X-bar and S charts, cumulative sum (CUSUM) charts and exponentially moving average (EWMA) charts, can be generalized to the multivariate or multiple-variable case. For example, in a univariate-single-variable orcharacteristic-X-bar chart, sample means and standard deviations are placed on simple line graphs, with control limits fine-tuned to provide the necessary sensitivity and acceptable false-alarm rate, where the chart indicates an out-of-control condition, when the process is really in-control.
The main motivation for multivariate control charting is that it is rarely practical to construct and monitor a large number of single-variable control charts. In addition, the false-alarm rate will increase dramatically as one monitors more variables, one-by-one. As a result, in a scenario with 20 or 30 X-bar charts, one can expect an erroneous out-of-control signal and condition virtually every time, and be perpetually alarmed. In practice, all alarms will soon be ignored by operators or process control engineers, rendering the control system less effective because of the large number of false alarms.
Multiple Data Streams
A special case of multivariate control charting occurs when multiple data streams emanate from identical independent processes. For example, a manufacturer may run identical production lines with identical machines. In that case, the data can be considered multiple-independent data streams; special quality control charts can then be constructed to monitor the extremes-highs and lows-across the multiple streams. When a particular stream is responsible for a large number of consecutively observed extreme values, it is likely that a quality problem has occurred or is developing with the respective process associated with that data stream.Simple and efficient to implement, multiple stream charts are created by overlaying the extreme plot points, across data streams, from standard univariate X, X-Bar, R (range), MR (moving range) or S (standard deviation) charts. The information displayed on multiple stream charts-runs of consecutive extreme values, means, standard deviations or ranges observed from a particular process-can easily be translated into actionable information. For example, the information can be used to determine which specific process and data stream is defective and responsible for an unexpectedly large number of extreme values.
Hotelling T2 Chart
The multivariate equivalent of X and X-bar and S charts is the Hotelling T2 chart and generalized variance chart. Instead of controlling single X values or means, and standard deviations, the Hotelling T2 chart allows for the control of a vector of means for multiple characteristics, and the variance/ covariance matrix of the variables to control process variability.These simple charts can be used to effectively monitor tens of variables simultaneously in a single chart. When an out-of-control condition occurs, a follow-up Pareto-like chart usually can be created to indicate which variables are most likely responsible for the alarm; these variables can be further examined in standard X or X-bar and S or R, MR charts.
Fundamentally, the Hotelling T2 is a distance chart. It tracks the distance of mean vectors, or samples, from an ideal point in this multivariate space, a multivariate centerline or centroid.
In addition to its simplicity, the Hotelling T2 chart can detect small movements or drift in multivariate space that could not be picked up as early using simple univariate control charting. In that sense, Hotelling T2 charts can provide a more sensitive and powerful control method for large numbers of variables, detecting even small shifts or drift that simultaneously affect all or most of these variables.
MEWMA Charts, MCUSUM Charts
When the detection of small shifts or drift in multivariate space is critical, for example, a precursor to more serious quality problems, the Multivariate Exponentially Weighted Moving Average (MEWMA) and Multivariate Cumulative Sum (MCUSUM) charts can be useful. Like their univariate counterparts, EWMA and CUSUM, these charts track smoothed multivariate means or multivariate cumulative sums of deviations, respectively.Like the Hotelling T2 chart, the resulting charts are simple to monitor and provide effective, simultaneous control over tens of variables.
Drawbacks
While often effective, multivariate control charts have a few key weaknesses.• Critical quality indicators. With the exception of the multiple stream chart, these charts are specifically designed to track a multivariate process in multivariate space; if only one variable-out of 30, for example-begins to go out of control, this may not be detected until its contribution to the overall multivariate test statistic becomes so large that an overall alarm is triggered, such as on the Hotelling T2 chart.
So, in practice, it is important to think about the variables used in control charting, and to consider how likely it is that a few variables will step out of control without affecting the rest and, secondly, how critical it is to detect an abnormal condition for a particular single variable. For example, if a single quality indicator is absolutely critical for final overall product quality, it is best to monitor that variable by itself, using traditional univariate quality control charting methods.
• Multivariate normality. Certain statistical assumptions go into the computations and, in particular, the determination of control limits for multivariate control charts. In short, it is assumed that the distribution of variables is multivariate-normal. The real problem is that it is not clear how multivariate control charts will be affected when this assumption is violated. However, using modern quality control software, it is simple enough to test how well these charts perform, and fine tune those charts and control limits based on actual experience.
Model-Based Control Charting
Multivariate control charting techniques have been known, and in some cases practiced, for some time. However, newer approaches to multivariate process monitoring and quality control have emerged more recently. These methods allow users to build a statistical mode showing how variables are related to each other during known in-control operation of a multivariate process. This model can then be applied to data collected from the ongoing process to identify two problems:• Is the process, as defined by the model, drifting out of control or away from the process as it was observed when the statistical model was built?
• Has the universe changed? Have the relationships between variables describing the process fundamentally changed, so that the model is no longer valid or applicable?
The first issue is similar to the types of process problems that can be identified using multivariate control charting, such as Hotelling T2 charts. The second type of process problem, though, is not as easily detected using those charts.
Linear Model
One simple statistical model for data is the linear model: Define some imaginary underlying variables or factors that cannot be directly observed, as linear combinations of variables that can be measured and observed (for example, f1 = .4*x1 + .5*x2....; f2 = .1*x1 -.2*x2 ...; here both f1 and f2 would not actually be observed, but are unobserved factors, which are defined as linear combinations of the x variables).Usually, fewer such factors exist than variables from which the linear equations are estimated. Another way to think of these linear equations is that they represent the observed data, or x variables, in lower-dimensional space, compared to the original space defined by all variables.
Principle Components Analysis (PCA) is a statistical method that will compute such linear combinations from the data-the observed x variables-subject to the constraint that the linear combinations are uncorrelated, and that the linear combinations extract the maximum amount of information, or variability, from the x variable.
Partial Least Squares (PLS) methods are similar to PCA, except that the unobservable factors are extracted with an additional constraint-they correlate as much as possible with one or more outcome or y variables.
• Quality control charting using PCA and PLS. In order to detect shifts or drift of the multivariate process model, standard multivariate control charting methods can be applied to the fewer factor values, or unobserved variables. The main difference in interpretation is that the multivariate charts will track the unobserved factor values; this computation is based on a statistical model-linear, in this case-that was built for the data collected when the process was in control, or running very well.
• Detecting when the universe changes. In addition, one would track how well the model can account for or predict the actual observed x-values. Remember, the models are built to account for as much variability in the x variables as possible. So, if the model estimated from the good process no longer applies, this will show up in the residuals-observed minus predicted values-for the x variables. Typically, in this type of model-based quality control charting, special charts are created and monitored to identify this condition, when the original model no longer applies.
Consider a typical batch process, which is common in chemical manufacturing. In a particular step, multiple variables are monitored, as a batch of product slowly matures in a specific production step. A PLS model was built for good batches, to describe how the multiple variables change in relation to maturation time. In this case, time is the y variable.
Two things can go wrong: 1) The maturation process, as captured by the model, proceeds too rapidly or too slowly, triggering an alarm in the Hotelling T2 chart of factor values. 2) The relationship between variables may change over time-for example, as contaminants interfere with normal fermentation-rendering the model no longer applicable. So, while two variables x1 and x2 are supposed to, according to the model, increase simultaneously over time, only one increases, while the other goes in the opposite-to the model prediction-direction. In other words, the universe has changed. The relationships between observed variables are inconsistent with the relationships between variables observed when the process was in control.
Nonlinear Models
So far, only linear PCA and PLS have been discussed: statistical models that identify linear combinations of observed variables to define unobserved factors, which are then used for quality control charting, and for tracking prediction residuals in the observed x variables.In practice, the relationship between variables describing complex processes in chemical manufacturing, operation of smelters, kilns or power generation equipment can rarely be summarized usefully in linear models. This is where nonlinear PCA and PLS become useful for multivariate quality control. Linear PCA and PLS identify linear correlations among the quality variables to reduce the dimensionality in the x variables, and to build linear models. However, nonlinear PCA and PLS are capable of modeling linear and nonlinear correlations, and of building nonlinear models for the x variables, or x and y variable. In fact, nonlinear PCA and PLS methods will rarely impose any restrictions on the nature of the nonlinearity in the data.
Nonlinear PCA and PLS is computationally more demanding to implement, compared to linear versions. Yet, with the computational power available today at a relatively low cost, this issue has become less of an obstacle for successful quality control applications. In fact, increasingly sophisticated forms of statistical and machine learning modeling tools are being applied to model-based quality control.
A variety of powerful, machine learning-based, universal approximators are available for any type of relationship between variables of the process, such as advanced versions of recursive partitioning or tree algorithms-such as stochastic gradient boosting of simple trees, average prediction methods for forests of trees-and neural network methods.
So, although computationally more complex, given modern software and computer hardware, the general approach to model-based process monitoring and quality control is the same as that described for the linear model, in practice.
Obviously, when implementing model-based process monitoring and control charting methods, it is critical to train operators how to interpret the charts and identify root causes for alarms. However, the typical workflow, to go from a universe-changed-type alarm to identify the actual variables that are most likely responsible, can usually be defined with sufficient detail, so that it can be automated. After sufficient experience with a process and the performance of the model-based approach has been collected, such systems can be fully automated to provide actionable feedback to operators when out-of-control conditions are encountered.
Monitoring Thousands of Variables Simultaneously
Throughout this article, the suitability of methods for monitoring tens of variables has been discussed. That choice of wording was deliberate because the techniques described here will lose their effectiveness and applicability when the number of variables that need to be monitored simultaneously exceeds several hundred, or reaches into the thousands, although linear PCA and PLS methods can sometimes successfully cope with very high-dimensional data. The reason: It becomes computationally difficult to compute some of these charts and build reliable, valid models that summarize the quality of the process.The most reasonable way to approach the problem of very high-dimensional data-the curse of dimensionality-is to apply some data-mining based pre-processing, to identify the important features or variables of the process that drive the key performance or quality indicators, and ignore the rest.
The term data mining describes the application of exploratory methods, usually some machine learning and pattern recognition algorithms, to identify the salient, or predictive features of a process and their interactions, which can be used to predict some quality criteria.
After such important variables have been identified and validated in a large number of candidate variables that one could monitor, the operator can focus on this subset of variables for subsequent multivariate control charting.
Within this approach, the first step is to identify observed variables that are related to final product quality or other key performance indicators. This step, also called feature extraction, often can apply computationally intensive search algorithms that will consider not only one variable at a time but interactions between variables. Typical algorithms used for this task involve genetic and recursive partitioning algorithms.
After a suitable subset of key process variables has been identified, standard or multivariate process monitoring and control charting methods can be applied.
A high-level overview of emerging methods and technologies for process monitoring and control in situations where a large number of process parameters need to be monitored simultaneously, to ensure process efficiency and final product quality has been provided. Because the cost of computing and data storage continues to fall at an amazing rate, these methods have become more practical and useful to control complex, automated manufacturing or production processes.
Many of the techniques described are available in commercial, off-the-shelf software. However, some are available only in specialized applications. Based on the trends in process control and monitoring practice observed over the past decade, these methods will certainly find increasing use, particularly in competitive industries and those where the cost of losses that can be predicted by uncovering trends hidden in high-dimensional data is high. Q
Paul Lewicki, Ph.D., Thomas Hill, Ph.D., and Cazhaow Qazaz, Ph.D. are members of the R&D Department for StatSoft Inc. (Tulsa, OK). For more information, e-mail [email protected] or visit www.statsoft.com.
Tech tips
• Extracting useful insights to implement effective process monitoring and control systems often requires the application of lesser known multivariate data analysis and data mining techniques.• The main motivation for multivariate control charting is that it is rarely practical to construct and monitor a large number of single-variable control charts.
