Operations Management / Supply Chain Management

Module 03.03 Key Concept: Forecast Accuracy

Forecasting is not complete without an assessment of forecast error.  The reason for tracking forecast errors is primarily to identify opportunities for improvement or assess the benefits and liabilities of different models. We introduced you the concept of error when we showed how to calculate exponential smoothing forecasts.

The text presents three methods for assessing forecast accuracy:

  • Period Forecast Error
  • Mean Absolute Deviation (MAD)
  • Mean Squared Error (MSE)
  • Mean Absolute Percent of Error (MAPE)
  • Tracking Signal (a measure of “Bias”)

Examples of each one are presented below

Period Forecast Error = (Actual – Forecast).  It can be expresses in units or as a percentage.

Given a forecast of sales in December of 120 units and the Actual Sales for December recorded as 98 units: the Period Forecast Error = 98 – 120 = 22 units.

Mean Absolute Deviation = Sum of Absolute Value of Period Forecast Error / number of Periods.  Thus, it is always a positive number that indicates the average value of forecast error during the time of evaluation.  It can be expressed in absolute terms or as a percentage.

Mean Square of Error = A measure of statistical variation in a forecast.  It is computed by squaring the forecast errors and then taking the average of the sum of the squared errors.

Mean Absolute Percent of Error = A measure of statistical variation in a forecast.  It is computed by dividing each absolute forecast error by the actual demand, multiplying that by 100 to get the absolute percentage
error, and computing the average.

 Tracking Signal = The ratio of the cumulative algebraic sum of the deviations between the forecasts and the actual values to the mean absolute deviation.  It is calculated by: (Sum of Forecast Errors / MAD).   It is used to signal when the validity of the forecasting model might be in doubt.

A common general convention: if the Tracking Signal is > 3 or < -3, the Forecast is likely to be biased and calls for a review.  Calculations and a graphical representation of Tracking Signal are shown in detail in the text.

When monitoring forecasts the following should be considered:

  • Is the forecast too high or too low?
    – Mean Error (bias)
  • What is the magnitude of the forecast error?
    –Mean Absolute Deviation (MAD)
    –Standard Deviation of forecast error = 1.25 x MAD
  • Is the Forecast Biased or do other systemic problems exist?  Measuring both bias and MAD is critical to understanding the quality of the forecast.

The ultimate goal of monitoring forecast accuracy is to highlight the need for improvement and track performance over time