Red noise estimation

This is a new set of procedures implemented in version 3 to handle the time series when red noise (or serial correlation) is present. Red noise is modeled by the first order autoregressive model (AR1). If serial correlation is present in the time series, it is necessary to either adjust the significance level of the shifts by calculating the effective degrees of freedom, or use a so-called "prewhitening" procedure prior to application of a regime shift detection method. In any case, it requires an estimate of the AR1 coefficient, which can be really tricky for the time series containing both red noise and regime shifts. Two methods of estimating AR1 have been implemented here. The first method, MPK, is based on the formula for the bias in the ordinary least squares (OLS) estimate of AR1 suggested by Marriott and Pope (1954) and Kendall (1954). The second method, called IP4 for short, is based on the assumption that the bias is approximately inversely proportional to the sample size, with four subsequent corrections applied. Both methods are described in Rodionov (2006).

Figure below schematically explains the options available in this section. If "None" is chosen, then no AR1 estimation is performed. The results are the same as in the previous version of the program. All other options require the AR1 estimate. Note that the OLS estimate is calculated using the entire time series. The MPK and IP4 methods break the time series into subsamples, estimate bias corrected AR1 for each subsample and then use the median value of all estimates. The suggested subsample size m is calculated as m = (l + 1)/3, where l is the cutoff length. It is recommended to experiment with different subsample sizes to see how it affects the AR1 estimate.

If the Prewhitening box is unchecked, the AR1 is estimated given the adjusted degrees of freedom (DF_adj) for the RSI: DF_adj = 2l_eq - 2, where l_eq is the equivalent cutoff length, calculated using the formula in Von Storch and Zwiers (1999, p. 115) for the equivalent sample size. This formula is also used to calculate the final significance level for the shifts adjusted for serial correlation.

Schematic for the red noise estimation options in the program.

If the Prewhitening box is checked, the regime shifts are detected for the filtered time series. No adjustments for serial correlation are made when calculating the DF. As an output, the user can see either the filtered time series (the Filtered box checked), or get back to the original time series (the Filtered box unchecked). In the latter case, the final significance level is adjusted using the equivalent sample size formula.