Basic Building Blocks of Yield Curve Smoothing, Part 11: The Shimko Test for Measuring Accuracy of Smoothing Techniques
In the first 10 installments of this series on yield curve smoothing, we committed the most common sin there is in the yield curve smoothing literature. We used one set of "made up" data instead of hundreds or thousands of real data points to judge the performance of yield curve smoothing techniques. In this blog, we explain why the test proposed by David Shimko is essential to judging the accuracy and realism of yield curve smoothing techniques. We dust off some old yield data from the attic to illustrate how the test works.
Introduction to the Shimko Test
In 1994, Kenneth J. Adams and I published "Fitting Yield Curves and Forward Rate Curves with Maximum Smoothness" (Journal of Fixed Income, June 1994). A copy of the article is available in the research section of www.kamakuraco.com with registration. Please note, by the way, that the proof in the original paper was corrected in van Deventer and Imai, Financial Risk Analytics (1996) thanks to comments by Volf Frishling of the Commonwealth Bank of Australia and by Kamakura's Managing Director Robert A. Jarrow. The corrected proof of the functional form of the maximum smoothness forward rate function is also reproduced in Appendix B of Chapter 8 of Advanced Financial Risk Management by van Deventer, Imai and Mesler (John Wiley & Sons, 2004).
In 1993, my good friend David Shimko responded to an early draft of the Adams and van Deventer paper by saying "I don't care about a mathematical proof of 'best,' I want something that would have best estimated a data point that I intentionally leave out of the smoothing process-this to me is proof of which technique is most realistic." A statistician would add, "And I want something that is most realistic on a very large sample of data." We agreed that Dr. Shimko's suggestion was the ultimate proof of the accuracy and realism of any smoothing technique. The common academic practice of using one set of fake data and then judging which technique "looks good" or "looks bad" is as ridiculous as it is common. Therefore, we atone for the same sin, which we have used in the first 10 installments of this series, by dusting off the original van Deventer and Adams data from our computer attic.
U.S. Dollar Swap Data, 1989-1992
Using interest rate swap data provided by Fuji Bank and Mitsubishi Bank, Adams and van Deventer analyzed the accuracy of six different smoothing methods on 848 days of yen interest rate swap data and 660 days of U.S. dollar interest rate swap data. We remind readers that, since interest rate swap curves reflect default risk, the proper way to smooth such curves is by smoothing credit spreads, not the absolute level of yields. We explain that in a later part of this series. We use the Adams and van Deventer U.S. dollar swap data merely to illustrate the process for applying the Shimko test for realism. Adams and van Deventer applied the Shimko test at both 3 year and 7 year maturities. We discuss the U.S. dollar 7 year Shimko test here.
The Shimko test works as follows. First we assemble a large data set, in this case with 660 days of swap data. Next, we select one of the maturities in that data set and leave it out of the smoothing process. For purposes of this example, we leave out the 7 year maturity because that leaves a wide 5 year gap in the swap data to be filled by the smoothing technique. The other maturities were 1 month, 3 months, 6 months, 1 year, 2 years, 3 years, 5 years and 10 years. We smooth the 660 yield curves one by one. Using the smoothing results and the zero coupon bond yields associated with the 14 semiannual payment dates of the 7 year interest rate swap, we calculate the 7 year swap rate implied by the smoothing process. We have 660 observations of this estimated 7 year swap rate, and we compare it to the actual 7 year swap rates that we left out of the smoothing process. The "best" smoothing technique is the one which most accurately estimates the omitted data point over the full sample.
This test can be performed on any of the maturities that were inputs to the smoothing process, and we strongly recommend that all maturities are used, one at a time.
This graph from the original Adams and van Deventer paper shows the wide variation in yield curve shapes over the 660 data points for U.S. dollar swaps in the 1989 to 1992 period:
From the original six smoothing methods compared by Adams and van Deventer, we focus on two in particular. The first technique is a cubic spline of zero coupon yields, which we discussed in part 8 of this series. We use an implementation of yield splines where the zero coupon yield curve is constrained to be flat at the right hand side (the 10 year maturity) of the yield curve, y'(10)=0. The other approach is the Adams and van Deventer 1994 implementation of the maximum smoothness forward rate approach.
The graph below plots the actual 7 year U.S interest rate swap yield versus the estimated 7 year swap rate using the yield spline (red) and maximum smoothness forward rate approach (green):
While it's hard to see unless you have exceptionally good eyesight, on the right hand side of the graph the cubic spline approach is very consistently and significantly different from the actual seven year swap rate, which the maximum smoothness approach fits quite well. The errors in fitting the actual 7 year swap rate are much easier to see if we plot the mean absolute errors that result over time. The graph below shows the cubic yield spline errors in blue and the maximum smoothness errors in red. On the left hand side of the graph, the maximum smoothness approach is generally slightly better, but on the right hand side of the graph, the advantage of the maximum smoothness approach becomes very large.
Over the full 660 observation sample, the maximum smoothness approach produces a mean absolute error of 5.73 basis points in estimating the omitted 7 year swap yield. The cubic yield spline produces a mean absolute error of 8.51 basis points. This is extremely powerful evidence that the maximum smoothness approach is just as attractive from a practical point of view as its mathematical appeal for being "the most smooth" forward rate curve that can be produced from this data. This empirical evidence helps to confirm that the mathematical criterion for best, maximum smoothness, in effect is the most realistic assumption about the nature of forward interest rates.
The graph above and the other results reported by Adams and van Deventer make it clear that the relative performance of smoothing techniques is highly dependent on the shape of the yield curve. When yields are flat, for example, all techniques will work equally well. The differences in smoothing performance will be most obvious when there are many sharp bends in the yield curve. That is why a large sample for performance measurement is essential. It is also further confirmation that the typical academic criterion for "best," asserting that a given functional form is best based on one fictitious set of yield inputs, is simply a fairy tale that may illustrate weaknesses of a given technique but that carries no weight of proof whatsoever. Only large masses of real data can do that. Seventeen years of working with multiple yield curve smoothing techniques with masses of daily data from many countries have consistently produced accuracy results very similar to those in the Adams and van Deventer paper. Because this test suggested by David Shimko is a powerful test applicable to any contending smoothing techniques, we strongly recommend that no assertion of superior performance be made without applying the Shimko test on a large amount of real data.
In the next post in this series, we step forward in realism by discussing how to use bond prices, rather than zero coupon bond yields, as inputs to the smoothing process.
Honolulu, January 13, 2010