How can I investigate historical effect of Rebalancing on Return and Standard Deviation?
"Do not reinvent the wheel! Historical data about stock market returns and standard deviations suffer from number of issues such as past-filling and mostly survivorship bias -- that the current answers do not consider at all. I suggest to read the paper ""A Century of Global Stock Markets"" by Philippe Jorion (UC Irvine) and William Goetzmann (Yale), here. William Bernstein comments the results here, notice that rebalancing is sometimes a good option but not always, his non-obvious finding where the low SD did not favour from rebalancing:  Look at the final page of the paper,  ""geometric returns -- represent returns to a buy-and-hold strategy"" and the ""arithmetic averages -- give equal weight to each observation interval."", where you can find your asked ""historical effect of Rebalancing on Return and Standard Deviation"". The paper nicely summarizes the results to this table:  The results in the table are from the interval 1921-1996, it is not that long-time but even longer term data has its own drawbacks. The starting year 1921 is interesting choice because it is around the times of social-economical changes and depressing moments, historical context can be realized from books such as Grapes Of Wrath (short summary here, although fiction to some extent, it has some resonance to the history). The authors have had to ignore some years because of different reasons such as political unrest and wars.  Instead of delving into marketed spam as suggested by one reply, I would look into this search here. Look at the number of references and the related papers to judge their value.  P.s. I encourage people to attack my open question here, hope we can solve it!" "Doesn't ""no rebalancing"" mean ""start with a portfolio and let it fly?""  Seems like incorporation of rebalancing is more sophisticated than not. Just ""buy"" your portfolio at the start and see where it ends up with no buying/selling, as compared with where it ends up if you do rebalance. Or is it not that simple?" "To answer your question directly..  you can investigate by using google or other means to look up research done in this area.  There's been a bunch of it Here's an example of search terms that returns a wealth of information. effect+of+periodic+rebalancing+on+portfolio+return I'd especially look for stuff that appears to be academic papers etc, and then raid the 'references' section of those.  Look for stuff published in industry journals such as ""Journal of Portfolio Management"" as an example. If you want to try out different models yourself and see what works and what doesn't, this Monte Carlo Simulator might be something you would find useful The basic theory for those that don't know is that various parts of a larger market do not usually move in perfect lockstep, but go through cycles.. one year tech might be hot, the next year it's healthcare.  Or for an international portfolio, one year korea might be doing fantastic only to slow down and have another country perform better the next year.    So the idea of re-balancing is that since these things tend to be cyclic, you can get a higher return if you sell part of a slice that is doing well (e.g. sell at the high) and invest it in one that is not (buy at the low) Because you do this based on some criteria, it helps circumvent the human tendency to 'hold on to a winner too long' (how many times have you heard someone say 'but it's doing so well, why do I want to sell now""? presuming trends will continue and they will 'lose out' on future gains, only to miss the peak and ride the thing down back into mediocrity.)  Depending on the volatility of the specific market, and the various slices, using re balancing can get you a pretty reasonable 'lift' above the market average, for relatively low risk. generally the more volatile the market, (such as say an emerging markets portfolio) the more opportunity for lift. I looked into this myself a number of years back, the concensus I came was that the most effective method was to rebalance based on 'need' rather than time.  Need is defined as one or more of the 'slices' in your portfolio being more than 8% above or below the average.  So you use that as the trigger. How you rebalance depends to some degree on if the portfolio is taxable or not.  If in a tax deferred account, you can simply sell off whatever is above baseline and use it to buy up the stuff that is below.   If you are subject to taxes and don't want to trigger any short term gains, then you may have to be more careful in terms of what you sell.  Alternatively if you are adding funds to the portfolio, you can alter how your distribute the new money coming into the portfolio in order to bring up whatever is below the baseline (which takes a bit more time, but incurs no tax hit)  The other question is how will you slice a given market?  by company size?  by 'sectors' such as tech/finance/industrial/healthcare, by geographic regions?" From Vanguard's Best practices for portfolio rebalancing:
My level of analysis is not quite that advanced. Can you share what that would show and why that particular measure is the one to use?  I've run regression on prices between the two. VIX prices have no correlation to the s&amp;p500 prices. Shouldn't true volatility result in the prices (more people putting options on the VIX during the bad times and driving that price up) correlate to the selloff that occurs within the S&amp;P500 during recessions and other events that would cause significant or minor volatility? My r2 showed no significance within a measurement of regression within Excel.  But, *gasp* I could be wrong, but would love to learn more about better ways of measurement :) Basically, diversifying narrows the spread of possible results, raising  the center of the returns bell-curve by reducing the likelihood of extreme results at either the high or low end.  It's largely a matter of basic statistics. Bet double-or-nothing on a single coin flip, and those are the only possible results, and your odds of a disaster (losing most or all of the money) are 50%. Bet half of it on each of two coin flips, and your odds of losing are reduced to 25% at the cost of reducing your odds of winning to 25%, with 50% odds that you retain your money and can try the game again. Three coins divides the space further; the extremes are reduced to 12.5% each, with the middle being most likely. If that was all there was, this would be a zero-sum game and pure gambling. But the stock market is actually positive-sum, since companies are delivering part of their profits to their stockholder owners. This moves the center of the bell curve up a bit from break-even, historically to about +8%. This is why index funds produce a profit with very little active decision; they treat the variation as mostly random (which seems to work statistically) and just try to capture average results of a (hopefully) slightly above-average bucket of stocks and/or bonds. This approach is boring. It will never double your money overnight. On the other hand, it will never wipe you out overnight. If you have patience and are willing to let compound interest work for you, and trust that most market swings regress to the mean in the long run, it quietly builds your savings while not driving you crazy worrying about it. If all you are looking for is better return than the banks, and you have a reasonable amount of time before you need to pull the funds out, it's one of the more reliably predictable risk/reward trade-off points. You may want to refine this by biasing the mix of what you're holding. The simplest adjustment is how much you keep in each of several major investment categories. Large cap stocks, small cap stocks, bonds, and real estate (in the form of REITs) each have different baseline risk/return curves, and move in different ways in response to news, so maintaining a selected ratio between these buckets and adding the resulting curves together is one simple way to make fairly predictable adjustments to the width (and centerline) of the total bell curve. If you think you can do better than this, go for it. But index funds have been outperforming professionally managed funds (after the management fees are accounted for), and unless you are interested in spending a lot of time researching and playing with your money the odds of your doing much better aren't great unless you're willing to risk doing much worse. For me, boring is good. I want my savings to work for me rather than the other way around, and I don't consider the market at all interesting as a game. Others will feel differently. Buy as much data as you can, make a model, automate the back testing, see if it works over the past decade or so. If the returns are offering a superior risk adjusted return then you have a valid model, if not try an other model.   If you don't know how to program, learn how to. "The question ""do they?"" is a fair one, but the answer, ""we can only observe the past, and that's what they did,"" may not be so satisfying to you.  It's safe to say that any longer term view of any market will show far less volatility than a short one. It only takes a glance at the return of the 2000's  2009 27.11   2008 -37.22  2007      5.46  2006  15.74 2005  4.79   2004 10.82   2003  28.72  2002    -22.27  2001 -11.98 2000     -9.11 (for the S&P) to see that in an awful decade containing -37% and -22% that the full decade was ""only"" down 9% in total or just less than 1% per year compounded. I'm not predicting any particular returns forward, just noting this is how the math works.  DCA performs well through such a decade, better than in a rising one. You are offered the opportunity to buy into a market selling below the long term trend.  Added note in response to Enno's answer below - On rereading the linked article, I see where the author cites Zvi Bodie who clearly made a logical error. He concludes that since a 20 month S&P put costs triple what a 2.3 mo put costs, that there's more risk the market falls over the longer period, not less. American options can be sold or exercised at any time. If a 2 year option were cheaper than a 2 month option, no one would buy the shorter term. It's pretty simple that the Options Pricing Models take time into account and their value, put or call, increases along with the time till expiration.  On a lighter note, when I take the S&P data for 1871-2012 (I know, no S&P back then, but it's Schiller's data) I get average 40 year returns of 44X, similar to the author's conclusion, $1K growing to $44K. But, the Standard deviation is 28. So the high end of +1 STDEV is $72K, not the author's $166K. Although, the low end 44-28=16 comes close to his $14K figure. $16K is a 7.18% long term return which today doesn't look bad. When the article was written, the author was looking at a 6% short term risk free rate." From an article I wrote a while back: “Dalbar Inc., a Boston-based financial services research firm, has been measuring the effects of investors’ decisions to buy, sell, and switch into and out of mutual funds since 1984. The key finding always has been that the average investor earns significantly less than the return reported by their funds. (For the 20 years ended Dec. 31, 2006, the average stock fund investor earned a paltry 4.3 average annual compounded return compared to 11.8 percent for the Standard & Poor’s 500 index.)” It's one thing to look at the indexes. But quite another to understand what other investors are actually getting. The propensity to sell low and buy high is proven by the data Dalbar publishes. And really makes the case to go after the magic S&P - 0.09% gotten from an ETF. "I think you're on the wrong track.  Getting more and more samples from the real world does not make your backtest more accurate, it just confirms that your strategy can withstand one particular sample path of a stochastic process. The reason why you find it simple to incorporate fees, commissions, taxes, etc. is because they're a static and constant process -- well they might change over time but most definitely uncorrelated to the markets. Modelling overnight returns or the top levels of the order book the next day is serious work.  First you have to select a suitable model (that's mostly theoretical work but experience can help a lot).  Then, in order to do it data-driven, you'd have to plough through thousands of days of sample data on a set of thousands of instruments to get a ""feeling"" (aka significant model parameters).   Apropos data mining, I think Excel might be the wrong tool for the job.  Level-2 data (even just the first 10 levels) is a massive blob.  For example, the NYSE OpenBook historical data weighs in at a massive 15 TB compressed (uncompressed 74 TB) for the last 10 years, and costs USD 200k.  Anyway, as for other factors to take into account: So how to account for all this in a backtest?  Personally, I would put in some penalty terms (as % on a return basis) for every factor you want to consider, don't hardcode them.  You can then run a stress test by exploring these parameters (i.e. assign some values in the range of 0 to whatever fits).  Explore them individually (only set one penalty term at a time) to get a feeling how the strategy might react to stress from that factor.  Then you can run the backtest with typical (or observed) combinations of penalty factors and slowly stress them altogether. Edit Just to avoid confusion about terminology.  A backtest in the strict sense (had I implemented this strategy X years ago, what would have happened?) won't benefit from any modelling simply because the real-world ""does the sampling"" for us. However, to evaluate a strategy's robustness you should account for the additional factors and run some stress tests.  If the strategy performs well in the real-world or no-stress scenario but produces losses once a tiny slippage occurs every now and again, you could conclude that the strategy is very fragile.  The key is to explore the maximum stress the strategy can handle (by whatever measure); if a lot you can call the strategy robust. The latter is what I personally call a backtest; the first procedure would go by the name ""extension towards the past"" or so. Some lightweight literature:" "To your first comment: yup. To your second comment, A = L + E. If E goes down, and L goes up, the net effect is 0. Then, if L goes down, and A goes up, the net effect is 0 and we are balanced once again. There is no ""rebalancing"" equity. You just have to make sure that, at the end of your journal entries, the accounting equation holds.  It's a very unintuitive concept to wrap your head around, but spend some time mapping out the flow of various journal entries. Once it clicks, you'll really understand the logic." What you were told isn't an absolute truth, so trying to counter something fundamentally flawed won't get you anywhere.  For example: chinese midcap equities are up 20% this year, even from their high of 100%. While the BSE Sensex in India is down several percentage points on the year. Your portfolio would have lost money this year taking advice from your peers. The fluctuation in the rupees and remnibi would not have changed this fact. What you are asking is a pretty common area of research, as in several people will write their dissertation on the exact same topic every year, and you should be able to find various analysis and theories on the subject. But the macroeconomic landscape changes, a lot. This link does it ok:  http://investexcel.net/1979/calculate-historical-volatility-excel/ Basically, you calculate percentage return by doing stock price now / stock price before.  You're not calculating the rate of return hence no subtraction of 100%.  The standard is to do this on a daily basis: stock price today / stock price yesterday. The most important and most misunderstood part is that you now have to analyze the data geometrically not arithmetically.  To easily do this, convert all percentage returns with the natural log, ln(). Next, you take the standard deviation of all of those results, and apply exp().  This answers the title of your question. For convenience's sake, it's best to annualize since volatility (implied or statistical) is now almost always quoted annualized.  There are ~240 trading days each year.  You multiply your stdev() result by (240 / # of trading days per return) ^ 0.5, so if you're doing this for daily returns, multiply the stdev() result by 240^0.5; if you were doing it weekly, you'd want to multiply by (240 / ~5)^0.5; etc.  This is your number for sigma.  This answers the intent of your question. For black-scholes, you do not convert anything back with exp(); BS is already set up for geometric analysis, so you need to stay there. The reason why analysis is done geometrically is because the distribution of stock returns is assumed to be lognormal (even though it's really more like logLaplace). you can check google scholar for some research reports on it.  depends how complex you want to get...  it is obviously a function of the size of the portfolio of each type of asset. do you have a full breakdown of securities held? you can get historical average volumes during different economic periods, categorized by interest rates for example, and then calculate the days required to liquidate the position, applying a discount on each subsequent day. I couldn't find historical data either, so I contacted Vanguard Canada and Barclays; Vanguard replied that  This index was developed for Vanguard, and thus historical information is available as of the inception of the fund. Unfortunately, that means that the only existing data on historical returns are in the link in your question. Vanguard also sent me a link to the methodology Barclay's uses when constructing this index, which you might find interesting as well.  I haven't heard from Barclays, but I presume the story is the same; even if they've been collecting data on Canadian bonds since before the inception of this index, they probably didn't aggregate it into an index before their contract with Vanguard (and if they did, it might be proprietary and not available free of charge). My thoughts from the top:  1. If you're already long, which not just buy-write? if you want to sit on the portfolio for some time with little rebalancing collecting some option premiums might not be bad?  2. You can replicate this portfolio on a rolling basis through bull spreads, which should hedge (or at least reduce) your theta and give you some of that sweet sweet convexity, however you will need a large enough size to cover your transaction costs.  3. I agree with maximizing mean / lower semi var (most kids max mean/var)) so good move there. However you need estimates of expected return (not easy!) and semi var (easier). Look into using GARCH (or a form of it) on the semi var for a better estimator.  4. Is this an MSR portfolio? How do you determine your weights?  5. In terms of sizing/leverage,  Remember the betting condition that equity needs to be positive for all time. How are you planning to limit your max DD? and how often do you plan on rebalancing/rebetting. "My answer is Microsoft Excel.  Google ""VBA for dummies"" (seriously) and find out if your brokerage offers an 'API'.  With a brief understanding of coding you can get a spreadsheet that is live connected to your brokers data stream.  Say you have a spreadsheet with the 1990 value of each in the first two columns (cells a1 and b1).  Maybe this formula could be the third column, it'll tell you how much to buy or sell to rebalance them.   then to iterate the rebalance, set both a2 and b2 to =C1 and drag the formula through row 25, one row for each year.  It'll probably be a little more work than that, but you get the idea." "There are some good answers about the benefits of diversification, but I'm going to go into what is going on mathematically with what you are attempting. I was always under the assumption that as long as two securities are less than perfectly correlated (i.e. 1), that the standard deviation/risk would be less than if I had put 100% into either of the securities. While there does exist a minimum variance portfolio that is a combination of the two with lower vol than 100% of either individually, this portfolio is not necessarily the portfolio with highest utility under your metric.  Your metric includes returns not just volatility/variance so the different returns bias the result away from the min-vol portfolio. Using the utility function: E[x] - .5*A*sig^2 results in the highest utility of 100% VTSAX. So here the Sharpe ratio (risk adjusted return) of the U.S. portfolio is so much higher than the international portfolio over the period tracked that the loss of returns from adding more international stocks outweigh the lower risk that you would get from both just adding the lower vol international stocks and the diversification effects from having a correlation less than one. The key point in the above is ""over the period tracked"".  When you do this type of analysis you implicitly assume that the returns/risk observed in the past will be similar to the returns/risk in the future.   Certainly, if you had invested 100% in the U.S. recently you would have done better than investing in a mix of US/Intl.  However, while the risk and correlations of assets can be (somewhat) stable over time relative returns can vary wildly! This uncertainty of future returns is why most people use a diversified portfolio of assets.  What is the exact right amount is a very hard question though." "It's easy for me to look at an IRA, no deposits or withdrawal in a year, and compare the return to some index. Once you start adding transactions, not so easy. Here's a method that answers your goal as closely as I can offer: SPY goes back to 1993. It's the most quoted EFT that replicates the S&P 500, and you specifically asked to compare how the investment would have gone if you were in such a fund. This is an important distinction, as I don't have to adjust for its .09% expense, as you would have been subject to it in this fund. Simply go to Yahoo, and start with the historical prices. Easy to do this on a spreadsheet. I'll assume you can find all your purchases inc dates & dollars invested. Look these up and treat those dollars as purchases of SPY. Once the list is done, go back and look up the dividends, issues quarterly, and on the dividend date, add the shares it would purchase based on that day's price. Of course, any withdrawals get accounted for the same way, take out the number of SPY shares it would have bought. Remember to include the commission on SPY, whatever your broker charges.  If I've missed something, I'm sure we'll see someone point that out, I'd be happy to edit that in, to make this wiki-worthy.  Edit - due to the nature of comments and the inability to edit, I'm adding this here. Perhaps I'm reading the question too pedantically, perhaps not. I'm reading it as ""if instead of doing whatever I did, I invested in an S&P index fund, how would I have performed?"" To measure one's return against a benchmark, the mechanics of the benchmarks calculation are not needed. In a comment I offer an example - if there were an ETF based on some type of black-box investing for which the investments were not disclosed at all, only day's end pricing, my answer above still applies exactly. The validity of such comparisons is a different question, but the fact that the formulation of the EFT doesn't come into play remains. In my comment below which I removed I hypothesized an ETF name, not intending it to come off as sarcastic. For the record, if one wishes to start JoesETF, I'm ok with it."