Standard Deviation with Asset Prices?
You can use google docs to create a spreadsheet. In field A2, I put Google will load the prices into the sheet. At that point, I add the following into C12, then copy that line all the way down to the botton of column C. You can find my spreadsheet here. It calculates the moving 10 day standard deviation as a percentage of average price for that time period. "Almost every online datasources provide historical prices on given company / index's performance; from this, you can easily calculate ""standard deviation"" by yourself. With that said, standard deviation presumes a fixed set of data. Most public corporations have data spanning multiple decades, during which a number of things have changed: For these reasons, I have doubts on simplistic measures, such as ""standard deviation"" measuring any reality on the underlying vehicle. Professional investors usually tend to more time-point data, such as P/E ratio." "Some years ago, two ""academics,"" Ibbotson and Sinquefield did these calculations. (Roger) Ibbotson, is still around. So Google Roger Ibbotson, or Ibbotson Associates. There are a number of entries so I won't provide all the links." James Roth provides a partial solution good for stock picking but let's speed up process a bit, already calculated historical standard deviations: Ibbotson, very good collection of research papers here, examples below Books
"He's calculating portfolio variance. The general formula for the variance of a portfolio composed of two securities looks like this:  where w_a and w_b are the weights of each stock in the portfolio and the sigmas represent the standard deviation/risk of each asset or portfolio.  In the case of perfect positive or negative correlation, applying some algebra to the formula relating covariance to the correlation coefficient (rho, the Greek letter that looks like ""p""):  tells us that the covariance we need in the original formula is simply the product of the standard deviations and the correlation coefficient (-1 in this case).  Combining that result with our original formula yields this calculation:  Technically we've calculated the portfolio's variance and not it's standard deviation/risk, but since the square root of 0 is still 0, that doesn't matter. The Wikipedia article on Modern Portfolio Theory has a section that describes the mathematical methods I used above. The entire article is worth a read, however." Metals and Mining is an interesting special case for stocks.  It's relationship to U.S. equity (SPX) is particularly weak (~0.3 correlation) compared to most stocks so it doesn't behave like equity.  However, it is still stock and not a commodities index so it's relation to major metals (Gold for instance) is not that strong either (-0.6 correlation). Metals and Mining stocks have certainly underperformed the stock market in general over the past 25years 3% vs 9.8% (annualized) so this doesn't look particularly promising.  It did have a spectacularly good 8 year period ('99-'07) though 66% (annualized).  It's worth remembering that it is still stock.  If the market did not think it could make a reasonable profit on the stock the price would decrease until the market thought it could make the same profit as other equity (adjusted slightly for the risk).  So is it reasonable to expect that it would give the same return as other stock on average?  Yes.. -ish.  Though as has been shown in the past 25 years your actual result could vary wildly both positive and negative. (All numbers are from monthly over the last 25 years using VGPMX as a M&M proxy) "VaR does not do what it is supposed to do which is give you a ""floor"" with confidence on your potential losses.  Even for portfolios with millions of instruments, it will not give you a metric that means anything.  The point im trying to convey is that VaR does not give any meaningful information as its horribly inaccurate and that we would be better off WITHOUT VaR." Is it POSSIBLE? Of course. I don't even need to do any research to prove that. Just some mathematical reasoning: Take the S&P 500. Find the performance of each stock in that list over whatever time period you want to use for your experiment. Now select some number of the best-performing stocks from the list -- any number less than 500. By definition, the X best must be better than or equal to the average. Assuming all the stocks on the S&P did not have EXACTLY the same performance, these 10 must be better than average. You now have a diversified portfolio that performed better than the S&P 500 index fund. Of course as they always say in a prospectus, past performance is not a guarantee of future performance.  It's certainly possible to do. The question is, if YOU selected the stocks making up a diversified portfolio, would your selections do better than an index fund? "Answering this question is weird, because it is not really precise in what you mean. Do you want all stocks in the US? Do you want a selection of stocks according to parameters?  Do you just want a cool looking graph? However, your possible misuse of the word derivative piqued my interest. Your reference to  gold and silver seems to indicate that you do not know what a derivative actually is. Or what it would do in a portfolio.  The straightforward way to ""see"" an efficient frontier is to do the following. For a set of stocks (in this case six ""randomly"" selected ones):      library(quantmod)  library(fPortfolio)  library(PerformanceAnalytics)     getSymbols(c(""STZ"", ""RAI"", ""AMZN"", ""MSFT"", ""TWX"", ""RHT""), from = ""2012-06-01"", to = ""2017-06-01"")     returns &lt;- NULL  tickerlist &lt;- c(""STZ"", ""RAI"", ""AMZN"", ""MSFT"", ""TWX"", ""RHT"")  for (ticker in tickerlist){   returns &lt;- cbind(returns, monthlyReturn(Ad(eval(as.symbol(ticker)))))  }  colnames(returns) &lt;- tickerlist  returns &lt;- as.timeSeries(returns)    frontier &lt;- portfolioFrontier(returns)  png(""frontier.png"", width = 800, height = 600)  plot(frontier, which = ""all"")  dev.off()    minvariancePortfolio(returns, constraints = ""LongOnly"")    Portfolio Weights:    STZ    RAI   AMZN   MSFT    TWX    RHT   0.1140 0.3912 0.0000 0.1421 0.1476 0.2051     Covariance Risk Budgets:    STZ    RAI   AMZN   MSFT    TWX    RHT   0.1140 0.3912 0.0000 0.1421 0.1476 0.2051     Target Returns and Risks:   mean    Cov   CVaR    VaR   0.0232 0.0354 0.0455 0.0360    https://imgur.com/QIxDdEI  The minimum variance portfolio of these six assets has a mean return is 0.0232 and variance is 0.0360. AMZN does not get any weight in the portfolio.  It kind of means that the other assets span it and it does not provide any additional diversification benefit.   Let us add two ETFs that track gold and silver to the mix, and see how little difference it makes:    getSymbols(c(""GLD"", ""SLV""), from = ""2012-06-01"", to = ""2017-06-01"")    returns &lt;- NULL  tickerlist &lt;- c(""STZ"", ""RAI"", ""AMZN"", ""MSFT"", ""TWX"", ""RHT"", ""GLD"", ""SLV"")  for (ticker in tickerlist){   returns &lt;- cbind(returns, monthlyReturn(Ad(eval(as.symbol(ticker)))))  }  colnames(returns) &lt;- tickerlist  returns &lt;- as.timeSeries(returns)   frontier &lt;- portfolioFrontier(returns)    png(""weights.png"", width = 800, height = 600)  weightsPlot(frontier)  dev.off()    # Optimal weights  out &lt;- minvariancePortfolio(returns, constraints = ""LongOnly"")  wghts &lt;- getWeights(out)  portret1 &lt;- returns%*%wghts  portret1 &lt;- cbind(monthprc, portret1)[,3]  colnames(portret1) &lt;- ""Optimal portfolio""    # Equal weights  wghts &lt;- rep(1/8, 8)  portret2 &lt;- returns%*%wghts  portret2 &lt;- cbind(monthprc, portret2)[,3]  colnames(portret2) &lt;- ""Equal weights portfolio""   png(""performance_both.png"", width = 800, height = 600)  par(mfrow=c(2,2))  chart.CumReturns(portret1, ylim = c(0, 2))  chart.CumReturns(portret2, ylim = c(0, 2))  chart.Drawdown(portret1, main = ""Drawdown"", ylim = c(-0.06, 0))  chart.Drawdown(portret2, main = ""Drawdown"", ylim = c(-0.06, 0))  dev.off()   https://imgur.com/sBHGz7s   Adding gold changes the minimum variance mean return to 0.0116 and the variance stays about the same 0.0332. You can see how the weights change at different return and variance profiles in the picture.  The takeaway is that adding gold decreases the return but does not do a lot for the risk of the portfolio.  You also notice that silver does not get included in the minimum variance efficient portfolio (and neither does AMZN).  https://imgur.com/rXPbXau  We can also compare the optimal weights to an equally weighted portfolio and see that the latter would have performed better but had much  larger drawdowns. Which is because it has a higher volatility, which might be undesirable.  ---  Everything below here is false, but illustrative.  So what about the derivative part? Let us assume you bought an out of the money call option with a strike of 50 on MSFT at the beginning of the  time series and held it to the end.  We need to decide on the the annualized cost-of-carry rate, the annualized rate of interest, the time to maturity is measured in years, the annualized  volatility of the underlying security is proxied by the historical volatility.   library(fOptions)   monthprc &lt;- Ad(MSFT)[endpoints(MSFT, ""months"")]  T &lt;- length(monthprc) # 60 months, 5 years  vol &lt;- sd(returns$MSFT)*sqrt(12) # annualized volatility    optprc &lt;- matrix(NA, 60, 1)  for (t in 1:60) {   s &lt;- as.numeric(monthprc[t])   optval &lt;- GBSOption(TypeFlag = ""c"", S = s, X = 50, Time = (T - t) / 12, r = 0.001, b = 0.001, sigma = vol)   optprc[t] &lt;- optval@price  }    monthprc &lt;- cbind(monthprc, optprc)  colnames(monthprc) &lt;- c(""MSFT"", ""MSFTCall50"")    MSFTCall50rets &lt;- monthlyReturn(monthprc[,2])  colnames(MSFTCall50rets) &lt;- ""MSFTCall50rets""    returns &lt;- merge(returns, MSFTCall50rets)    wghts &lt;- rep(1/9, 9)    portret3 &lt;- returns%*%wghts  portret3 &lt;- cbind(monthprc, portret3)[,3]  colnames(portret3) &lt;- ""Equal weights derivative portfolio""    png(""performance_deriv.png"", width = 800, height = 600)  par(mfrow=c(2,2))  chart.CumReturns(portret2, ylim = c(0, 4.5))  chart.CumReturns(portret3, ylim = c(0, 4.5))  chart.Drawdown(portret2, main = ""Drawdown"", ylim = c(-0.09, 0))  chart.Drawdown(portret3, main = ""Drawdown"", ylim = c(-0.09, 0))  dev.off()  https://imgur.com/SZ1xrYx   Even though we have a massively profitable instrument in the derivative. The portfolio analysis does not include it  because of the high volatility. However, if we just use equal weighting and essentially take a massive position in  the out of the money call (which would not be possible in real life), we get huge drawdowns and volatility, but  the returns are almost two fold. But nobody will sell you a five year call.  Others can correct any mistakes or misunderstandings in the above. It hopefully gives a starting point.  Read more at:   https://en.wikipedia.org/wiki/Modern_portfolio_theory https://en.wikipedia.org/wiki/Option_(finance)  The imgur album:  https://imgur.com/a/LoBEY" Yes, more leverage increases the variance of your individual portfolio (variance of your personal net worth). The simple way to think about it is that if you only own only 50% of your risky assets, then you can own twice as many risky assets. That means they will move around twice as much (in absolute terms). Expected returns and risk (if risk is variance) both go up. If you lend rather than borrow, then you might have only half your net worth in risky assets, and then your expected returns and variation in returns will go down. Note, the practice of using leverage differs from portfolio theory in a couple important ways. Your question is a moving target. And my answer will be subject to revision. I disagree with the votes to close, as you are asking (imho) what role commodities and specifically oil, play in one's asset allocation. Right? How much may be opinion, but there's a place to ask if.  I'm looking at this chart, and thinking, long term, the real return is zero. The discussion regarding gold has been pretty exhausted. For oil, it's not tough to make the case that it will fluctuate, but long term, there's no compelling reason to believe its price will rise any faster than inflation over the really long term. "Let me start by giving you a snippet of a report that will floor you. Beat the market? Investors lag the market by so much that many call the industry a scam.   This is the 2015 year end data from a report titled Quantitive Analysis of Investor Behavior by a firm, Dalbar. It boggles the mind that the disparity could be this bad. A mix of stocks and bonds over 30 years should average 8.5% or so. Take out fees, and even 7.5% would be the result I expect. The average investor return was less than half of this. Jack Bogle, founder of Vanguard, and considered the father of the index fund, was ridiculed. A pamphlet I got from Vanguard decades ago quoted fund managers as saying that ""indexing is a path to mediocrity."" Fortunately, I was a numbers guy, read all I could that Jack wrote and got most of that 10.35%, less .05, down to .02% over the years. To answer the question: psychology. People are easily scammed as they want to believe they can beat the market. Or that they'll somehow find a fund that does it for them. I'm tempted to say ignorance or some other hint at lack of intelligence, but that would be unfair to the professionals, all of which were scammed by Madoff. Individual funds may not be scams, but investors are partly to blame, buy high, sell low, and you get the results above, I dare say, an investor claiming to use index funds might not fare much better than the 3.66% 30 year return above, if they follow that path, buying high, selling low. Edit - I am adding this line to be clear - My conclusion, if any, is that the huge disparity cannot be attributed to management, a 6.7% lag from the S&P return to what the average investor sees likely comes from bad trading. To the comments by Dave, we have a manager that consistently beats the market over any 2-3 year period. You have been with him 30 years and are clearly smiling about your relationship and investing decision. Yet, he still has flows in and out. People buy at the top when reading how good he is, and selling right after a 30% drop even when he actually beat by dropping just 22%. By getting in and out, he has a set of clients with a 30 year record of 6% returns, while you have just over 11%. This paragraph speaks to the behavior of the investor, not managed vs indexed." Ok, so disregarding passivity, could you help me through a simplified example?   Say I only had two assets, SPY and TLT, with a respective weight of 35 and 65% and I want want to leverage this to 4x.  Additionally, say daily return covar is:  * B/B .004% * B/S -.004% * S/S .02%   Now, if I read correctly, I should buy ATM calls xxx days in the future.  Which may look like: Ticker, S, K, Option Price, Delta, Lambda  * TLT  $126.04  $126.00  $4.35  0.50  14.5 * SPY  $134.91  $134.00  $6.26  0.55  11.8  ^ This example is pretty close but some assets are far off.   I feel like I'm on the wrong track so I'll stop here.    I just want to lever up my risk-parity. Margin rates are too high and I'm docked by Reg-T. Use the Black-Scholes formula. If you know the current price, an options strike price, time until expiration, and risk-free interest rate, then knowing the market price of the option will tell you what the market's estimation of the volatility is. This does rely on a few assumptions, such as Gaussian random walk, but those are reasonable assumptions for most stocks. You can also get a list of past stock prices, put them in Excel, and ask Excel to calculate the standard deviation with stdev.s(), but that gives you the past volatility. The market's estimate of future volatility is more relevant. Variance of a single asset is defined as follows: σ2 = Σi(Xi - μ)2 where Xi's represent all the possible final market values of your asset and μ represents the mean of all such market values.  The portfolio's variance is defined as σp2 = Σiwi2σi2 where, σp is the portfolio's variance, and wi stands for the weight of the ith asset.  Now, if you include the borrowing in your portfolio, that would classify as technically shorting at the borrowing rate. Thus, this weight would (by the virtue of being negative) increase all other weights. Moreover, the variance of this is likely to be zero (assuming fixed borrowing rates). Thus, weights of risky assets rise and the investor's portfolio's variance will go up. Also see, CML at wikipedia. "When you have multiple assets available and a risk-free asset (cash or borrowing) you will always end up blending them if you have a reasonable objective function.  However, you seem to have constrained yourself to 100% investment.  Combine that with the fact that you are considering only two assets and you can easily have a solution where only one asset is desired in the portfolio.  The fact that you describe the US fund as ""dominating"" the forign fund indicates that this may be the case for you.  Ordinarily diversification benefits the overall portfolio even if one asset ""dominates"" another but it may not in your special case. Notice that these funds are both already highly diversified, so all you are getting is cross-border diversification by getting more than one.  That may be why you are getting the solution you are.  I've seen a lot of suggested allocations that have weights similar to what you are using. Finding an optimal portfolio given a vector of expected returns and a covariance matrix is very easy, with some reliable results.  Fancy models get pretty much the same kinds of answers as simple ones.  However, getting a good covariance matrix is hard and getting a good expected return vector is all but impossible.  Unfortunately portfolio results are very sensitive to these inputs.  For that reason, most of us use portfolio theory to guide our intuition, but seldom do the math for our own portfolio.  In any model you use, your weak link is the expected return and covariance.  More sophisticated models don't usually help produce a more reasonable result. For that reason, your original strategy (80-20) sounds pretty good to me.  Not sure why you are not diversifying outside of equities, but I suppose you have your reasons." Yes, it's a risk. To put it in perspective, If we look at the data for S&P returns since 1871, we get a CAGR of 10.72%. But, that comes with a SDev (Standard deviation) of 18.67%. This results in 53 of the 146 years returning less than 4%.  Now if we repeat the exercise over rolling 8 year periods, the CAGR drops to 9.22%, but the SDev drops to 5.74%. This results in just 31 of the 139 periods returning less than 4%. On the flip side, 26 periods had an 8 year return of over 15% CAGR.  From the anti-DS article you linked, I see that you like a good analogy. For me, the returns of the S&P over the long term are like going to Vegas, and finding that after you run the math of their craps (dice rolling game) you find the expected return is 10%. You can still lose on a given roll. But over a series of a larger number of rolls, you're far ahead.  To D Stanley - I agree that returns are not quite normal, but they are not so far off. Of the 139 rolling returns, we'd expect about 68% or 95 results to be 1 SDev away. We get 88 returns +/-1SDev. 2 SDevs? We'd expect only 5% to lie outside this range, and in fact, I only get one result on the low side and 4 on the high side, 5 results vs the 7 total we'd expect. The results are a bit better (more profitable) than the Normal Bell Curve fit would suggest. I  found that  an application already exists which does virtually everything I want to do with a reasonable interface.  Its called My Personal Index.  It has allowed me  to look at my asset allocation all in one place. I'll have to enter: The features which solve my problems above include:  Note - This is related to an earlier post I made regarding dollar cost averaging and determining rate of returns.   (I finally got off my duff and did something about it) &gt; One is whether prices are correlated to each other for long periods of time as a preliminary study suggested (which would go against efficient markets hypothesis, since you could use that info to game the market) or if that result is illusory and the long term returns are close to a standard normal distribution which would follow the effiecient markets hypo.  The fascinating thing about this is that the returns themselves show no correlation at all (at any time scale), but the *absolute* returns do.  i.e. following a sharp rise/drop in price, you can predict that a sharp rise/drop is likely to follow, you just can't say in which direction.  And this effect carries over for long periods.  Given that by the central limit theorem the sum of identically distributed random variables converges to a gaussian, leads one to think that short term returns *ought* to be gaussian also.  However, they're not.  Evidently there is something very subtle going on.