<jupyter_start><jupyter_text>Forecasting Dividendsby [<NAME>](http://www.hvass-labs.org/)/ [GitHub](https://github.com/Hvass-Labs/FinanceOps) / [Videos on YouTube](https://www.youtube.com/playlist?list=PL9Hr9sNUjfsmlHaWuVxIA0pKL1yjryR0Z)  IntroductionIn a [previous paper](https://github.com/Hvass-Labs/FinanceOps/blob/master/01C_Theory_of_Long-Term_Stock_Forecasting.ipynb) we found a mathematical formula for doing long-term stock forecasting. The formula was derived from the definition of annualized return and separated the stock-return into 3 components: Dividends, change in the Sales Per Share, and change in the P/Sales ratio. If you can predict these 3 components, then you can predict the future stock-return.This paper is a basic statistical study of how to predict the future Dividends and Dividend Growth. [Another previous paper](https://github.com/Hvass-Labs/FinanceOps/blob/master/05_Forecasting_PSales_Ratio.ipynb) studied how to predict the future P/Sales ratio, and [yet another paper](https://github.com/Hvass-Labs/FinanceOps/blob/master/06_Forecasting_Sales_Growth.ipynb) studied how to predict the future Sales Growth.  Imports<jupyter_code>%matplotlib inline
import matplotlib.pyplot as plt
from IPython.display import display_jpeg

import pandas as pd
import numpy as np
import seaborn as sns
from scipy.stats import linregress
import statsmodels.api as sm

# SimFin imports.
import simfin as sf
from simfin.names import *
# Version of the SimFin Python API.
sf.__version__<jupyter_output><empty_output><jupyter_text>ConfigSetup and configure the various Python packages we are using.<jupyter_code># SimFin data-directory.
sf.set_data_dir('~/simfin_data/')
# SimFin load API key or use free data.
sf.load_api_key(path='~/simfin_api_key.txt', default_key='free')
# Seaborn set plotting style.
sns.set_style("whitegrid")<jupyter_output><empty_output><jupyter_text>Data-HubWe use [SimFin](https://github.com/SimFin/simfin) to easily load and process financial data with the following settings:<jupyter_code>hub_args = \
{
    # We are interested in the US stock-market.
    'market': 'us',

    # Use last-known values to fill in missing values.
    'fill_method': 'ffill',

    # Refresh the fundamental datasets (Income Statements etc.)
    # every 30 days.
    'refresh_days': 30,

    # Refresh the dataset with shareprices every 10 days.
    'refresh_days_shareprices': 10
}<jupyter_output><empty_output><jupyter_text>We can then create a `StockHub` object to handle all the data and signal processing:<jupyter_code>%%time
hub = sf.StockHub(**hub_args)<jupyter_output>CPU times: user 11 µs, sys: 2 µs, total: 13 µs
Wall time: 15.7 µs<jupyter_text>SignalsWe can now use the stock-hub to calculate the signals that we will be using in our analysis, such as Sales Growth, Net Profit Margin, etc.<jupyter_code>%%time
# Calculate Growth Signals.
# We set variant='quarterly' to get 4 data-points per year,
# but the data used to calculate the growth signals is TTM.
df_growth_signals = hub.growth_signals(variant='quarterly')

# Calculate Financial Signals. Also 4 data-points per year.
df_fin_signals = hub.fin_signals(variant='quarterly')

# Calculate the 1-year change in the Financial Signals.
df_fin_signals_chg = hub.fin_signals(variant='quarterly',
                                     func=sf.rel_change_ttm_1y)

# Rename columns for the 1-year changes.
def rename_chg(s):
    return s + ' (1Y Change)'
df_fin_signals_chg.rename(mapper=rename_chg, axis='columns', inplace=True)

# Combine all the signals we have calculated.
dfs = [df_growth_signals, df_fin_signals, df_fin_signals_chg]
df_signals1 = pd.concat(dfs, axis=1)

# Remove outliers using "Winsorization".
# The outliers are removed and not "clipped" because it distorts
# correlation-measures and line-fittings.
# We can also exclude some columns from the Winsorization.
exclude_columns = [LOG_REVENUE]
df_signals1 = sf.winsorize(df=df_signals1, clip=False,
                           exclude_columns=exclude_columns)<jupyter_output>Dataset "us-income-ttm" on disk (16 days old).
- Loading from disk ... Done!
Dataset "us-income-quarterly" on disk (16 days old).
- Loading from disk ... Done!
Dataset "us-balance-ttm" on disk (16 days old).
- Loading from disk ... Done!
Dataset "us-balance-quarterly" on disk (16 days old).
- Loading from disk ... Done!
Dataset "us-cashflow-ttm" on disk (16 days old).
- Loading from disk ... Done!
Dataset "us-cashflow-quarterly" on disk (16 days old).
- Loading from disk ... Done!
Cache-file 'growth_signals-52a73c0c.pickle' on disk (8 days old).
- Loading from disk ... Done!
Cache-file 'fin_signals-52a73c0c.pickle' on disk (8 days old).
- Loading from disk ... Done!
Cache-file 'fin_signals-6d9b5e29.pickle' on disk (8 days old).
- Loading from disk ... Done!
CPU times: user 3.56 s, sys: 157 ms, total: 3.72 s
Wall time: 3.71 s<jupyter_text>Extra SignalsLet us now create a new Pandas DataFrame for the extra signals that we need, which are not calculated by SimFin's built-in functions above. The new signals are:- `DIVIDEND_1Y_PAST` is the cash dividends paid in the PAST 1 year.- `DIVIDEND_1Y_FUTURE` is the cash dividends paid in the FUTURE 1 year.- `LOG_DIVIDEND_1Y_PAST` is the Log10 of `DIVIDEND_1Y_PAST`.- `LOG_DIVIDEND_1Y_FUTURE` is the Log10 of `DIVIDEND_1Y_FUTURE`.- `DIVIDEND_GROWTH_1Y_PAST` is the PAST 1-year growth in dividends.- `DIVIDEND_GROWTH_1Y_FUTURE` is the FUTURE 1-year growth in dividends.- `DIVIDEND_GROWTH_3Y_PAST` is the PAST 1-3 year average growth in dividends.- `DIVIDEND_GROWTH_3Y_FUTURE` is the FUTURE 1-3 year average growth in dividends.Some of these signals will be calculated using SimFin's functions `sf.rel_change` and `sf.mean_log_change`. See the SimFin documentation [here](https://simfin.readthedocs.io/en/latest/rel_change.html) and [here](https://github.com/SimFin/simfin-tutorials/blob/master/03_Growth_Returns.ipynb) for a detailed explanation of what these functions are calculating, as it would be too lengthy to explain it here.<jupyter_code>%%time
# Create a new DataFrame to hold the other signals.
# This will have the same index as the other DataFrame.
df_signals2 = pd.DataFrame(index=df_growth_signals.index)

# Load the Cash-Flow Statements TTM data.
df_cashflow_ttm = hub.load_cashflow(variant='ttm')

# Get the data for Cash Dividends Paid TTM.
# Note the negation because these amounts are negative.
df_dividend = -df_cashflow_ttm[DIVIDENDS_PAID]

# Cash Dividends Paid in the PAST year.
DIVIDEND_1Y_PAST = 'Dividend 1Y PAST'
df_signals2[DIVIDEND_1Y_PAST] = df_dividend

# Cash Dividends Paid in the FUTURE year.
DIVIDEND_1Y_FUTURE = 'Dividend 1Y FUTURE'
df_signals2[DIVIDEND_1Y_FUTURE] = \
    df_dividend.groupby(TICKER).shift(-4)

# Log10 of the Cash Dividends Paid in the PAST year.
LOG_DIVIDEND_1Y_PAST = 'Log Dividend 1Y PAST'
# Replace zeros with NaN to avoid inf-values.
df_div_nonzero = df_dividend.replace(0, np.nan)
df_log_div = np.log10(df_div_nonzero)
df_signals2[LOG_DIVIDEND_1Y_PAST] = df_log_div

# Log10 of the Cash Dividends Paid in the FUTURE year.
LOG_DIVIDEND_1Y_FUTURE = 'Log Dividend 1Y FUTURE'
df_signals2[LOG_DIVIDEND_1Y_FUTURE] = \
    df_log_div.groupby(TICKER).shift(-4)

# Calculate the PAST 1-year Dividend Growth.
DIVIDEND_GROWTH_1Y_PAST = 'Dividend Growth 1Y PAST'
df_signals2[DIVIDEND_GROWTH_1Y_PAST] = \
    sf.rel_change(df=df_div_nonzero, freq='ttm',
                  years=1, future=False)

# Calculate the FUTURE 1-year Dividend Growth.
DIVIDEND_GROWTH_1Y_FUTURE = 'Dividend Growth 1Y FUTURE'
df_signals2[DIVIDEND_GROWTH_1Y_FUTURE] = \
    sf.rel_change(df=df_div_nonzero, freq='ttm',
                  years=1, future=True)

# Calculate the PAST 3-year average Dividend Growth.
# These are ANNUALIZED growth-rates.
DIVIDEND_GROWTH_3Y_PAST = 'Dividend Growth 3Y Avg. PAST'
df_signals2[DIVIDEND_GROWTH_3Y_PAST] = \
    sf.mean_log_change(df=df_div_nonzero, freq='ttm', future=False,
                       min_years=1, max_years=3, annualized=True)

# Calculate the FUTURE 3-year average Dividend Growth.
# These are ANNUALIZED growth-rates.
DIVIDEND_GROWTH_3Y_FUTURE = 'Dividend Growth 3Y Avg. FUTURE'
df_signals2[DIVIDEND_GROWTH_3Y_FUTURE] = \
    sf.mean_log_change(df=df_div_nonzero, freq='ttm', future=True,
                       min_years=1, max_years=3, annualized=True)

# Limit / clip outliers for the Dividend Growth signals only.
columns = [DIVIDEND_GROWTH_1Y_PAST, DIVIDEND_GROWTH_1Y_FUTURE,
           DIVIDEND_GROWTH_3Y_PAST, DIVIDEND_GROWTH_3Y_FUTURE]
df_signals2 = sf.winsorize(df=df_signals2, clip=False,
                           columns=columns)

# Combine all the signals into one DataFrame.
dfs = [df_signals1, df_signals2]
df_signals = pd.concat(dfs, axis=1)<jupyter_output>/home/magnus/anaconda3/envs/simfin/lib/python3.6/site-packages/pandas/core/series.py:679: RuntimeWarning: invalid value encountered in log10
  result = getattr(ufunc, method)(*inputs, **kwargs)
/home/magnus/anaconda3/envs/simfin/lib/python3.6/site-packages/pandas/core/series.py:679: RuntimeWarning: invalid value encountered in log
  result = getattr(ufunc, method)(*inputs, **kwargs)<jupyter_text>These are the resulting signals for ticker MSFT:<jupyter_code>df_signals.loc['MSFT'].dropna(how='all').tail()<jupyter_output><empty_output><jupyter_text>Data YearsLet us see how many years of data we have for all the different stocks. First we need a small helper-function:<jupyter_code>def data_years(df):
    """
    Calculate the number of years of data in DataFrame `df`.
    
    :param df:
        Pandas DataFrame assumed to have TTM data and be
        grouped by TICKER, and not have any empty NaN rows.

    :return:
        Pandas Series with number of years for each TICKER.
    """

    # Count the number of data-points for each ticker.
    df_len_data = df.groupby(TICKER).apply(lambda df_grp: len(df_grp))

    # Calculate the number of years of data for each ticker.
    # TTM data has 4 data-points per year.
    df_data_years = df_len_data / 4
    
    return df_data_years<jupyter_output><empty_output><jupyter_text>On average we have about 6 years of Dividend TTM data for all these stocks:<jupyter_code># Calculate number of years of Sales Growth data for all stocks.
df = df_signals[DIVIDEND_1Y_PAST].dropna()
df_data_years = data_years(df=df)

# Show statistics.
df_data_years.describe()<jupyter_output><empty_output><jupyter_text>We can also plot a histogram, so we can see the distribution of how many years of data we have for all the individual companies:<jupyter_code>df_data_years.plot(kind='hist', bins=50);<jupyter_output><empty_output><jupyter_text>In some of the plots further below, we will compare e.g. the PAST 1-year Dividend Growth to the FUTURE 3-year average Dividend Growth. The summary statistics below shows that on average there is about 3.5 years of such data per company, for a total of about 600 companies:<jupyter_code>columns = [DIVIDEND_GROWTH_1Y_PAST, DIVIDEND_GROWTH_3Y_FUTURE]
df = df_signals[columns].dropna(how='any')
data_years(df=df).describe()<jupyter_output><empty_output><jupyter_text>Further below, we will also compare the PAST 3-year average Dividend Growth to the FUTURE 3-year average Dividend Growth. The statistics below show that there was only about 2.5 years of data-points per company, for a total of about 500 companies:<jupyter_code>columns = [DIVIDEND_GROWTH_3Y_PAST, DIVIDEND_GROWTH_3Y_FUTURE]
df = df_signals[columns].dropna(how='any')
data_years(df=df).describe()<jupyter_output><empty_output><jupyter_text>This is a fairly short data-period, because the SimFin database currently does not have any more data. What this means for our analysis, is that we should interpret the results with some caution, as the data may contain trends that are unique for that period in time.  Plotting FunctionsThese are small helper-functions for making plots.<jupyter_code>def plot_scatter(df, x, y, hue=None, num_samples=5000):
    """
    Make a scatter-plot using a random sub-sample of the data.
    
    :param df:
        Pandas DataFrame with columns named `x`, `y` and `hue`.

    :param x:
        String with column-name for the x-axis.

    :param y:
        String with column-name for the y-axis.

    :param hue:
        Either None or string with column-name for the hue.

    :param num_samples:
        Int with number of random samples for the scatter-plot.

    :return:
        matplotlib Axes object
    """

    # Select the relevant columns from the DataFrame.
    if hue is None:
        df = df[[x, y]].dropna()
    else:
        df = df[[x, y, hue]].dropna()

    # Only plot a random sample of the data-points?
    if num_samples is not None and len(df) > num_samples:
        idx = np.random.randint(len(df), size=num_samples)
        df = df.iloc[idx]

    # Ensure the plotting area is a square.
    plt.figure(figsize=(5,5))

    # Make the scatter-plot.
    ax = sns.scatterplot(x=x, y=y, hue=hue, s=20,
                         data=df.reset_index())

    # Move legend for the hue.
    if hue is not None:
        ax.legend(loc='center left', bbox_to_anchor=(1, 0.5), ncol=1)
    
    return ax
def plot_scatter_fit(df, x, y, num_samples=5000):
    """
    Make a scatter-plot and fit a line through the points.
    
    If there are many data-points, you can use a random
    sample for the scatter-plot, but the linear formula
    is still found using all the data-points.
    
    :param df:
        Pandas DataFrame with columns named `x` and `y`.

    :param x:
        String with column-name for the x-axis.

    :param y:
        String with column-name for the y-axis.

    :param num_samples:
        Int with number of random samples for the scatter-plot.

    :return:
        matplotlib Axes object
    """
    
    # Select the relevant columns from the DataFrame.
    df = df[[x, y]].dropna(how='any').reset_index()

    # Fit a line through all the data-points and get stats.
    slope, intercept, r_value, p_value, std_err = \
        linregress(x=df[x], y=df[y])

    # Show the fitted line and its stats.
    msg = 'y = {0:.2f} * x + {1:.2f} (R^2={2:.2f}, p={3:.0e})'
    msg = msg.format(slope, intercept, r_value**2, p_value)
    print(msg)
    
    # Only plot a random sample of the data-points?
    if num_samples is not None and len(df) > num_samples:
        idx = np.random.randint(len(df), size=num_samples)
        df = df.iloc[idx]

    # Make the scatter-plot with a fitted line.
    # This uses the smaller sample of data-points.
    ax = sns.jointplot(x=x, y=y, kind='reg', data=df,
                       line_kws={'color': 'red'},
                       scatter_kws={'s': 2})

    return ax<jupyter_output><empty_output><jupyter_text>Regression FunctionThis is a small helper-function for doing multiple linear regression.<jupyter_code>def regression(df, y, standardize=True, use_constant=True):
    """
    Perform multiple linear-regression on the given DataFrame.
    
    :param df:
        Pandas DataFrame with signals and returns.
        
    :param y:
        String with column-name for the dependent variable.
        This will be taken from the DataFrame `df`.
        
    :param standardize:
        Boolean whether to standardize the predictor variables
        so they have 0 mean and 1 standard deviation.

    :param use_constant:
        Boolean whether to add a 'Constant' column to
        find the bias.
    
    :return:
        StatsModels Regression Results.
    """
    
    # Remove rows with missing values.
    df = df.dropna(how='any').copy()

    # DataFrame for the x-signals.
    df_x = df.drop(columns=[y])
    
    # DataFrame for the y-signal.
    df_y = df[y]

    # Standardize the signals so they have mean 0 and std 1.
    if standardize:
        df_x = (df_x - df_x.mean()) / df_x.std()

    # Add a "constant" column so the regression can find the bias.
    if use_constant:
        df_x['Constant'] = 1.0

    # Perform the regression on this data.
    model = sm.OLS(df_y, df_x).fit()
    
    return model<jupyter_output><empty_output><jupyter_text>Past vs. Future DividendsLet us first compare the PAST and FUTURE Dividends. The first scatter-plot shows the raw cash amounts in USD. This fits a straight line with $R^2$ nearly 1, so it is a really good fit. But it has a tremendous range of values between zero and 14b USD, and the majority of points are concentrated in a small value-range.<jupyter_code>plot_scatter_fit(df=df_signals,
                 x=DIVIDEND_1Y_PAST,
                 y=DIVIDEND_1Y_FUTURE);<jupyter_output>y = 1.05 * x + 13152720.73 (R^2=0.98, p=0e+00)<jupyter_text>Let us instead try and plot the Log10 dividend amounts. This effectively shows the "number of zeros" so that 1 million becomes the value 6 because it has 6 zeros, and 1 billion becomes the value 9 because it has 9 zeros, etc.This creates a more narrow range of values for the dividend amounts. But the line-fit is still nearly diagonal with $R^2$ near 1. This shows that the dividend amount tends to be almost the same from one year to the next, with only minor variation.<jupyter_code>plot_scatter_fit(df=df_signals,
                 x=LOG_DIVIDEND_1Y_PAST,
                 y=LOG_DIVIDEND_1Y_FUTURE);<jupyter_output>y = 0.98 * x + 0.21 (R^2=0.94, p=0e+00)<jupyter_text>Dividend Growth StatisticsBefore we investigate whether some of the signals can be used to predict FUTURE Dividend Growth, let us consider some basic statistics. This shows that on average, the dividends have grown about 10.2% per year. Note that these are the total Cash Dividends Paid from the company's Cash-Flow Statement, it is NOT per-share numbers!<jupyter_code>df_signals[DIVIDEND_GROWTH_1Y_FUTURE].describe()<jupyter_output><empty_output><jupyter_text>We can also show the distribution of 1-year Dividend Growth in a histogram:<jupyter_code>df_signals[DIVIDEND_GROWTH_1Y_FUTURE].hist(bins=100);<jupyter_output><empty_output><jupyter_text>We can also show the basic statistics for 3-year average Dividend Growth. These are annualized numbers and show that on average the dividend grew about 10.3% when considering 3-year periods.<jupyter_code>df_signals[DIVIDEND_GROWTH_3Y_FUTURE].describe()<jupyter_output><empty_output><jupyter_text>Once again we can show the distribution in a histogram:<jupyter_code>df_signals[DIVIDEND_GROWTH_3Y_FUTURE].hist(bins=100);<jupyter_output><empty_output><jupyter_text>Signal CorrelationLet us now investigate if any of the signals can be used to predict the FUTURE Dividend Growth for a company. First we calculate the linear correlations between the various signals and the FUTURE Dividend Growth.<jupyter_code>%%time
df_corr = df_signals.corr()

# New column names.
SIGNALS_1Y = '1-Year Dividend Growth'
SIGNALS_3Y = '3-Year Avg. Dividend Growth'

# Create a new DataFrame with the correlations.
data = \
{
    SIGNALS_1Y: df_corr[DIVIDEND_GROWTH_1Y_FUTURE],
    SIGNALS_3Y: df_corr[DIVIDEND_GROWTH_3Y_FUTURE]
}
df = pd.DataFrame(data=data)<jupyter_output>CPU times: user 428 ms, sys: 0 ns, total: 428 ms
Wall time: 430 ms<jupyter_text>We can then show the correlations between the various signals and the FUTURE 1-year Dividend Growth and the FUTURE 3-year average Dividend Growth.A correlation coefficient of 1 means the correlation is perfect so the two variables always move together, while a correlation of 0 means there is no linear relation between the two variables, and a correlation of -1 means the two variables always move perfectly opposite to each other.We will show the absolute correlation values, because we are only concerned about the strength of the correlation and not its direction here.<jupyter_code>df.abs().sort_values(by=SIGNALS_3Y, ascending=False)<jupyter_output><empty_output><jupyter_text>Let us try and fit a Linear Regression Model to some of the signals with highest correlation. The reason we don't fit the regression model to all the signals, is that a lot of them contain NaN (Not-a-Number), which results in the entire rows with NaN being removed, so the dataset becomes much smaller, and the correlation numbers already show that many of the signals are not linearly related to the FUTURE Dividend Growth anyway.The regression model has $R^2 = 0.13$ which is quite weak. Because the data is standardized to having zero mean and one standard deviation before the regression model is fitted, the coefficients show us which signals are most important in predicting the FUTURE 1-year Dividend Growth, and that is by far the PAST 3-year average Dividend Growth, followed by the PAST 1-year Dividend Growth.<jupyter_code># Only use some of the signals in the regression.
columns = [DIVIDEND_GROWTH_1Y_FUTURE, DIVIDEND_GROWTH_1Y_PAST,
           DIVIDEND_GROWTH_3Y_PAST, SALES_GROWTH,
           PAYOUT_RATIO, ASSETS_GROWTH_YOY]

# Fit the regression model.
model = regression(df=df_signals[columns],
                   y=DIVIDEND_GROWTH_1Y_FUTURE, standardize=True)

# Show the results.
model.summary()<jupyter_output><empty_output><jupyter_text>Past vs. Future Dividend GrowthLet us now study whether the PAST Dividend Growth can be used to predict the FUTURE Dividend Growth.First compare the PAST and FUTURE 1-year Dividend Growth. The fitted line in this scatter-plot shows a positive correlation with $R^2 = 0.09$. Note that the plot only contains 5000 data-points that are randomly sampled from the full dataset of more than 13,000 points. But the linear formula printed above the plot is fitted to the entire dataset.Each dot in the scatter-plot shows a single data-point, that is, the x-axis is the 1-year PAST Dividend Growth for a single company and one of its financial reports. The y-axis shows the 1-year Dividend Growth starting in the same quarter and going 1 year into the future.<jupyter_code>plot_scatter_fit(df=df_signals,
                 x=DIVIDEND_GROWTH_1Y_PAST,
                 y=DIVIDEND_GROWTH_1Y_FUTURE);<jupyter_output>y = 0.28 * x + 0.06 (R^2=0.09, p=8e-298)<jupyter_text>Let us now compare the PAST 3-year average Dividend Growth to the FUTURE 3-year average Dividend Growth. The plot looks quite similar to the one above for 1-year Dividend Growth. Once again there is a positive correlation and now with slightly lower $R^2 = 0.07$.<jupyter_code>plot_scatter_fit(df=df_signals,
                 x=DIVIDEND_GROWTH_3Y_PAST,
                 y=DIVIDEND_GROWTH_3Y_FUTURE);<jupyter_output>y = 0.27 * x + 0.06 (R^2=0.07, p=6e-80)<jupyter_text>Assets Growth YOYLet us now consider if the Year-Over-Year (YOY) Assets Growth can predict the FUTURE 1-year Dividend Growth. This does not work very well, as the fitted line only has $R^2 = 0.03$.<jupyter_code>plot_scatter_fit(df=df_signals,
                 x=ASSETS_GROWTH_YOY,
                 y=DIVIDEND_GROWTH_1Y_FUTURE);<jupyter_output>y = 0.19 * x + 0.09 (R^2=0.03, p=4e-90)<jupyter_text>We can also try and see if the YOY Assets Growth can predict the FUTURE 3-year average Dividend Growth. Although there is a positive correlation, once again we get a very low $R^2$ near zero.<jupyter_code>plot_scatter_fit(df=df_signals,
                 x=ASSETS_GROWTH_YOY,
                 y=DIVIDEND_GROWTH_3Y_FUTURE);<jupyter_output>y = 0.15 * x + 0.09 (R^2=0.03, p=4e-62)<jupyter_text>However, if we zoom in on the part of the plot where the YOY Assets Growth is greater than 0.2 (or 20%), we see that most FUTURE 1-year average Dividend Growths are greater than zero. This might be useful knowledge.<jupyter_code># Only use some of the data-points.
mask = (df_signals[ASSETS_GROWTH_YOY] > 0.2)
df = df_signals.loc[mask]

plot_scatter_fit(df=df,
                 x=ASSETS_GROWTH_YOY,
                 y=DIVIDEND_GROWTH_1Y_FUTURE);<jupyter_output>y = -0.04 * x + 0.15 (R^2=0.00, p=3e-01)<jupyter_text>We get a very similar plot when we compare the YOY Assets Growth that are greater than 0.2 to the FUTURE 3-year average Dividend Growth.<jupyter_code>plot_scatter_fit(df=df,
                 x=ASSETS_GROWTH_YOY,
                 y=DIVIDEND_GROWTH_3Y_FUTURE);<jupyter_output>y = -0.03 * x + 0.14 (R^2=0.00, p=4e-01)<jupyter_text>Sales GrowthLet us now consider if the TTM Sales Growth can predict the FUTURE 1-year Dividend Growth. There is a positive correlation but the fitted line only has $R^2 = 0.03$.<jupyter_code>plot_scatter_fit(df=df_signals,
                 x=SALES_GROWTH,
                 y=DIVIDEND_GROWTH_1Y_FUTURE);<jupyter_output>y = 0.24 * x + 0.09 (R^2=0.03, p=2e-102)<jupyter_text>We can also try and see if the TTM Sales Growth can predict the FUTURE 3-year average Dividend Growth. Once again there is a positive correlation but we get a very low $R^2$ near zero.<jupyter_code>plot_scatter_fit(df=df_signals,
                 x=SALES_GROWTH,
                 y=DIVIDEND_GROWTH_3Y_FUTURE);<jupyter_output>y = 0.21 * x + 0.09 (R^2=0.05, p=9e-98)<jupyter_text>However, if we zoom in on the part of the plot where the TTM Sales Growth is greater than 0.1 (or 10%), we see that most FUTURE 1-year Dividend Growths are significantly greater than zero. This might be useful knowledge.<jupyter_code># Only use data-points with positive Sales Growth.
mask = (df_signals[SALES_GROWTH] > 0.1)
df = df_signals.loc[mask]

plot_scatter_fit(df=df,
                 x=SALES_GROWTH,
                 y=DIVIDEND_GROWTH_1Y_FUTURE);<jupyter_output>y = 0.04 * x + 0.13 (R^2=0.00, p=2e-01)<jupyter_text>We get a similar plot when we compare the positive TTM Sales Growth to the FUTURE 3-year average Dividend Growth.<jupyter_code>plot_scatter_fit(df=df,
                 x=SALES_GROWTH,
                 y=DIVIDEND_GROWTH_3Y_FUTURE);<jupyter_output>y = 0.01 * x + 0.13 (R^2=0.00, p=8e-01)<jupyter_text>Dividend Payout RatioThe Dividend Payout Ratio measures how much of the company's earnings are paid out as dividends to shareholders. We use a slightly different formula for the Payout Ratio than people normally use, as we use the Free Cash Flow instead of the reported Net Income:$$Payout\ Ratio = \frac{Dividends\ Paid\ TTM}{Free\ Cash\ Flow\ TTM} = \frac{Dividends\ Paid\ TTM}{Net\ Cash\ From\ Operations\ TTM - Capital\ Expenditures\ TTM}$$The scatter-plot shows a tendency for higher Dividend Payout Ratios to result in lower FUTURE 3-year average Dividend Growth, although the $R^2$ is nearly zero because the data-points are so dispersed.<jupyter_code>plot_scatter_fit(df=df_signals,
                 x=PAYOUT_RATIO,
                 y=DIVIDEND_GROWTH_3Y_FUTURE);<jupyter_output>y = -0.09 * x + 0.14 (R^2=0.03, p=1e-67)
{'fraction_non_alphanumeric': 0.08415598169371005, 'fraction_numerical': 0.021105185521388858, 'mean_word_length': 4.001891789632992, 'pattern_counts': {'":': 0, '<': 121, '<?xml version=': 0, '>': 125, 'https://': 8, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '34', 'dir': 'jupyter-structured-clean-dedup', 'id': '140860', 'license': 'MIT', 'n_tokens_mistral': 9421, 'n_tokens_neox': 8826, 'n_words': 2878, 'path': '07_Forecasting_Dividends.ipynb', 'repo_name': 'planetceres/FinanceOps', 'size': '26459'}