<jupyter_start><jupyter_text>* 请在环境变量中设置`DB_URI`指向数据库* 请在环境变量中设置`DATAYES_TOKEN`作为通联数据登陆凭证<jupyter_code>%matplotlib inline
import os
from matplotlib import pyplot as plt
import uqer
import numpy as np
import pandas as pd
from uqer import DataAPI as api
from alphamind.api import *
from alphamind.data.neutralize import neutralize

plt.style.use('ggplot')
_ = uqer.Client(token=os.environ['DATAYES_TOKEN'])
ref_date = '2017-06-23'
factor = 'EPS'

engine = SqlEngine(os.environ['DB_URI'])
universe = Universe('zz800')<jupyter_output><empty_output><jupyter_text>Algorithm Description-------------------------- 猜测的 ``neutralize`` 残差$\bar Res$计算公式：$$\bar Res_{i,k} = \bar f_{i,k} - \sum_j \beta_{j,k} \times \bar Ex_{i, j, k}$$其中：$k$为行业分类，$i$为该行业中第$i$只股票，$j$为第$j$个风险因子。$\bar f$为因子序列，$\bar Ex$为风险暴露矩阵。系数$\beta_{j,k}$由OLS确定。下面的章节，我们分别比较三种``neutralize``的方法差别：* **UQER Neutralize**    使用优矿的SDK计算因子残差。* **Alpha-Mind Neutralize**    使用alpha-mind计算因子残差，alpha-mind可以由以下地址安装：        ```    https://github.com/wegamekinglc/alpha-mind    ```* **Direct Weighted Least Square Fit Implementation**    直接使用scikit-learn的线性回归功能来计算因子残差。  Raw Data---------------------------<jupyter_code>codes = engine.fetch_codes(ref_date, universe)
factor_data = engine.fetch_factor(ref_date, factor, codes)
risk_cov, risk_expousre = engine.fetch_risk_model(ref_date, codes)
total_data = pd.merge(factor_data, risk_expousre, on=['code']).dropna()
total_data['ticker'] = total_data.code.apply(lambda x: '{0:06}'.format(x))
total_data.set_index('ticker', inplace=True)
len(total_data)<jupyter_output><empty_output><jupyter_text>UQER Neutralize-----------------------<jupyter_code>%%timeit
neutralized_factor_uqer = uqer.neutralize(total_data[factor],
                                          target_date=ref_date.replace('-', ''),
                                          industry_type='short')
neutralized_factor_uqer = uqer.neutralize(total_data[factor],
                                          target_date=ref_date.replace('-', ''),
                                          industry_type='short')
df = pd.DataFrame(neutralized_factor_uqer, columns=['uqer'])
df.head(10)
len(neutralized_factor_uqer)
risk_exposure_uqer = uqer.DataAPI.RMExposureDayGet(tradeDate=ref_date.replace('-', '')).set_index('ticker')
targeted_secs = risk_exposure_uqer.loc[neutralized_factor_uqer.index]

style_exposure = neutralized_factor_uqer.values @ targeted_secs[risk_styles].values
industry_exposure = neutralized_factor_uqer.values @ targeted_secs[industry_styles].values

exposure = pd.Series(np.concatenate([style_exposure, industry_exposure]), index=risk_styles+industry_styles)
exposure<jupyter_output><empty_output><jupyter_text>Alpha-Mind Neutralize--------------------------<jupyter_code>x = total_data[risk_styles + industry_styles].values
y = total_data[factor].values
%%timeit
neutralized_factor_alphamind = neutralize(x, y, weights=np.ones(len(y)))
neutralized_factor_alphamind = neutralize(x, y, weights=np.ones(len(y)))
alphamind_series = pd.Series(neutralized_factor_alphamind.flatten(), index=total_data.index)
df['alpha-mind'] = alphamind_series
df.head()
len(alphamind_series)<jupyter_output><empty_output><jupyter_text>The Ticker Missing in UQER but Still in Alpha-Mind-----------------------------------<jupyter_code>missed_codes = [c for c in alphamind_series.index if c not in neutralized_factor_uqer.index]
total_data.loc[missed_codes]<jupyter_output><empty_output><jupyter_text>Direct Weighted Least Square Fit Implementation------------------------<jupyter_code>import statsmodels.api as sm
mod = sm.WLS(y, x, weights=np.ones(len(y))).fit()
lg_series = pd.Series(mod.resid, index=total_data.index)
df['ols'] = lg_series<jupyter_output><empty_output><jupyter_text>Comparison------------------<jupyter_code>df['uqer - ols'] = df['uqer'] - df['ols']
df['alphamind - ols'] = df['alpha-mind'] - df['ols']
df[['uqer - ols', 'alphamind - ols']].plot(figsize=(14, 7), ylim=(-1e-4, 1e-4))
df.head()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.17764971185166625, 'fraction_numerical': 0.005762966675018792, 'mean_word_length': 4.575418994413408, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '89236', 'license': 'MIT', 'n_tokens_mistral': 1559, 'n_tokens_neox': 1534, 'n_words': 201, 'path': 'notebooks/Example 8 - Neutralize Algorithm Comparison.ipynb', 'repo_name': 'iLampard/alpha-mind', 'size': '3991'}