<jupyter_start><jupyter_text>**Briefings**1. Two packages:*   https://www.statsmodels.org/dev/generated/statsmodels.sandbox.regression.gmm.IV2SLS.html*   https://bashtage.github.io/linearmodels/doc/iv/examples/advanced-examples.html2. Four variables:Dependent Variables: YEndogenous Variable: XControl variables: Winstruments: Z3. Two regressions*   First: X~Z*   Second: Y~(X+W)  background readings: https://cryptobriefing.com/earning-20-apy-with-yield-app/  Data Preparations  Date Sources https://docs.defipulse.com/overview-of-api-endpointshttps://data-beta.defipulse.com/dashboard/egshttps://docs.defipulse.com/api-docs-by-provider/defi-pulse-data/total-value-locked/total-value-locked  Import the Historical Data of Total Value Locked<jupyter_code>import pandas as pd
import requests
import json 
from datetime import datetime
TVL= requests.get('https://data-api.defipulse.com/api/v1/defipulse/api/GetHistory?resolution=history&project=all&api-key=10095a3e336c5beb50c0e30ef3ab8ebdcde3e0c08f65f3ae462f69a6902a')
TVL= TVL.json()
TVL=pd.DataFrame(TVL)
TVL.head()<jupyter_output><empty_output><jupyter_text>Epoch & Unix Timestamp Conversion Tools: https://www.epochconverter.com/<jupyter_code>import datetime
TVL['timestamp']=TVL['timestamp'].astype(int)
TVL.dtypes
TVL['time']=TVL['timestamp'].apply(lambda x: datetime.datetime.utcfromtimestamp(x).replace(tzinfo=datetime.timezone.utc))
TVL.head()
TVL['date'] = pd.to_datetime(TVL['time']).dt.date
TVL.head()
TVL=TVL.drop(columns='time')
TVL['date']=pd.to_datetime(TVL['date'])
TVL.dtypes<jupyter_output><empty_output><jupyter_text>Import the data for Ethereum Prices [Data Metrics](https://github.com/coinmetrics-io/data/blob/master/csv/metrics.csv)<jupyter_code>df_eth=pd.read_csv("https://raw.githubusercontent.com/coinmetrics-io/data/master/csv/eth.csv",usecols=(["time","PriceUSD","VtyDayRet30d","TxTfrValAdjUSD","AdrBalCnt","NVTAdj90","SplyCur"]))
df_eth['time']=pd.to_datetime(df_eth['time'])
df_eth.dtypes<jupyter_output><empty_output><jupyter_text>Merge the two Data Frame<jupyter_code>df =df_eth.merge(TVL, left_on='time', right_on='date')
df.head()
df=df[["date","tvlUSD","VtyDayRet30d","SplyCur","AdrBalCnt"]]<jupyter_output><empty_output><jupyter_text>Test Instrument ValiditiesY: "SplyCur"X: "VtyDayRet30d"Z: "SplyCur"W: "AdrBalCnt"<jupyter_code>df[["VtyDayRet30d","SplyCur"]].corr()<jupyter_output><empty_output><jupyter_text>Using the Linear Model Packagehttps://bashtage.github.io/linearmodels/doc/iv/index.html<jupyter_code>pip install linearmodels
from linearmodels import IV2SLS<jupyter_output><empty_output><jupyter_text>run an OLS<jupyter_code>df['const'] = 1
ivolsmod = IV2SLS(df.tvlUSD, df[["const","VtyDayRet30d","SplyCur","AdrBalCnt"]], None, None)
res_ols = ivolsmod.fit()
print(res_ols)<jupyter_output>OLS Estimation Summary                            
==============================================================================
Dep. Variable:                 tvlUSD   R-squared:                      0.7385
Estimator:                        OLS   Adj. R-squared:                 0.7380
No. Observations:                1423   F-statistic:                    1460.9
Date:                Sat, Jun 26 2021   P-value (F-stat)                0.0000
Time:                        07:25:18   Distribution:                  chi2(3)
Cov. Estimator:                robust                                         
                                                                              
                              Parameter Estimates                               
================================================================================
              Parameter  Std. Err.     T-stat    P-value    Lower CI    Upper CI
----------------------------------------------[...]<jupyter_text>run an IV2SLS<jupyter_code>ivmod = IV2SLS(df.tvlUSD, df[["const","AdrBalCnt"]], df["VtyDayRet30d"], df["SplyCur"])
res_2sls = ivmod.fit()
print(res_2sls.summary)<jupyter_output>IV-2SLS Estimation Summary                          
==============================================================================
Dep. Variable:                 tvlUSD   R-squared:                     -3.3208
Estimator:                    IV-2SLS   Adj. R-squared:                -3.3269
No. Observations:                1423   F-statistic:                    238.39
Date:                Sat, Jun 26 2021   P-value (F-stat)                0.0000
Time:                        07:25:18   Distribution:                  chi2(2)
Cov. Estimator:                robust                                         
                                                                              
                              Parameter Estimates                               
================================================================================
              Parameter  Std. Err.     T-stat    P-value    Lower CI    Upper CI
----------------------------------------------[...]<jupyter_text>comparison of the results<jupyter_code>from collections import OrderedDict
from linearmodels.iv.results import compare
res = OrderedDict()
res['OLS'] = res_ols
res['2SLS'] = res_2sls
print(compare(res))<jupyter_output>Model Comparison                
=================================================
                               OLS           2SLS
-------------------------------------------------
Dep. Variable               tvlUSD         tvlUSD
Estimator                      OLS        IV-2SLS
No. Observations              1423           1423
Cov. Est.                   robust         robust
R-squared                   0.7385        -3.3208
Adj. R-squared              0.7380        -3.3269
F-statistic                 1460.9         238.39
P-value (F-stat)            0.0000         0.0000
==================     ===========   ============
const                    7.003e+11     -1.154e+11
                          (27.919)      (-9.3888)
VtyDayRet30d             7.566e+10      1.911e+12
                          (5.7469)       (8.3447)
SplyCur                    -7595.4               
                         (-28.485)               
AdrBalCnt                   3672.8         856.60
[...]<jupyter_text>Testing endogeneity The Durbin test is a classic of endogeneity which compares OLS estimates with 2SLS and exploits the fact that OLS estimates will be relatively efficient. Durbin’s test is not robust to heteroskedasticity.<jupyter_code>res_2sls.durbin()<jupyter_output><empty_output><jupyter_text>Wooldridge’s regression-based test of exogeneity is robust to heteroskedasticity since it inherits the covariance estimator from the model. Here there is little difference.<jupyter_code>res_2sls.wooldridge_regression<jupyter_output><empty_output><jupyter_text>print first stage regression<jupyter_code>print(res_2sls.first_stage)<jupyter_output>First Stage Estimation Results    
=======================================
                           VtyDayRet30d
---------------------------------------
R-squared                        0.0589
Partial R-squared                0.0552
Shea's R-squared                 0.0552
Partial F-statistic              72.210
P-value (Partial F-stat)         0.0000
Partial F-stat Distn            chi2(1)
========================== ============
const                            0.4444
                               (9.6514)
AdrBalCnt                     1.534e-09
                               (8.0798)
SplyCur                      -4.138e-09
                              (-8.4976)
---------------------------------------

T-stats reported in parentheses
T-stats use same covariance type as original model
{'fraction_non_alphanumeric': 0.18667017220980675, 'fraction_numerical': 0.042329433416589984, 'mean_word_length': 2.0334928229665072, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 45, 'https://': 11, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '320663', 'license': 'MIT', 'n_tokens_mistral': 2472, 'n_tokens_neox': 2196, 'n_words': 381, 'path': '2SLS_section.ipynb', 'repo_name': 'JeJe620/NYU-SUMMER-2021', 'size': '7607'}