<jupyter_start><jupyter_text>Overcast CRE vs. cloud type as vertically stacked plots We plot the overcast CRE vs. cloud type. Overcast CRE means that each cloud type radiative effects is calculated for a hypothetical situation of 100% cloud cover for this type.  Load Standard and User Libraries<jupyter_code>%matplotlib inline

import os, sys, copy, glob
import numpy as np
import datetime
import seaborn as sns
import pylab as pl
import pandas as pd
import scipy.optimize

import nawdex_analysis.analysis.ave_cre
import nawdex_analysis.io.collector

import nawdex_analysis.plot.stacked_analysis_plots
import nawdex_analysis.plot.legend
reload( nawdex_analysis.plot.stacked_analysis_plots)

from nawdex_analysis.plot.stacked_analysis_plots import vert_stacked_exp_plot, get_exp_kws, get_plotting_order
pl.rcParams['figure.figsize'] = (16.0, 8.0)
pl.rcParams['font.size'] = 24.0
pl.rcParams['lines.linewidth'] = 3
import xarray as xr<jupyter_output><empty_output><jupyter_text>Read Radiation Fluxes This is a slight jump forward in time. Now, average radiation fluxes are available for different CT categories.<jupyter_code>rset= nawdex_analysis.io.collector.get_radflux4set( 'all', method = 'strict' )
#rset= nawdex_analysis.io.collector.get_radflux4set( 'all', method = 'all' )
#rset = nawdex_analysis.io.collector.get_radflux4set( 2 )<jupyter_output><empty_output><jupyter_text>Convert Flux to CRE (Scaling !!!!)<jupyter_code>nawdex_analysis.analysis.ave_cre.radflux2cre( rset, scaling = True, new_factor = 0.88, old_factor = 0.9 )
rset<jupyter_output><empty_output><jupyter_text>Adding fractional and very low clouds<jupyter_code>cset = rset[['afrac']].sel(ct = 'very low') + rset[['afrac']].sel(ct = 'fractional')
cset = cset.expand_dims('ct')
cset['ct'] = ['very low / fractional']

for vname in ['scre_ave', 'lcre_ave', 'net_cre_ave']:
    cset[vname] = ( rset[vname].sel(ct = 'very low') * rset['afrac'].sel( ct = 'very low') +
                    rset[vname].sel(ct = 'fractional') * rset['afrac'].sel( ct = 'fractional') ) / cset['afrac'] 

                    
#cset[['afrac', 'scre_ave', 'lcre_ave']]
cset
cset = xr.concat( [cset, rset[['afrac', 'scre_ave', 'lcre_ave', 'net_cre_ave']]], dim = 'ct' )<jupyter_output><empty_output><jupyter_text>Rename CT labels<jupyter_code>ctnames = cset.ct.data
ctnames[8] = 'semi. moderately thick'
ctnames[4] = 'mid-level'<jupyter_output><empty_output><jupyter_text>Get rid of data which contain too many NaNs The problems with the NaNs is that the average values and std of shortwave fluxes change strongly if some NaNs mask out a certain time at the day.We try to fit shortwave CREs with a cosine function and fillin Nans.  A Short Test Plot<jupyter_code>cset.sel(  ct = 'very high opaque', idname = '80km_twoMom_Conv' )['scre_ave'].plot()<jupyter_output>/vols/fs1/store/senf/.conda/python27mod/lib/python2.7/site-packages/pandas/plotting/_converter.py:129: FutureWarning: Using an implicitly registered datetime converter for a matplotlib plotting method. The converter was registered by pandas on import. Future versions of pandas will require you to explicitly register matplotlib converters.

To register the converters:
	>>> from pandas.plotting import register_matplotlib_converters
	>>> register_matplotlib_converters()
  warnings.warn(msg, FutureWarning)<jupyter_text>Functions for Data Interpolation<jupyter_code>def cos(x, a, xm, sig = 0.66):
    y = 2 * np.pi * ( x - xm) / sig
    
    f = a * (1 - np.cos(  y  ))
    m = np.logical_or( (y < 0) , (y > 2 * np.pi) )
    f[m] = 0
    return f
def cosfit_for_day( dset, date, ct = 'very high opaque', idname = '80km_twoMom_Conv' ):
    
    d = dset.sel( ct = ct, idname =  idname, time = date) 
    
    y = d['scre_ave'].data
    t = np.linspace( 0, 1, len(y))

    m = np.isfinite( y )
    tm = t[m]
    ym = y[m]
    
    popt, pcov = scipy.optimize.curve_fit( cos, tm, ym, p0 = [-250,0.6] )
    
    # print( popt)
    # pl.plot( t, y )
    
    y_update = y.copy()
    y_update[~m] = cos(t[~m], *popt)
    
    d['scre_ave'].data[:] = y_update
    return 
    

def cosfit_update(dset, ct = 'very high opaque', idname = '80km_twoMom_Conv' ):
    
    d_range = pd.date_range( '20160921', '20161005')

    for d in d_range:
        try:
            cosfit_for_day( dset, d.strftime('%Y%m%d'), ct = ct, idname = idname )
        except:
            print('no fit at %s' % d)
cosfit_update( cset, idname = '80km_twoMom_Conv' )
cosfit_update( cset, idname = '80km_oneMom_Conv' )<jupyter_output>no fit at 2016-09-26 00:00:00
no fit at 2016-09-27 00:00:00
no fit at 2016-09-28 00:00:00
no fit at 2016-09-29 00:00:00
no fit at 2016-10-01 00:00:00
no fit at 2016-10-02 00:00:00
no fit at 2016-09-26 00:00:00
no fit at 2016-09-27 00:00:00
no fit at 2016-09-28 00:00:00
no fit at 2016-09-29 00:00:00<jupyter_text>An Other Short Test Plot<jupyter_code>cset.sel(  ct = 'very high opaque', idname = '80km_twoMom_Conv' )['scre_ave'].plot()
#mask = cset.isnull().sum('time') < 10
#cset = cset.where(mask)
cset.isnull()<jupyter_output><empty_output><jupyter_text>Calculate Mean and STD<jupyter_code>catlist = list( cset.ct.data )

remove_list = ['clear_ocean', 'semi. above', 'very low', 'fractional']

for rm_cat in remove_list:
    catlist.remove(rm_cat)


idlist = list( cset.idname.data )
idlist.remove( 'msevi-not_scaled' )

dset = cset.sel(ct = catlist, idname = idlist)
daily_mean = dset.groupby('time.day').mean('time')
# dvar = daily_mean.quantile([0.25, 0.5, 0.75], dim = 'day').diff('quantile')
dvar = daily_mean.std('day')
mvar = dset.mean('time')
mvar['net_cre_ave'] = mvar['scre_ave'] + mvar['lcre_ave']
dvar_q = ( daily_mean.quantile(0.84, 'day') - daily_mean.quantile(0.16, 'day')  ) /2.<jupyter_output><empty_output><jupyter_text>Calculate Standard Error of Mean<jupyter_code>nday = daily_mean.dims['day']
var_sem = ( daily_mean.quantile(0.84, 'day') - daily_mean.quantile(0.16, 'day')  ) / np.sqrt( nday )<jupyter_output><empty_output><jupyter_text>Plotting<jupyter_code>mvar.ct.data = ['very low /\nfractional', 'low', 'mid-level', 'high opaque',
       'very high\nopaque', 'semi. thin', 'semi.\nmoderately thick',
       'semi. thick']
dvar.ct.data = mvar.ct.data
dvar_q.ct.data = mvar.ct.data
var_sem.ct.data = mvar.ct.data
grey_fontsize = 21
fac = 0.6


fig, ax = pl.subplots( ncols = 2, figsize = (30*fac, 35*fac), )

pl.sca(ax[0])
vert_stacked_exp_plot( mvar, 'scre_ave' , var_err = var_sem, doffset =0.06)
vert_stacked_exp_plot( mvar, 'lcre_ave' , var_err = var_sem, doffset =0.06)

xlim = pl.xlim()
pl.text( -100, -0.7, 'Short-\nwave', 
            color = 'gray', ha = 'center', va = 'center', alpha = 0.7, 
            fontweight = 'bold', fontsize = grey_fontsize)

pl.text( 80, -0.7, 'Long-\nwave', 
            color = 'gray', ha = 'center', va = 'center', alpha = 0.7, 
            fontweight = 'bold', fontsize = grey_fontsize)
pl.xticks([-100,100])

sns.despine(ax = ax[0])
pl.title('(a) Short- & Longwave')


pl.sca(ax[1])
vert_stacked_exp_plot( mvar, 'net_cre_ave' , var_err = var_sem, doffset =0.06)
sns.despine(left = True, ax = ax[1])
pl.title('     (b) Net')
pl.yticks([])
xlim = pl.xlim(-80, 60)

pl.text( -45, -0.8, 'Cooling', 
            color = 'gray', ha = 'center', va = 'center', alpha = 0.7, 
            fontweight = 'bold', fontsize = grey_fontsize)

pl.text( 50, -0.8, 'Warming', 
            color = 'gray', ha = 'center', va = 'center', alpha = 0.7, 
            fontweight = 'bold', fontsize = grey_fontsize)

#plegend()

for a in ax:
    a.axvline(0, color = 'gray', lw = 5, alpha = 0.3)
    a.set_xlabel('CRE ($\mathrm{W\, m^{-2}}$)')
    a.set_ylim(-1,None)

#nawdex_analysis.plot.legend.plegend( fontsize = 22,  bbox_to_anchor = (3.2, 0.5) )
pl.suptitle('Overcast CRE', fontsize = 'x-large', fontweight = 'bold')

pl.subplots_adjust( left = 0.3, right = 0.6, top = 0.91 )
pl.savefig( '../pics/cre_overcast-vs-ct.png', dpi = 300,)<jupyter_output><empty_output><jupyter_text>Make a Plot for the Legend This extra plot is made to get a legend that can be included in merged plots later.<jupyter_code>fig, ax = pl.subplots( ncols = 1, nrows = 1, figsize = (12, 8), )
pl.subplots_adjust(top = 0.9, bottom = 0.8)
vert_stacked_exp_plot( mvar, 'scre_ave' , var_err = dvar, doffset =0.06, catlist=['low'])
nawdex_analysis.plot.legend.plegend( fontsize = 16,  bbox_to_anchor = (1, -2.5), ncol = 3 )
pl.savefig('../pics/legend.png', dpi = 400)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.150795535502256, 'fraction_numerical': 0.04025172168131085, 'mean_word_length': 4.126597687157639, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 54, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '251748', 'license': 'BSD-3-Clause-Clear', 'n_tokens_mistral': 3590, 'n_tokens_neox': 3275, 'n_words': 785, 'path': 'nbooks/07-overcast_CRE-per-ct_plots.ipynb', 'repo_name': 'fsenf/nbook.CRE-2020-paper-plots', 'size': '8422'}