<jupyter_start><jupyter_text>Created this notebook to play around with synthetic data generation. I am using this source as my inspiration: https://www.youtube.com/watch?time_continue=187&v=VJBY2eVtf7o&feature=emb_logo&ab_channel=KeithGalliOnce I play with it I am using this for my tasks in FAD and also PAD project:Following steps in video I first import the modules that I need:<jupyter_code>#Importing modules: 

import numpy as np
import pandas as pd
import datetime 
import random

#now I create the columns using my example from task 4 in FAD tasks: 

# creating the Numpy array 

np.random.seed(42) 

# creating the dataframe 
womenapplied=pd.DataFrame({'Number of applicants':np.random.randint(0,10,size=30)})
womenaccepted=pd.DataFrame({'Accepted':np.random.randint(0,5,size=30)})

df=pd.DataFrame(womenapplied + womenaccepted)

#creating categorical variable for women and men applicants and accepted candidates

#source: https://pandas.pydata.org/pandas-docs/stable/user_guide/categorical.html?highlight=index%20values 
print(df) 

for i in range(50):
    Department_A= random.choice(list(Department_A.keys()))
    df.loc[i]=[i,Department_A]
    
    df.to_csv('test_data.csv')
#Trying the same using a np array ( not random module):

women_applied=np.random.randint(1,20, size=30)
women_accepted=np.random.randint(1,15, size=30)

men_applied=np.random.randint(1,10, size=30)

men_accepted=np.random.randint(1,20, size=30)

Data_women=women_applied, women_accepted
Data_men=men_applied,men_accepted



#Company employment divided by gender dataframe : 

df = pd.DataFrame((da women_applied, women_accepted, men_applied, men_accepted),
                   columns=['Women applied', 'Women accepted', 'Men applied', 'Men accepted'])
    
print(df)
# creating the dataframe 
df=pd.DataFrame({'Number of applicants':np.random.randint(0,10,size=25)})


  
# displaying the dataframe 

print(df)<jupyter_output>Number of applicants
0                      0
1                      1
2                      0
3                      4
4                      4
5                      6
6                      8
7                      8
8                      2
9                      2
10                     2
11                     3
12                     7
13                     5
14                     7
15                     0
16                     7
17                     3
18                     0
19                     7
20                     3
21                     5
22                     7
23                     3
24                     2<jupyter_text>***** Playing with synthetic data for task 4 , simpsons paradox: ***Following approach from this source now: https://easystats.github.io/correlation/reference/simulate_simpson.html<jupyter_code>%matplotlib inline
#I firt import the modules needed and I create my variables:


import numpy as np
import pandas as pd
import random
import matplotlib.pyplot as plt
import seaborn as sns 

#Creating Synthetic data 
np.random.seed(0)

a=np.random.randint(1,25,30)#number of observations for applied women group
b=np.random.randint(1,10,30)#number of observations for women accepted in the job

print(a)
print(b)



#Plotting this data


x=np.linspace( 1,30,num=30,endpoint=False)
y=a


plt.style.use('ggplot')

plt.plot(x, y)

plt.show()
#Using shuffle to modify 'a'and 'b'


a_s=np.random.shuffle(a)


print(a_s)

#Trying again.
#Now I am creating and plotting data for adults that tried a certain drug and got cured vs adults that were not cured. 


import numpy as np
import pandas as pd
import random
import matplotlib.pyplot as plt
import seaborn as sns 

np.random.seed(0)

a=np.random.randint(3,20,30)#number of observations of cured adults with drug A
b=np.random.randint(1,10,30)#number of observations of non cured adults with drug B
print(a)
print(b)



x = np.linspace(1, 30, 30)
y1=a
y2=b

sns.scatterplot(x,y1)



#plt.barchar(x, y1, y2)
#yfit = [a + b * xi for xi in x]
#plt.plot(x,y1,y2)
plt.show()

#Creating and plotting data from men that applied and were accepted in role: 

np.random.seed(0)

a=np.random.randint(3,15,30)#number of observations for applied women group
b=np.random.randint(5,26,30)#number of observations for women accepted in the job

x = np.linspace(1, 30, 30)
y1=a
y2=b


sns.regplot(x,y1 )


plt.show()


#Using multiple subplots to expose which number of adults got cured with drug A:

#setting variables: 


a=np.random.randint(5,25,100)#number of adults that were cured with Drug A
b=np.random.randint(5,68,100)#number of adults not cured with Drug A

#Plotting: 

x1=np.linspace(5, 30, 100)
x2=np.linspace(5, 30, 100)
y1=np.random.randint(5,25,100)
y2=np.random.randint(5,68,100)



fig, (ax1, ax2) = plt.subplots(2, 1)
fig.suptitle('Effects on adults that tried drug A')

ax1.plot(x1, y1, 'o-')
ax1.set_ylabel('Cured')

ax2.plot(x2, y2, '.-')
ax2.set_xlabel('Count')
ax2.set_ylabel('Non cured')

plt.show()
#Effects of the drug in children: 


#setting variables: 


a=np.random.randint(2,30,70)#number of children that were cured with Drug A
b=np.random.randint(7,15,40)#number of children not cured with Drug A

#Plotting: 

x1=np.linspace(5, 30, 100)
x2=np.linspace(5, 30, 100)
y1=np.random.randint(2,30,100)
y2=np.random.randint(7,15,100)



fig, (ax1, ax2) = plt.subplots(2, 1)
fig.suptitle('Effects on children that tried drug A')

ax1.plot(x1, y1, 'o-')
ax1.set_ylabel('Cured')

ax2.plot(x2, y2, '.-')
ax2.set_xlabel('Count')
ax2.set_ylabel('Non cured')

plt.show()
#combining both: 

a=np.random.randint(5,25,100)#Adults cured 
b=np.random.randint(2,30,100)#number of children that were cured with Drug A

#Plotting for adults cured compared with children cured: 

x=np.linspace(1, 30, 100)
y=a*x+b

sns.lmplot(x, y)


plt.show()<jupyter_output><empty_output><jupyter_text>Attempting this again:Now I am using an example from this source: https://www.youtube.com/watch?v=wgLUDw8eLB4&feature=emb_logo&ab_channel=singingbanana We have two drugs A and B and 100 people trying them during a testing time of 2 days. The first and second day drug B seem to have better results, however looking at the data from the two doys combined drugs A cures more people overall. Lets reproduce this and plot it using Python:<jupyter_code>#Creating the synthetic data based on this example: 

import numpy as np

#Data for day 1: 

#Drug A:

Drug_A=np.array([2,4,7,8,9,3,2,4,3,6,7,8,1,9,4,3,2,1,6,7])

print(Drug_A)

#Finding sum of Drug A array: 

Drug_Asum=np.sum(Drug_A)

print(Drug_Asum)

#Data for day 1: 

#Drug B: 

Drug_B=np.array([3,5,6,8,1,3,1,3,1,0])


print(Drug_B)

#Finding sum of Drug A array: 

Drug_Bsum=np.sum(Drug_B)

print(Drug_Bsum)

#Plotting both drugs results for day 1: 

import matplotlib.pyplot as plt


#Organizing data: 

x=np.linspace(1,2,100)

y1=Drug_A*x
y2=Drug_B*x



plt.bar(x,y1,y2)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10866715222141296, 'fraction_numerical': 0.049526584122359794, 'mean_word_length': 3.378826530612245, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '491598', 'license': 'MIT', 'n_tokens_mistral': 2736, 'n_tokens_neox': 2453, 'n_words': 698, 'path': 'Drafts/Playing with synthetic data creation.ipynb', 'repo_name': 'Ainara12/FundamentalsDA-tasks', 'size': '6865'}