<jupyter_start><jupyter_text>Warsztaty Python w Data Science  Data Wrangling z Biblioteką Pandas  Część 1 z 3 - wprowadzenie  ***  https://github.com/MichalKorzycki/PythonDataScience ***  Data Wrangling 1- Data wrangling, munging, tidying- Biblioteka Pandas – wprowadzenie- Wybieranie wierszy- Formatowanie --- Zadanie 1 Posortować `dluga_lista` po wartości bezwględnej elementów listy interpretowanych jako wartość całkowita<jupyter_code>dluga_lista = [ str(i) for i in range(21)] + [ str(i) for i in range(-1, -21, -1)]
str(dluga_lista)
abs(-1)
abs('-1')
def bezwzgledna(x):
    x = float(x)
    return x if x > 0 else -x
bezwzgledna('-1')
dluga_lista.sort(key=bezwzgledna)
str(dluga_lista)
dluga_lista =  [ str(i) for i in range(-1, -21, -1)] + [ str(i) for i in range(21)]
str(dluga_lista)
dluga_lista.sort(key=bezwzgledna)
str(dluga_lista)
def bezwzgledna(x):
    x = float(x)
    return (x, x if x > 0 else -x)<jupyter_output><empty_output><jupyter_text>Zadanie 2 Wygenerować listę liczb z zakresu `1-1000` które nie są podzielne przez żadną cyfrę od `2` do `9`.<jupyter_code>[ (num,[div for div in range(2,10) if num%div == 0])  for num in range(1,21)]<jupyter_output><empty_output><jupyter_text>``` python[num for num in range(1,1001) if not [div for div in range(2,10) if num%div == 0]]```<jupyter_code>str([num for num in range(1,1001) if not [div for div in range(2,10) if num%div == 0]][:21])
str([num for num in range(1,1001) if not [div for div in range(2,10) if num%div == 0]][-20:])<jupyter_output><empty_output><jupyter_text>Zadanie 3 Stworz funkcję która usunie "wyplaszczy" zagnieżdzone listyl = [4, 1, [ 2, [7] ], 3]wyplaszcz(l)l[4, 1, 2, 7, 3]<jupyter_code>def wyplaszcz(lista):
    ret = []
    for x in lista:
        if type(x)==type([]):
            for y in wyplaszcz(x):
                ret.append(y)
        else:
            ret.append(x)
    return ret
l = [4, 1, [ 2, [7] ], 3]
l = wyplaszcz(l)
l
def wyplaszcz(lista):
    kopia = list(lista)
    for i in range(len(lista)-1,-1,-1):
        del lista[i]
    for x in kopia:
        if type(x)==type([]):
            for y in wyplaszcz(x):
                lista.append(y)
        else:
            lista.append(x)
    return lista
l = [4, 1, [ 2, [7] ], 3]
wyplaszcz(l)
l<jupyter_output><empty_output><jupyter_text>*** ![](img/road.jpg)  Data Wrangling, Munging, Tidying   Data Wrangling- __Discovering__ (eksploracja danych)- __Structuring__ (przygotowanie danych do konkretnego zadania)- __Cleaning__ (ujednolicenie danych, usuwanie danych niepotrzebnych itd.)- Enriching (łączenie danych)- Validating (sprawdzenie poprawności merytorycznej)- Publishing (publikacja wyników)źródło: https://www.onlinewhitepapers.com/information-technology/six-core-data-wrangling-activities/  Tidy Data Wickham, Hadley - _"Tidy Data"_https://www.jstatsoft.org/index.php/jss/article/view/v059i10/v59i10.pdf - __Each variable you measure should be in one column.__- __Each different observation of that variable should be in a different row.__- There should be one table for each "kind" of variable.- If you have multiple tables, they should include a column in the table that allows them to be linked.  Pandas https://pandas.pydata.org/Pandas_Cheat_Sheet.pdf<jupyter_code>import pandas
data = {'year': [2010, 2011, 2012, 2011, 2012, 2010, 2011, 2012],
        'team': ['Bears', 'Bears', 'Bears', 'Packers', 'Packers', 'Lions',
                 'Lions', 'Lions'],
        'wins': [11, 8, 10, 15, 11, 6, 10, 4],
        'losses': [5, 8, 6, 1, 5, 10, 6, 12]}
import pandas 

football = pandas.DataFrame(data)
print (football)
football
football.describe()
football.dtypes
football.head()
football.tail()
football.sample(5)
football['year']
football.year
football[['year', 'team', 'wins']]<jupyter_output><empty_output><jupyter_text>--- Row Selection Wybieranie wierszy1. Slicing2. Individual index (iloc / loc)3. Boolean indexing4. Combination  Slicing<jupyter_code>football
football[3:5]<jupyter_output><empty_output><jupyter_text>Individual index  Iloc- An integer, e.g. 5.- A list or array of integers, e.g. [4, 3, 0].- A slice object with ints, e.g. 1:7.- A boolean array.- A function<jupyter_code>football.iloc[[0,3]]<jupyter_output><empty_output><jupyter_text>Loc- A single label- A list or array of labels, e.g. ['a', 'b', 'c'].- A slice object with labels, e.g. 'a':'f' __(WARNING - both the start and the stop are included)__ - A boolean array- A callable function<jupyter_code>import numpy as np
import pandas as pd

index = pd.date_range('1/1/2000', periods=8)
df = pd.DataFrame(np.random.randn(8, 3), index=index, columns=['A', 'B', 'C'])
df
df.loc['2000-01-03']
df.loc['2000-01-03': '2000-01-04']<jupyter_output><empty_output><jupyter_text>Boolean indexing<jupyter_code>football[football.wins > 10]<jupyter_output><empty_output><jupyter_text>Combination<jupyter_code>football
football[(football.wins > 10) & (football.team == "Packers")]
football[(football.wins > 10) | (football.team == "Packers")]<jupyter_output><empty_output><jupyter_text>*** Formatowanie<jupyter_code>import numpy as np
import pandas as pd

index = pd.date_range('1/1/2000', periods=8)
df = pd.DataFrame(np.random.randn(8, 3), index=index, columns=['A', 'B', 'C'])
df
df.values
df.index
df.columns
newcols = []
for i in range(len(df.columns)):
    newcols.append(df.columns[i].lower())
df.columns = newcols
df
df.style
def color_negative_red(val):
    color = 'red' if val < 0 else 'white'
    return 'color: %s' % color
df.style.applymap(color_negative_red)
df.index = pd.to_datetime(df.index, format = '%Y-%m-%d').strftime('%Y-%m-%d')
df.style.applymap(color_negative_red)
def highlight_max(s):
    is_max = s == s.max()
    return ['background-color: blue' if v else '' for v in is_max]
df.style.apply(highlight_max)
df.style.apply(highlight_max, axis=1)
df.style.apply(highlight_max, axis=0)
df.style.apply(highlight_max, axis=0).applymap(color_negative_red)<jupyter_output><empty_output><jupyter_text>Nowe kolumny<jupyter_code>df.suma = df.a + df.b + df.c
df['suma'] = df.a + df.b + df.c
df<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14991762767710048, 'fraction_numerical': 0.03657331136738056, 'mean_word_length': 4.579963235294118, 'pattern_counts': {'":': 0, '<': 50, '<?xml version=': 0, '>': 54, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '157126', 'license': 'Unlicense', 'n_tokens_mistral': 2520, 'n_tokens_neox': 2366, 'n_words': 587, 'path': '03.Data_Wrangling_1z3.ipynb', 'repo_name': 'MichalKorzycki/PythonDataScience', 'size': '6070'}