<jupyter_start><jupyter_text>Data reading<jupyter_code>import pandas as pd 
import matplotlib.pyplot as plt

data = pd.read_csv('data/titanic_train.csv')
data.head(2)
data.describe()
data.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 891 entries, 0 to 890
Data columns (total 12 columns):
PassengerId    891 non-null int64
Survived       891 non-null int64
Pclass         891 non-null int64
Name           891 non-null object
Sex            891 non-null object
Age            714 non-null float64
SibSp          891 non-null int64
Parch          891 non-null int64
Ticket         891 non-null object
Fare           891 non-null float64
Cabin          204 non-null object
Embarked       889 non-null object
dtypes: float64(2), int64(5), object(5)
memory usage: 83.6+ KB<jupyter_text>Series and dataframesSeries, one-dimensional arrays of data. They are like lists, but have another behavior.<jupyter_code>series = pd.Series([1,2,3])
print(series)
print(series * 2)<jupyter_output>0    1
1    2
2    3
dtype: int64
0    2
1    4
2    6
dtype: int64<jupyter_text>The first column is id<jupyter_code>print(series[0])<jupyter_output>1<jupyter_text>You can use another values for id<jupyter_code>ids = [1,2,3,4]
values = [2,3,4,5]

data = pd.Series(data=values, index=ids)
data<jupyter_output><empty_output><jupyter_text>Dataframes are tables. They have rows, columns and cells. Columns in dataframes are series.<jupyter_code>info = {
    'first': [1,2,3,4],
    'second': [5,6,7,8],
    'third': [9,10,11,12]
}

df = pd.DataFrame(data=info, index=ids)
df
df_example = [
    {'a': 1, 'b': 2},
    {'a': 1, 'b': 2},
    {'a': 1, 'b': 2}
]

type(df_example)
df_test = pd.DataFrame(df_example)
print(df_test)
print(type(df_test))<jupyter_output>a  b
0  1  2
1  1  2
2  1  2
<class 'pandas.core.frame.DataFrame'><jupyter_text>Uploaded data researching<jupyter_code>data = pd.read_csv('data/titanic_train.csv')
data.head()<jupyter_output><empty_output><jupyter_text>Atributes<jupyter_code>print("Shape: " + str(data.shape) + '\n')
print("Columns: " + str(data.columns) + '\n')
print("Indexies: " + str(data.index) + '\n')
print("Data types: " + str(data.dtypes) + '\n')
print("Data description: " + str(data.describe))<jupyter_output>Shape: (891, 12)

Columns: Index(['PassengerId', 'Survived', 'Pclass', 'Name', 'Sex', 'Age', 'SibSp',
       'Parch', 'Ticket', 'Fare', 'Cabin', 'Embarked'],
      dtype='object')

Indexies: RangeIndex(start=0, stop=891, step=1)

Data types: PassengerId      int64
Survived         int64
Pclass           int64
Name            object
Sex             object
Age            float64
SibSp            int64
Parch            int64
Ticket          object
Fare           float64
Cabin           object
Embarked        object
dtype: object

Data description: <bound method NDFrame.describe of      PassengerId  Survived  Pclass  \
0              1         0       3   
1              2         1       1   
2              3         1       3   
3              4         1       1   
4              5         0       3   
5              6         0       3   
6              7         0       1   
7              8         0       3   
8              9         1       3   
9             10         1       2 [...]<jupyter_text>Only columns and rows<jupyter_code>print(data['Age'].head(2))
print(data[['Sex', 'Age']].head(2))<jupyter_output>Sex   Age
0    male  22.0
1  female  38.0<jupyter_text>Filters<jupyter_code>show_data = [3, 4, 5]
data.loc[show_data]<jupyter_output><empty_output><jupyter_text>Conditions<jupyter_code>filter_cond = data['Age'] > 70.0
data.loc[filter_cond]<jupyter_output><empty_output><jupyter_text>Queries<jupyter_code>data.query('Age > 60 & Survived == 1')<jupyter_output><empty_output><jupyter_text>Metrics<jupyter_code>res = ((data['Survived'] == 1).sum()/((data['Survived'] == 0).sum() + (data['Survived'] == 1).sum())) * 100
print(str(res.round(1)) + '% is survived')
data.groupby('Sex')['Survived'].sum()<jupyter_output><empty_output><jupyter_text>Null data<jupyter_code>data.dropna(axis=1, thresh=1).head(2)
data['Pclass'] = data['Pclass'].astype(str)
data.dtypes<jupyter_output><empty_output><jupyter_text>Operations with columns<jupyter_code>data.columns
data.columns[:-1]
data[data.columns[:-1]].head()<jupyter_output><empty_output><jupyter_text>iloc - это выбор столбцов по позициям<jupyter_code>data.iloc[:, :-1].head(2)
# we don't touch the rows, but we choose all columns despite the last one<jupyter_output><empty_output><jupyter_text>loc - именной выбор<jupyter_code>data.loc[:, data.columns[:-1]].head(2)
# take all despite the last one
data.drop(['Cabin'], axis=1).head(2)
# axis=0 for rows, 1 - for columns<jupyter_output><empty_output><jupyter_text>Missing dataLook at the missings in our data<jupyter_code>data.isnull().any().any()<jupyter_output><empty_output><jupyter_text>We don't have missings. In this columns we have missings (where is true)<jupyter_code>data.isnull().any() # define: columns
?pd.isnull<jupyter_output>[1;31mSignature:[0m [0mpd[0m[1;33m.[0m[0misnull[0m[1;33m([0m[0mobj[0m[1;33m)[0m[1;33m[0m[0m
[1;31mDocstring:[0m
Detect missing values for an array-like object.

This function takes a scalar or array-like object and indicates
whether values are missing (``NaN`` in numeric arrays, ``None`` or ``NaN``
in object arrays, ``NaT`` in datetimelike).

Parameters
----------
obj : scalar or array-like
    Object to check for null or missing values.

Returns
-------
bool or array-like of bool
    For scalar input, returns a scalar boolean.
    For array input, returns an array of boolean indicating whether each
    corresponding element is missing.

See Also
--------
notna : Boolean inverse of pandas.isna.
Series.isna : Detect missing values in a Series.
DataFrame.isna : Detect missing values in a DataFrame.
Index.isna : Detect missing values in an Index.

Examples
--------
Scalar arguments (including strings) result in a scalar boolean.

>>> pd.isna('dog')
False

>>> pd.isna(np[...]<jupyter_text>ConditionsUnique values of the column<jupyter_code>data['Age'].unique()<jupyter_output><empty_output><jupyter_text>Number of unique values<jupyter_code>data['Age'].nunique()
data['Sex'].value_counts()
data[(data['Age'] == 7) & (data['Sex'] == 'female')].head()
data[(data['Age'] == 50) & (data['Sex'] == 'female')]['Pclass'].value_counts()<jupyter_output><empty_output><jupyter_text>Visualization<jupyter_code>data['Age'].hist() # distribution of ages in dataset<jupyter_output><empty_output><jupyter_text>average number<jupyter_code>data['Age'].mean()<jupyter_output><empty_output><jupyter_text>Groups<jupyter_code>data_class = data.groupby('Pclass')
data_class.describe()
data_class.mean()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14336492890995262, 'fraction_numerical': 0.039691943127962086, 'mean_word_length': 3.816690442225392, 'pattern_counts': {'":': 0, '<': 89, '<?xml version=': 0, '>': 96, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '572074', 'license': 'MIT', 'n_tokens_mistral': 2591, 'n_tokens_neox': 2441, 'n_words': 545, 'path': 'Instruments/Pandas.ipynb', 'repo_name': 'SunnyientDev/kaggle_contests', 'size': '6752'}