<jupyter_start><jupyter_text>NOTE: Typically we will just be either reading csv files directly or using pandas-datareader to pull data from the web. Consider this lecture just a quick overview of what is possible with pandas (we won't be working with SQL or Excel files in this course)  Data Input and OutputThis notebook is the reference code for getting input and output, pandas can read a variety of file types using its pd.read_ methods. Let's take a look at the most common data types:<jupyter_code>import numpy as np
import pandas as pd<jupyter_output><empty_output><jupyter_text>Check out the references here! **This is the best online resource for how to read/write to a variety of data sources!**https://pandas.pydata.org/pandas-docs/stable/user_guide/io.html-------- Format TypeData DescriptionReaderWritertextCSVread_csvto_csvtextJSONread_jsonto_jsontextHTMLread_htmlto_htmltextLocal clipboardread_clipboardto_clipboardbinaryMS Excelread_excelto_excelbinaryOpenDocumentread_excel&160;binaryHDF5 Formatread_hdfto_hdfbinaryFeather Formatread_featherto_featherbinaryParquet Formatread_parquetto_parquetbinaryMsgpackread_msgpackto_msgpackbinaryStataread_statato_statabinarySASread_sas&160;binaryPython Pickle Formatread_pickleto_pickleSQLSQLread_sqlto_sqlSQLGoogle Big Queryread_gbqto_gbq ---------  CSVComma Separated Values files are text files that use commas as field delimeters.Unless you're running the virtual environment included with the course, you may need to install xlrd and openpyxl.In your terminal/command prompt run:    conda install xlrd    conda install openpyxlThen restart Jupyter Notebook.(or use pip install if you aren't using the Anaconda Distribution) Understanding File PathsYou have two options when reading a file with pandas:1. If your .py file or .ipynb notebook is located in the **exact** same folder location as the .csv file you want to read, simply pass in the file name as a string, for example:            df = pd.read_csv('some_file.csv')        2. Pass in the entire file path if you are located in a different directory. The file path must be 100% correct in order for this to work. For example:        df = pd.read_csv("C:\\Users\\myself\\files\\some_file.csv")  Print your current directory file path with pwd<jupyter_code>pwd<jupyter_output><empty_output><jupyter_text>List the files in your current directory with ls<jupyter_code>ls<jupyter_output>Volume in drive C is Windows
 Volume Serial Number is 6AAE-CCF2

 Directory of C:\Users\yasho\TensorFlow2.0\02-Pandas

14-08-2020  16:23    <DIR>          .
14-08-2020  16:23    <DIR>          ..
14-08-2020  15:57    <DIR>          .ipynb_checkpoints
14-08-2020  15:57             1,117 00-Intro-to-Pandas.ipynb
14-08-2020  15:59             8,484 01-Series.ipynb
14-08-2020  16:13            65,574 02-DataFrames.ipynb
14-08-2020  16:15            14,760 03-Missing-Data.ipynb
14-08-2020  16:18            34,347 04-Groupby.ipynb
14-08-2020  16:23            57,413 05-Operations.ipynb
14-08-2020  15:57            20,129 06-Data-Input-and-Output.ipynb
14-08-2020  15:57            32,486 07-Pandas-Exercises.ipynb
14-08-2020  15:57            33,196 08-Pandas-Exercises-Solutions.ipynb
14-08-2020  15:57            66,984 african_econ_crises.csv
14-08-2020  15:57           375,542 bank.csv
14-08-2020  15:57                51 example.csv
14-08-2020  15:57             5,064 Excel_Sample.xlsx
14-0[...]<jupyter_text>CSV Input<jupyter_code>df = pd.read_csv('example.csv')
df<jupyter_output><empty_output><jupyter_text>CSV Output<jupyter_code>df.to_csv('example.csv',index=False)<jupyter_output><empty_output><jupyter_text>HTMLPandas can read table tabs off of HTML. This only works if your firewall isnb't blocking pandas from accessing the internet!Unless you're running the virtual environment included with the course, you may need to install lxml, htmllib5, and BeautifulSoup4.In your terminal/command prompt run:    conda install lxml    conda install html5lib    conda install beautifulsoup4Then restart Jupyter Notebook.(or use pip install if you aren't using the Anaconda Distribution)  HTML InputPandas read_html function will read tables off of a webpage and return a list of DataFrame objects:<jupyter_code>tables = pd.read_html('http://www.fdic.gov/bank/individual/failed/banklist.html')
tables[0].head()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07223372440763745, 'fraction_numerical': 0.06809293765815505, 'mean_word_length': 4.170035671819263, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 27, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '640266', 'license': 'MIT', 'n_tokens_mistral': 1570, 'n_tokens_neox': 1355, 'n_words': 433, 'path': '0 - Visualization-and-analysis/2. Pandas/06-Data-Input-and-Output.ipynb', 'repo_name': 'yashodeepchikte/Machine-Learning', 'size': '4347'}