<jupyter_start><jupyter_text>Getting help or usageLet us explore details about how to list the commands and get the help or usage for given command.<jupyter_code>%%HTML
<iframe width="560" height="315" src="https://www.youtube.com/embed/_9hUiuzcgsI?rel=0&amp;controls=1&amp;showinfo=0" frameborder="0" allowfullscreen></iframe><jupyter_output><empty_output><jupyter_text>* Even though we can run commands from almost all the nodes in the clusters, we should only use Gateway to run HDFS Commands.* First we need to make sure designated Gateway server is Gateway for HDFS service so that we can run commands from Gateway node. In our case we have designated **gw02.itversity.com** or **gw03.itversity.com** as Gateways.* Typically Namenode process will be running on port number 8020. We can also pass namenode URI to access HDFS.<jupyter_code>%%sh

head -20 /etc/hadoop/conf/core-site.xml
%%sh

hdfs dfs -ls /user/${USER}
%%sh

hdfs dfs -ls hdfs://nn01.itversity.com:8020/user/${USER}<jupyter_output><empty_output><jupyter_text>* `hadoop fs` or `hdfs dfs` – list all the commands available* `hadoop fs -usage` – will give us basic usage for given command* `hadoop fs -help` – will give us additional information for all the commands. It is same as just running `hadoop fs` or `hdfs dfs`.* We can run help on individual commands as well - example: `hadoop fs -help ls` or `hdfs dfs -help ls`<jupyter_code>%%sh

hdfs dfs -help
%%sh

hdfs dfs -usage ls
%%sh

hdfs dfs -help ls
%%sh

hdfs dfs -ls /public/retail_db<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1025974025974026, 'fraction_numerical': 0.01753246753246753, 'mean_word_length': 5.01953125, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 1}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '282241', 'license': 'MIT', 'n_tokens_mistral': 529, 'n_tokens_neox': 502, 'n_words': 201, 'path': '02_overview_of_hdfs/03_getting_help_or_usage.ipynb', 'repo_name': 'itversity/spark-sql', 'size': '1540'}