<jupyter_start><jupyter_text>HDFS File System Command Note > **Make directory** `hdfs dfs -mkdir dir_name` > **Copy file from local file system to hdfs file system** `hdfs dfs -copyFromLocal local_filename hdfs_new_filename` `hdfs dfs -copyFromLocal local_filename target_folder/hdfs_new_filename`* Copy directly into a target folder `hdsf dfs -put local_filename hdfs_new_filename`* -put is hte same as -copyFromLocal > **Copy file from hdfs file system to local file system** `hdfs dfs -copyToLocal hdfs_filename local_new_filename` > **Copy from on hdfs** `hdfs dfs -cp original_file new_file` > **Move file to a folder** `hdfs dfs -mv file newdir` > **cd: How to navigate directories in Hadoop HDFS** `hdfs dfs -ls /user/username/app1/subdir/`* There is no cd (change directory) command in hdfs file system. You can only list the directories and use them for reaching the next directory. [stackoverflow](https://stackoverflow.com/questions/33055403/how-to-navigate-directories-in-hadoop-hdfs) > **get pwd** [stackoverflow](https://stackoverflow.com/questions/21539694/is-there-an-equivalent-to-pwd-in-hdfs) there is no pwd in hdfs There is no pwd in hdfsThe home directory of hdfs is always `/user/myname`, in my case, it'll be `/user/jchiu` > **Create new file in hdfs** `hdfs dfs -touchz new_file.txt` > **write text from comand line into Hadoop** `echo "hello world" | hadoop fs -appendToFile - /dir/hadoop/hello_world.txt``echo "hello world" | hadoop fs -put - /dir/hadoop/hello_world.txt``hdfs dfs -cat /dir/hadoop/hello_world.txt | hdfs dfs -appendToFile - /dir/hadoop/hello_world.txt`* note: appendToFile or put command is to append single src, or multiple srcs from **local file system to the destination file system**. [stackoverflow](https://stackoverflow.com/questions/31107065/write-text-from-comand-line-into-hadoop) > **getmerge file back to local file system**<jupyter_code># format
hdfs dfs -getmerge hdfs_folder_path local_file_path

# example
hdfs dfs -getmerge hw1/exercise1/ hw1-exercise1/exercise1_output.txt<jupyter_output><empty_output><jupyter_text>> remove and skip trash<jupyter_code>hdfs dfs -rm -skipTrash -r hddfs_folder_path<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10603290676416818, 'fraction_numerical': 0.013711151736745886, 'mean_word_length': 5.290229885057471, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 20, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '299257', 'license': 'MIT', 'n_tokens_mistral': 748, 'n_tokens_neox': 719, 'n_words': 235, 'path': 'others/hdfs_command_note.ipynb', 'repo_name': 'johnnychiuchiu/Machine-Learning', 'size': '2188'}