<jupyter_start><jupyter_text>Introduction to the Command Line The command line, or *shell*, is a powerful way to interact with the internals of the computer. From the command line you can do many things you normally do with a computer, such as navigate the filesystem and run programs. In addition to these common tasks, the command line allows you to easily automate repetitive tasks and use software programs for science that can only be installed and run via this interface. There are different types of shells:- sh: Bourne shell- csh: C shell- ksh: Korn shell- bash: Bourne again shell- tch: Tenex C shellAmong these types, bash is the most widely used.  Navigating the Shell You can access the shell by opening a terminal emulator (“terminal” for short) on a Linux or MacOS computer.   Exercise: Open a Terminal 1. Search your computer’s programs to find one called Terminal. 2. Open an instance of that program. You’re in the shell and you should see something that looks like: `user@host $` The dollar sign (`$`) is called the *prompt*, and it indicates that the *shell* is ready for your input. When typing commands, either from these lessons or from other sources, do not type the prompt, only the commands that follow it.- $: regular user- \: super user (root)- Customizable via Environment Variables PS1.export PS1=”\u@\h:\w\$ ”The shell starts your session from a special directory called home directory (tilde `~`)  Paths and `pwd` The first command we will learn about is `pwd`, which stands for print working directory. Typing this command into the command line will tell you where you are in your computer's filesystem.<jupyter_code>pwd<jupyter_output><empty_output><jupyter_text>What the shell returns to us from typing `pwd` is the absolute path to where we are in the computers filesystem. Absolute paths begin at the top of the filesystem directory tree, called the root directory. The `/` syntax is used at the beginning of a path to indicate the root or top-level directory. The following `/`'s are used to separate the names of directories in a path. Another path we need to be familar with is the relative path which tells us where something is in relation to where we are, intead of from the root of the filesystem. The relative path for the current directory is `.` and the parent directory (*e.g.,* `Users`) is `..`  Changing Directories (`cd`) *note - to follow along with the code below please download the example filesystem we will be working with from here: https://github.com/physics-codes/seminar/blob/ch01/filespace.zip?raw=true and unzip it with `unzip` command syntax```unzip file.zip -d destination_folder```and save it to your Computer's Desktop.* We are now going to navigate to the fission directory in Professor Meitner’s home directory. We will use do this using `cd`, which stands for change directory.   `$ cd Desktop/filespace/people/l/lisemeitner/fission/` As you can see in this example, executing the cd command with no arguments results in a new prompt. The prompt reflects the new current working directory, `fission`. To double-check that we are where we think we are we can use `pwd`. ~~~$ pwd~~~~~~/Users/tiffanytimbers/Desktop/filespace/people/l/lisemeitner/fission~~~ You can also use `cd` with a relative path (*e.g.,* `cd ..` to navigate up to the parent directory), and we wil demonstrate this and more after we introduce the `ls` command.  Listing the Contents (`ls`) To find out what is inside the a directory we can use `ls`, which means list. Typing `ls` on its own will tell you the contents of the current working directory, in our case the `fission` directory. ~~~$ ls~~~~~~applications		heat-generation.txt	neutron-production.txt~~~ We see that the shell returns three things, two of which end in `.txt` and thus look like files, and one item that has no file extension, and thus is likely a directory. But files are not required to have file extensions, and so we cannot be sure. To be sure, we can use the `-F` option with `ls` to ask the shell to clearly specify which items are files and which are directories. ~~~$ ls -F~~~~~~applications/		heat-generation.txt	neutron-production.txt~~~ Now we see that `applications` ends with a `/`, clearly indicating that it is in fact a directory. We can also list the contents of directories other than our current directory by providing an argument to the `ls` command. To list the contents of the applications directory without entering it, we can execute: ~~~$ ls applications~~~~~~power	propulsion		weapons~~~To display a list of all files, including the hidden files, we can use the flag -a :<jupyter_code>ls -a ~/Desktop<jupyter_output>[34m$RECYCLE.BIN[m[m/
[34m.[m[m/
[34m..[m[m/
.DS_Store
.localized
[34mAustralia2017[m[m/
Glacial-Ice-Hazards-Working-Group15-Feb-2016.pdf
desktop.ini
[34mfilespace[m[m/<jupyter_text>More Changing Directories (`cd`) Previously we changed directories to the `fission` directory using the `cd` and the absolute path. As mentioned above, we can also change directories using the relative path, for example to navigate into the `applications` directory we can type: ~~~$ cd applications$ pwd~~~~~~/Users/tiffanytimbers/Desktop/filespace/people/l/lisemeitner/fission/applications~~~ We can also move back to the `fission` directory by using the relative path to move to the parent directory by typing: ~~~$ cd ..$ pwd~~~~~~/Users/tiffanytimbers/Desktop/filespace/people/l/lisemeitner/fission~~~ The `applications` directory had to be present in the `fission` directory for `cd applications` to work.If a directory does not exist, bash will not be able to change into that location and will report an error message, as seen here. Notice that bash stays in the original directory, as you might expect: ~~~$ cd biology~~~~~~-bash: cd: biology: No such file or directory~~~~~~$ pwd~~~~~~/Users/tiffanytimbers/Desktop/filespace/people/l/lisemeitner/fission~~~ What happens when you type `cd` alone, without giving an absolute or relative path? When you do this, the cd command assumes you want to go to your home directory, so that’s where it takes you: ~~~$ cd$ pwd~~~~~~/Users/tiffanytimbers~~~<jupyter_code>cd ~/Desktop<jupyter_output>/Users/jmunroe/Desktop<jupyter_text>Exercise: Open a Terminal1. Open a new Terminal instance.2. Type `cd ..` at the command prompt and press **Enter** to move from your home directory to the directory above it.3. Move back into your home directory using a relative path.4. If you have downloaded Lise’s directory tree from the book’s GitHub repository, can you navigate to that directory using what you know about ls, cd, and pwd?  File Inspection (head and tail) When using scientific computing programs, you often need to peak at the input and/or output files (for instance, to check some important input parameter or see if your run completed successfully). To demonstrate this, let's first navigate to the `power` directory, and examine the contents of this directory: ~~~$ cd Desktop/filespace/people/l/lisemeitner/fission/applications/power$ ls~~~~~~reactor.txt~~~  The command head prints the first 10 lines of the given file: ~~~$ head -10 reactor.txt~~~~~~ Fission Power IdeaThe heat from the fission reaction could be used to heat fluids. Inthe same way that coal power starts with the production heat whichturns water to steam and spins a turbine, so too, nuclear fissionmight heat fluid that pushes a turbine. If somehow there were a way tohave many fissions in one small space, the heat from those fissionscould be used to heat quite a lot of water.~~~ As you might expect, the tail command prints the last 10: ~~~$ tail -10 reactor.txt~~~~~~the same way that coal power starts with the production heat whichturns water to steam and spins a turbine, so too, nuclear fissionmight heat fluid that pushes a turbine. If somehow there were a way tohave many fissions in one small space, the heat from those fissionscould be used to heat quite a lot of water.Of course, it would take quite a lot of fissions. Perhaps Professors Rutherford, Curie, or Fermi have some ideas on thistopic. ~~~  Exercise: Open a Terminal1. Open a new Terminal instance.2. Navigate to a plain text file (*.e.g.,* not a Word or Pages document).3. Use `head` and `tail` to print the first and last lines to the terminal.  Manipulating Files and Directories Beyond navigating the filesystem, the shell can also be used to do other common file manipulation tasks, such as creating, copying, moving and deleting files.  Creating an empty file (`touch`) The `touch` command is used to create an empty text file. The `touch` command, followed by a filename, will create an empty file with that name. Suppose we want to create a file to act as a placeholder for a new idea for a nuclear fission application, like providing heat sources for remote locations such as Siberia. We can create that file with the touch command. Let's first navigage to the directory to where we want to create that file, in this case, the `applications` directory. ~~~$ cd ..$ pwd~~~~~~/Users/tiffanytimbers/Desktop/filespace/people/l/lisemeitner/fission/applications~~~ Let's also take a look at what is currently in the directory. ~~~$ ls~~~~~~power	propulsion		weapons~~~ Now we can use the `touch` command to create the file, and then `ls` to confirm that the file was created. ~~~$ touch remote_heat.txt$ ls~~~~~~power	propulsion   remote_heat.txt   weapons~~~  The simplest text editor (cat and >) Creating empty files can be useful sometimes, but most often computational scientists want to write code by adding text to code source files. This process relies on text editors. At the command line, the simplest way to do this is to use a program called `cat` and the shell syntax `>`, which is called redirection. We will introduce `cat` first and then put these together to create a text file.The cat command is meant to help concatenate files together. Given a filename as its argument, cat will print the full contents of the file to the terminal window. For example, to print all of the text from reactor.txt to the screen, Lise could use `cat` as follows: ~~~$ cat power/reactor.txt~~~~~~ Fission Power IdeaThe heat from the fission reaction could be used to heat fluids. Inthe same way that coal power starts with the production heat whichturns water to steam and spins a turbine, so too, nuclear fissionmight heat fluid that pushes a turbine. If somehow there were a way tohave many fissions in one small space, the heat from those fissionscould be used to heat quite a lot of water.Of course, it would take quite a lot of fissions. Perhaps <NAME>, Curie, or Fermi have some ideas on thistopic. ~~~ We can use this feature of cat and combine it with a redirection (>) to send the output of one file into another. *Note - if you specify the name of an existing file, its contents will be overwritten. If the file does not already exist, it will be created.* For example, the following syntax pushes the contents of `reactor.txt` into a new file called `reactor_copy.txt`: ~~~$ cat power/reactor.txt > power/reactor_copy.txt$ ls power~~~~~~reactor.txt		reactor_copy.txt~~~ We can use cat to check that `reactor_copy.txt` is not empty and that its contents is the same as `reactor.txt`. ~~~$ cat power/reactor_copy.txt~~~~~~ Fission Power IdeaThe heat from the fission reaction could be used to heat fluids. Inthe same way that coal power starts with the production heat whichturns water to steam and spins a turbine, so too, nuclear fissionmight heat fluid that pushes a turbine. If somehow there were a way tohave many fissions in one small space, the heat from those fissionscould be used to heat quite a lot of water.Of course, it would take quite a lot of fissions. Perhaps <NAME>, Curie, or Fermi have some ideas on thistopic. ~~~ To use `cat` and `>` as a text editor where we can type our own thoughts or code, we call `cat` but do not give it any files to operate on. When we do this `cat` accepts input from the text we type into command prompt. To end our text file we need to type **Ctrl-d**. ~~~$ cat > new_ideas_for_applications.txtThis file will be a list for new ideas for new applications.      They are to be listed below:         1. 2.3.4.5.~~~ Again, we can use `ls` and `cat` to see that the file exists and what it's contents are: ~~~$ ls~~~~~~new_ideas_for_applications.txtpower				remote_heat.txtpropulsion			weapons~~~~~~$ cat new_ideas_for_applications.txt~~~~~~This file will be a list for new ideas for new applications.They are to be listed below:1.2.3.4.5.~~~  Exercise: Learn About a Command1. Open a terminal.2. Type `cat` and press **Enter**. The cursor will move to a blank line.3. Try typing some text. Note how every time you press **Enter**, a copy of your text is repeated.4. To exit, type **Ctrl-d**. That is, hold down the Control key and press the lowercase d key at the same time.5. Type `cat > mynewfile.txt` and create your own file.6. Verify to yourself that your new file was created and that it contains the expected contents.  More powerful text editors (nano, emacs, and vim) As illustrated above, using cat is a very simple way to add text to a file, but this method doesn't let you easily edit the file. Thus, we recommend the use of text editors for this purpose. nano is a simple text editor that is recommended for first-time users and we will use it for the examples below. To use the nano text editor to open or create the `remote_heat.txt` file, we type nano and then the file name we want to create or edit. We can then type and/or edit text in nano. To save changes press the **Cntrl + o** keys. To exit, press **Cntrl + x** keys. You can view what you typed by either opening the file with nano again, or printing the text of the file to the screen using `cat`. ~~~$ nano remote_heat.txt~~~ Inside nano, type the text below, and save the changes and exit out of nano.~~~This is some new text.~~~ View the changes to the file using `cat`~~~$ cat remote_heat.txt~~~~~~This is some new text.~~~To use `vim`, ou can use either the command `vi` or the command `vim`. Similarly, to use `emacs`, you should use the command `emacs`  Exercise: Open nano1. Open the Terminal.2. Execute the command nano.3. Add some text to the file.4. Use the instructions at the bottom of the window to name and save the file, then exit nano.  Copying and Renaming Files (`cp` and `mv`) Now that we know how to make and edit files, we can learn how to copy, move and rename them. The `cp` command is used to copy files from one place to another. The first argument passed to `cp` is the source file to be copied, and the second argument is the destination file (the copy to be made). To make a copy of the `reactors.txt` file in the `power` directory, let's navigate there and then us `cp` to do this. ~~~$ cd power$ ls~~~~~~reactor.txt		reactor_copy.txt~~~ ~~~$ cp reactor.txt reactor_copy2.txt$ ls~~~~~~reactor.txt		reactor_copy.txt	reactor_copy2.txt~~~ We can also copy the file when we create it by specifying where to copy it to. For example to create a copy of `remote.txt` in the parent directory (`applications`) called `reactor_copy3.txt` we would type: ~~~$ cp reactor.txt ../reactor_copy3.txt$ ls~~~~~~reactor.txt		reactor_copy.txt	reactor_copy2.txt~~~~~~$ ls ..~~~~~~new_ideas_for_applications.txt	reactor_copy3.txtpower				remote_heat.txt~~~ *note - if the copy destination is in another directory, that directory must already exist or cp will throw an error* If you want to rename a file without copying it, you use the `mv` command. For example, in browsing through the filesystem, we notice an idea for a nuclear plane in the propulsion directory: ~~~$ cd ../propulsion$ ls ~~~~~~nuclear_plane.txt~~~ This is really a bad idea, a nuclear plane would probably be too heavy to ever fly. Let's rename the idea as `bad_idea.txt`, as a warning to others. To do this we give the `mv` command two arguments: the original file path and then the new file path. ~~~$ mv nuclear_plane.txt bad_idea.txt$ ls~~~~~~bad_idea.txt~~~ We can also use `mv` to move a file, and we can either keep the filename the same or rename it. For example, here's how we would move the `bad_idea.txt` file to the `applications` directory: ~~~$ mv bad_idea.txt ../bad_idea.txt$ ls ..~~~~~~bad_idea.txt			reactor_copy3.txtnew_ideas_for_applications.txt	remote_heat.txtpower				weaponspropulsion~~~  Making Directories (mkdir) In addition to creating files, we can also create directories using the `mkdir` command. For example, to create a new directory called `nuclear` to hold ideas and code related to a new class of theories about the nucleus, we will first navigate to the `theories` directory: ~~~$ cd ../../../theories$ ls~~~~~~README.md~~~ As you can see, the `theories` directory is currently only has one file named `README.md`, now let's make the `nucleus` directory using `mkdir`: ~~~$ mkdir nuclear$ ls~~~~~~nuclear~~~ Using our usual path conventions, you can make directories anywhere, not just in your current working directory. For example, we can create a directory called `fission` within the `nuclear` directory we just created without being in the `nuclear` directory: ~~~$ mkdir nuclear/fission$ ls nuclear~~~~~~fission~~~  Deleting Files and Directories (rm) In addition to creating files and directories, you also can delete files and directories in the Shell. This is done using the `rm` command. We can use this to delete our bad idea in the `applications` directory: ~~~$ cd ../fission/applications/$ ls~~~~~~bad_idea.txt            reactor_copy3.txtnew_ideas_for_applications.txt    remote_heat.txtpower                weaponspropulsion~~~~~~$ rm bad_idea.txt$ ls~~~~~~reactor_copy3.txt  new_ideas_for_applications.txt    remote_heat.txt  power                weapons  propulsion~~~ Important note! There is no such this as undo, or a trash can or recycling bin in the Shell. When you delete something in the Shell, it is deleted FOREVER.  As mentioned above, you can also delete directories. To delete the `propulsion` directory we can try: ~~~$ rm propulsion ~~~~~~rm: propulsion: is a directory~~~ But we get an error because `propulsion` is a directory. This is a safety feature to prevent mistakes. To delete directories you need to use a special arguement, `-r`. This flag tells `rm` to go into the directory and execute the command all the way down the tree, deleting all files and folders below propulsion: ~~~$ rm -r propulsion$ ls~~~~~~reactor_copy3.txt  new_ideas_for_applications.txt    remote_heat.txt  power                weapons~~~  Exercise: Make and Remove Files and Directories1. Open the Terminal.2. Use mkdir to create a directory with a few empty subdirectories.3. Use touch to create five empty files in those directories, and use ls to inspect your work.4. With one command (hint: it will have to be recursive), remove the whole directory.   Wildcards Often times you want to apply a command in the shell to many files and/or directories. For example, if we want to apply the ls command to everything in the fission directory we can use a wildcard, `*`, to match all the things in that directory: ~~~$ cd ..$ ls *~~~~~~heat-generation.txt	neutron-release.txtapplications:power		propulsion	weapons~~~ Why is this different from just typing `ls`?  ~~~$ ls~~~~~~applications		heat-generation.txt	neutron-release.txt~~~ When we type `ls *` the Shell first expands the wildcard, `*`, matching everything listed in that directory. Thus typing `ls *` is really like typing: ~~~$ ls applications heat-generation.txt neutron-release.txt~~~~~~heat-generation.txt	neutron-release.txtapplications:power		propulsion	weapons~~~ We can also use wildcards to filter files. For example, if we only want to list the files ending in .txt in the fission directory, we can type: ~~~$ ls *.txt~~~~~~heat-generation.txt	neutron-release.txt~~~  Getting Help How do you know what flags to use? Or what exactly a command does? Your best bet is the program `man` (manual). For example, to see the documentation for the `ls` command and all of the possible arguments/flags that can be used with it, type: ~~~$ man ls~~~~~~LS(1)                     BSD General Commands Manual                    LS(1)NAME     ls -- list directory contentsSYNOPSIS     ls [-ABCFGHLOPRSTUW@abcdefghiklmnopqrstuwx1] [file ...]DESCRIPTION     For each operand that names a file of a type other than directory, ls     displays its name as well as any requested, associated information.  For     each operand that names a file of type directory, ls displays the names     of files contained within that directory, as well as any requested, asso-     ciated information.     If no operands are given, the contents of the current directory are dis-     played.  If more than one operand is given, non-directory operands are     displayed first; directory and non-directory operands are sorted sepa-     rately and in lexicographical order.     The following options are available:~~~ When you use the `man` command, the help documents are opened in a program called `less`. A few helpful things to know about `less` are:* Use the **up and down arrows** to scroll up and down.* Use **Page Up**` and **Page Down** (or the **space bar**) to move up or down by an entire page.* Use a forward slash (**/**) followed by a search term and then **Enter** to search for a particular word. The letter **n** (next) toggles through each occurrence.* Use **h** to display help inside less—this displays all the possible commands that less understands.* Use **q** to quit.* Less can be used to look at text files too (just type: `less [filename]`).  *note - on some Shells the `man` command might not work, try ` [command] --help` in these cases, for example:* ~~~$ ls --help~~~  Exercise: Use the man Pages with less1. Open the Terminal.2. Use the man command and the preceding notes on less to learn about the commands covered already in this chapter (e.g., mkdir, touch, mv, cp, etc.)  Finding the Right Hammer (apropos) How do you find help if you don't know the name of the command? You can use a tool called `apropos` which searches the `man` pages for keywords. For example, to find out what text editors are available on your computer you could search for the keyword "text editor": ~~~$ apropos "text editor"~~~~~~ed(1), red(1)            - text editorvim(1)                   - Vi IMproved, a programmers text editor~~~  Exercise: Find and Learn About a Command1. Open the Terminal.2. Search your computer for commands by using apropos and a keyword.3. Take some time to explore the man page of a command we’ve discussed or of another command or program you know of. Learn about a couple of new arguments or options and try them out. Practice killing or interrupting programs if neces‐ sary.  Combining Utilities with Redirection and Pipes (>, >>, and |) The Shell starts to become really powerful when you start capturing the output of commands to files, and combining (called piping) commands together. We've already encountered the single arrow (`>`), which we can use to send the output of a command to a file. Here's another example of how to use it to output the first line of the `reactor.txt` file in the `power` directory to a new file called `reactor_title.txt`: ~~~$ cd applications/power/$ head -1 reactor.txt > reactor_title.txt$ ls~~~~~~reactor.txt		reactor_title.txt~~~ We can use `cat` to look at what is now in `reactor_title.txt` ~~~$ cat reactor_title.txt~~~~~~ Fission Power Idea~~~ The single arrow above created a **new** file with the outputs of `head -1 reactor.txt` and saved them to a file called `reactor_title.txt`. Had a file already existed with that name, it would have overwritten the contents. Two arrows (`>>`) has a slightly different meaning, it creates a file if none with that name previously existed. But, if one does exist, it does not overwrite it, it appends the commands output to the end of the file.  For example, if we wanted add add the last line of `reactor.txt` to `reactor_title.txt`, while keeping the title, we would type: ~~~$ tail -1 reactor.txt >> reactor_title.txt$ cat reactor_title.txt~~~~~~ Fission Power Ideatopic. ~~~ As you might have guessed, since `head -1` gives first line of a file, `tail -1` gives us the last.  To chain two commands together in one line, we use pipes ( | ). For example, to print the 11th line from `reactor.txt` we can type: ~~~$ head -11 reactor.txt | tail -1~~~~~~Of course, it would take quite a lot of fissions.~~~  Excercise1. Open a terminal2. Use arrows and pipes to save lines 4-9 of `reactor.txt` to a file called `reactor_body.txt`  Arguments, options, and variables- An argument simply gets added after the command. - You can add multiple arguments if the command expect this behavior. - `cd` expect single argument - `cp` expect multiple arguments (as sources). - flags, also called switches or options tell the command to run in a predefined way. They are specified with (`-`) before them.    - `ls -r .` to print the list of files and directories in the current working directory in reverse order.    - `rm -r ` to recursively remove ``.    - Some options fllows `--` (minus minus). For instance,    ```     To sort directory contents by file modification date    ls --sort=time      ```    - Alternatively, you can use    ```    ls -lrth     ```  Combining Utilities with Redirection and Pipes (`>`, `>>`, and `|`)- Combine these simple utilities into more complex algorithms very quickly.- Send the output of a command into a file with `>` (create a new file) or `>>` (append to a file)    ```    /fission/applications/power $ head -1 reactor.txt > reactor_title.txt         Now, the content of reactor_title.txt is simply:    Fission Power Idea    ```- To chain programs together, the pipe ( | ) command can be used in a similar fashion.```/fission/applications/power $ head -1 reactor.txt | tail -1Of course, it would take quite a lot of fissions.```-More:```you@host:~ $ ./myprog 2> errors.log > out.logyou@host:~ $ ./mymistake 2>&1 > all.log``` Permissions and Sharing- Different people can be given different types of access to a given file, program, or computer.- At the higest level, filesystems is only available to users with a user account on the computer. For example:```    ssh [user@host] connects to another computer.    scp [file] [user@host]:path copies files from one computer to another.     These commands work only if the user have account on the host computer```- The “different types of people” are the individual user (`u`) who owns the file, the group (`g`) who’s been granted special access to it, and all others (`o`). The “different types of access” are permission to read (`r`), write to (`w`), or execute (`x`) a file or directory.- `ls -l` is to display the permissons of the files and directories.- `chown [-R] [[user]][:group] target1 [[target2 ..]]` changes the individual user and group ownership of the target(s), recursively if `-R` is used and one or more targets are directories.- `chmod [options] mode[,mode] target1 [[target2 ...]]` changes or sets the permissions for the given target(s) to the given mode(s). ExampleAt the path `filespace/people/l/lisemeitner/fission`<jupyter_code>ls -l filespace/people/l/lisemeitner/fission/<jupyter_output>total 8
drwxr-xr-x  3 jmunroe  staff  96  7 May 10:07 [34mapplications[m[m/
-rw-r--r--@ 1 jmunroe  staff  19 29 Sep  2015 heat-generation.txt
-rw-r--r--@ 1 jmunroe  staff   0 29 Sep  2015 neutron-production.txt<jupyter_text>Take the file heat-generation.txt for example:- The first bit is `d` for directory, `l` for link and `-` otherwise.- The next three bits (`rw-`) indicate that the user owner (`Shahrazadmalek`) can read (`r`) or write (`w`) this file, but not execute it, i.e., cannot run it, (`-`). - The following three bits indicate the same permissions (`r--`) for the group owner (`staff`).- The final three bits (`r--`) indicate read (`r`) but not write or execute permissions for everyone else. Setting Ownership (`chown`)Assume there is another local user with name `Fred` on the computer, and you want to change the owner of `heat-generation.txt` to `Fred`. You can type:```~/fission $ chown Fred heat-generation.txt~/fission $ ls -l heat-generation.txt-rw-r--r-- 1 Fred Stuff 80 May 30 15:08 heat-generation.txt```Now say you want to change the ownership to a group called `research````~/fission $ chown :research heat-generation.txt~/fission $ ls -l heat-generation.txt-rw-r--r-- 1 Fred research 80 May 30 15:08 heat-generation.txt``` Setting Permissions (`chmod`)The `chmod` syntax is ```chmod -R  ```Now say you want to change the group permssions of `heat-generation.txt` to (`rwx`), then you can type:<jupyter_code># Note that ! is just to access bash commands through python notebook
!chmod g+rwx filespace/people/l/lisemeitner/fission/heat-generation.txt
!ls -l filespace/people/l/lisemeitner/fission/heat-generation.txt<jupyter_output>-rwxrwx---@ 1 jmunroe  staff  19 29 Sep  2015 [31mfilespace/people/l/lisemeitner/fission/heat-generation.txt[m[m<jupyter_text>The mode entry `g+rwx` meanswe add the read and execution bits to the group’s permissions for the file. Creating Links (`ln`)Rather than storing multiple copies of large datasets in multiplelocations, you can link to the datasets using:```ln -s  ``` ExampleAt the path `filespace/people/l/lisemeitner/programs/hello`Try to type:~~~$ ls -l-rwxr-xr-x@ 1 shahrazadmalek  staff  20  8 May 20:39 hello-rwxr-xr-x@ 1 shahrazadmalek  staff  20  8 May 20:39 helloworld~~~~~~ Notice that both files are executable~~~Say you cant remember whether it is `hello` ot `helloagain`, and you tried to run `helloagain`, then you get error ```$ ./helloagain-bash: ./helloagain: No such file or directory```But if linked it to hello:~~~$ ln -s hello helloagain$ ./helloagainHello World!~~~Note that linking is very useful to avoid coping huge files to run codes on different directories. Connecting to Other computers (`ssh` and `scp`)This powerful feature of the command line, providing access to networked andremote filesystems, is key to high-performance computing tasks.If you have the right credentials, you can even get access to another machine throughthe shell. You can do this using a program called `ssh` (secure shell). ExampleIf I (with username `shahrazad`) have an account on host machine called `bern`, which is a computer located on the remote network domain `physics.mun.ca`, then Ican connect to that computer from my home computer with the full location of thecomputer in its domain:```~ $ ssh <EMAIL>```Once logged into the computer, I has access to the files and directories in theremote filesystem and can interact with them just as I do locally.I can use the `scp` (secure copy) command to copy files and directories from onecomputer to another. It has the syntax scp  [[user@]host]:. So, to copy a notes.txt file from my local computer to the `Desktop` directoryon the `bern.physics.<EMAIL>.ca` filesystem, I would execute:```~ $ scp notes.txt <EMAIL>:~/Desktop/notes.txt``` The EnviromentThe bash shell defines a computing environment. This computing environmentcan be customized using environment variables. We can investigate our environmentwith a program called `echo`. The echo command prints arguments to the terminal. Inthe case of a string argument, the string is printed verbatim:```~ $ echo "Hello World"```Other examples of enviromental variables:<jupyter_code>! echo $USER
! echo $PWD<jupyter_output>/Users/jmunroe/Desktop<jupyter_text>In bash, you can createyour own variables and change existing variables with the export command:<jupyter_code>! export cmsc6950="Computational Computer Based Research Tools and Applications" &&  echo ${cmsc6950}<jupyter_output>Computational Computer Based Research Tools and Applications<jupyter_text>Saving Enviroment Variables (`.bashrc`)- A number of files in a bash shell store the environment variables that are active in each terminal session. - They are plain-text files containing bash commands. - These commands are executed every time a terminal window is opened. - Any environment variables set with export commands in those files are active for every new terminal session.- To configure and customize your environment, environment variables can be added or edited in `~/.bashrc`.- The `source` command can be used to make changes to `.bashrc` take effect immediately, in the current session.```$ source ~/.bashrc``` Nicknaming Commands (`alias`)In the same way that you can create variables for shortening long strings, you can create shorthand aliases for commands. Example, Say you like colors to distinguish the files and directories in your shell session, you’ll always want touse the `--color` variable when calling `ls`. However, `ls --color` is quite a lot to type.It is preferable to reset the meaning of ls so that it behaves like `ls --color`. The`alias` command allows you to do just that. To replace ls with `ls --color`, youwould type:```alias ls 'ls --color'```and you can add this change to `~/.bashrc`. Scripting with Bash- Repeating processes on the command line is made easy with files (like `.bashrc`) thatcan store many commands and be executed at will. - Any such series of commands canbe placed into a file called a `script`. - This type of file, like a program, can be writtenonce and executed many times.- Bash scripts typically end in the `.sh` extension.- First step for creating a bashscript is to create such a file.  ExampleOpen the script file `explore.sh` with `nano````~ $ nano explore.sh```Then inside the file type:``` explore.sh explore the three directories above this one print a status messageecho "Initial Directory:" print the working directorypwd list the contents of this directorylsecho "Parent Directory:" ascend to the parent directorycd ..pwdlspwd```Note that comments are preceded by a \ symbol. They are reference text only and are notexecuted. Now change the mode of the file to become executable using:```chmod a+x explore.sh```where (`a`) stands for `ugo` Then run it using```./explore.sh```What do you get?You can also have functions and loops in script, and you can send inputs to it as well. For example, see the script `scripting.sh` :<jupyter_code>cd ~/GitHub/CMSC6950.github.io/lectures/scripts-DATA/
! cat scripting.sh
! ./scripting.sh<jupyter_output>Factorial part ................
24
Simple for loop ................
Simple for loop --- reading from file ................
0
1
2
3
4
For loop with var ................
 25878 — PID number ($$)
 0 — exit status of last command ($?)
 3 — number of options/arguments ($#)
 ./scripting.sh, 1, 2, 9  — name of calling program, its arguments ($0, $1...$n)
 1 2 9, 1 2 9 — list of all arguments (man bash for details) ( $*, $@)
1
3
5
7
9
If statement ................
wrong! .... # of arguments < 3
{'fraction_non_alphanumeric': 0.08842014636246233, 'fraction_numerical': 0.006485865977902138, 'mean_word_length': 4.256599788806758, 'pattern_counts': {'":': 1, '<': 32, '<?xml version=': 0, '>': 58, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '93682', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 9735, 'n_tokens_neox': 9084, 'n_words': 5125, 'path': 'lectures/ch01-the-command-line.ipynb', 'repo_name': 'CMSC6950/CMSC6950.github.io', 'size': '34915'}