Changes between Initial Version and Version 1 of Workshops/IntroToHpc2015/BasicLinuxComands


Ignore:
Timestamp:
Oct 12, 2015 3:49:50 PM (6 years ago)
Author:
pdejesus
Comment:

Legend:

Unmodified
Added
Removed
Modified
  • Workshops/IntroToHpc2015/BasicLinuxComands

    v1 v1  
     1= Linux Commands =
     2
     3== Basic Linux Commands ==
     4Here is a curated list of the most common Linux commands you will use. This list is by no means extensive, but these commands will allow you to navigate the clusters as well as create, destroy, and manipulate files and directories.
     5
     6=== man ===
     7Display online manual pages. Most Linux commands have a manual page with detailed instructions on use. Replace '''<command name>''' below with command you would like information on.
     8
     9{{{
     10[tulaneID@cypress1 ~]$ man <command name>
     11}}}
     12
     13=== pwd ===
     14On a *nix system, directories are containers for files and objects. The '''pwd''' command lists the present working directory. This is the "where am I?" command.
     15{{{
     16[tulaneID@cypress1 ~]$ pwd
     17/home/tulaneID
     18}}}
     19
     20=== cd ===
     21
     22The '''cd''' command changes directory.
     23
     24{{{
     25[tulaneID@cypress1 ~]$ cd NextDirectoryDown
     26[tulaneID@cypress1 ~]$ pwd
     27/home/tulaneID/NextDirectoryDown
     28}}}
     29
     30As you can see, !NextDirectoryDown is a ''subdirectory'' of tulaneID. And tulaneID is itself a subdirectory of home, which is a subdirectory of the root directory (/). Thus, the directories form a downward facing tree with all directories stemming from the root directory. You can move one level up in the tree by typing '''cd ..''
     31
     32{{{
     33[tulaneID@cypress1 ~]$ pwd
     34/home/tulaneID/NextDirectoryDown
     35[tulaneID@cypress1 ~]$ cd ..
     36[tulaneID@cypress1 ~]$ pwd
     37/home/tulaneID
     38}}}
     39
     40And you can navigate using either relative or absolute paths. That is, you can enter directory paths relative to your current location, or you can entire the entire path starting with the root directory.
     41
     42{{{
     43[tulaneID@cypress1 ~]$ pwd
     44/home
     45[tulaneID@cypress1 ~]$ cd tulaneID/NextDirectoryDown
     46[tulaneID@cypress1 ~]$ pwd
     47/home/tulaneID/NextDirectoryDown
     48[tulaneID@cypress1 ~]$ cd /home
     49[tulaneID@cypress1 ~]$ pwd
     50/home
     51}}}
     52
     53Lastly, if you ever get lost you can use the tilde (~) to return to your HOME directory.
     54{{{
     55[tulaneID@cypress1 ~]$ cd ~
     56[tulaneID@cypress1 ~]$ pwd
     57/home/tulaneID
     58}}}
     59
     60=== ls ===
     61The '''ls''' command will list files in the current directory.
     62{{{
     63[tulaneID@cypress1 ~]$ ls
     64a.out  code.c  Makefile
     65}}}
     66
     67If you would like to see if a directory contains a specific file, you can pass the directory path and file name to '''ls''' as an argument
     68{{{
     69[tulaneID@cypress1 ~]$ ls a.out
     70a.out
     71}}}
     72
     73Setting the ''-l'' flag will display files along with their permissions and ownership.
     74{{{
     75[tulaneID@cypress1 ~]$ ls -l
     76total 0
     77-rw-r--r-- 1 tulaneID MyGroup 0 Aug 14 10:57 a.out
     78-rw-r--r-- 1 tulaneID MyGroup 0 Aug 14 10:57 code.c
     79-rw-r--r-- 1 tulaneID MyGroup 0 Aug 14 10:57 Makefile
     80}}}
     81
     82=== mkdir ===
     83Create new directory.
     84{{{
     85[tulaneID@cypress1 ~]$ cd testdir
     86-bash: cd: testdir: No such file or directory
     87[tulaneID@cypress1 ~]$ mkdir testdir
     88[tulaneID@cypress1 ~]$ cd testdir
     89[tulaneID@cypress1 ~]$ pwd
     90/home/tulaneID/testdir
     91}}}
     92
     93=== rm ===
     94Remove files and directories. CAUTION: THERE IS NO UNDOING THIS COMMAND.
     95{{{
     96[tulaneID@cypress1 ~]$ rm testfile
     97}}}
     98If you would like to remove a directory and all of its contents use the following command:
     99{{{
     100[tulaneID@cypress1 ~]$ rm -ri testdir
     101rm: descend into directory `testdir'? y
     102rm: remove regular empty file `testdir/Makefile'? y
     103rm: remove regular empty file `testdir/code.c'? y
     104rm: remove regular empty file `testdir/a.out'? y
     105rm: remove directory `testdir'? y
     106[tulaneID@cypress1 ~]$
     107}}}
     108The ''r'' flag specifies to remove files/directories recursively. ''i'' specifies to prompt before deleting each file.
     109
     110If you are confident with the command line and your understanding of file locations you can use the ''f'' flag instead of ''i'' to force deletion of all files without prompting:
     111{{{
     112[tulaneID@cypress1 ~]$ rm -rf testdir
     113[tulaneID@cypress1 ~]$ ls testdir
     114ls: cannot access testdir: No such file or directory
     115}}}
     116
     117== rmdir ==
     118Removes directories. The '''rmdir''' is a safer approach than using '''rm -r''' as it will only act on empty directories.
     119
     120{{{
     121[tulaneID@cypress1 ~]$ ls NextDirectoryDown/
     122file.txt
     123[tulaneID@cypress1 ~]$ rmdir NextDirectoryDown/
     124rmdir: failed to remove `NextDirectoryDown/': Directory not empty
     125[tulaneID@cypress1 ~]$ rm NextDirectoryDown/file.txt
     126[tulaneID@cypress1 ~]$ rmdir NextDirectoryDown/
     127[tulaneID@cypress1 ~]$ ls
     128[tulaneID@cypress1 ~]$
     129
     130}}}
     131
     132=== cp ===
     133Copy file to new location.
     134
     135In the example below sourcefile already exists, and destinationfile may or may not exist and will become a copy of sourcefile.
     136{{{
     137[tulaneID@cypress1 ~]$ cp sourcefile destinationfile
     138}}}
     139To copy a directory and all of its contents we use the recursive flag '''-r''':
     140{{{
     141[tulaneID@cypress1 ~]$ cp -r sourcedir destinationdir
     142}}}
     143
     144Take note of the order here: source first, destination second. This is the standard order across most *nix commands.
     145=== mv ===
     146Move a file to a new location.
     147{{{
     148[tulaneID@cypress1 ~]$ mv file1 file2
     149[tulaneID@cypress1 ~]$ ls file1
     150ls: cannot access file1: No such file or directory
     151[tulaneID@cypress1 ~]$ ls file2
     152file2
     153}}}
     154Unlike '''cp''', the '''mv''' command does not create a second instance of the file or directory.
     155
     156=== cat ===
     157Print the entire contents of a file (short for conCATenate).
     158{{{
     159[tulaneID@cypress1 ~]$ cat daysofweek.txt
     160monday
     161tuesday
     162wednesday
     163thursday
     164friday
     165saturday
     166sunday
     167}}}
     168
     169=== tail ===
     170Displays lines from the end of a file. Useful for viewing recent results in an output file.
     171{{{
     172[tulaneID@cypress1 ~]$ cat animals
     173dog
     174Dog
     175cat
     176Racoon
     177DOG
     178bullfrog
     179Little Doggie
     180
     181[tulaneID@cypress1 ~]$ tail -3 animals
     182DOG
     183bullfrog
     184Little Doggie
     185}}}
     186The ''-f'' flag can be used to tail a file interactively. New additions to the end of the file will be printed to your screen.
     187
     188=== head ===
     189'''head''' is much like '''tail''', except it prints from the top of a file.
     190{{{
     191[tulaneID@cypress1 ~]$ head -2 animals
     192dog
     193Dog
     194}}}
     195
     196=== more ===
     197'''more''' is like '''cat''', except it prints the file one page at a time. The spacebar is used to continue on to the next page.
     198
     199== Basic Linux Commands Exercise ==
     200
     201Lets take a break from lecture to practice some of the commands we've just learned
     202
     203[[cypress/BasicLinuxComands/Exercise|Basic Commands Exercise]]
     204
     205== Not Basic Linux Commands ==
     206
     207=== Star Wildcard ===
     208Shell commands can often make use of the wildcard characters. The most universal of these is the asterisk or star wildcard, *. This acts as a stand-in for any and all strings.
     209 
     210=== ls again ===
     211Suppose we have a directory containing many files, but we only want to list files ending with the .txt extension. We can use the * wildcard to request a list of any file ending in .txt
     212
     213{{{
     214[tulaneID@cypress1 ~]$ ls
     215a.out  NextDirectoryDown helloworld.c  textfile01.txt  textfile02.txt
     216[tulaneID@cypress1 ~]$ ls *.txt
     217textfile01.txt  textfile02.txt
     218[tulaneID@cypress1 ~]$
     219}}}
     220
     221Additionally, the behavior of the '''ls''' command can be modified by the addition of option flags. By default '''ls''' does not list any "hidden" files and subdirectories. To display all files and subdirectories one must add the ''-a'' flag.
     222
     223{{{
     224[tulaneID@cypress1 ~]$ ls -l
     225total 4
     226-rw-r--r-- 1 tulaneID workshop    0 Aug 18 21:50 a.out
     227drwxr-xr-x 2 tulaneID workshop 4096 Aug 18 21:37 NextDirectoryDown
     228-rw-r--r-- 1 tulaneID workshop    0 Aug 18 21:50 helloworld.c
     229-rw-r--r-- 1 tulaneID workshop    0 Aug 18 21:50 textfile01.txt
     230-rw-r--r-- 1 tulaneID workshop    0 Aug 18 21:50 textfile02.txt
     231[tulaneID@cypress1 ~]$ ls -al
     232total 56
     233drwx------ 6 tulaneID workshop 4096 Aug 18 20:03 .
     234drwxr-xr-x 5 root     root     4096 Aug 12 13:00 ..
     235-rw------- 1 tulaneID workshop  107 Aug 18 16:59 .bash_history
     236-rw-r--r-- 1 tulaneID workshop   18 Jul 18  2013 .bash_logout
     237-rw-r--r-- 1 tulaneID workshop  176 Jul 18  2013 .bash_profile
     238-rw-r--r-- 1 tulaneID workshop  124 Sep 30  2014 .bashrc
     239-rw-r--r-- 1 tulaneID workshop  500 May  7  2013 .emacs
     240drwxr-xr-x 2 tulaneID workshop 4096 Nov 11  2010 .gnome2
     241-rw-r--r-- 1 tulaneID workshop  171 Aug  6  2014 .kshrc
     242drwxr-xr-x 4 tulaneID workshop 4096 Aug  6  2014 .mozilla
     243drwxr-xr-x 2 tulaneID workshop 4096 Aug 18 16:17 NextDirectoryDown
     244drwx------ 2 tulaneID workshop 4096 Aug 13 16:18 .ssh
     245-rw------- 1 tulaneID workshop 1786 Aug 18 20:03 .viminfo
     246-rw------- 1 tulaneID workshop   54 Aug 13 16:18 .Xauthority
     247[tulaneID@cypress1 ~]$
     248}}}
     249
     250=== quota ===
     251Display disk quotas.
     252
     253The '-s' flag translates the output into a readable format. If your blocks column is equal to or greater than the quota column, you have exceeded your available disk space.
     254{{{
     255[tulaneID@cypress1 ~]$quota -s
     256Disk quotas for user tulaneID (uid 12345):
     257     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
     258master.10ge.cluster:/home
     259                  6459M  10000M  10000M           23260    250k    250k       
     260master.10ge.cluster:/share/apps
     261                  6459M  10000M  10000M           23260    250k    250k     
     262}}}
     263
     264=== grep ===
     265Find text in a file. '''grep''' will return every line in the file that matches your search term.
     266{{{
     267[tulaneID@cypress1 ~]$ cat animals
     268dog
     269Dog
     270cat
     271Racoon
     272DOG
     273bullfrog
     274Little Doggie
     275
     276[tulaneID@cypress1 ~]$ grep dog animals
     277dog
     278}}}
     279Use the ''-v'' flag to print out every line excluding those containing the search term. Notice that "dog" (lowercase) is missing from the results.
     280{{{
     281[tulaneID@cypress1 ~]$ grep -v dog animals
     282Dog
     283cat
     284Racoon
     285DOG
     286bullfrog
     287Little Doggie
     288}}}
     289The ''-i'' flag will search without case-sensitivity.
     290{{{
     291[tulaneID@cypress1 ~]$ grep -i dog animals
     292dog
     293Dog
     294DOG
     295Little Doggie
     296}}}
     297The ''-i'' and ''-v'' flag can be combined to exclude all lines containing the search term regardless of capitalization.
     298{{{
     299[tulaneID@cypress1 ~]$ grep -iv dog animals
     300cat
     301Racoon
     302bullfrog
     303}}}
     304
     305=== find ===
     306Searches for files in a directory hierarchy. The '''find''' takes a directory as an argument and will search in that directory and all of it's subdirectories for files that match the search criteria. If no directory is specified, the current directory is used.
     307
     308You can search for a file by name using the '''-name''' flag as your search criteria
     309{{{
     310[tulaneID@cypress1 ~]$ find /home/tulaneID/ -name file1.txt
     311/home/tulaneID/dir1/file1.txt
     312[tulaneID@cypress1 ~]$
     313}}}
     314
     315One can also search for all files that have a string of text in any portion of their name by using the star wildcard, e.g. we can find all the .txt files in our HOME directory and it's subdirectories. Note that quotes are required around our criteria as we are now using a ''regular expression''
     316
     317{{{
     318[tulaneID@cypress1 ~]$ find /home/tulaneID/ -name "*.txt"
     319/home/tulaneID/textfile02.txt
     320/home/tulaneID/dir1/file1.txt
     321/home/tulaneID/examples/alphabet.txt
     322/home/tulaneID/examples/numbers.txt
     323/home/tulaneID/textfile01.txt
     324[tulaneID@cypress1 ~]$
     325}}}
     326
     327These are just a couple examples of the many ways '''find''' can be used. A nice short tutorial on '''find''' with a lot of examples can be found at http://alvinalexander.com/unix/edu/examples/find.shtml
     328
     329
     330=== Pipe ( | ) ===
     331
     332The pipe command, '''|''', takes the output of one command and passes it to another. For example we could "pipe" the output of '''cat''' to '''grep''' in order to search a file for a particular string
     333
     334{{{
     335[tulaneID@cypress1 examples]$ head -5 numbers.txt | grep 1
     33601
     337[tulaneID@cypress1 examples]$
     338}}}
     339
     340=== history ===
     341Displays the command history list with line numbers. You can add an optional integer, n, argument to display only the last n commands
     342
     343{{{
     344[tulaneID@cypress1 examples]$ history 2
     345  299  cat numbers.txt | grep 1
     346  300  history 2
     347[tulaneID@cypress1 examples]$
     348}}}
     349
     350Note: '''history''', '''|''', and '''grep''' when used together are a powerful combination as they allow you to search for old commands that you may not remember the exact syntax for:
     351
     352{{{
     353[tulaneID@cypress1 examples]$ history | grep chmod
     354  153  chmod u+x myscript.sh
     355  204  chmod +x myexecutable.sh
     356  271  chmod +x myscript.sh
     357  273  chmod -x myscript.sh
     358  301  history | grep chmod
     359[tulaneID@cypress1 examples]$
     360}}}
     361
     362=== chmod  ===
     363The '''chmod''' commands allow a user to modify the permissions of files and directories that they own. To see the permissions of a file/directory we can use the '''ls -l''' command
     364
     365{{{
     366[tulaneID@cypress1 ~]$ ls -l
     367total 4
     368-rw-r--r-- 1 tulaneID workshop    0 Aug 18 21:50 a.out
     369-rw-r--r-- 1 tulaneID workshop    0 Aug 18 21:50 helloworld.c
     370-rw-r--r-- 1 tulaneID workshop    0 Aug 19 07:15 myscript.sh
     371drwxr-xr-x 2 tulaneID workshop 4096 Aug 18 21:37 NextDirectoryDown
     372-rw-r--r-- 1 tulaneID workshop    0 Aug 18 21:50 textfile01.txt
     373-rw-r--r-- 1 tulaneID workshop    0 Aug 18 21:50 textfile02.txt
     374[tulaneID@cypress1 ~]$
     375}}}
     376
     377The ten left most characters describe your permissions. The first character describes the file type (- for a file, d for a directory). The next three describe the permissions for the user who owns the file, the middle three describe the permissions for the group that owns the file, and the last three describe the permissions for "others". The "r" denotes permission to read, the "w" means permission to write, and the "x" indicates permission to execute the file/directory.
     378
     379The change mode or '''chmod''' command allows us to alter those permissions. In the past one had to remember (or google) numeric codes that corresponded to permission states, but modern syntax allows for the addition and subtraction of permissions with greater ease. Now one needs only indicate whose permissions to change and what changes to make. For example, let's add executable permissions for the user to the file myscript.sh
     380
     381{{{
     382[tulaneID@cypress1 ~]$ chmod u+x myscript.sh
     383[tulaneID@cypress1 ~]$ ls -l
     384total 4
     385-rw-r--r-- 1 tulaneID workshop    0 Aug 18 21:50 a.out
     386-rw-r--r-- 1 tulaneID workshop    0 Aug 18 21:50 helloworld.c
     387-rwxr--r-- 1 tulaneID workshop    0 Aug 19 07:15 myscript.sh
     388drwxr-xr-x 2 tulaneID workshop 4096 Aug 18 21:37 NextDirectoryDown
     389-rw-r--r-- 1 tulaneID workshop    0 Aug 18 21:50 textfile01.txt
     390-rw-r--r-- 1 tulaneID workshop    0 Aug 18 21:50 textfile02.txt
     391}}}
     392
     393The user (and only the user) can now run the script myscript.sh as an executable. For a more complete introduction see http://alvinalexander.com/linux-unix/linux-chmod-command-permissions-file-directories
     394
     395
     396= Next Section =
     397[[cypress/FileEditingSoftware|File editing software]]