shell script basics

Note:

Well,

Bash is intepreted line by line as it runs and depends on calling a lot of external progs (depending on what you want to do).

You often have to use temp files as intermediate storage for result sets.
It (shell) was originally designed to talk to the system and automate cmd sequences (shell files).

Perl is more like C, it's largely self contained with a huge library of free code and it's compiled , so it runs much faster, eg about 80-90% speed of C, but easier to program (eg variable sizes are dynamic).
Syntax is similar to C. 

1)UNIX Commands:

Although a shell script can make use of any unix commands here are a number of commands which are more often used than others. These commands can generally be described as commands for file and text manipulation.

Command syntax	Purpose
echo "some text"	write some text on your screen
ls	list files
wc -l file wc -w file wc -c file	count lines in file or count words in file or count number of characters
cp sourcefile destfile	copy sourcefile to destfile
mv oldname newname	rename or move file
rm file	delete a file
grep 'pattern' file	search for strings in a file Example: grep 'searchstring' file.txt
cut -b colnum file	get data out of fixed width columns of text Example: get character positions 5 to 9 cut -b5-9 file.txt Do not confuse this command with "cat" which is something totally different
cat file.txt	write file.txt to stdout (your screen)
file somefile	describe what type of file somefile is
read var	prompt the user for input and write it into a variable (var)
sort file.txt	sort lines in file.txt
uniq	remove duplicate lines, used in combination with sort   since uniq removes only duplicated consecutive lines Example: sort file.txt | uniq
expr	do math in the shell Example: add 2 and 3 expr 2 "+" 3
find	search for files Example: search by name: find . -name filename -print This command has many different possibilities and options. It is unfortunately too much to explain it all in this article.
tee	write data to stdout (your screen) and to a file Normally used like this: somecommand | tee outfile It writes the output of somecommand to the screen and to the file outfile
basename file	return just the file name of a given name and strip the directory path Example: basename /bin/tux returns just tux
dirname file	return just the directory name of a given name and strip the actual file name Example: dirname /bin/tux returns just /bin
head file	print some lines from the beginning of a file
tail file	print some lines from the end of a file
sed	sed is basically a find and replace program. It reads text from standard input (e.g from a pipe) and writes the result to stdout (normally the screen). The search pattern is a regular expression (see references). This search pattern should not be confused with shell wildcard syntax. To replace the string linuxfocus with LinuxFocus in a text file use: cat text.file | sed 's/linuxfocus/LinuxFocus/' > newtext.file This replaces the first occurance of the string linuxfocus in each line with LinuxFocus. If there are lines where linuxfocus appears several times and you   want to replace all use: cat text.file | sed 's/linuxfocus/LinuxFocus/g' > newtext.file
awk	Most of the time awk is used to extract fields from a text line. The default field separator is space. To specify a different one use the option -F.  cat file.txt | awk -F, '{print $1 "," $3 }'   Here we use the comma (,) as field separator and print the first and third ($1 $3) columns. If file.txt has lines like: Adam Bor, 34, India Kerry Miller, 22, USA   then this will produce: Adam Bor, India Kerry Miller, USA   There is much more you can do with awk but this is a very common use.

Simple Stuff:

First, a convention. I'll list things for you to type in this format:

$ date

I will list the computer's reply like this:

Tue Dec 23 10:52:51 PST 2003

To put this another way, enter this example:

# whoami
root

Where are you?

As you may be aware, a Linux filesystem is in the form of a large tree with many branches called "subdirectories". When you issue a shell command,

$ pwd
/path/path/path

You can decide where you are in the tree. Type this example:

$ cd ~
$ pwd
/home/username

Listing Files:

Directories contain files, and you can list them in a simple, compact format:

$ ls
filename filename filename ...

Or you can list them in more detail:

$ ls -la
(detailed list, one file per line)

And, very important, to find out what a command's options are, use the "man" (manual) command:

$ man ls
(manual page for "ls")

To find files by name:

$ find . -name '*.jpg'
(list of files with .jpg suffix in current and all child directories)

To create a text diagram of the directory tree:

$ tree -d .
(diagram of the directory tree from the current directory)

Examining Files:

There are a number of things you can do to find out more about the files in the list. Here are just a few:

The "file" command tries to identify files by examining their contents:

$ file tux_small.png
tux_small.png: PNG image data, 128 x 151, 8-bit/color RGB, non-interlaced

The next example uses the obscurely named "cat" command. It prints the contents of a file. Unfortunately if the file's contents are not readable, they get printed anyway.

$ cat zipcodes.txt
(prints the entire contents of a file named "zipcodes.txt")

If a file is too long to be viewed on one page, you can say:

$ more zipcodes.txt
(prints file one screenful at a time)

You can also use "grep" to print only those parts of a file you are interested in:

$ grep 10001 zipcodes.txt
(prints only those lines that have the character string "10001" in them)

The "grep" command is very useful, unfortunately it has a difficult-to-remember name. Be sure to:

$ man grep

Pipelines and Redirection:

Enter this command:

$ echo "cherry apple peach"
cherry apple peach

$ echo "cherry apple peach" | tr " " "\n"
cherry
apple
peach

$ echo "cherry apple peach" | tr " " "\n" | sort
apple
cherry
peach

$ echo "cherry apple peach" | tr " " "\n" | sort -r
peach
cherry
apple

Remember: A pipeline ("|") takes the output of one command and makes it the input to another command.

• Normally the output from commands is printed on the screen. But using the symbol ">", you can redirect the output to a file:
  $ date > RightNow.txt
  $ cat RightNow.txt
  Tue Dec 23 14:43:33 PST 2003
• The above example used ">" to replace the content of any existing file having the name "RightNow.txt". To append new data to an existing file, use ">>" instead:
  $ date >> RightNow.txt
  $ cat RightNow.txt
  Tue Dec 23 14:43:33 PST 2003
  Tue Dec 23 14:46:10 PST 2003
• 
  
• Remember: Use ">" to overwrite any existing file, use ">>" to append to any existing file. In both cases, if no file exists, one is created.

• Many commands have inputs as well as outputs. The input defaults to the keyboard, the output defaults to the screen.
• To redirect the output to a file, use ">" or ">>" as shown above.
• To make the output of a command serve as the input of another command, use "|".
• To make the contents of a file serve as the input to a command, use "<":
  $ wc < RightNow.txt
  2 12 58
  As is so often the case in shell programming, there is at least one other way to produce the above result:
  $ cat RightNow.txt | wc
  2 12 58

Shell Script Basics:

To be executable, a shell script file must meet some conditions:

• The file must have a special first line that names an appropriate command processor. For this tutorial, the following will work in most cases:
  #!/bin/bash
• If this example doesn't work, you will need to find out where your Bash shell executable is located and substitute that location in the above example. Here is one way to find out:
  $ whereis bash
• 
  
• The file must be made executable by changing its permission bits. An example:
  $ chmod +x (shell script filename)
• One normally executes a shell script this way:
  $ ./scriptname.sh

First Shell Script

1. Run your choice of editor and type the following lines:

$vi myscript.sh

 #!/bin/bash
 echo "Hello, world."

$ chmod +x myscript.sh

$ ./myscript.sh


Tests and Branching

Bash shell scripts can perform, and act on, various kinds of tests. This will be just the most basic introduction 

Here is an example of a test and branch:

if [ -e . ]
then
 echo "Yes."
else
 echo "No."
fi
              
              

Run the test script:
$ ./myscript.sh
Yes.

We created a test (the part of the script between "[" and "]") which tested whether a particular element existed ("-e"). Because the symbol "." in this context means the current directory, the test succeeded. Try replacing the "." with something that is not present in the current directory, example "xyz". See how the outcome changes.


It is important to realize that "[" is an alias for the command "test". The script could have been written as:

if test -e .
then
 echo "Yes."
else
 echo "No."
fi
              
              

$man test

test - check file types and compare values

Otherwise, EXPRESSION is true or false and sets exit status.  

It is one of:

       ( EXPRESSION )
              EXPRESSION is true

       ! EXPRESSION
              EXPRESSION is false

       EXPRESSION1 -a  EXPRESSION2
              both EXPRESSION1 and EXPRESSION2 are true

       EXPRESSION1 -o EXPRESSION2

             either EXPRESSION1 or EXPRESSION2 is true

-n STRING

              the length of STRING is nonzero

       

-z STRING

              the length of STRING is zero

STRING1 = STRING2

              the strings are equal

STRING1 != STRING2

              the strings are not equal

INTEGER1 -eq INTEGER2

              INTEGER1 is equal to INTEGER2

INTEGER1 -ge INTEGER2

              INTEGER1 is greater than or equal to INTEGER2

INTEGER1 -gt INTEGER2

              INTEGER1 is greater than INTEGER2

INTEGER1 -le INTEGER2

              INTEGER1 is less than or equal to INTEGER2

INTEGER1 -lt INTEGER2

              INTEGER1 is less than INTEGER2

INTEGER1 -ne INTEGER2

              INTEGER1 is not equal to INTEGER2

FILE1 -ef FILE2

              FILE1 and FILE2 have the same device and inode numbers

FILE1 -nt FILE2

              FILE1 is newer (modification date) than FILE2

FILE1 -ot FILE2

              FILE1 is older than FILE2

-b FILE

              FILE exists and is block special

-c FILE

              FILE exists and is character special

-d FILE

              FILE exists and is a directory

-e FILE

              FILE exists

-f FILE

              FILE exists and is a regular file

-g FILE

              FILE exists and is set-group-ID

-G FILE

              FILE exists and is owned by the effective group ID

-h FILE

              FILE exists and is a symbolic link (same as -L)

-k FILE

              FILE exists and has its sticky bit set

-L FILE

              FILE exists and is a symbolic link (same as -h)

-O FILE

              FILE exists and is owned by the effective user ID

-p FILE

              FILE exists and is a named pipe

-r FILE

              FILE exists and read permission is granted

-s FILE

              FILE exists and has a size greater than zero

-S FILE

              FILE exists and is a socket

-t FD  file descriptor FD is opened on a terminal

-u FILE

              FILE exists and its set-user-ID bit is set

-w FILE

              FILE exists and write permission is granted

-x FILE

              FILE exists and execute (or search) permission is granted

Note:when a test is conducted or a command returns a result value, the numerical value for "true" is 0, and "false" is 1. 

$ echo $(( 0 && 0 ))
0

$ echo $(( 1 && 0 ))
0
$ echo $(( 0 && 1 ))
0
$ echo $(( 1 && 1 ))
1

A couple of rules about logical operators used as branches:

• If you write "test && command", the command will only be executed if the test succeeds.
• If you write "test || command", the command will only be executed if the test fails.

Run these tests:
$ true && echo "Yes."
Yes.

$ false || echo "Yes."
Yes.

========================================================================


Loops and Repetition:

1. Here are some examples of loop operators:

for fn in *; #

do
   echo "$fn"
done

==>the "*" is expanded by the shell to a list of all the files in the current directory.