Thursday 5 January 2012

shell script basics

Note:

Well,
Bash is intepreted line by line as it runs and depends on calling a lot of external progs (depending on what you want to do).
You often have to use temp files as intermediate storage for result sets.
It (shell) was originally designed to talk to the system and automate cmd sequences (shell files).

Perl is more like C, it's largely self contained with a huge library of free code and it's compiled , so it runs much faster, eg about 80-90% speed of C, but easier to program (eg variable sizes are dynamic).
Syntax is similar to C. 

1)UNIX Commands:

Although a shell script can make use of any unix commands here are a number of commands which are more often used than others. These commands can generally be described as commands for file and text manipulation.

Command syntax
Purpose
echo "some text"
write some text on your screen
ls
list files
wc -l file
wc -w file
wc -c file
count lines in file or
count words in file or
count number of characters
cp sourcefile destfile
copy sourcefile to destfile
mv oldname newname
rename or move file
rm file
delete a file
grep 'pattern' file
search for strings in a file
Example: grep 'searchstring' file.txt
cut -b colnum file
get data out of fixed width columns of text
Example: get character positions 5 to 9
cut -b5-9 file.txt
Do not confuse this command with "cat" which is something totally different
cat file.txt
write file.txt to stdout (your screen)




file somefile
describe what type of file somefile is
read var
prompt the user for input and write it into a variable (var)
sort file.txt
sort lines in file.txt
uniq
remove duplicate lines, used in combination with sort   since uniq removes only duplicated consecutive lines
Example: sort file.txt | uniq
expr
do math in the shell
Example: add 2 and 3
expr 2 "+" 3
find
search for files
Example: search by name:
find . -name filename -print
This command has many different possibilities and options. It is unfortunately too much to explain it all in this article.
tee
write data to stdout (your screen) and to a file
Normally used like this:
somecommand | tee outfile
It writes the output of somecommand to the screen and to the file outfile
basename file
return just the file name of a given name and strip the directory path
Example: basename /bin/tux
returns just tux
dirname file
return just the directory name of a given name and strip the actual file name
Example: dirname /bin/tux
returns just /bin
head file
print some lines from the beginning of a file
tail file
print some lines from the end of a file
sed
sed is basically a find and replace program. It reads text from standard input (e.g from a pipe) and writes the result to stdout (normally the screen). The search pattern is a regular expression (see references). This search pattern should not be confused with shell wildcard syntax. To replace the string linuxfocus with LinuxFocus in a text file use:
cat text.file | sed 's/linuxfocus/LinuxFocus/' > newtext.file
This replaces the first occurance of the string linuxfocus in each line with LinuxFocus. If there are lines where linuxfocus appears several times and you   want to replace all use:
cat text.file | sed 's/linuxfocus/LinuxFocus/g' > newtext.file
awk
Most of the time awk is used to extract fields from a text line. The default field separator is space. To specify a different one use the option -F.
 cat file.txt | awk -F, '{print $1 "," $3 }'
 
Here we use the comma (,) as field separator and print the first and third ($1 $3) columns. If file.txt has lines like:
Adam Bor, 34, India
Kerry Miller, 22, USA
 
then this will produce:
Adam Bor, India
Kerry Miller, USA
 
There is much more you can do with awk but this is a very common use.

Simple Stuff:

First, a convention. I'll list things for you to type in this format:
$ date
I will list the computer's reply like this:
Tue Dec 23 10:52:51 PST 2003

To put this another way, enter this example:
# whoami
root

Where are you?

As you may be aware, a Linux filesystem is in the form of a large tree with many branches called "subdirectories". When you issue a shell command,
$ pwd
/path/path/path

You can decide where you are in the tree. Type this example:
$ cd ~
$ pwd
/home/username


Listing Files:

Directories contain files, and you can list them in a simple, compact format:
$ ls
filename filename filename ...

Or you can list them in more detail:
$ ls -la
(detailed list, one file per line)

And, very important, to find out what a command's options are, use the "man" (manual) command:
$ man ls
(manual page for "ls")


To find files by name:
$ find . -name '*.jpg'
(list of files with .jpg suffix in current and all child directories)

To create a text diagram of the directory tree:
$ tree -d .
(diagram of the directory tree from the current directory)


Examining Files:

There are a number of things you can do to find out more about the files in the list. Here are just a few:
The "file" command tries to identify files by examining their contents:

$ file tux_small.png
tux_small.png: PNG image data, 128 x 151, 8-bit/color RGB, non-interlaced


The next example uses the obscurely named "cat" command. It prints the contents of a file. Unfortunately if the file's contents are not readable, they get printed anyway.
$ cat zipcodes.txt
(prints the entire contents of a file named "zipcodes.txt")


If a file is too long to be viewed on one page, you can say:
$ more zipcodes.txt
(prints file one screenful at a time)


You can also use "grep" to print only those parts of a file you are interested in:
$ grep 10001 zipcodes.txt
(prints only those lines that have the character string "10001" in them)


The "grep" command is very useful, unfortunately it has a difficult-to-remember name. Be sure to:
$ man grep



Pipelines and Redirection:

Enter this command:
$ echo "cherry apple peach"
cherry apple peach

$ echo "cherry apple peach" | tr " " "\n"
cherry
apple
peach

$ echo "cherry apple peach" | tr " " "\n" | sort
apple
cherry
peach


$ echo "cherry apple peach" | tr " " "\n" | sort -r
peach
cherry
apple

Remember: A pipeline ("|") takes the output of one command and makes it the input to another command.


  • Normally the output from commands is printed on the screen. But using the symbol ">", you can redirect the output to a file:
    $ date > RightNow.txt
    $ cat RightNow.txt
    Tue Dec 23 14:43:33 PST 2003
  • The above example used ">" to replace the content of any existing file having the name "RightNow.txt". To append new data to an existing file, use ">>" instead:
    $ date >> RightNow.txt
    $ cat RightNow.txt
    Tue Dec 23 14:43:33 PST 2003
    Tue Dec 23 14:46:10 PST 2003

  • Remember: Use ">" to overwrite any existing file, use ">>" to append to any existing file. In both cases, if no file exists, one is created.

  • Many commands have inputs as well as outputs. The input defaults to the keyboard, the output defaults to the screen.
  • To redirect the output to a file, use ">" or ">>" as shown above.
  • To make the output of a command serve as the input of another command, use "|".
  • To make the contents of a file serve as the input to a command, use "<":
    $ wc < RightNow.txt
    2 12 58
    As is so often the case in shell programming, there is at least one other way to produce the above result:
    $ cat RightNow.txt | wc
    2 12 58
Shell Script Basics:

To be executable, a shell script file must meet some conditions:

  • The file must have a special first line that names an appropriate command processor. For this tutorial, the following will work in most cases:
    #!/bin/bash
  • If this example doesn't work, you will need to find out where your Bash shell executable is located and substitute that location in the above example. Here is one way to find out:
    $ whereis bash

  • The file must be made executable by changing its permission bits. An example:
    $ chmod +x (shell script filename)
  • One normally executes a shell script this way:
    $ ./scriptname.sh
First Shell Script

1. Run your choice of editor and type the following lines:

$vi myscript.sh



 #!/bin/bash
 echo "Hello, world."


$ chmod +x myscript.sh

$ ./myscript.sh


Tests and Branching

Bash shell scripts can perform, and act on, various kinds of tests. This will be just the most basic introduction 

Here is an example of a test and branch:


if [ -e . ]
then
 echo "Yes."
else
 echo "No."
fi
              
              
Run the test script:
$ ./myscript.sh
Yes.


We created a test (the part of the script between "[" and "]") which tested whether a particular element existed ("-e"). Because the symbol "." in this context means the current directory, the test succeeded. Try replacing the "." with something that is not present in the current directory, example "xyz". See how the outcome changes.


It is important to realize that "[" is an alias for the command "test". The script could have been written as:


if test -e .
then
 echo "Yes."
else
 echo "No."
fi
              
              
$man test

test - check file types and compare values

Otherwise, EXPRESSION is true or false and sets exit status.  
It is one of:

       ( EXPRESSION )
              EXPRESSION is true

       ! EXPRESSION
              EXPRESSION is false

       EXPRESSION1 -a  EXPRESSION2
              both EXPRESSION1 and EXPRESSION2 are true

       EXPRESSION1 -o EXPRESSION2
             either EXPRESSION1 or EXPRESSION2 is true

-n STRING
              the length of STRING is nonzero

       
-z STRING
              the length of STRING is zero

STRING1 = STRING2
              the strings are equal

STRING1 != STRING2
              the strings are not equal

INTEGER1 -eq INTEGER2
              INTEGER1 is equal to INTEGER2

INTEGER1 -ge INTEGER2
              INTEGER1 is greater than or equal to INTEGER2

INTEGER1 -gt INTEGER2
              INTEGER1 is greater than INTEGER2

INTEGER1 -le INTEGER2
              INTEGER1 is less than or equal to INTEGER2

INTEGER1 -lt INTEGER2
              INTEGER1 is less than INTEGER2

INTEGER1 -ne INTEGER2
              INTEGER1 is not equal to INTEGER2

FILE1 -ef FILE2
              FILE1 and FILE2 have the same device and inode numbers

FILE1 -nt FILE2
              FILE1 is newer (modification date) than FILE2

FILE1 -ot FILE2
              FILE1 is older than FILE2

-b FILE
              FILE exists and is block special

-c FILE
              FILE exists and is character special

-d FILE
              FILE exists and is a directory

-e FILE
              FILE exists

-f FILE
              FILE exists and is a regular file

-g FILE
              FILE exists and is set-group-ID

-G FILE
              FILE exists and is owned by the effective group ID

-h FILE
              FILE exists and is a symbolic link (same as -L)

-k FILE
              FILE exists and has its sticky bit set

-L FILE
              FILE exists and is a symbolic link (same as -h)

-O FILE
              FILE exists and is owned by the effective user ID

-p FILE
              FILE exists and is a named pipe

-r FILE
              FILE exists and read permission is granted

-s FILE
              FILE exists and has a size greater than zero

-S FILE
              FILE exists and is a socket

-t FD  file descriptor FD is opened on a terminal

-u FILE
              FILE exists and its set-user-ID bit is set

-w FILE
              FILE exists and write permission is granted

-x FILE
              FILE exists and execute (or search) permission is granted

Note:when a test is conducted or a command returns a result value, the numerical value for "true" is 0, and "false" is 1. 

$ echo $(( 0 && 0 ))
0


$ echo $(( 1 && 0 ))
0

$ echo $(( 0 && 1 ))
0

$ echo $(( 1 && 1 ))
1


A couple of rules about logical operators used as branches:
  • If you write "test && command", the command will only be executed if the test succeeds.
  • If you write "test || command", the command will only be executed if the test fails.
Run these tests:
$ true && echo "Yes."
Yes.


$ false || echo "Yes."
Yes.
========================================================================


Loops and Repetition:

1. Here are some examples of loop operators:


for fn in *; #

do
   echo "$fn"
done

==>the "*" is expanded by the shell to a list of all the files in the current directory.

            

No comments:

Post a Comment