Monday, 29 October 2012

Reference Cards to Shell Script(Over View): PART 6

Reference Cards:

Table 1. Special Shell Variables:

$0Filename of script
$1Positional parameter #1
$2 - $9Positional parameters #2 - #9
${10}Positional parameter #10
$#Number of positional parameters
"$*"All the positional parameters (as a single word) *
"$@"All the positional parameters (as separate strings)
${#*}Number of positional parameters
${#@}Number of positional parameters
$?Return value
$$Process ID (PID) of script
$-Flags passed to script (using set)
$_Last argument of previous command
$!Process ID (PID) of last job run in background
* Must be quoted, otherwise it defaults to $@.

Table 2. TEST Operators: Binary Comparison:

Arithmetic ComparisonString Comparison
-eqEqual to=Equal to
==Equal to
-neNot equal to!=Not equal to
-ltLess than\<Less than (ASCII) *
-leLess than or equal to
-gtGreater than\>Greater than (ASCII) *
-geGreater than or equal to
-zString is empty
-nString is not empty
Arithmetic Comparisonwithin double parentheses (( ... ))
>Greater than
>=Greater than or equal to
<Less than
<=Less than or equal to
* If within a double-bracket [[ ... ]] test construct, then no escape \ is needed.

Table 3. TEST Operators: Files:

OperatorTests Whether-----OperatorTests Whether
-eFile exists-sFile is not zero size
-fFile is a regular file
-dFile is a directory-rFile has read permission
-hFile is a symbolic link-wFile has write permission
-LFile is a symbolic link-xFile has execute permission
-bFile is a block device
-cFile is a character device-gsgid flag set
-pFile is a pipe-usuid flag set
-SFile is a socket-k"sticky bit" set
-tFile is associated with a terminal
-NFile modified since it was last readF1 -nt F2File F1 is newer than F2 *
-OYou own the fileF1 -ot F2File F1 is older than F2 *
-GGroup id of file same as yoursF1 -ef F2Files F1 and F2 are hard links to the same file *
!NOT (inverts sense of above tests)
* Binary operator (requires two operands).

Table 4. Parameter Substitution and Expansion:

${var}Value of var (same as $var)
${var-DEFAULT}If var not set, evaluate expression as $DEFAULT *
${var:-DEFAULT}If var not set or is empty, evaluate expression as $DEFAULT *
${var=DEFAULT}If var not set, evaluate expression as $DEFAULT *
${var:=DEFAULT}If var not set, evaluate expression as $DEFAULT *
${var+OTHER}If var set, evaluate expression as $OTHER, otherwise as null string
${var:+OTHER}If var set, evaluate expression as $OTHER, otherwise as null string
${var?ERR_MSG}If var not set, print $ERR_MSG and abort script with an exit status of 1.*
${var:?ERR_MSG}If var not set, print $ERR_MSG and abort script with an exit status of 1.*
${!varprefix*}Matches all previously declared variables beginning with varprefix
${!varprefix@}Matches all previously declared variables beginning with varprefix
* If var is set, evaluate the expression as $var with no side-effects.

Table 5. String Operations:

${#string}Length of $string
${string:position}Extract substring from $string at $position
${string:position:length}Extract $length characters substring from $string at $position [zero-indexed, first character is at position 0]
${string#substring}Strip shortest match of $substring from front of $string
${string##substring}Strip longest match of $substring from front of $string
${string%substring}Strip shortest match of $substring from back of $string
${string%%substring}Strip longest match of $substring from back of $string
${string/substring/replacement}Replace first match of $substring with $replacement
${string//substring/replacement}Replace all matches of $substring with $replacement
${string/#substring/replacement}If $substring matches front end of $string, substitute $replacement for $substring
${string/%substring/replacement}If $substring matches back end of $string, substitute $replacement for $substring
expr match "$string" '$substring'Length of matching $substring* at beginning of $string
expr "$string" : '$substring'Length of matching $substring* at beginning of $string
expr index "$string" $substringNumerical position in $string of first character in $substring* that matches [0 if no match, first character counts as position 1]
expr substr $string $position $lengthExtract $length characters from $string starting at $position [0 if no match, first character counts as position 1]
expr match "$string" '\($substring\)'Extract $substring*, searching from beginning of $string
expr "$string" : '\($substring\)'Extract $substring* , searching from beginning of $string
expr match "$string" '.*\($substring\)'Extract $substring*, searching from end of $string
expr "$string" : '.*\($substring\)'Extract $substring*, searching from end of $string
* Where $substring is a Regular Expression.

Table 6. Miscellaneous Constructs:

if [ CONDITION ]Test construct
if [[ CONDITION ]]Extended test construct
Array[1]=element1Array initialization
[a-z]Range of characters within a Regular Expression
Curly Brackets
${variable}Parameter substitution
${!variable}Indirect variable reference
{ command1; command2; . . . commandN; }Block of code
{string1,string2,string3,...}Brace expansion
{a..z}Extended brace expansion
{}Text replacement, after find and xargs
( command1; command2 )Command group executed within a subshell
Array=(element1 element2 element3)Array initialization
result=$(COMMAND)Command substitution, new style
>(COMMAND)Process substitution
<(COMMAND)Process substitution
Double Parentheses
(( var = 78 ))Integer arithmetic
var=$(( 20 + 5 ))Integer arithmetic, with variable assignment
(( var++ ))C-style variable increment
(( var-- ))C-style variable decrement
(( var0 = var1<98?9:21 ))C-style ternary operation
"$variable""Weak" quoting
'string''Strong' quoting
Back Quotes
result=`COMMAND`Command substitution, classic style

This is a very brief introduction to the sed and awk text processing utilities.

sed: a non-interactive text file editor
awk: a field-oriented pattern processing language with a C-style syntax

Operator Precedence
Table 7. Operator Precedence
var++ var--post-increment, post-decrementC-style operators
++var --varpre-increment, pre-decrement
! ~negationlogical / bitwise, inverts sense of following operator
**exponentiationarithmetic operation
* / %multiplication, division, moduloarithmetic operation
+ -addition, subtractionarithmetic operation
<< >>left, right shiftbitwise
-z -nunary comparisonstring is/is-not null
-e -f -t -x, etc.unary comparisonfile-test
< -lt > -gt <= -le >= -gecompound comparisonstring and integer
-nt -ot -efcompound comparisonfile-test
== -eq != -neequality / inequalitytest operators, string and integer
^XORexclusive OR, bitwise
&& -aANDlogicalcompound comparison
|| -oORlogical, compound comparison
?:trinary operatorC-style
=assignment(do not confuse with equality test)
*= /= %= += -= <<= >>= &=combination assignmenttimes-equal, divide-equal, mod-equal, etc.
,commalinks a sequence of operations

Table 8. Job identifiers
%NJob number [N]
%SInvocation (command-line) of job begins with string S
%?SInvocation (command-line) of job contains within it string S
%%"current" job (last job stopped in foreground or started in background)
%+"current" job (last job stopped in foreground or started in background)
%-Last job
$!Last background process

Exit Codes With Special Meanings

Table . Reserved Exit Codes:

Exit Code NumberMeaningExampleComments
1Catchall for general errorslet "var1 = 1/0"Miscellaneous errors, such as "divide by zero" and other impermissible operations
2Misuse of shell builtins (according to Bash documentation)empty_function() {}Missing keyword or command
126Command invoked cannot execute/dev/nullPermission problem or command is not an executable
127"command not found"illegal_commandPossible problem with $PATH or a typo
128Invalid argument to exitexit 3.14159exit takes only integer args in the range 0 - 255 (see first footnote)
128+nFatal error signal "n"kill -9 $PPID of script$? returns 137 (128 + 9)
130Script terminated by Control-CCtl-CControl-C is fatal error signal 2, (130 = 128 + 2, see above)
255*Exit status out of rangeexit -1exit takes only integer args in the range 0 - 255


1. Out of range exit values can result in unexpected exit codes. An exit value greater than 255 returns an exit code modulo 256. For example, exit 3809 gives an exit code of 225 (3809 % 256 = 225).

2.An update of /usr/include/sysexits.h allocates previously unused exit codes from 64 - 78. It may be anticipated that the range of unallotted exit codes will be further restricted in the future.

A Detailed Introduction to I/O and I/O Redirection

A command expects the first three file descriptors to be available. 

The first, fd 0 (standard input, stdin), is for reading.
The other two (fd 1stdout and fd 2,stderr) are for writing.

There is a stdinstdout, and a stderr associated with each command. ls 2>&1 means temporarily connecting the stderr of the ls command to the same"resource" as the shell's stdout.

[sankar@new-host ~]$ cat /etc/passwd >&-
cat: standard output: Bad file descriptor

Standard Command-Line Options:

The two most widely-accepted options are:

-h or (--help) ==> Help: Give usage message and exit.

-v or (--version) ==> Version: Show program version and exit.

-r or -R (--recursive) ==> Recursive: Operate recursively (down directory tree).

-v or (--verbose) ==> Verbose: output additional information to stdout or stderr.

Important Files

startup files
These files contain the aliases and environmental variables made available to Bash running as a user shell and to all Bash scripts invoked after system initialization.

Systemwide defaults, mostly setting the environment (all Bourne-type shells, not just Bash)

systemwide functions and aliases for Bash

user-specific Bash environmental default settings, found in each user's home directory (the local counterpart to /etc/profile)

user-specific Bash init file, found in each user's home directory (the local counterpart to /etc/bashrc). Only interactive shells and user scripts read this file. 

logout file

user-specific instruction file, found in each user's home directory. Upon exit from a login (Bash) shell, the commands in this file execute.

data files

A listing of all the user accounts on the system, their identities, their home directories, the groups they belong to, and their default shell. Note that the user passwords are not stored in this file, 
But in /etc/shadow in encrypted form.

system configuration files

Listing and description of attached hardware devices. This information is in text form and can be extracted and parsed.

bash$ grep -A 5 AUDIO /etc/sysconfig/hwconf       
 class: AUDIO
 bus: PCI
 detached: 0
 driver: snd-intel8x0
 desc: "Intel Corporation 82801CA/CAM AC'97 Audio Controller"
 vendorId: 8086

Important System Directories

Sysadmins and anyone else writing administrative scripts should be intimately familiar with the following system directories.

  • /bin
    Binaries (executables). Basic system programs and utilities (such as bash).
  • /usr/bin 
    More system binaries.
  • /usr/local/bin
    Miscellaneous binaries local to the particular machine.
  • /sbin
    System binaries. Basic system administrative programs and utilities (such as fsck).
  • /usr/sbin
    More system administrative programs and utilities.
  • /etc
    Et cetera. Systemwide configuration scripts.
    Of particular interest are the /etc/fstab (filesystem table), /etc/mtab (mounted filesystem table), and the /etc/inittab files.
  • /etc/rc.d
    Boot scripts, on Red Hat and derivative distributions of Linux.
  • /usr/share/doc
    Documentation for installed packages.
  • /usr/man
    The systemwide manpages.
  • /dev
    Device directory. Entries (but not mount points) for physical and virtual devices.
  • /proc
    Process directory. Contains information and statistics about running processes and kernel parameters.
  • /sys
    Systemwide device directory. Contains information and statistics about device and device names. This is newly added to Linux with the 2.6.X kernels.
  • /mnt
    Mount. Directory for mounting hard drive partitions, such as /mnt/dos, and physical devices. In newer Linux distros, the /media directory has taken over as the preferred mount point for I/O devices.
  • /media
    In newer Linux distros, the preferred mount point for I/O devices, such as CD/DVD drives or USB flash drives.
  • /var
    Variable (changeable) system files. This is a catchall "scratchpad" directory for data generated while a Linux/UNIX machine is running.
  • /var/log
    Systemwide log files.
  • /var/spool/mail
    User mail spool.
  • /lib
    Systemwide library files.
  • /usr/lib
    More systemwide library files.
  • /tmp
    System temporary files.
  • /boot
    System boot directory. The kernel, module links, system map, and boot manager reside here.

    History Commands

    Bash history commands:

    1. history
    2. fc

    bash$ history
       1  mount /mnt/cdrom
        2  cd /mnt/cdrom
        3  ls

    Internal variables associated with Bash history commands:

    1. $HISTCMD
    4. $HISTFILE
    6. $HISTSIZE
    7. $HISTTIMEFORMAT (Bash, ver. 3.0 or later)
    8. !!
    9. !$
    10. !#
    11. !N
    12. !-N
    13. !STRING
    14. !?STRING?
    15. ^STRING^string^

    Unfortunately, the Bash history tools find no use in scripting.

    # A (vain) attempt to use the 'history' command in a script.
    history                      # No output.
    var=$(history); echo "$var"  # $var is empty.
    # History commands disabled within a script.

    bash$ ./
    (no output)  

    Converting DOS Batch Files to Shell Scripts

    Table 1. Batch file keywords / variables / operators, and their shell equivalents

    Batch File OperatorShell Script EquivalentMeaning
    %$command-line parameter prefix
    /-command option flag
    \/directory path separator
    ===(equal-to) string comparison test
    !==!!=(not equal-to) string comparison test
    @set +vdo not echo current command
    **filename "wild card"
    >>file redirection (overwrite)
    >>>>file redirection (append)
    <<redirect stdin
    %VAR%$VARenvironmental variable
    NOT!negate following test
    NUL/dev/null"black hole" for burying command output
    ECHOechoecho (many more option in Bash)
    ECHO.echoecho blank line
    ECHO OFFset +vdo not echo command(s) following
    FOR %%VAR IN (LIST) DOfor var in [list]; do"for" loop
    :LABELnone (unnecessary)label
    GOTOnone (use a function)jump to another location in the script
    PAUSEsleeppause or wait an interval
    CHOICEcase or selectmenu choice
    IF EXIST FILENAMEif [ -e filename ]test if file exists
    IF !%N==!if [ -z "$N" ]if replaceable parameter "N" not present
    CALLsource or . (dot operator)"include" another script
    COMMAND /Csource or . (dot operator)"include" another script (same as CALL)
    SETexportset an environmental variable
    SHIFTshiftleft shift command-line argument list
    SGN-lt or -gtsign (of integer)
    ERRORLEVEL$?exit status
    CONstdin"console" (stdin)
    PRN/dev/lp0(generic) printer device
    LPT1/dev/lp0first printer device
    COM1/dev/ttyS0first serial port
    Batch files usually contain DOS commands. These must be translated into their UNIX equivalents in order to convert a batch file into a shell script.

    Table 2. DOS commands and their UNIX equivalents

    DOS CommandUNIX EquivalentEffect
    ASSIGNlnlink file or directory
    ATTRIBchmodchange file permissions
    CDcdchange directory
    CHDIRcdchange directory
    CLSclearclear screen
    COMPdiff, comm, cmpfile compare
    COPYcpfile copy
    Ctl-CCtl-Cbreak (signal)
    Ctl-ZCtl-DEOF (end-of-file)
    DELrmdelete file(s)
    DELTREErm -rfdelete directory recursively
    DIRls -ldirectory listing
    ERASErmdelete file(s)
    EXITexitexit current process
    FCcomm, cmpfile compare
    FINDgrepfind strings in files
    MDmkdirmake directory
    MKDIRmkdirmake directory
    MOREmoretext file paging filter
    PATH$PATHpath to executables
    RENmvrename (move)
    RENAMEmvrename (move)
    RDrmdirremove directory
    RMDIRrmdirremove directory
    SORTsortsort file
    TIMEdatedisplay system time
    TYPEcatoutput file to stdout
    XCOPYcp(extended) file copy

No comments:

Post a Comment