Getting started with Shell Programming

In this tutorial you are introduce to shell programming, how to write script, execute them etc. We will getting started with writing small shell script, that will print Knowledge is Power on screen.

How to write shell script

Following steps required to write shell script

1. Use any editor like vi or mcedit to write shell script.
   
2. After writing shell script set execute permission for your script as follows
   syntax: 
   chmod         permission             your-script-name
   
   Examples
   $ chmod     +x         your-script-name
   $ chmod    755        your-script-name
   
   Note This will set read write execute(7) permission for owner, for group and other permission is read and execute only(5).    
   
3. Execute your script as
   syntax: 
   bash    your-script-name
   sh       your-script-name
   ./your-script-name
   
   Examples 
   $ bash   bar
   $ sh   bar
   $ ./bar
   
   NOTE In the last syntax ./ means current directory, But only . (dot) means execute given command file in current shell without starting the new copy of shell, The syntax for . (dot) command is as follows
   Syntax  
   .  command-name
   
   Example
   $ . foo

Here we are writing our first shell script. See the common vi command list, if you are new to vi.

Now we write our first script that will print "Knowledge is Power" on screen.

After saving the above script, you can run the script as follows
$ ./first.sh

This will not run script since we have not set Execute permission for our script first.sh; to do this type command
$ chmod 755 first.sh
$ ./first.sh

First screen will be clear, then Knowledge is Power is printed on screen. 

How Shell Locates the file (My own bin directory to execute script)

Exercise
1) Now write following shell script and note down the it's output.

Variables in Linux

Sometimes to process our data/information, data must be kept in computers RAM memory. RAM memory is divided into small locations, and each location had unique number called memory location/address, which is used to hold our data. Programmer can give a unique name to this memory location/address called memory variable or variable (Its a named storage location that may take different values, but only one at a time). 

In Linux (Shell), there are two types of variable
1) System variables - Created and maintained by Linux itself. This type of variable defined in CAPITAL LETTERS.
2) User defined variables (UDV) - Created and maintained by user. This type of variable defined in lower letters.

You can see system variables by giving command like $ set, some of the important System variables are

NOTE: Here 'value' is assigned to given 'variable name' and Value must be on right side = sign.
 
Example
$ no=10        # this is ok
$ 10=no       # Error, NOT Ok, Value must be on right side of = sign.
To define variable called 'vech' having value Bus
$ vech=Bus  
To define variable called n having value 10
$ n=10         

Rules for Naming variable name (Both UDV and System Variable)

(1) Variable name must begin with Alphanumeric character or underscore character (_), followed by one or more Alphanumeric character. For e.g. Valid shell variable are as follows
HOME
SYSTEM_VERSION
vech
no

(2) Don't put spaces on either side of the equal sign when assigning value to variable. For e.g.. In following variable declaration there will be no error
$ no=10
But here there will be problem for following
$ no     =10
$ no=     10
$ no     =     10

(3) Variables are case-sensitive, just like filename in Linux. For e.g.
$ no=10
$ No=11
$ NO=20
$ nO=2
Above all are different variable name, so to print value 20 we have to use $ echo $NO and not any of the following
$ echo $no        # will print 10 but not 20
$ echo $No        # will print 11 but not 20
$ echo $nO        # will print 2 but not 20

(4) You can define NULL variable as follows (NULL variable is variable which has no value at the time of definition) For e.g.
$ vech=
$ vech=""
Try to print it's value $ echo $vech , Here nothing will be shown because variable has no value i.e. NULL variable.

(5) Do not use ?,* etc, to name your variable names.

How to print or access value of UDV (User defined variables)

To print or access UDV use following syntax
Syntax:  
$variablename
For eg. To print contains of variable 'vech'
$ echo $vech
It will print 'Bus' (if previously defined as vech=Bus) ,To print contains of variable 'n' 
$ echo $n
It will print '10' (if previously defined as n=10)

Caution: Do not try $ echo vech It will print vech instead its value 'Bus' and $ echo n, It will print n instead its value '10', You must use $ followed by variable name.

Exercise
Q.1.How to Define variable x with value 10 and print it on screen.
Q.2.How to Define variable xn with value Rani and print it on screen
Q.3.How to print sum of two numbers, let's say 6 and 3
Q.4.How to define two variable x=20, y=5 and then to print division of x and y (i.e. x/y)
Q.5.Modify above and store division of x and y to variable called z
Q.6.Point out error if any in following script

For Answers Click here

echo [options] [string, variables...]
Displays text or variables value on screen.
Options
-n Do not output the trailing new line.
-e Enable interpretation of the following backslash escaped characters in the strings:
\a alert (bell)
\b backspace
\c suppress trailing new line
\n new line
\r carriage return
\t horizontal tab
\\ backslash

For e.g. $ echo -e "An apple a day keeps away \a\t\tdoctor\n"

How to display color with echo, bold and blink effects, how to print text on any row, column on screen, click here for more!

Shell Arithmetic

Use to perform arithmetic operations.

Syntax:  
expr   op1  math-operator   op2
Examples
$ expr 1 + 3
$ expr 2 - 1
$ expr 10 / 2
$ expr 20 % 3 # remainder read as 20 mod 3 and remainder is 2
$ expr 10 \* 3 # Multiplication use \* not * since its wild card
$ echo `expr 6 + 3`

For the last statement not the following points

(1) First, before expr keyword we used ` (back quote) sign not the (single quote i.e. ') sign. Back quote is generally found on the key under tilde (~) on PC keyboard OR to the above of TAB key.

(2) Second, expr is also end with ` i.e. back quote.

(3) Here expr 6 + 3 is evaluated to 9, then echo command prints 9 as sum

(4) Here if you use double quote or single quote, it will NOT work 
For e.g.
$ echo "expr 6 + 3"    # It will print expr 6 + 3
$ echo 'expr 6 + 3'      # It will work

See Parameter substitution - To save your time.

More about Quotes

There are three types of quotes

Example
$ echo "Today is date"
Can't print message with today's date.
$ echo "Today is `date`".
Now it will print today's date as, Today is Tue Jan ....,Note that the `date` statement uses back quote. 

Exit Status

By default in Linux if particular command is executed, it return two type of values which is  used to see whether command is successful or not. 

(1)If return value is zero (0), command is successful.
(2)If return value is nonzero , command is not successful or some sort of error executing command/shell script.

This value is know as Exit Status of that command. 

To determine this exit Status we use $? variable of shell. 

For e.g. 
$ rm unknow1file
It will show error as follows
rm: cannot remove `unkowm1file': No such file or directory
and after that if you give command 
$ echo $?
it will print nonzero value to indicate error. Now give command
$ ls
$ echo $?
It will print 0 to indicate command is successful. 

Exercise
Try the following commands and not down there exit status
$ expr 1 + 3
$ echo $?

$ echo Welcome
$ echo $?

$ wildwest canwork?
$ echo $?

$ date
$ echo $?

$ echon $?
$ echo $?

$? useful variable, want to know more such Linux variables click here to explore them! 

Use to get input (data from user) from keyboard and store (data) to variable.
Syntax: read variable1, variable2,...variableN

Run it as follows
$ chmod  755 sayH
$ ./sayH

This script first ask you your name and then waits to enter name from the user, Then user enters name from keyboard (After giving name you have to press ENTER key) and entered name through keyboard is stored (assigned) to variable fname. 

Wild cards (Filename Shorthand or meta Characters) 

Note:
[..-..] A pair of characters separated by a minus sign denotes a range.

Example
$ ls /bin/[a-c]*  

Will show all files name beginning with letter a,b or c like

   /bin/arch           /bin/awk           /bin/bsh     /bin/chmod           /bin/cp
   /bin/ash           /bin/basename   /bin/cat      /bin/chown           /bin/cpio
   /bin/ash.static   /bin/bash          /bin/chgrp   /bin/consolechars  /bin/csh

But
$ ls /bin/[!a-o]
$ ls /bin/[^a-o]

If the first character following the [ is a ! or a ^ ,then any character not enclosed is matched i.e. do not show us file name that beginning with a,b,c,e...o, like

   /bin/ps            /bin/rvi              /bin/sleep /bin/touch      /bin/view
   /bin/pwd           /bin/rview        /bin/sort   /bin/true        /bin/wcomp
   /bin/red           /bin/sayHello     /bin/stty   /bin/umount   /bin/xconf
   /bin/remadmin  /bin/sed           /bin/su      /bin/uname     /bin/ypdomainname
   /bin/rm            /bin/setserial    /bin/sync   /bin/userconf  /bin/zcat
   /bin/rmdir         /bin/sfxload      /bin/tar    /bin/usleep
   /bin/rpm           /bin/sh            /bin/tcsh    /bin/vi

Syntax:command1;command2
To run two command with one command line.

Examples 
$ date;who  
Will print today's date followed by users who are currently login. Note that You can't use 
$ date who 
for same purpose, you must put semicolon in between date and who command.

Command Line Processing
Now try following command (assumes that the file "grate_stories_of" is not exist on your system)

$ ls  grate_stories_of

It will print message something like - grate_stories_of: No such file or directory.

ls is the name of an actual command and shell executed this command when you type command at shell prompt. Now it creates one question What are commands? What happened when you type $ ls grate_stories_of ? 

The first word on command line, ls, is name of the command to be executed. 
Everything else on command line is taken as arguments to this command. For e.g.
$ tail   +10  myf
Here the name of command is tail, and the arguments are +10 and myf.

Exercise
Try to determine command and arguments from following commands
$ ls     foo
$ cp    y     y.bak
$ mv    y.bak     y.okay
$ tail    -10     myf
$ mail     raj
$ sort     -r    -n     myf
$ date
$ clear

Answer

NOTE: $# holds number of arguments specified on command line. And $* or $@ refer to all arguments passed to script. 

Why Command Line arguments required

1. Telling the command/utility which option to use
2. Informing the utility/command which file to read/write etc.

Let's take rm command, which is used to remove file, But which file you want to remove and how you will tail this to rm command (Even rm command does not ask you name of file that you would like to remove). So what we do is we write command as follows
$ rm       {file-name}
Here rm is command and file-name is file which you would like to remove. This way you tail to rm command which file you would like to remove. So we are doing one way communication with our command by specifying file-name. Also you can pass command line arguments to your script to make it more users friendly. But how we access command line argument in our script.

Lets take ls command
$ ls -a  /*
This command has 2 command line argument -a and /* is another. For shell script,
$ myshell  foo   bar

Shell Script name i.e. myshell
First command line argument passed to myshell i.e. foo
Second command line argument passed to myshell i.e. bar

In shell if we wish to refer this command line argument we refer above as follows

myshell it is  $0
foo it is  $1
bar it is  $2

Here $# (built in shell variable ) will be 2 (Since foo and bar only two Arguments), Please note at a time such 9 arguments can be used from $1..$9, You can also refer all of them by using $* (which expand to `$1,$2...$9`) 

Exercise
Try to write following for commands, 
Shell Script Name ($0), 
No. of Arguments (i.e. $#), 
And actual argument (i.e. $1,$2 etc)
$ sum   11   20
$ math   4   -   7
$ d
$ bp   -5   myf   +20
$ ls   *
$ cal
$ findBS   4   8   24   BIG

Answer

 

Shell Script Name	No. Of Arguments to script	Actual Argument ($1,..$9)
$0	$#	$1	$2	$3	$4	$5
sum	2	11	20
math	3	4	-	7
d	0
bp	3	-5	myf	+20
ls	1	*
cal	0
findBS	4	4	8	24	BIG

 

For e.g. now will write script to print command ling argument and we will see how to access them

Run it as follows

Set execute permission,
$ chmod  755  demo

Run it & test it,
$ ./demo  Hello World

If test successful, copy script to your own bin directory (Install script for private use)
$ cp demo ~/bin

Check whether it is working or not (?) 
$ demo
$ demo Hello World

NOTE After this, For any script you have to used above command, in sequence, I am not going to show you all of the above for rest of Tutorial.

Mostly all command gives output on screen or take input from keyboard, but in Linux it's possible to send output to file or to read input from file. 

For e.g. 
$ ls command gives output to screen; to send output to file of ls command give command 
 
$ ls > filename. It means put output of ls command to filename. 

There are three main redirection symbols >,>>,<

(1) > Redirector Symbol
Syntax:  Linux-command > filename
To output Linux-commands result to file. Note that If file already exist, it will be overwritten else new file is created. For e.g. To send output of ls command give
$ ls > myfiles
Now if 'myfiles' file exist in your current directory it will be overwritten without any type of warning. 

(2) >> Redirector Symbol
Syntax: Linux-command >> filename
To output Linux-commands result to END of file. Note that If file exist , it will be opened and new information/data will be written to END of file, without losing previous information/data, And if file is not exist, then new file is created. For e.g. To send output of date command to already exist file give command
$ date >> myfiles

(3) < Redirector Symbol
Syntax: Linux-command < filename
To take input to Linux-command from file instead of key-board. For e.g. To take input for cat command give
$ cat < myfiles

Click here to learn more about I/O Redirection

You can also use above redirectors simultaneously as follows
Create text file sname as follows

$cat > sname
vivek
ashish
zebra
babu
Press CTRL + D to save.

Now issue following command.

$ sort  < sname  > sorted_names
$ cat  sorted_names
ashish
babu
vivek
zebra

In above example sort command takes input from sname file and output of sort command (i.e. sorted names) is redirected to sorted_names file.

Now try following command
$ tr  "[a-z]"  "[A-Z]"  < sname  > cap_names
$ cat   cap_names
VIVEK
ASHISH
ZEBRA
BABU

Here tr command is used to translate all lower case characters to upper-case letters. It take input from sname file, and tr's output is redirected to cap_names file.

:-) Now try following command and find out most important point
$ sort  > new_sorted_names   < sname  
$ cat new_sorted_names 

Pips
A pipe is a way to connect the output of one program to the input of another program without any temporary file.

If a Linux command accepts its input from the standard input and produces its output on standard output is know as a filter. 

A filter performs some kind of process on the input and gives output. 

For e.g.. Suppose we have file called 'hotel.txt' with 100 lines data, And from 'hotel.txt' we would like to print contains from line number 20 to line number 30 and store this result to file called 'hlist' then give command

$ tail +20 < hotel.txt | head -n30 >hlist

Here head is filter which takes its input from tail command (tail command start selecting from line number 20 of given file i.e. hotel.txt) and passes this lines as input to head, whose output is redirected to 'hlist' file.

Now consider following example
$ sort < sname | uniq  > u_sname

Here uniq is filter which takes its input from sort command and passes this lines as input to uniq; Then uniqs output is redirected to "u_sname" file.

What is Processes

Process is kind of program or task carried out by your PC. 

For e.g.
$ ls -lR 
ls command or a request to list files in a directory and all subdirectory in your current directory. It is a process. A process is program (command given by user) to perform specific Job. In Linux when you start process, it gives a number (called PID or process-id), PID starts from 0 to 65535.

Why Process required

Linux is multi-user, multitasking Os. It means you can run more than two process simultaneously if you wish. For e.g. To find how many files do you have on your system you may give command like
$ ls / -R | wc -l
This command will take lot of time to search all files on your system. So you can run such command in Background or simultaneously by giving command like
$ ls / -R | wc -l  &
The ampersand (&) at the end of command tells shells start process (ls / -R | wc -l) and run it in background takes next command immediately. An instance of running command is called process and the number printed by shell is called process-id (PID), this PID can be use to refer specific running process.

Linux Command Related with Process

NOTE that you can only kill process which are created by yourself. A Administrator can almost kill 95-98% process. But some process can not be killed, such as VDU Process.

Shells (bash) structured Language Constructs

if-then-fi for decision making is shell script

Before making any decision in Shell script you must know following things Type bc at $ prompt to start Linux calculator program
$ bc
After this command bc is started and waiting for your commands, i.e. give it some calculation as follows type 5 + 2 as
5 + 2
7
7 is response of bc i.e. addition of 5 + 2 you can even try
5 - 2
5 / 2
Now what happened if you type 5 > 2 as follows
5 > 2
0
0 (Zero?) is response of bc, How? Here it compare 5 with 2 as, Is 5 is greater then 2, (If I ask same question to you, your answer will be YES) 

In Linux (bc) gives this 'YES' answer by showing 0 (Zero) value. It means when ever there is any type of comparison in Linux Shell It gives only two answer one is YES and NO is other.

Now run it.
$ chmod 755 showfile
$./showfile foo
Here our shell script name is showfile($0) and foo is argument (which is $1).Now we compare as follows
if cat $1 (i.e. if cat foo)
if cat command finds foo file and if its successfully shown on screen, it means our cat command is successful and its exist status is 0 (indicates success), So our if condition is also true and hence statement echo -e "\n\nFile $1, found and successfully echoed" is proceed by shell. Now if cat command is not successful then it returns non-zero value (indicates some sort of failure) and this statement echo -e "\n\nFile $1, found and successfully echoed" is skipped by our shell.

Exercise
Try to write answer for following

(Press Ctrl + d to save)
$ chmod  755 trmif
Now answer the following
A) There is file called foo, on your disk and you give command, $ ./trmfi   foo what will be output.
B) If bar file not present on your disk and you give command, $ ./trmfi   bar what will be output.
C) And if you type $ ./trmfi, What will be output.

For Answer click here.

test command or [ expr ]

test command or [ expr ] is used to see if an expression is true, and if it is true it return zero(0), otherwise returns nonzero(>0) for false. 
Syntax: test   expression OR   [   expression    ]
Now will write script that determine whether given argument number is positive. Write script as follows

Run it as follows
$ chmod 755 ispostive

$ ispostive 5
Here o/p : 5 number is positive

$ispostive -45
Here o/p : Nothing is printed

$ispostive
Here o/p : ./ispostive: test: -gt: unary operator expected
The line, if test $1 -gt 0 , test to see if first command line argument($1) is greater than 0. If it is true(0) then test will return 0 and output will printed as 5 number is positive but for -45 argument there is no output because our condition is not true(0) (no -45 is not greater than 0) hence echo statement is skipped. And for last statement we have not supplied any argument hence error ./ispostive: test: -gt: unary operator expected is generated by shell , to avoid such error we can test whether command line argument is supplied or not. 

test or [ expr ] works with
1.Integer ( Number without decimal point)
2.File types
3.Character strings
For Mathematics use following operator in Shell Script

NOTE: == is equal, != is not equal.
For string Comparisons use

Shell also test for file and directory types

Logical Operators
Logical operators are used to combine two or more condition at a time

           if  condition   
           then
                     command1 if condition is true or if exit status
                     of condition is 0(zero)
                     ...
                     ...     
           else
                     command2 if condition is false or if exit status
                     of condition is (nonzero)
                      ...
                      ...             
           fi

Try it as follows
$ chmod 755 isnump_n
$ isnump_n 5
Here o/p : 5 number is positive
$ isnump_n -45
Here o/p : -45 number is negative
$ isnump_n
Here o/p : ./ispos_n : You must give/supply one integers
$ isnump_n 0
Here o/p : 0 number is negative

Here first we see if no command line argument is given then it print error message as "./ispos_n : You must give/supply one integers". if statement checks whether number of argument ($#) passed to script is not equal (-eq) to 0, if we passed any argument to script then this if statement is false and if no command line argument is given then this if statement is true. The echo command i.e.
echo "$0 : You must give/supply one integers"
         |              |
         |              |
         1              2
1 will print Name of script
2 will print this error message
And finally statement exit 1 causes normal program termination with exit status 1 (nonzero means script is not successfully run) 

The last sample run $ isnump_n 0 , gives output as "0 number is negative", because given argument is not > 0, hence condition is false and it's taken as negative number. To avoid this replace second if statement with if test $1 -ge 0.

Multilevel if-then-else

Syntax:

           if condition
           then
                       condition is zero (true - 0)
                       execute all commands up to elif statement
           elif condition1
                       condition1 is zero (true - 0)
                       execute all commands up to elif statement
           elif condition2
                       condition2 is zero (true - 0)
                       execute all commands up to elif statement
           else
                       None of the above condtion,condtion1,condtion2 are true (i.e.
                       all of the above nonzero or false)
                       execute all commands up to fi
           fi

Try above script with  
$ chmod    755    elf 
$ ./elf   1 
$ ./elf   -2 
$ ./elf   0 
$ ./elf   a  
Here o/p for last sample run: 
./elf: [: -gt: unary operator expected 
./elf: [: -lt: unary operator expected 
./elf: [: -eq: unary operator expected 
Opps! a is not number, give number 
Above program gives error for last run, here integer comparison is expected therefore error like "./elf: [: -gt: unary operator expected" occurs, but still our program notify this thing to user by providing message "Opps! a is not number, give number". 

Loops in Shell Scripts

Computer can repeat particular instruction again and again, until particular condition satisfies. A group of instruction that is executed repeatedly is called a loop.

(a) for loop Syntax:

            for { variable name } in { list }
            do
                     execute one for each item in the list until the list is
                     not finished (And repeat all statement between do and done)
            done

Run it as,
$ chmod +x testfor
$ ./testfor
The for loop first creates i variable and assigned a number to i from the list of number from 1 to 5, The shell execute echo statement for each assignment of i. (This is usually know as iteration) This process will continue until all the items in the list were not finished, because of this it will repeat 5 echo statements. Now try script as follows

Save and Run it as
$ chmod  755  mtable
$ ./mtable 7
$ ./mtable
For first run, Above program print multiplication table of given number where i = 1,2 ... 10 is multiply by given n (here command line argument 7) in order to produce multiplication table as
7 * 1 = 7
7 * 2 = 14
...
..
7 * 10 = 70 
And for Second run, it will print message -
Error - Number missing form command line argument
Syntax : ./mtable number
Use to print multiplication table for given number

This happened because we have not supplied given number for which we want multiplication table, Hence script is showing Error message, Syntax and usage of our script. This is good idea if our program takes some argument, let the user know what is use of this script and how to used it. Note that to terminate our script we used 'exit 1' command which takes 1 as argument (1 indicates error and therefore script is terminated)

(b)while loop
Syntax:

           while [ condition ]
           do
                 command1
                 command2
                 command3
                 ..
                 ....
            done

Save it and try as
$ chmod 755  nt1
$./nt1 7
Above loop can be explained as follows

(a) First, the variable used in loop condition must be initialized, then execution of the loop begins.

(b) A test (condition) is made at the beginning of each iteration.

(c) The body of loop ends with a statement that modifies the value of the test (condition) variable.

The case Statement

The case statement is good alternative to Multilevel if-then-else-fi statement. It enable you to match several values against one variable. Its easier to read and write.
Syntax:

           case  $variable-name  in
                pattern1)   command
                                ...
                                ..
                                command;;
                pattern2)   command
                                ...
                                ..
                                command;;
                patternN)   command
                                ...
                                ..
                                command;;
                *)             command
                                ...
                                ..
                                command;;
           esac

Save it by pressing CTRL+D
$ chmod +x car
$ car van
$ car car
$ car Maruti-800

Here first we will check, that if $1(first command line argument) is not given set value of rental variable to "*** Unknown vehicle ***",if value given then set it to given value. The $rental is compared against the patterns until a match is found. 

Here for first run its match with van and it will show output For van Rs.10 per k/m. 

For second run it print, "For car Rs.20 per k/m". 

And for last run, there is no match for Maruti-800, hence default i.e. *) is executed and it prints, "Sorry, I can not gat a Maruti-800 for you". Note that esac is always required to indicate end of case statement.

How to de-bug the shell script?

While programming shell sometimes you need to find the errors (bugs) in shell script and correct the errors (remove errors - debug). For this purpose you can use -v and -x option with sh or bash command to debug the shell script. General syntax is as follows
Syntax
sh       option      { shell-script-name }
OR
bash    option      { shell-script-name }
Option can be
-v     Print shell input lines as they are read.    

-x     After expanding each simple-command, bash displays the expanded value of PS4 system variable, followed by the command and its expanded arguments.

Examples
$ cat > dsh1.sh
#
# Script to show debug of shell
#
tot=`expr $1 + $2`
echo $tot

Press ctrl + d to save, and run it as
$ chmod 755  dsh1.sh
$ ./dsh1.sh   4 5
9
$ sh -x  dsh1.sh   4 5
#
# Script to show debug of shell
#
tot=`expr $1 + $2`
expr $1 + $2
++ expr 4 + 5
+ tot=9
echo $tot
+ echo 9
9

See the above output, -x shows the exact values of variables (or statements are shown on screen with values).

$ sh -v  dsh1.sh   4 5

Use this to debug complex shell script.