Bash scripting and shell programming linux part 4

Data manipulations and text transformations with SED:

• SED=Stream Editor
• A stream is data that travels from one process to another through pipe, one file to another as redirect or one device to another.
• Sed performs text transformation on streams.
• Examples:
• substitute some text with other text
• removes lines
• Append text after given lines
• Insert text before certain lines

 type -a sed

sed is /usr/bin/sed

sed is /bin/sed

Syntax:

sed 's/search-pattern/replacement-pattern/flags' file_name

Example:

echo 'I love my wife.' > love.txt

cat love.txt

I love my wife.

sed 's/my wife/sed/' love.txt

I love sed.

Scripting is case sensitive so we can use use "i" flag to make it  case insensitive with sed

sed 's/MY WIFE/sed/I' love.txt

or

sed 's/MY WIFE/sed/i' love.txt

to append a line to a existing a file:

echo 'this is line 2.' >> love.txt

cat love.txt

I love my wife.

this is line 2.

Change my wife with sed

cat love.txt

I love my wife.

this is line 2.

I love my wife with all of my heart.

sed 's/my wife/sed/' love.txt

I love sed.

this is line 2.

I love sed with all of my heart.

sed 's/my wife/sed/' love.txt

I love sed.

this is line 2.

I love sed with all of my heart.

I love sed and my wife loves me too

globally change all the words "my wife" with sed:

sed 's/my wife/sed/g' love.txt

I love sed.

this is line 2.

I love sed with all of my heart.

I love sed and sed loves me too

change the 2nd word "my wife" in a line

sed 's/my wife/sed/2' love.txt

I love my wife.

this is line 2.

I love my wife with all of my heart.

I love my wife and sed loves me too

Save in a file

sed 's/my wife/sed/2' love.txt >ex.txt

cat ex.txt

I love my wife.

this is line 2.

I love my wife with all of my heart.

I love my wife and sed loves me too

delete a line starting with "this"

sed '/this/d' love.txt

I love my wife.

I love my wife with all of my heart.

I love my wife and my wife loves me too

using escape characters:

echo '/home/test' | sed 's/\/home\/test/\/home\/var\/test/'

/home/var/test

echo '/home/test' | sed 's:\/home\/test:\/home\/var\/test:'

/home/var/test

To delete a line based on a word/char starting with the line.

echo '#User to run service as.' > config

echo 'User apache' >> config

echo '#Group to run service as ' >> config

echo 'Group apache' >>config

 cat config

#User to run service as.

User apache

#Group to run service as

Group apache

sed '/^#/d' config

User apache

Group apache

awk:

• Awk is a scripting language used for manipulating data and generating reports.The awk command programming language requires no compiling, and allows the user to use variables, numeric functions, string functions, and logical operators.
• Awk is a utility that enables a programmer to write tiny but effective programs in the form of statements that define text patterns that are to be searched for in each line of a document and the action that is to be taken when a match is found within a line. Awk is mostly used for pattern scanning and processing. It searches one or more files to see if they contain lines that matches with the specified patterns and then performs the associated actions.
• Awk is abbreviated from the names of the developers – Aho, Weinberger, and Kernighan.

Syntax:

awk options 'selection _criteria {action }' input-file > output-file

Options:

-f program-file : Reads the AWK program source from the file 

                  program-file, instead of from the 

                  first command line argument.

-F fs : Use fs for the input field separator

To print every lines of a file:

awk '{print}' config

#User to run service as.

User apache

#Group to run service as

Group apache

Print the lines which matches with the given pattern.

awk '/User/ {print}' config

#User to run service as.

User apache

Splitting a Line Into Fields :

For each record i.e line, the awk command splits the record delimited by whitespace character by default and stores it in the $n variables. If the line has 4 words, it will be stored in $1, $2, $3 and $4 respectively. Also, $0 represents the whole line.

awk '{print $1, $3}' config

#User run

User

#Group run

Group

Built In Variables In Awk:

Awk’s built-in variables include the field variables—$1, $2, $3, and so on ($0 is the entire line) — that break a line of text into individual words or pieces called fields.

NR: NR command keeps a current count of the number of input records. Remember that records are usually lines. Awk command performs the pattern/action statements once for each record in a file.

NF: NF command keeps a count of the number of fields within the current input record.

FS: FS command contains the field separator character which is used to divide fields on the input line. The default is “white space”, meaning space and tab characters. FS can be reassigned to another character (typically in BEGIN) to change the field separator.

RS: RS command stores the current record separator character. Since, by default, an input line is the input record, the default record separator character is a newline.

OFS: OFS command stores the output field separator, which separates the fields when Awk prints them. The default is a blank space. Whenever print has several parameters separated with commas, it will print the value of OFS in between each parameter.

ORS: ORS command stores the output record separator, which separates the output lines when Awk prints them. The default is a newline character. print automatically outputs the contents of ORS at the end of whatever it is given to print.

awk '{print NR,$1}' config

1 #User

2 User

3 #Group

4 Group

 awk '{print NR,$0}' config

1 #User to run service as.

2 User apache

3 #Group to run service as

4 Group apache

awk '{print NR,NF,$0}' config

1 5 #User to run service as.

2 2 User apache

3 5 #Group to run service as

4 2 Group apache

To print the first item along with the row number(NR) separated with ” – “

awk '{print NR "-" $1}' config

1-#User

2-User

3-#Group

4-Group

To print any non empty line if present

awk 'NF > 0' config

#User to run service as.

User apache

#Group to run service as

Group apache

 To find the length of the longest line present in the file:

 awk '{ if (length($0) > max) max = length($0) } END { print max }' config

25

To count the lines in a file:

awk 'END {print NR} ' config

4

Printing lines with more than 10 characters:

 awk 'length($0) > 10' config

#User to run service as.

User apache

#Group to run service as

Group apache

To print the squares of first numbers from 1 to n say 6:

awk 'BEGIN { for(i=1;i<=6;i++) print "square of", i, "is",i*i; }'

square of 1 is 1

square of 2 is 4

square of 3 is 9

square of 4 is 16

square of 5 is 25

square of 6 is 36

Bash scripting and shell programming linux part 3

The Part#3 includes:

• Variable scope
• let command
• case statements
• eval command
• wildcards
• Logger
• Reading a file line by line
• expr command
• loop control statements - Break and Continue
• Debugging

variable scope:

Global:

• By default, variables are global.
• define variables before use it
• If you define a global variable in a function, it will be only be available after the function is called.

Local:

• Can only be accessed within a function using a local keyword, example: local LOCAL_VARIABLE=1

let command:

• The let command is an arithmetic operator. It is almost same as (( )). Only difference is that, let is an arithmetic command while (( )) is a compound command.
• It is a built-in command which instructs shell to perform an evaluation of arithmetic expressions. No spaces should be used around the arithmetic operand with let command.

 let y=y+1

 echo $y

1

let y="34 +45"

echo $y

79

case statements:

A case construct helps us to simplify nested if statement. You can match several variables against one variable. Each case is an expression matching a certain pattern.

Syntax:

case $var in

pattern_1)

    Commands

;;

pattern_N)

    Commands

;;

esac

Example:

cat > case.sh

#!/usr/bin/bash

echo "Enter the name of the State"

read state

case $state in

"Uttar Pradesh")

echo "Capital id Lucknow"

;;

"Bihar")

echo " Capital id Patna"

;;

*)

echo " You discovered an unknown state"

;;

esac

chmod +x case.sh

./case.sh

Enter the name of the State

Bihar

 Capital id Patna

./case.sh

Enter the name of the State

bihar

 You discovered an unknown state

eval command:

The eval command is a built-in command. It takes a string as its argument and evaluate it, then run the command stored in the argument. It allows using the value of a variable as a variable.

eval echo \${$User}

cat > eval.sh

#!/usr/bin/bash

hello=Mr.x

user=hello

echo \${$user}

eval echo "\${$user}"

chmod +x eval.sh

./eval.sh

$[hello}

Mr.x

But command "eval echo \${$User}" runs the parameter passed to eval. After expansion, remaining parameters are echo and ${Hello}. Hence eval command runs the command echo ${Hello}. And so the output is Mr. X

Wild cards:

• A character or string used for pattern matching.
• wildcards can be used with most of the commands like cp, ls, rm etc.

* : Matches zero or more characters.

? : Matches exactly one character

Character classes[]: Matches any of the characters included between the brackets.matches exactly one character.

Example:

[aeiou]

ca[nt]* : can, cat, candy, catter etc.

Character classes [!]: Matches any of the characters not included between the brackets. matches exactly once character.

Example:

[!aeiou] : Devdas, criccket.

Ranges: use two chars separated by a hypen to create a range in a character class.

[a-g]* : matches all files starting with a,b,c,d,e,f,g

[1-5]*: matches all files started with 1,2,3,4,5

Named character class: 

[[:alpha:]]

[[:alnum:]]

[[:digit:]]

[[:lower:]]

[[:upper:]]

[[:space::]]

escape character: "\" is called escape character. use it to match a wildcard character.

Example:

*\? : match all files that end with a question mark.

Example:

 ls -l a*

-rwxr-xr-x    1 oracle   dba              22 Aug 11 03:20 a.sh

ls c[aeiou]*

case.sh    config.sh

ls c[!aeiou]*

c.sh

ls [a-d]*

a.sh       b.sh       c.sh       case.sh    config.sh

ls [[:digit:]]*

20200819mycat.jpg

ls *.txt

test.txt

ls ?est.txt

test.txt

copying all the html files from current directory to /a directory.

#!/usr/bin/bash

for FILE in *.html

do

echo "copying $FILE"

cp $FILE /a

done

Logging:

Syslog:

• The syslog standards uses facilities and severities to categorize messages.
• Facilities: kern,mail,user,daemon,auth,local0
• Severities: alert, emerg,crit,err,warning,notice,info debug
• Log files locations are configureables
• /var/log/messages
• /var/log/syslog

Logging with logger:

logger "Message"

logger -p local0.info "Message"

logger -t myscript -p local0.info "Message"

logger -i -t myscript -p local0.info "Message"

Reading a file line by line:

#!/usr/bin/bash

LINE_NUM=1

while read LINE

do

echo "$LINE_NUM : $LINE"

((LINE_NUM++))

done < /test/test.txt

expr command:

• The expr command in Unix evaluates a given expression and displays its corresponding output. It is used for: Basic operations like addition, subtraction, multiplication, division, and modulus on integers.
• Evaluating regular expressions, string operations like substring, length of strings etc.

Syntax:

$expr expression

Performing operations on variables inside a shell script

Example: Adding two numbers in a script

echo "Enter two numbers"

read x 

read y

sum=`expr $x + $y`

echo "Sum = $sum"

Note: expr is an external program used by Bourne shell. It uses expr external program with the help of backtick. The backtick(`) is actually called command substitution.

Comparing two expressions

Example:

x=10

y=20

# matching numbers with '='

res=`expr $x = $y`

echo $res

# displays 1 when arg1 is less than arg2

res=`expr $x \< $y`

echo $res

# display 1 when arg1 is not equal to arg2

res=`expr $x \!= $y`

echo $res

For String operations

Example: Finding length of a string

x=geeks

len=`expr length $x`

echo $len

Example: Finding substring of a string

x=geeks

sub=`expr substr $x 2 3` 

#extract 3 characters starting from index 2

echo $sub

Break Statement:

The break statement is used to terminate the execution of the entire loop, after completing the execution of all of the lines of code up to the break statement. It then steps down to the code following the end of the loop.

Syntax:

The following break statement is used to come out of a loop −

break

The break command can also be used to exit from a nested loop using this format −

break n

Here n specifies the nth enclosing loop to the exit from.

Example 1:

Here is a simple example which shows that loop terminates as soon as a becomes 5 −

#!/usr/bin/bash

a=0

while [ $a -lt 10 ]

do

echo "$a"

if [ $a -eq 5 ]

then 

break

fi

a=`expr $a + 1`

done

Upon execution, you will receive the following result −

0

1

2

3

4

5

Example 2 of nested for loop. This script breaks out of both loops if var1 equals 2 and var2 equals 0 −

#!/usr/bin/sh

for var1 in 1 2 3

do

   for var2 in 0 5

   do

      if [ $var1 -eq 2 -a $var2 -eq 0 ]

      then

         break 2

      else

         echo "$var1 $var2"

      fi

   done

done

Upon execution, you will receive the following result. In the inner loop, you have a break command with the argument 2. This indicates that if a condition is met you should break out of outer loop and ultimately from the inner loop as well.

1 0

1 5

The continue statement:

The continue statement is similar to the break command, except that it causes the current iteration of the loop to exit, rather than the entire loop.

This statement is useful when an error has occurred but you want to try to execute the next iteration of the loop.

Syntax

continue

Like with the break statement, an integer argument can be given to the continue command to skip commands from nested loops.

continue n

Here n specifies the nth enclosing loop to continue from.

Example 1:

The following loop makes use of the continue statement which returns from the continue statement and starts processing the next statement −

#!/bin/sh

NUMS="1 2 3 4 5 6 7"

for NUM in $NUMS

do

   Q=`expr $NUM % 2`

   if [ $Q -eq 0 ]

   then

      echo "Number is an even number!!"

      continue

   fi

   echo "Found odd number"

done

Upon execution, you will receive the following result −

Found odd number

Number is an even number!!

Found odd number

Number is an even number!!

Found odd number

Number is an even number!!

Found odd number

Debugging:

Debugging is required to fix bugs or errors and to determine the root of unexpected behavior.

Technique 1:

cat >debug.sh

#!/usr/bin/bash -x

VAR_TEST="test"

echo "$VAR_TEST"

chmod +x debug.sh

 ./debug.sh

Output:

+ VAR_TEST=test

+ echo test

test

Technique 2:

#!/usr/bin/bash

VAR_TEST="test"

set -x

echo $VAR_TEST

set +x

hostname

output:

+echo test

linuxTestServ

-e option:

• Exit on error
• can be used with other options
• #!/usr/bin/bash -ex
• #!/usr/bin/bash -xe
• #!/usr/bin/bash -e -x
• #!/usr/bin/bash -x -e

Example 1:

#!/usr/bin/bash -e

FILE_NAME="/invalid"

ls $FILE_NAME

echo $FILE_NAME

Output:

ls: 0653-341 The file /invalid does not exist.

/invalid

Example 2:

#!/usr/bin/bash -ex

FILE_NAME="/invalid"

ls $FILE_NAME

echo $FILE_NAME

Output:

+ FILE_NAME=/invalid

+ ls /invalid

ls: 0653-341 The file /invalid does not exist.

-v option: Prints shell input lines as they are read.

#!/usr/bin/bash -v

VAR_TEST="test"

echo "$VAR_TEST"

Output:

#!/usr/bin/bash -v

VAR_TEST="test"

echo "$VAR_TEST"

test

#!/usr/bin/bash -vx

VAR_TEST="test"

echo "$VAR_TEST"

Output:

#!/usr/bin/bash -vx

VAR_TEST="test"

+ VAR_TEST="test"

echo "$VAR_TEST"

+ echo test

test

PS4:

• Controls what is displayed before a line when using teh "-x" option.
• the default value is "+"
• Bash variables BASH_SOURCE, LINENO etc.

Example:

cat > ps4.sh

#!/usr/bin/bash -x

PS4='+ $BASH_SOURCE : $LINENO : '

TEST_VAR="test"

echo "$TEST_VAR"

chmod +x ps4.sh

./ps4.sh

+ PS4='+ $BASH_SOURCE " $LINENO " '

+ ./ps4.sh : 3 : TEST_VAR=test

+ ./ps4.sh : 4 : echo test

test

#!/usr/bin/bash -x

PS4='+ $BASH_SOURCE : $LINENO : $FUNCNAME[0]()'

debug(){

echo "Executing: $@"

$@

}

debug ls

+ PS4='+ $BASH_SOURCE : $LINENO : $FUNCNAME[0]()'

+ ./ps41.sh : 7 : [0]()debug ls

+ ./ps41.sh : 4 : debug[0]()echo 'Executing: ls'

Executing: ls

+ ./ps41.sh : 5 : debug[0]()ls

20200819mycat.jpg 

DOS VS Linux  file types:

A file can contain CRLF/Carriage return, Line feed. So when we are moving file from DOS system to Linux, we should use " dos2unix script.sh " command to make it compatible with BASH executable.

To continue, go to the next page, bash-scripting-and-shell-programming-part4.

Bash scripting and shell programming linux part 2

 The Part#2 includes:

• Positional Parameters
• Shell Scripting Shift Through Parameters
• Shell Scripting Sourcing a file
• Accepting User input (STDIN)
• Shell Scripting Sourcing a config file
• Return codes / Exit statuses
• EXIT Command
• Functions

Positional Parameters:

• A bash shell script have parameters. These parameters start from $1 to $9.
• When we pass arguments into the command line interface, a positional parameter is assigned to these arguments through the shell.
• The first argument is assigned as $1, second argument is assigned as $2 and so on...
• If there are more than 9 arguments, then tenth or onwards arguments can't be assigned as $10 or $11.
• You have to either process or save the $1 parameter, then with the help of shift command drop parameter 1 and move all other arguments down by one. It will make $10 as $9, $9 as $8 and so on.

$1-$9 Represent positional parameters for arguments one to nine

${10}-${n} Represent positional parameters for arguments after nine

$0 Represent name of the script

$∗ Represent all the arguments as a single string

$@ Same as $∗, but differ when enclosed in (")

$# Represent total number of arguments

$$ PID of the script

$? Represent last return code

Example:

cat > positional.sh

#!/usr/bin/bash

echo "The script name: $0"

echo "The 1st argument: $1"

echo "The 2nd argument: $2"

echo "The pid of the script: $$"

echo "The total numbe rof arguments: $#"

echo "All the arguments: $*"

echo "The last return code: $?"

chmod +x positional.sh

./positional.sh

The script name: ./positional.sh

The 1st argument: 1

The 2nd argument: 4

The pid of the script: 4129020

The total numbe rof arguments: 2

All the arguments: 1 4

The last return code: 0

Shell Scripting Shift Through Parameters:

Shift command is a built-in command. Command takes number as argument. Arguments shift down by this number.

For example, if number is 5, then $5 become $1, $6 become $2 and so on.

The shift command is mostly used when arguments are unknown. Arguments are processed in a while loop with a condition of (( $# )). this condition holds true as long as arguments are not zero. Number of arguments are reduced each time as the shift command executes.

cat > shift.sh

#!/usr/bin/bash

if [ "$#" == "0" ]

then

echo pass at least one parameter

exit 1

fi

while (( $# ))

do

echo you gave me $1

shift

done

 chmod +x shift.sh

 ./shift.sh

pass at least one parameter

./shift.sh 1 e

you gave me 1

you gave me e

Shell Scripting Sourcing a file:

A file is sourced in two ways. 

• source <fileName>
•   . ./<filename> 

When a file is sourced, the code lines are executed as if they were printed on the command line.

The difference between sourcing and executing a script is that, while executing a script it runs in a new shell whereas while sourcing a script, file will be read and executed in the same shell.

source ./variable.sh

jai viru

echo $var1

jai

. ./variable.sh

jai viru

echo $var2

viru

Accepting User input (STDIN):

The read command allows a user to provide the runtime input.

 cat > read.sh

#!/usr/bin/bash

echo enter your name:

read name

echo "Welcome $name "

chmod +x read.sh

 ./read.sh

enter your name:

sri

Welcome sri

Syntax:

read -p "Prompt" variable

Example:

#!/usr/bin/bash

read -p "Enter your name" Name

echo " Hello $Name"

Output:

Enter your name

rabi

Hello rabi

Shell Scripting Sourcing a config file:

Many programs use external configuration files. Use of external configuration files prevents a user from making changes to a script. Config file is added with the help of source command.

If a script is shared in many users and every user need a different configuration file, then instead of changing the script each time simply include the config files.

We have two files, one is parent file (main.sh) and other is configuration file (config.sh). We have to source this configuration file into our parent file

Example 1:

Config.sh

#!/usr/bin/bash

user =java

id=201

Main.sh

#!/usr/bin/bash

source config.sh

echo “this is user $user with $id”

chmod +x main.sh

 ./main.sh

 this is user abc with 101.

Note: We can also use ( . config.sh ) command instead of ( source config.sh ) .

Example 2:

cat > config.sh

#! /usr/bin/bash

user=abc

id=101

 cat > main1.sh

#! /usr/bin/bash

. config.sh

echo "this is user $user"

chmod +x main1.sh

./main1.sh

Output:

this is user abc

Return codes / Exit statuses:

• Every command returns an exit codes, range from 0 to 255.
• 0=success and other than zero is a error condition.
• We can use this exit status as error checking purposes.
• $? returns the return code of the previously executed command.

Example:

#!/usr/bin/bash

HOST="google.com"

ping -c 1 $HOST

RETURN_CODE=$?

if [ $RETURN_CODE -eq "0" ]

then 

echo "$HOST is reachable"

else

echo "$HOST is not reachable"

fi

chmod +x ret.sh

./ret.sh

PING google.com: (xxx.xxx.x.xx): 56 data bytes

64 bytes from xxx.xxx.x.xx: icmp_seq=0 ttl=111 time=42 ms

--- google.com ping statistics ---

1 packets transmitted, 1 packets received, 0% packet loss

round-trip min/avg/max = 42/42/42 ms

google.com is reachable

&& and ||:

#!/usr/bin/bash

HOST="google.com"

#If 1st condition is true then it executes the second command.

ping -c 1 $HOST && echo " $HOST is reachable"

#If 1st condition is false then it executes the second command.

ping -c 1 $HOST || echo "$HOST is unreachbale"

The Semicolon:

It separates the commands with a semicolon to ensure they all get executed.

cp test.txt /test/bak/ ; cp tets.txt  /test/

EXIT Command:

Using EXIT keyword we can explicitly define the return code. The default value is that of the last command executed. exit 0, exit 1, ....exit 255 etc.

Example:

#!/usr/bin/bash

HOST="google.com"

ping -c 1 $HOST

if [ $HOST -ne "0" ]

then 

echo " $HOST unreachable"

exit 1

fi

exit 0

More examples:

 ping -c 1 abc.com

PING abc.com: : 56 data bytes

64 bytes from : icmp_seq=0 ttl=238 time=42 ms

--- abc.com ping statistics ---

1 packets transmitted, 1 packets received, 0% packet loss

round-trip min/avg/max = 42/42/42 ms

 echo $?

0

ping -c 1 ttrert.com

0821-062 ping: host name ttrert.com NOT FOUND

echo $?

1

 ping -c 1 amazon.com.1

0821-062 ping: host name amazon.com.1 NOT FOUND

 echo $?

127

Functions:

With the help of functions, overall functionality of a function can be divided into smaller or logical parts, which can be called to perform their task. It helps us to check our program part by part. We can reuse the function where ever we want.

• Don't Repeat yourself(DRY)
• Write once, use many times
• easier to maintain and troubleshoot

Creating a function:

Syntax:

function functionName () {  

        Commands to be executed  

}

or

functionName () {  

        Commands to be executed  

}

You will call your function with their function name.

Example 1:

cat > function.sh

#!/usr/bin/bash

function helloWorld(){

echo "hello"

}

helloWorld

chmod +x function.sh

./function.sh

hello

Example 2:

cat > functest.sh

#!/usr/bin/bash

function hello(){

for NAME in $@

do

echo "Hello $NAME"

done

}

hello sri jinx pepa luis

chmod +x functest.sh

./functest.sh

Hello sri

Hello jinx

Hello pepa

Hello luis

Passing Parameters:

You can pass one or more parameters in a function. Parameters will be defined as $1, $2 and so on.

 cat > funcparam.sh

#!/usr/bin/bash

function passParam(){

echo "param1= $1 and param2= $2"

}

#call function

passParam test 123

 chmod +x funcparam.sh

 ./funcparam.sh

param1= test and param2= 123

Returning Values from Functions:

• If you execute an exit command from inside a function, its effect is not only to terminate execution of the function but also of the shell program that called the function.

Based on the situation you can return any value from your function using the return command whose syntax is as follows −

return code

cat > funcreturn.sh

#!/usr/bin/bash

Hello(){

echo "hello $1 $2"

return 10

}

Hello Sri Das

ret=$?

echo "return vaue is $ret"

chmod +x funcreturn.sh

./funcreturn.sh

hello Sri Das

return vaue is 10

Nested Functions:

One of the more interesting features of functions is that they can call themselves and also other functions. A function that calls itself is known as a recursive function.

Following example demonstrates nesting of two functions −

#!/bin/sh

# Calling one function from another

number_one () {

   echo "This is the first function speaking..."

   number_two

}

number_two () {

   echo "This is now the second function speaking..."

}

# Calling function one.

number_one

To continue, go to the blog page  part 3: bash-scripting-and-shell-programming_part3

Bash scripting and shell programming linux part 1

The Part#1 includes:

• What is a Shell & Types of shells
• What is Script and how to determine shell
• She-Bang & comments
• Steps to create a shell script and execution
• Variables & usages, assign output command to a variable, Variables Names
• File and String Test Operators
• Decision making statements if, if else, if elif else
• Shell Scripting loops for, while, infinite loop, until

 What is a Shell?

An Operating system is made of many components, but its two prime components are -

• Kernel
• Shell

A Kernel is at the nucleus of a computer. It makes the communication between the hardware and software possible. While the Kernel is the innermost part of an operating system, a shell is the outermost one.

A shell in a Linux operating system takes input from you in the form of commands, processes it, and then gives an output. It is the interface through which a user works on the programs, commands, and scripts. A shell is accessed by a terminal which runs it. When you run the terminal, the Shell issues a command prompt (usually $), where you can type your input, which is then executed when you hit the Enter key. The output or the result is thereafter displayed on the terminal.

The Shell wraps around the delicate interior of an Operating system protecting it from accidental damage. Hence the name Shell.

A shell is a command-line interpreter and typical operations performed by shell scripts include file manipulation, program execution, and printing text.

Types of Shell

There are two main shells in Linux:

1. The Bourne Shell: The prompt for this shell is $ and its derivatives are listed below:

• POSIX shell also is known as sh
• Korn Shell also knew as sh
• Bourne Again SHell also knew as bash (most popular)

2. The C shell: The prompt for this shell is %, and its subcategories are:

• C shell also is known as csh
• Tops C shell also is known as tcsh

Scripts:

• contains a series of commands
• An interpreter executes commands in the scripts
• anything you can type at the command line, you can put in a script.
• great for automating tasks.

How to determine Shell:

oracle@test> echo $SHELL

/usr/bin/bash

The $ sign stands for a shell variable, echo will return the text whatever you typed in.

Shell Scripting She-bang:

The sign #! is called she-bang and is written at top of the script. It passes instruction to program /bin/sh.

To run your script in a certain shell (shell should be supported by your system), start your script with #! followed by the shell name.

Example:

#!/bin/bash  

echo Hello World  

• The first line of a shell script typically starts with a shebang followed by the path to an interpreter that will be used to execute the commands in the script. 
• If you do not supply a shebang and specify an interpreter on the first line of the script the commands in the script will be executed using your current shell.

Shell Scripting Comments:

Any line starting with a hash (#) becomes comment. Comment means, that line will not take part in script execution. It will not show up in the output.

Steps to create a script and execute:

• create a .sh file using cat or touch command
• write the body and saved it
• provide the execute permission using chmod 777 or chomd +x command
• execute by ./scriptname.sh

Example:

cat > variable.sh

#!/usr/bin/bash

var1=jai

var2=viru

#Echo

echo "$var1 $var2"

chmod 777 variable.sh

./variable.sh

Output: jai viru

Variables:

• Storage location that have a name. By convention variables are upper case.
• Name value pair like variable_name = "Value"
• Variables are case sensitive

Variable usage:

#! /bin/bash

MY_SHELL="bash"

echo " I like the $MY_SHELL shell"

or 

echo "I like the ${MY_SHELL} shell"

Output:

I like the bash shell

Assign command output to a variable:

#!/bin/bash

SERVER=$(hostname)

echo "The script is running on ${SERVER}"

or 

#!/bin/bash

SERVER=`hostname`

echo "The script is running on ${SERVER}"

Output:

The script is running on LinxTestServer

Variables Names:

• It contains combination of uppercase and lowercase alphabets, digits and underscore as follows:

FIRST3VARIABLE

FIRST_VARIABLE

FirstVarable

• Variable names should not start with a digit or not use any special characters other than underscore. For instance,

2FirstVariable

First-Varable

FIRST@VARIABLE

Read-only Variables:

Shell provides a way to mark variables as read-only by using the read-only command. After a variable is marked read-only, its value cannot be changed.

For example, the following script generates an error while trying to change the value of NAME −

 cat > read.sh

#!/usr/bin/bash

NAME="Sri"

readonly NAME

NAME="Rudra"

echo "$NAME"

The above script will generate the following result −

./read.sh: line 4: NAME: readonly variable

Sri

Unsetting Variable:

Unsetting or deleting a variable directs the shell to remove the variable from the list of variables that it tracks. Once you unset a variable, you cannot access the stored value in the variable.

Following is the syntax to unset a defined variable using the unset command −

unset variable_name

The above command unsets the value of a defined variable. Here is a simple example that demonstrates how the command works −

 cat > read1.sh

#!/usr/bin/bash

NAME="Sri"

unset NAME

echo $NAME

The above example does not print anything. You cannot use the unset command to unset variables that are marked readonly. 

 File Operators(Tests):

-d FILE : True if file is a directory

-e FILE : True if file exists

-f FILE : True if file exists and is a regular file

-r FILE : True if the file is readable by you

-s FILE : True if file exists and not empty

-w FILE : True if file is writable by you

-x FILE : True if is executable by you 

String Operators(tests):

-z STRING : True if string is empty

-n STRING : True if string is not empty

String1 = String2 : True if the strings are equal

String1 != String2 : True if the strings are not equal.

Making Decisions:

The if statement:

syntax:

if [ condition is true ]

then 

commands

fi

Example:

#! /bin/bash

MY_SHELL="bash"

if [ $MY_SHELL = "bash" ]

then 

echo "You like the bash shell"

fi

if-else statement:

Syntax:

if [ condition is true ]

then

commands

else

commands

fi

Example:

#! /bin/bash

MY_SHELL="csh"

if [ $MY_SHELL="bash" ]

then

echo " You like bash shell"

else

echo " you dont like bash shell"

fi

if elif else statements:

Syntax:

if [ condition is true ]

then

commands

elif [ condition is true ]

then 

commands

else

commands

fi

Example:

#! /bin/bash

MY_SHELL="bash"

if [ $MY_SHELL="bash" ]

then

echo " You like bash shell"

elif [ $MY_SHELL="csh" ]

then

echo " you like csh shell"

else

echo " you dont like bash or csh shells"

fi

Shell Scripting for loop:

The for loop moves through a specified list of values until the list is exhausted.

Syntax 1:

Syntax of for loop using in and list of values is shown below. This for loop contains a number of variables in the list and will execute for each item in the list. For example, if there are 10 variables in the list, then loop will execute ten times and value will be stored in varname.

for table in {min_value..Max_value..Increment_value}

do

commands

done

Example:

cat >for1.sh

#!/usr/bin/bash

for table in {2..20..2}

do

echo "$table"

done

chmod +x for1.sh

 ./for1.sh

Output:

2

4

6

8

10

12

14

16

18

20

Syntax 2:

for (( initialization;condition;increment/decrement))

do

commands

done

Example:

cat > for2.sh

#!/usr/bin/bash

for (( i=10; i >= 1; i--))

do

echo "$i"

done

chmod +x for2.sh

./for2.sh

Output:

10

9

8

7

6

5

4

3

2

1

Syntax 3:

for Var_name in Item_1 ItemN

do

commands

done

Example:

#! /bin/bash

COLORS="RED GREEN BLUE"

for COLOR in $COLORS

do 

echo "COLOR: $COLOR"

done

Output:

COLOR:RED
COLOR:GREEN
COLOR:BLUE

Scripting while loop:

There is a condition in while. And commands are executed till the condition is valid. Once condition becomes false, loop terminates.

Syntax:

While [ condition ]

Do

commands

done

example:

cat >while.sh

#!/usr/bin/bash

i=10;

while [ $i -ge 0 ] ;

do

echo " reverse order number $i"

let i--;

done

chmod +x while.sh

 ./while.sh

Output: 

reverse order number 10

 reverse order number 9

 reverse order number 8

 reverse order number 7

 reverse order number 6

 reverse order number 5

 reverse order number 4

 reverse order number 3

 reverse order number 2

 reverse order number 1

 reverse order number 0

while infinite loop:

Infinite loop is also called endless loop. It is made with while true (it means condition will always be true) or while : (it means an empty expression), where colon (:) is equivalent to no operation.

 cat whileinfinite.sh

#!/usr/bin/bash

while true

do

echo " this is infinite loop. press ctrl + c to exit"

done

Shell Scripting until loop:

It is similar to while loop. The only difference is that until statement executes its code block while its conditional expression is false, and while statement executes its code block while its conditional expression is true.

Difference between while and until:

Until loop always executes at least once. Loop while executes till it returns a zero value and until loop executes till it returns non-zero value.

Syntax:

Until [ condition ]

Do

Commands

Done

Example:

 cat > until.sh

#!/usr/bin/bash

i=5;

until [ $i -gt 15 ];

do

echo "number $i"

let i++;

done

 chmod +x until.sh

 ./until.sh

Output:

number 5

number 6

number 7

number 8

number 9

number 10

number 11

number 12

number 13

number 14

number 15

Example: Rename all the.jpg files of the current folder to YYYY-MM-DD- filename.

#! /bin/bash

LIST_PICTURES=$(ls *.jpg)

DATE=`date +%F`

for PICTURE in LIST_PICTURES

do

echo "Renaming $PICTURE to $DATE-$PICTURE"

mv $PICTURE $DATE-$PICTURE

done

To continue, go to the next blog page bash-scripting-and-shell-programming-part-2.