Bash scripting and shell programming linux part 3

The Part#3 includes:

• Variable scope
• let command
• case statements
• eval command
• wildcards
• Logger
• Reading a file line by line
• expr command
• loop control statements - Break and Continue
• Debugging

variable scope:

Global:

• By default, variables are global.
• define variables before use it
• If you define a global variable in a function, it will be only be available after the function is called.

Local:

• Can only be accessed within a function using a local keyword, example: local LOCAL_VARIABLE=1

let command:

• The let command is an arithmetic operator. It is almost same as (( )). Only difference is that, let is an arithmetic command while (( )) is a compound command.
• It is a built-in command which instructs shell to perform an evaluation of arithmetic expressions. No spaces should be used around the arithmetic operand with let command.

 let y=y+1

 echo $y

1

let y="34 +45"

echo $y

79

case statements:

A case construct helps us to simplify nested if statement. You can match several variables against one variable. Each case is an expression matching a certain pattern.

Syntax:

case $var in

pattern_1)

    Commands

;;

pattern_N)

    Commands

;;

esac

Example:

cat > case.sh

#!/usr/bin/bash

echo "Enter the name of the State"

read state

case $state in

"Uttar Pradesh")

echo "Capital id Lucknow"

;;

"Bihar")

echo " Capital id Patna"

;;

*)

echo " You discovered an unknown state"

;;

esac

chmod +x case.sh

./case.sh

Enter the name of the State

Bihar

 Capital id Patna

./case.sh

Enter the name of the State

bihar

 You discovered an unknown state

eval command:

The eval command is a built-in command. It takes a string as its argument and evaluate it, then run the command stored in the argument. It allows using the value of a variable as a variable.

eval echo \${$User}

cat > eval.sh

#!/usr/bin/bash

hello=Mr.x

user=hello

echo \${$user}

eval echo "\${$user}"

chmod +x eval.sh

./eval.sh

$[hello}

Mr.x

But command "eval echo \${$User}" runs the parameter passed to eval. After expansion, remaining parameters are echo and ${Hello}. Hence eval command runs the command echo ${Hello}. And so the output is Mr. X

Wild cards:

• A character or string used for pattern matching.
• wildcards can be used with most of the commands like cp, ls, rm etc.

* : Matches zero or more characters.

? : Matches exactly one character

Character classes[]: Matches any of the characters included between the brackets.matches exactly one character.

Example:

[aeiou]

ca[nt]* : can, cat, candy, catter etc.

Character classes [!]: Matches any of the characters not included between the brackets. matches exactly once character.

Example:

[!aeiou] : Devdas, criccket.

Ranges: use two chars separated by a hypen to create a range in a character class.

[a-g]* : matches all files starting with a,b,c,d,e,f,g

[1-5]*: matches all files started with 1,2,3,4,5

Named character class: 

[[:alpha:]]

[[:alnum:]]

[[:digit:]]

[[:lower:]]

[[:upper:]]

[[:space::]]

escape character: "\" is called escape character. use it to match a wildcard character.

Example:

*\? : match all files that end with a question mark.

Example:

 ls -l a*

-rwxr-xr-x    1 oracle   dba              22 Aug 11 03:20 a.sh

ls c[aeiou]*

case.sh    config.sh

ls c[!aeiou]*

c.sh

ls [a-d]*

a.sh       b.sh       c.sh       case.sh    config.sh

ls [[:digit:]]*

20200819mycat.jpg

ls *.txt

test.txt

ls ?est.txt

test.txt

copying all the html files from current directory to /a directory.

#!/usr/bin/bash

for FILE in *.html

do

echo "copying $FILE"

cp $FILE /a

done

Logging:

Syslog:

• The syslog standards uses facilities and severities to categorize messages.
• Facilities: kern,mail,user,daemon,auth,local0
• Severities: alert, emerg,crit,err,warning,notice,info debug
• Log files locations are configureables
• /var/log/messages
• /var/log/syslog

Logging with logger:

logger "Message"

logger -p local0.info "Message"

logger -t myscript -p local0.info "Message"

logger -i -t myscript -p local0.info "Message"

Reading a file line by line:

#!/usr/bin/bash

LINE_NUM=1

while read LINE

do

echo "$LINE_NUM : $LINE"

((LINE_NUM++))

done < /test/test.txt

expr command:

• The expr command in Unix evaluates a given expression and displays its corresponding output. It is used for: Basic operations like addition, subtraction, multiplication, division, and modulus on integers.
• Evaluating regular expressions, string operations like substring, length of strings etc.

Syntax:

$expr expression

Performing operations on variables inside a shell script

Example: Adding two numbers in a script

echo "Enter two numbers"

read x 

read y

sum=`expr $x + $y`

echo "Sum = $sum"

Note: expr is an external program used by Bourne shell. It uses expr external program with the help of backtick. The backtick(`) is actually called command substitution.

Comparing two expressions

Example:

x=10

y=20

# matching numbers with '='

res=`expr $x = $y`

echo $res

# displays 1 when arg1 is less than arg2

res=`expr $x \< $y`

echo $res

# display 1 when arg1 is not equal to arg2

res=`expr $x \!= $y`

echo $res

For String operations

Example: Finding length of a string

x=geeks

len=`expr length $x`

echo $len

Example: Finding substring of a string

x=geeks

sub=`expr substr $x 2 3` 

#extract 3 characters starting from index 2

echo $sub

Break Statement:

The break statement is used to terminate the execution of the entire loop, after completing the execution of all of the lines of code up to the break statement. It then steps down to the code following the end of the loop.

Syntax:

The following break statement is used to come out of a loop −

break

The break command can also be used to exit from a nested loop using this format −

break n

Here n specifies the nth enclosing loop to the exit from.

Example 1:

Here is a simple example which shows that loop terminates as soon as a becomes 5 −

#!/usr/bin/bash

a=0

while [ $a -lt 10 ]

do

echo "$a"

if [ $a -eq 5 ]

then 

break

fi

a=`expr $a + 1`

done

Upon execution, you will receive the following result −

0

1

2

3

4

5

Example 2 of nested for loop. This script breaks out of both loops if var1 equals 2 and var2 equals 0 −

#!/usr/bin/sh

for var1 in 1 2 3

do

   for var2 in 0 5

   do

      if [ $var1 -eq 2 -a $var2 -eq 0 ]

      then

         break 2

      else

         echo "$var1 $var2"

      fi

   done

done

Upon execution, you will receive the following result. In the inner loop, you have a break command with the argument 2. This indicates that if a condition is met you should break out of outer loop and ultimately from the inner loop as well.

1 0

1 5

The continue statement:

The continue statement is similar to the break command, except that it causes the current iteration of the loop to exit, rather than the entire loop.

This statement is useful when an error has occurred but you want to try to execute the next iteration of the loop.

Syntax

continue

Like with the break statement, an integer argument can be given to the continue command to skip commands from nested loops.

continue n

Here n specifies the nth enclosing loop to continue from.

Example 1:

The following loop makes use of the continue statement which returns from the continue statement and starts processing the next statement −

#!/bin/sh

NUMS="1 2 3 4 5 6 7"

for NUM in $NUMS

do

   Q=`expr $NUM % 2`

   if [ $Q -eq 0 ]

   then

      echo "Number is an even number!!"

      continue

   fi

   echo "Found odd number"

done

Upon execution, you will receive the following result −

Found odd number

Number is an even number!!

Found odd number

Number is an even number!!

Found odd number

Number is an even number!!

Found odd number

Debugging:

Debugging is required to fix bugs or errors and to determine the root of unexpected behavior.

Technique 1:

cat >debug.sh

#!/usr/bin/bash -x

VAR_TEST="test"

echo "$VAR_TEST"

chmod +x debug.sh

 ./debug.sh

Output:

+ VAR_TEST=test

+ echo test

test

Technique 2:

#!/usr/bin/bash

VAR_TEST="test"

set -x

echo $VAR_TEST

set +x

hostname

output:

+echo test

linuxTestServ

-e option:

• Exit on error
• can be used with other options
• #!/usr/bin/bash -ex
• #!/usr/bin/bash -xe
• #!/usr/bin/bash -e -x
• #!/usr/bin/bash -x -e

Example 1:

#!/usr/bin/bash -e

FILE_NAME="/invalid"

ls $FILE_NAME

echo $FILE_NAME

Output:

ls: 0653-341 The file /invalid does not exist.

/invalid

Example 2:

#!/usr/bin/bash -ex

FILE_NAME="/invalid"

ls $FILE_NAME

echo $FILE_NAME

Output:

+ FILE_NAME=/invalid

+ ls /invalid

ls: 0653-341 The file /invalid does not exist.

-v option: Prints shell input lines as they are read.

#!/usr/bin/bash -v

VAR_TEST="test"

echo "$VAR_TEST"

Output:

#!/usr/bin/bash -v

VAR_TEST="test"

echo "$VAR_TEST"

test

#!/usr/bin/bash -vx

VAR_TEST="test"

echo "$VAR_TEST"

Output:

#!/usr/bin/bash -vx

VAR_TEST="test"

+ VAR_TEST="test"

echo "$VAR_TEST"

+ echo test

test

PS4:

• Controls what is displayed before a line when using teh "-x" option.
• the default value is "+"
• Bash variables BASH_SOURCE, LINENO etc.

Example:

cat > ps4.sh

#!/usr/bin/bash -x

PS4='+ $BASH_SOURCE : $LINENO : '

TEST_VAR="test"

echo "$TEST_VAR"

chmod +x ps4.sh

./ps4.sh

+ PS4='+ $BASH_SOURCE " $LINENO " '

+ ./ps4.sh : 3 : TEST_VAR=test

+ ./ps4.sh : 4 : echo test

test

#!/usr/bin/bash -x

PS4='+ $BASH_SOURCE : $LINENO : $FUNCNAME[0]()'

debug(){

echo "Executing: $@"

$@

}

debug ls

+ PS4='+ $BASH_SOURCE : $LINENO : $FUNCNAME[0]()'

+ ./ps41.sh : 7 : [0]()debug ls

+ ./ps41.sh : 4 : debug[0]()echo 'Executing: ls'

Executing: ls

+ ./ps41.sh : 5 : debug[0]()ls

20200819mycat.jpg 

DOS VS Linux  file types:

A file can contain CRLF/Carriage return, Line feed. So when we are moving file from DOS system to Linux, we should use " dos2unix script.sh " command to make it compatible with BASH executable.

To continue, go to the next page, bash-scripting-and-shell-programming-part4.

Bash scripting and shell programming linux part 2

 The Part#2 includes:

• Positional Parameters
• Shell Scripting Shift Through Parameters
• Shell Scripting Sourcing a file
• Accepting User input (STDIN)
• Shell Scripting Sourcing a config file
• Return codes / Exit statuses
• EXIT Command
• Functions

Positional Parameters:

• A bash shell script have parameters. These parameters start from $1 to $9.
• When we pass arguments into the command line interface, a positional parameter is assigned to these arguments through the shell.
• The first argument is assigned as $1, second argument is assigned as $2 and so on...
• If there are more than 9 arguments, then tenth or onwards arguments can't be assigned as $10 or $11.
• You have to either process or save the $1 parameter, then with the help of shift command drop parameter 1 and move all other arguments down by one. It will make $10 as $9, $9 as $8 and so on.

$1-$9 Represent positional parameters for arguments one to nine

${10}-${n} Represent positional parameters for arguments after nine

$0 Represent name of the script

$∗ Represent all the arguments as a single string

$@ Same as $∗, but differ when enclosed in (")

$# Represent total number of arguments

$$ PID of the script

$? Represent last return code

Example:

cat > positional.sh

#!/usr/bin/bash

echo "The script name: $0"

echo "The 1st argument: $1"

echo "The 2nd argument: $2"

echo "The pid of the script: $$"

echo "The total numbe rof arguments: $#"

echo "All the arguments: $*"

echo "The last return code: $?"

chmod +x positional.sh

./positional.sh

The script name: ./positional.sh

The 1st argument: 1

The 2nd argument: 4

The pid of the script: 4129020

The total numbe rof arguments: 2

All the arguments: 1 4

The last return code: 0

Shell Scripting Shift Through Parameters:

Shift command is a built-in command. Command takes number as argument. Arguments shift down by this number.

For example, if number is 5, then $5 become $1, $6 become $2 and so on.

The shift command is mostly used when arguments are unknown. Arguments are processed in a while loop with a condition of (( $# )). this condition holds true as long as arguments are not zero. Number of arguments are reduced each time as the shift command executes.

cat > shift.sh

#!/usr/bin/bash

if [ "$#" == "0" ]

then

echo pass at least one parameter

exit 1

fi

while (( $# ))

do

echo you gave me $1

shift

done

 chmod +x shift.sh

 ./shift.sh

pass at least one parameter

./shift.sh 1 e

you gave me 1

you gave me e

Shell Scripting Sourcing a file:

A file is sourced in two ways. 

• source <fileName>
•   . ./<filename> 

When a file is sourced, the code lines are executed as if they were printed on the command line.

The difference between sourcing and executing a script is that, while executing a script it runs in a new shell whereas while sourcing a script, file will be read and executed in the same shell.

source ./variable.sh

jai viru

echo $var1

jai

. ./variable.sh

jai viru

echo $var2

viru

Accepting User input (STDIN):

The read command allows a user to provide the runtime input.

 cat > read.sh

#!/usr/bin/bash

echo enter your name:

read name

echo "Welcome $name "

chmod +x read.sh

 ./read.sh

enter your name:

sri

Welcome sri

Syntax:

read -p "Prompt" variable

Example:

#!/usr/bin/bash

read -p "Enter your name" Name

echo " Hello $Name"

Output:

Enter your name

rabi

Hello rabi

Shell Scripting Sourcing a config file:

Many programs use external configuration files. Use of external configuration files prevents a user from making changes to a script. Config file is added with the help of source command.

If a script is shared in many users and every user need a different configuration file, then instead of changing the script each time simply include the config files.

We have two files, one is parent file (main.sh) and other is configuration file (config.sh). We have to source this configuration file into our parent file

Example 1:

Config.sh

#!/usr/bin/bash

user =java

id=201

Main.sh

#!/usr/bin/bash

source config.sh

echo “this is user $user with $id”

chmod +x main.sh

 ./main.sh

 this is user abc with 101.

Note: We can also use ( . config.sh ) command instead of ( source config.sh ) .

Example 2:

cat > config.sh

#! /usr/bin/bash

user=abc

id=101

 cat > main1.sh

#! /usr/bin/bash

. config.sh

echo "this is user $user"

chmod +x main1.sh

./main1.sh

Output:

this is user abc

Return codes / Exit statuses:

• Every command returns an exit codes, range from 0 to 255.
• 0=success and other than zero is a error condition.
• We can use this exit status as error checking purposes.
• $? returns the return code of the previously executed command.

Example:

#!/usr/bin/bash

HOST="google.com"

ping -c 1 $HOST

RETURN_CODE=$?

if [ $RETURN_CODE -eq "0" ]

then 

echo "$HOST is reachable"

else

echo "$HOST is not reachable"

fi

chmod +x ret.sh

./ret.sh

PING google.com: (xxx.xxx.x.xx): 56 data bytes

64 bytes from xxx.xxx.x.xx: icmp_seq=0 ttl=111 time=42 ms

--- google.com ping statistics ---

1 packets transmitted, 1 packets received, 0% packet loss

round-trip min/avg/max = 42/42/42 ms

google.com is reachable

&& and ||:

#!/usr/bin/bash

HOST="google.com"

#If 1st condition is true then it executes the second command.

ping -c 1 $HOST && echo " $HOST is reachable"

#If 1st condition is false then it executes the second command.

ping -c 1 $HOST || echo "$HOST is unreachbale"

The Semicolon:

It separates the commands with a semicolon to ensure they all get executed.

cp test.txt /test/bak/ ; cp tets.txt  /test/

EXIT Command:

Using EXIT keyword we can explicitly define the return code. The default value is that of the last command executed. exit 0, exit 1, ....exit 255 etc.

Example:

#!/usr/bin/bash

HOST="google.com"

ping -c 1 $HOST

if [ $HOST -ne "0" ]

then 

echo " $HOST unreachable"

exit 1

fi

exit 0

More examples:

 ping -c 1 abc.com

PING abc.com: : 56 data bytes

64 bytes from : icmp_seq=0 ttl=238 time=42 ms

--- abc.com ping statistics ---

1 packets transmitted, 1 packets received, 0% packet loss

round-trip min/avg/max = 42/42/42 ms

 echo $?

0

ping -c 1 ttrert.com

0821-062 ping: host name ttrert.com NOT FOUND

echo $?

1

 ping -c 1 amazon.com.1

0821-062 ping: host name amazon.com.1 NOT FOUND

 echo $?

127

Functions:

With the help of functions, overall functionality of a function can be divided into smaller or logical parts, which can be called to perform their task. It helps us to check our program part by part. We can reuse the function where ever we want.

• Don't Repeat yourself(DRY)
• Write once, use many times
• easier to maintain and troubleshoot

Creating a function:

Syntax:

function functionName () {  

        Commands to be executed  

}

or

functionName () {  

        Commands to be executed  

}

You will call your function with their function name.

Example 1:

cat > function.sh

#!/usr/bin/bash

function helloWorld(){

echo "hello"

}

helloWorld

chmod +x function.sh

./function.sh

hello

Example 2:

cat > functest.sh

#!/usr/bin/bash

function hello(){

for NAME in $@

do

echo "Hello $NAME"

done

}

hello sri jinx pepa luis

chmod +x functest.sh

./functest.sh

Hello sri

Hello jinx

Hello pepa

Hello luis

Passing Parameters:

You can pass one or more parameters in a function. Parameters will be defined as $1, $2 and so on.

 cat > funcparam.sh

#!/usr/bin/bash

function passParam(){

echo "param1= $1 and param2= $2"

}

#call function

passParam test 123

 chmod +x funcparam.sh

 ./funcparam.sh

param1= test and param2= 123

Returning Values from Functions:

• If you execute an exit command from inside a function, its effect is not only to terminate execution of the function but also of the shell program that called the function.

Based on the situation you can return any value from your function using the return command whose syntax is as follows −

return code

cat > funcreturn.sh

#!/usr/bin/bash

Hello(){

echo "hello $1 $2"

return 10

}

Hello Sri Das

ret=$?

echo "return vaue is $ret"

chmod +x funcreturn.sh

./funcreturn.sh

hello Sri Das

return vaue is 10

Nested Functions:

One of the more interesting features of functions is that they can call themselves and also other functions. A function that calls itself is known as a recursive function.

Following example demonstrates nesting of two functions −

#!/bin/sh

# Calling one function from another

number_one () {

   echo "This is the first function speaking..."

   number_two

}

number_two () {

   echo "This is now the second function speaking..."

}

# Calling function one.

number_one

To continue, go to the blog page  part 3: bash-scripting-and-shell-programming_part3

Bash scripting and shell programming linux part 1

The Part#1 includes:

• What is a Shell & Types of shells
• What is Script and how to determine shell
• She-Bang & comments
• Steps to create a shell script and execution
• Variables & usages, assign output command to a variable, Variables Names
• File and String Test Operators
• Decision making statements if, if else, if elif else
• Shell Scripting loops for, while, infinite loop, until

 What is a Shell?

An Operating system is made of many components, but its two prime components are -

• Kernel
• Shell

A Kernel is at the nucleus of a computer. It makes the communication between the hardware and software possible. While the Kernel is the innermost part of an operating system, a shell is the outermost one.

A shell in a Linux operating system takes input from you in the form of commands, processes it, and then gives an output. It is the interface through which a user works on the programs, commands, and scripts. A shell is accessed by a terminal which runs it. When you run the terminal, the Shell issues a command prompt (usually $), where you can type your input, which is then executed when you hit the Enter key. The output or the result is thereafter displayed on the terminal.

The Shell wraps around the delicate interior of an Operating system protecting it from accidental damage. Hence the name Shell.

A shell is a command-line interpreter and typical operations performed by shell scripts include file manipulation, program execution, and printing text.

Types of Shell

There are two main shells in Linux:

1. The Bourne Shell: The prompt for this shell is $ and its derivatives are listed below:

• POSIX shell also is known as sh
• Korn Shell also knew as sh
• Bourne Again SHell also knew as bash (most popular)

2. The C shell: The prompt for this shell is %, and its subcategories are:

• C shell also is known as csh
• Tops C shell also is known as tcsh

Scripts:

• contains a series of commands
• An interpreter executes commands in the scripts
• anything you can type at the command line, you can put in a script.
• great for automating tasks.

How to determine Shell:

oracle@test> echo $SHELL

/usr/bin/bash

The $ sign stands for a shell variable, echo will return the text whatever you typed in.

Shell Scripting She-bang:

The sign #! is called she-bang and is written at top of the script. It passes instruction to program /bin/sh.

To run your script in a certain shell (shell should be supported by your system), start your script with #! followed by the shell name.

Example:

#!/bin/bash  

echo Hello World  

• The first line of a shell script typically starts with a shebang followed by the path to an interpreter that will be used to execute the commands in the script. 
• If you do not supply a shebang and specify an interpreter on the first line of the script the commands in the script will be executed using your current shell.

Shell Scripting Comments:

Any line starting with a hash (#) becomes comment. Comment means, that line will not take part in script execution. It will not show up in the output.

Steps to create a script and execute:

• create a .sh file using cat or touch command
• write the body and saved it
• provide the execute permission using chmod 777 or chomd +x command
• execute by ./scriptname.sh

Example:

cat > variable.sh

#!/usr/bin/bash

var1=jai

var2=viru

#Echo

echo "$var1 $var2"

chmod 777 variable.sh

./variable.sh

Output: jai viru

Variables:

• Storage location that have a name. By convention variables are upper case.
• Name value pair like variable_name = "Value"
• Variables are case sensitive

Variable usage:

#! /bin/bash

MY_SHELL="bash"

echo " I like the $MY_SHELL shell"

or 

echo "I like the ${MY_SHELL} shell"

Output:

I like the bash shell

Assign command output to a variable:

#!/bin/bash

SERVER=$(hostname)

echo "The script is running on ${SERVER}"

or 

#!/bin/bash

SERVER=`hostname`

echo "The script is running on ${SERVER}"

Output:

The script is running on LinxTestServer

Variables Names:

• It contains combination of uppercase and lowercase alphabets, digits and underscore as follows:

FIRST3VARIABLE

FIRST_VARIABLE

FirstVarable

• Variable names should not start with a digit or not use any special characters other than underscore. For instance,

2FirstVariable

First-Varable

FIRST@VARIABLE

Read-only Variables:

Shell provides a way to mark variables as read-only by using the read-only command. After a variable is marked read-only, its value cannot be changed.

For example, the following script generates an error while trying to change the value of NAME −

 cat > read.sh

#!/usr/bin/bash

NAME="Sri"

readonly NAME

NAME="Rudra"

echo "$NAME"

The above script will generate the following result −

./read.sh: line 4: NAME: readonly variable

Sri

Unsetting Variable:

Unsetting or deleting a variable directs the shell to remove the variable from the list of variables that it tracks. Once you unset a variable, you cannot access the stored value in the variable.

Following is the syntax to unset a defined variable using the unset command −

unset variable_name

The above command unsets the value of a defined variable. Here is a simple example that demonstrates how the command works −

 cat > read1.sh

#!/usr/bin/bash

NAME="Sri"

unset NAME

echo $NAME

The above example does not print anything. You cannot use the unset command to unset variables that are marked readonly. 

 File Operators(Tests):

-d FILE : True if file is a directory

-e FILE : True if file exists

-f FILE : True if file exists and is a regular file

-r FILE : True if the file is readable by you

-s FILE : True if file exists and not empty

-w FILE : True if file is writable by you

-x FILE : True if is executable by you 

String Operators(tests):

-z STRING : True if string is empty

-n STRING : True if string is not empty

String1 = String2 : True if the strings are equal

String1 != String2 : True if the strings are not equal.

Making Decisions:

The if statement:

syntax:

if [ condition is true ]

then 

commands

fi

Example:

#! /bin/bash

MY_SHELL="bash"

if [ $MY_SHELL = "bash" ]

then 

echo "You like the bash shell"

fi

if-else statement:

Syntax:

if [ condition is true ]

then

commands

else

commands

fi

Example:

#! /bin/bash

MY_SHELL="csh"

if [ $MY_SHELL="bash" ]

then

echo " You like bash shell"

else

echo " you dont like bash shell"

fi

if elif else statements:

Syntax:

if [ condition is true ]

then

commands

elif [ condition is true ]

then 

commands

else

commands

fi

Example:

#! /bin/bash

MY_SHELL="bash"

if [ $MY_SHELL="bash" ]

then

echo " You like bash shell"

elif [ $MY_SHELL="csh" ]

then

echo " you like csh shell"

else

echo " you dont like bash or csh shells"

fi

Shell Scripting for loop:

The for loop moves through a specified list of values until the list is exhausted.

Syntax 1:

Syntax of for loop using in and list of values is shown below. This for loop contains a number of variables in the list and will execute for each item in the list. For example, if there are 10 variables in the list, then loop will execute ten times and value will be stored in varname.

for table in {min_value..Max_value..Increment_value}

do

commands

done

Example:

cat >for1.sh

#!/usr/bin/bash

for table in {2..20..2}

do

echo "$table"

done

chmod +x for1.sh

 ./for1.sh

Output:

2

4

6

8

10

12

14

16

18

20

Syntax 2:

for (( initialization;condition;increment/decrement))

do

commands

done

Example:

cat > for2.sh

#!/usr/bin/bash

for (( i=10; i >= 1; i--))

do

echo "$i"

done

chmod +x for2.sh

./for2.sh

Output:

10

9

8

7

6

5

4

3

2

1

Syntax 3:

for Var_name in Item_1 ItemN

do

commands

done

Example:

#! /bin/bash

COLORS="RED GREEN BLUE"

for COLOR in $COLORS

do 

echo "COLOR: $COLOR"

done

Output:

COLOR:RED
COLOR:GREEN
COLOR:BLUE

Scripting while loop:

There is a condition in while. And commands are executed till the condition is valid. Once condition becomes false, loop terminates.

Syntax:

While [ condition ]

Do

commands

done

example:

cat >while.sh

#!/usr/bin/bash

i=10;

while [ $i -ge 0 ] ;

do

echo " reverse order number $i"

let i--;

done

chmod +x while.sh

 ./while.sh

Output: 

reverse order number 10

 reverse order number 9

 reverse order number 8

 reverse order number 7

 reverse order number 6

 reverse order number 5

 reverse order number 4

 reverse order number 3

 reverse order number 2

 reverse order number 1

 reverse order number 0

while infinite loop:

Infinite loop is also called endless loop. It is made with while true (it means condition will always be true) or while : (it means an empty expression), where colon (:) is equivalent to no operation.

 cat whileinfinite.sh

#!/usr/bin/bash

while true

do

echo " this is infinite loop. press ctrl + c to exit"

done

Shell Scripting until loop:

It is similar to while loop. The only difference is that until statement executes its code block while its conditional expression is false, and while statement executes its code block while its conditional expression is true.

Difference between while and until:

Until loop always executes at least once. Loop while executes till it returns a zero value and until loop executes till it returns non-zero value.

Syntax:

Until [ condition ]

Do

Commands

Done

Example:

 cat > until.sh

#!/usr/bin/bash

i=5;

until [ $i -gt 15 ];

do

echo "number $i"

let i++;

done

 chmod +x until.sh

 ./until.sh

Output:

number 5

number 6

number 7

number 8

number 9

number 10

number 11

number 12

number 13

number 14

number 15

Example: Rename all the.jpg files of the current folder to YYYY-MM-DD- filename.

#! /bin/bash

LIST_PICTURES=$(ls *.jpg)

DATE=`date +%F`

for PICTURE in LIST_PICTURES

do

echo "Renaming $PICTURE to $DATE-$PICTURE"

mv $PICTURE $DATE-$PICTURE

done

To continue, go to the next blog page bash-scripting-and-shell-programming-part-2.

12c OSB - Dynamic routing part1 with Proxy services

Here, we are going to showcase a simple way of how can we do dynamic routing from your OSB pipeline flow to other proxy services.

We will mock the response from those proxy services just to showcase that how the control comes into a particular proxy based on dynamic routing.

Implementation steps:

Step1: Create a New OSB project (New ⇾ Project ⇾ Service Bus Project ⇾ Give project Name DynamicRoutingSBProject)

Step2: On project ⇾ New ⇾ From galary ⇾ XML Schema ⇾ File name Employee.xsd. and target namespace if you can change.

<?xml version="1.0" encoding="windows-1252" ?>

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.example.org/Employee"

            targetNamespace="http://www.example.org/Employee" elementFormDefault="qualified">

  <xsd:element name="Employee">

    <xsd:complexType>

      <xsd:sequence>

        <xsd:element name="name" type="xsd:string"/>

        <xsd:element name="department" type="xsd:string"/>

        <xsd:element name="URL" type="xsd:string"/>

      </xsd:sequence>

    </xsd:complexType>

  </xsd:element>

    <xsd:element name="EmployeeResp">

    <xsd:complexType>

      <xsd:sequence>

        <xsd:element name="output" type="xsd:string"/>

      </xsd:sequence>

    </xsd:complexType>

  </xsd:element>

</xsd:schema>

Step 3 : Create Main proxy service. Go to the servicebus.sboverview and In proxy services lane, Insert Transports ⇾ HTTP ⇾ Provide Proxy service name as ProxyServiceMain and pipeline as ProxyServiceMainPipeline ⇾ Next ⇾ Select ⇾ Messaging and Request as XML and choose the created XSD for request and Response ⇾ Next ⇾ Finish. Main Proxy and pipeline done.

Step4 : Now create 2 proxy services ProxyService1 and ProxyService2. In proxy services lane, Insert Transports ⇾ HTTP ⇾ Provide Proxy service name as ProxyService1 and ProxyService1 and pipeline as ProxyService1Pipeline and ProxyService2Pipeline⇾ Next ⇾ Select ⇾Any XML⇾ Next ⇾ Finish. 

Step5: Go to the ProxyService1pipeline  and ProxyService2pipeline and drag and drop pipeline pair node and in the response stage, insert a replace activity and sending a response as below snaps.

step6 : Go to the main proxy pipeline and take a pipeline pair node and take 1st assign and add the routing rules xml into varURLMap variable.

<routing>

<row>

<department>IT</department>

<endpoint>DynamicRoutingSBProject/ProxyService1</endpoint>

</row>

<row>

<department>Sales</department>

<endpoint>DynamicRoutingSBProject/ProxyService2</endpoint>

</row>

</routing>

step7: Take another variable varDept and store the department from the proxy requst.

Step8 : Take another variable varRoute and store the following:

<ctx:route>

<ctx:service isProxy="true">{$varURLMap/*:row[department=$varDept]/*:endpoint}</ctx:service>

</ctx:route>

Note: 'isProxy' will be true only and only if we are routing to some or the other proxy service only.

In case you want to route to business service(s), 'isProxy' will be false.

Step9: Drag and drop Dynamic routing activity and provide Service =$varRoute.

Now test it. If you give department as IT. then it will route to ProxyService1 an provide the response what we configured in proxyservice1pipeline.

12c SOA - BPEL dynamic XSLT

Here we will see how to do dynamic XSLT that means to dynamically choose an xslt to execute on runtime.

Logical steps:

• Create a DVM for instance that contains a column on which you want to select, and a second column that lists the particular xslt to be executed.
• Then create string variable 'varXSLT' and an assign before the transformation, in which you do an assign using the LookupDVM() to translate the selection code to the actual xslt file with the varXSLT variable as target.
• From the source view move the copy rule of the transformation to the assign with the LookupDVM() function or in a different assign, remove the transformation activity. In the expression builder replace the reference to the xslt with the varXSLT variable. 
• Use ora:processXSLT() function instead of using ora:doXSLTransformForDoc(). If you use ora:doXSLTransformForDoc() function, it will not allow you to map/use this varXSLT variable as it needs a string literal as input value.
• Later, you can move the xsl files to MDS and use the oramds xpath to use the MDS reference xsl files dynamically. This way if you need to change anything in the xsl file , just change in MDS and no need to change in composite itself.

Note: If you create a transformation activity in BPEL, choose the sources and targets, create the xslt etc. then if you go to the source view, you will discover that it is actually an assign activity. The execution of the xslt is done through an xpath expression that references the particular xslt file.

 Implementation steps:

Create a SOA project.

Create a XSD

<?xml version="1.0" encoding="windows-1252" ?>

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.example.org"

            targetNamespace="http://www.example.org" elementFormDefault="qualified">

  <xsd:element name="OrderRequest">

    <xsd:complexType>

      <xsd:sequence>

        <xsd:element name="OrderId" type="xsd:string"/>

        <xsd:element name="OrderType" type="xsd:string"/>

      </xsd:sequence>

    </xsd:complexType>

  </xsd:element>

    <xsd:element name="OrderResponse">

    <xsd:complexType>

      <xsd:sequence>

        <xsd:element name="Status" type="xsd:string"/>

      </xsd:sequence>

    </xsd:complexType>

  </xsd:element>

</xsd:schema>

Use this schema and create a Syn BPEL

Create a DVM

here i added two rows containing 2 xsl files.

Create a Variable varXSLT

Take a assign and Use dvm:lookupValue() function to fetch teh xsl names based on the orderType.

Take a Transformation activity and create 2 xslt files as defined in the dvm.

XSLT1

XSLT2

Use ora:processXSLT() instead of using ora:doXSLTransformForDoc() functions.

Testing:

We can push the xsl files to MDS and use them dynamically from MDS too.

First i have kept the xsl files in the SOA design time repository local files,

Publish the files to MDS

Use the oramds path to the ora:processXSLT() function like this and it works.

Web Services Security at Transport Level and Message Level

There are two ways with which we can ensure security with Web Services:

Transport level security, such as HTTP Basic/Digest and SSL/TLS, is the usual "first line of defence", as securing the transport mechanism itself makes Web services inherently secure.  The trade-off is transport dependency (Web services are more tightly coupled to the network transport layer).

Authentication

The basic authentication scheme is by passing the credentials (userid & password) in the http header. This can be improved using password digest: the credentials are hashed (so that the attacker can not read the password) & using nonce (to prevent reply attack)

SSL

Certificates can be used for authentication, encryption and signature (non repudiation)

How

by setting in the web server (e.g. Weblogic, Apache, Tomcat): basically enabling the https listening port and register the location of keystore/certificates.

Message level security, such as WS-Security, SAML, XML Digital Signatures, and XML Encrypttion,  can be more effective and has the added flexibility that the message can be sent over any transport.

Message standard for SOAP web services security e.g. WS-Security (WSS), WS-Policy.

How:

Java: using handler/adapter to insert WSS header in the request and remove the WSS header in the received response. The handler also encrypt/decrypt the data.

Java using Rampart/Axis2 framework: set security context (e.g. keystore) in the request, define security policy in the wsdl. Futher info: read book by Tong.

OSB: using OWSM by defining policy. 

Why message-level security (e.g. WS-Security) is better than transport-level security (e.g. TLS/SSL):

• End-to-end security: message-level XML-Encryption protects sensitive data also in the intermediaries / external proxies. The point-to-point security TLS/SSL doesn't prevent the intermediaries to read the sensitive data.
• With WS-Encryption it's also possible to encrypt only a part of the messages for flexibility (e.g. in case the intermediary proxy need to peek the unencrypted part) or performance (it's cheaper to encrypt/decrypt only portions of the messages).
• The message-level security (e.g. WSS Authentication, XML-Encryption, XML-Signature) is independent to the protocols thus it offers more flexibility to send SOAP messages across different protocols (e.g. http, jms, ftp).

On the other hand, message-level security has also disadvantages:

• Performance (encrypt/decrypt, validate): processing time & increased message size
• Configuration & Maintenance (but can be easier using declarative policy)
• Can not peek the message values during development & debug
• More complex, more difficult to find developers who master

For more details, you can traverse to Web Services Security

12c SOA - Weblogic - Oracle apps adapter connection factory creation

In this blog we will see how we can create an oracle apps adapter connection factory.

Deployments⇾OracleAppsAdapter

Configuration

Outbound Connection Pools

Click New

select Outbound Connection Groups and Next

Provide the JNDI Name.

Apps Connection factory created. Open it

Give the XADataSourceName

Save

Go to the deployment again and select OracleAppsAdapter

Update

Select Update option to update the changes in plan.xml file.

Finish

Activate Changes

If it is a clustered environment then using putty, we need to copy the updated new apps plan.xml file under the deployment plan path for all the nodes.