Should I “bash” Or “bash”? Or writing scripts in Bourne Again Shell

[Giniu 0]

Should I âBashâ or âbashâ?

Or: âWriting scripts in GNU's Bourne Again Shellâ.

 

For the begining â short story and what is what...

 

In this tutorial I assume, that you don't have any basic knowledge, but you know Linux (or your other UNIX system) to understand what I'm talking about...

 

Bash stands for Bourne Again Shell, it is child of Bourne Shell (sh), that was improved with elements of Korn Shell (ksh) and C Shell (csh) to gain power and new possibilities in programming and interactive use. It is quite easy while powerful... but what it is?

 

What is shell? It is command line interpreter... and what is it? It is something in which you put code to execute... in UNIX command line interpreter is for example sh, bash, ksh, csh and many more... approximately about 30 more, in Windows command line interpreter is MS-DOS... we can say that... but it is quite primitive... anyway... So you knew what shell is, but now you also know that it is called shell... But what the heck is shell script? You heard many times that people was talking... âYou can write shell script that would do it for you...â What the... ? So... Let me explain â When you type commands in shell in interactive mode you give them one by one â seeing result after every command... so imagine that you have to do something twice... oh man borring... and when you have to repeat twenty command for hundred files in different directories? Mission impossible... so this is place when you have to write script â a file containing commands you would normally execute one by one, and when you run it, each command is run in right order so you can go get some coffee and relax while shell is doing hard job for you. Popular script file extensions are .sh (for UNIX-like systems, but you mainly don't need to give them any name â you can just make them executable and everything works just like it should) and .bat (for MS-DOS), so you probably saw few shell scripts in your life, but now you know the name...

 

In this tutorial I wouldn't talk about MS-DOS' scripts, so from now on suppose only info about bash...

 

 

Before I start â getting ready place to work...

 

First of all, unpack your computer from box, look for some place on desk to put it on, then see if you have all wires... but serious â I want to talk about getting ready place to work... but on hard drive not on desk â scripts are files like others so you have to get some disk space where you write and test them... There are many schools â some people say, that best place for scripts is /bin directory... others â that itâs /usr/bin or /usr/local/bin... I think best place for scripts is /usr/scripts â that you would create â with symbolic links to /usr/local/bin so you can have all your scripts in one place and you can run them like they would be just in /usr/local/bin or even /bin â this is where I have them, and writing this tutorial I also assume that you would put them there, but of course you can modify this path and everything would work also great â this is you choice... If you decide to choose my way, create script directory:

 

mkdir /usr/script

I assume you know about permissions and choose them right â this is not place for this â If you want to know more about permissions refer to âDemystifying "chmod" - Part 1, & why viruses get so retarded on Linuxâ by microscopic^earthling.

 

Let the game begin â very basics of bash scripting...

 

So first of all â you have to know about variables. They are used exactly to what their name describes... they are used to store data... To use some variable you have to assign value to it... you just turned on your computer... you think that you haven't any variable in system? Then type in shell:

 

echo $SHELL

you should see

 

/bin/bash

if not â execute /bin/bash and repeat this... Well so â there is one variable... maybe there are others? Type:

 

echo $LONGNAMEecho $PWD

and how is that? Yes â there are many more... you can see what variables are set by typing echo $ and pressing Tab key twice. So â now let's set some variable...:

 

export MYVAR=âMy first variableâ

then type:

 

echo MYVAR

and you should see... MYVAR? Yes... because you don't have to invoke text MYVAR, but refer to value stored in MYVAR... now type:

 

echo $MYVAR

uff... now it is working So now you see how can variables be used... in Scripts declaration are more easy, you have only to write:

 

MYVAR=âMy frist variableâ

But usage is same, you must refer to it by adding $ at the beginning... So Let's write some easy and useless script... Create text file in /usr/script of desired name ending with .sh, then make it executable, then edit it with your favourite text editor and put into it something like this (don't worry you don't know everything yet... I would explain it in a moment, just remember to put only empty line at end of file):

 

#!/bin/bash# Now set some variable...:MYVAR=âmy first scriptâANOTHERONE=$MYVAR ':)'# and tell whole world about it, let's there be echo from this !!!echo this is $MYVAR.

so... as you see this is useless program, but I will now explain it... first line... this strange #!/bin/bash... let's say that your default shell is sh or ksh â so scripts in bash shouldn't work for you? This line tells system âthis script is written in bash, execute it in bash interpreter â run /bin/bashâ. So â this is now clean what this mean... Then I left empty line â for scripts there doesn't mater how many empty lines you put into script... They are just for easy of read. Next one is strange again... I used hash like in first line, but you see that it must do something differ â and you are right â this is a comment â it also doesn't mater how many comments you put into your script... just when you read it after some time you know what do what... there is other comment in this example... I used in it echo â but it isn't consider as command echo â everything after â#â is ignored... Next line â I set variable... as you see I don't have to use export... you must be warn, that variables that are set in script doesn't exist without it â so you cannot use script to set system variables. But in next line you see, you can use variables to set variables â I used there single quotation to add smile to it... I used it because â()â are registered and reserved to gather code, so when you use it, it thinks you want to do is and not write ) - so I quoted it â they wouldn't be visible. But how can you quote ' ? Just double quote it â ' â and to quote â type ' â ' - easy, right? At last line I send text to default out â terminal (shell) and as you can see you can use variables and strings (text) in one line without any troubles... also adding dot at end doesn't matter â MYVAR is still interpreted as âMYVARâ, not as âMYVAR.â. And this is all... save it and run it... you should get:

 

this is my first script

 

So I'm keep going â expresions and I/O redirections...

 

Right... so now let's make some arithmetics â now not in script but in shell, just like last time... Let's say sum 2 and 5... it should be 7 if I'm not wrong... type:

 

expr 2 + 5

as you can see + is divided from 2 and 5 by spaces... and the result is... who knows... 7... In bash you also can do multiplications /* (* cannot be used since it used as wildcard in file names), subtraction -, you can divide numbers / (integer results, floats result will be discussed later), count rest while dividing %. But there is one problem... you cannot send it to variable by typing:

 

export SUM=expr 2 + 5

since you would get error â also typing

 

export SUM=âexpr 2 + 5â

or

 

export SUM='expr 2 + 5'

won't work since you would get âexpr 2 + 5â not â7â... so if single and double quotation doesn't work what can be done? There is way â use backquote â ` â... try this:

 

export SUM=`expr 2 + 5'echo $SUM

and who knows... we get result! So there is explanation â using backquote sends result to variable... This can be assigned straight to variable, but if you want to use it without assigning result to variable, use $( command ) command, like:

 

echo $( expr 2 + 5 )

but it doesn't take care for new lines, try this:

 

echo `ls`

as you can see this is usesfull for short results processing. But what if you wan't to process some long text? Like files list above? You have to do some operations on it and as you know â you can make operations on files... so you can send output not ditectly to shell but to file... this works for any command... just use:

 

command >> file

to send command results to file, or

 

command << file

to send file to command using >> or << would add new strings at end of file, using < or > would overwrite file (and create new if there isn't one)... you can also send command results stright to command in two ways just like they would be usual files. Let's see some example that shows above two things... Let's write script that counts factorial from 9 to11 and send results to file. We would make it better when I would go to loops... But now let's get to work:

 

#!/bin/bash#let's get first 9!FAC=`expr 2 \* 3 \* 4 \* 5 \* 6 \* 7 \* 8 \* 9`# send it to file creating new oneecho 9! = $FAC > factorial.result#count 10!FAC=`expr $FAC \* 10`#append it to fileecho 10! = $FAC >> factorial.result#do same as above for 11!FAC=`expr $FAC \* 11`echo 11! = $FAC >> factorial.result

when you would execute this script in directory where you executed it you have now file âfactorial.resultâ that contains desired results... quick and easy... now when you want to count factorial, just see what result it is...

 

Something more â results processing...

 

On UNIX-like systems there are many utilities that can be used to scripting, I won't talk about commands like cp, ls, rm, mv, chmod, su or cat since I think they was covered in prerequisites of this tutorial â this is very basic UNIX commands and without them you almost cannot use system... But there are few utilities about which I would write. First of them is wc:

 

wc comes from âword countâ and it counts lines, words and characters in file... the result is formatted like:

 

lines words characters file

 

and can be invoked using

 

wc file

second utility I would mention is sort â it names says all, it sorts file, sending result to default stream (terminal) so you can redirect it to do example file. You can Invoke it by typing:

 

sort file

while you want to sort number, you probably want to use:

 

sort -n file

to make 1000 larger that 2, you can also sort in inverse order, by typing:

 

sort -r file

Now what if you want to change something in file? There is sed... Most common use of sed is:

 

sed -e 's/FROM/TO/g' old_file > new_file

that form substitutes all FROM concurrencies to TO in old_file and saves modified file under name new_file... for example:

 

sed -e 's/welcome/bye/g' welcome.sh > bye.sh

There is also similar utility, but it can translate single letters, it is tr. Mainly tr is used to change one letter to another:

 

tr 'letter1' 'letter2' < old_file > new_file

changes all letter1 ocurences to letter2 occurences in old_file and saves it under new_file. It can be used also with ranges, like in this example that converts lowercases to uppercases in file lower and saves it under file named upper:

 

tr [a-b] [A-B] < lower > upper

Another useful utility for formatting and parsing is tail â it make file read from specified line to end of file, it can be invoked by:

 

tail +line_number file

tail +1 file doesn't make any difference than cat file, but tail +2 file don't show first line of file. Another thing that can be used to format output are pipes â thing that allow you to format it at time... There are many pipes but I would cover three of them. Pipes are used by using structure command | command, you can use multiple pipes â command | command | command... . First of commands mainly used in pipes is awk...

 

awk is used to process column data, so you can display only one column of result, you can do this like that:

 

ls -l | awk '{print $5}'

not there are single quotes, not backquotes... mainly this looks like:

 

command(s) | awk '{print $collumn_number}'

of course awk have much more functionality, check man awk to see how powerful it is... Next utility is much smaller, but also useful for processing â it is head â it shows only n lines from file, for example:

 

ls -l | head -3

shows only first three lines of directory listing, mainly use of it looks like:

 

command(s) | head -n

so you can show only n lines from command's result. Next and last pipe I would talk about is grep â it is used to show only lines containing specified text, while using from shell it is very usefull while checking in checking pid of application to kill, then we use:

 

ps x | grep application_name

and we get pid, then we are killing troublesome application. Mainly this pipe looks like:

 

command(s) | grep text

and shows only lines from result that contains text. Let's look at some example... first of all we would list files in current directory that are owned by root, then we would take their size, sort it and give in result size of largest and in separate block form second to fifth size. We write results to file, count characters and append it at end of file...

 

#!/bin/bashecho larger file in $PWD directory have `ls -l | grep root | awk '{print $5}' | sort -n -r | head -1` bytes > sizeanalizer.resultsecho others have: >> sizeanalizer.resultsls -l | grep root | awk '{print $5}' | sort -n -r | tail +2 | head -5 >> sizeanalizer.resultsecho and maybe less... >> sizeanalizer.resultsSIZE=`wc sizeanalizer.results | awk '{print $3}'`echo -------------------------------------------------------- >> sizeanalizer.resultsecho >> sizeanalizer.resultsecho to display text above this line, $SIZE characters were used >> sizeanalizer.results

it could be made easier, but the point is to demonstrate how these tools can be used...

 

And what if... - Test constructs...

 

As you see you can create now even complicated things... but what if you want to create file that exist or allow script to work only under exact circumstances? There you need to use if-else and case to do some test and steer script execution... right now our scripts were plain, now they got some depth and not always can execute same... it depends... but lets talk about test first...

 

test string must be contained between [ ... ] or after test command â can be said this returns true or false, but don't shows them as strings. So lets see how test can be made (note spaces in [] - they are important !!!):

 

STRING TESTS:

[ stringA == stringB ] is true if strings A and B are equal

[ stringA != stringB ] is true if strings A and B aren't equal

[ -z string ] is true if string have zero length

[ string ] or [ -n string ] is true if string has non-zero length

[ stringA < stringB ] is true if string A is before string B (lexicographicaly)

[ stringA > stringB ] is true if string A is after string B (lexicographicaly)

 

NUMERIC TESTS:

[ num1 -eq num2 ] stands for equal

[ num1 -ne num2 ] stands for non-equal

[ num1 -lt num2 ] stands for less than

[ num1 -le num2 ] stands for less or equal

[ num1 -gt num2 ] stands for greather than

[ num1 -ge num2 ] stands for greater or equal

 

FILE TESTS:

[ -e file ] true if âfileâ exists

[ -d file ] true if âfileâ exists and is directory

[ -f file ] true if âfileâ exists and is regular file

[ -L file ] true if âfileâ exists and is symbolic link

[ -r file ] true if âfileâ exists and is readable

[ -w file ] true if âfileâ exists and is writable

[ -x file ] true if âfileâ exists and is executable

[ -s file ] true if âfileâ exists and it's length is non-zero value

[ -N file ] true if âfileâ have been modified since it was last read

[ file1 -nt file2 ] true if âfile1â is newer than âfile2â

[ file1 -ot file2 ] true if âfile1â is older than âfile2â

 

OTHER TESTS:

[ test1 -o test2 ] is true if true is any of tests

[ test1 -a test2 ] is true if true are both of tests...

true always true

false always false

 

warning, while using neasted tests don't use [ [ ... ] -o [ ... ] ] but only [ ... -o ... ], same for â-aâ!!!

 

As you see there are many test and that aren't everything â they are only commonly used ones... So when you know how to make some test, let's go to if-else statement.

 

Mainly if-else structure looks like this (consider tabs and spaces to make it easier to read):

 

if expressionthen	commandselse	commandsfi

when i write commands I mean any command and by expression some test or file execution... just let me show you some of possibilities:

 

if [ $a == $b ]then	echo a is equal to belse	echo a isn't equal to bfi

this is simplest way... it also can be written like that:

 

if test $a == $bthen	echo a is equal to belse	echo a isn't equal to bfi

if you want to see if some application returned value 0 (exited normally) use:

 

if commandthen	echo command exited normallyelse	echo command exited abnormal, possible errors...fi

of course commands can be another if...:

 

if [ $a == $b ]then	echo a is equal to belse	if [ $a == $c ]then  echo a is equal to c	else  echo a isn't equal to b or c	fifi

this can be written short:

 

if [ $a == $b ]then	echo a is equal to belif [ $a == $c ]then	echo a is equal to celse	echo a isn't equal to b or cfi

As you see this can grow larger and larger every if-else... so what can we do about it? We can use case... It looks like this:

 

case variable in	value1 )  commands1;;	value2 )  commands2;;...	* )  default_commands;;esac

so â when variable is equal to value1, commands1 will be executed, if variable is equal value2, commands2 will be executed... if there isn't suited value, default_cammands will be executed... There is some example:

 

case $number in	1 )  echo number is one;;	2 )  echo number is two;;	3 )  echo number is three;;	* )  echo I can count only to three;;esac

case values looks a little different than test... there is what can be used:

 

strict value, like 1, 2, âsomethingâ

logicaly added values, like value1 | value2 that stands for value1 or value2

range value, like [1-4] that stands for 1, 2, 3, 4 or [a-c] that stands for a, b, c

 

just case allows us to create useful code fragments, like this:

 

case $( arch ) in	i386 )   echo 80386-based machine;;	i486 )   echo 80486-based machine;;	i586 )   echo Pentium-based machine;;	i686 )   echo Pentium2+-based machine;;	*    )   echo Other type of machine;;esac

how this would look using if-else? I can make you sure it would look longer... But what if you want to execute some fragment of code many times? You must use loops â loops that are allowed in bash are for and while, let me explain them... for should go first, it stands for âforeachâ and looks like:

 

for variable in listdo	commandsdone

how it works? It works like:

 

variable = first in list

commands

variable = second in list

commands

...

 

so â what is list? This is easy... lists are any variables or wildcards â exactly, variables that looks like:

 

variable=âvalueâ

have one list element, but variable:

 

variable=â value  value value ...â

have many elements â and this is variable list... you can also create file wildcard list... that looks like:

 

*.sh

that contains all file names ending with .sh in current directory, or

 

/usr/locale/games/mygame/*my*.dat?

that contains all filenames that contains my and ends with .dat1, .dat2, .data or .dat^, but not .dat or .dat10 in directory /usr/locale/games/mygame

 

Let me show you some example of for usage:

 

LIST=".c .cpp .h"for EXTENSION in $LISTdo        for FILE in *$EXTENSION        do                echo $FILE        donedone

that would list all files ending for .c, .cpp and .h in current directory, so you see for is very powerful, but it can't be executed exact times if we don't know the list or if we don't need to use any list... and there with help comes while loop:

 

while test_expressiondo	commandsdone

test expressions can be similar to if-else, but mainly there are used string and number tests, files tests are rather useless in this command... There is part simple example that counts from 1 to 100:

 

COUNTER=1while [ $COUNTER -le 100 ]do	echo $COUNTER	COUNTER=`expr $COUNTER + 1`done

Every loop (for and while) can be stopped at any moment if it executes break command it goes behind loop. There is example of using break:

 

while truedo	echo you think this would be looping infinite?	breakdoneecho WRONG ':)'

Now I would give you some example of use what we learned till now... I won't talk what it does, you should now what, if not, get back to earlier parts...

 

#!/bin/bashEXTENSIONS=".tex .abc"DIRECTORIES="design flash"if [ -e /tmp/countertemp ]; then	echo "delete /tmp/countertemp to continue"	exit 1fimkdir /tmp/countertempCOUNTERALL="0"COUNTER="1"LINESALL="0"for DIRECTORY in $DIRECTORIESdo	for EXTENSION in $EXTENSIONS	do  COUNTERALL=`expr $COUNTERALL + 1`  find $DIRECTORY -name *$EXTENSION -fprint /tmp/countertemp/_"$COUNTERALL"_.files	donedonewhile [ $COUNTER -le $COUNTERALL ]do	FILES="/tmp/countertemp/_"$COUNTER"_.files"	LINES=$(wc "$FILES"|awk '{print $1}')	LINE_NUMBER="1"	while [ $LINE_NUMBER -le $LINES ]	do  FILENOW=$(tail +$LINE_NUMBER "$FILES"|head -1)  LINENOW=$(wc "$FILENOW"|awk '{print $1}')  LINESALL=`expr $LINESALL + $LINENOW`  LINE_NUMBER=`expr $LINE_NUMBER + 1`	done	COUNTER=`expr $COUNTER + 1`doneecho $LINESALLrm -r /tmp/countertemp

A little more advanced thing for the end â read, bc, arguments, exit, functions...

 

This is almost everything â there are few small things left first of all â a float operations... this can be silly when you try to count percentage of something and get result 99/100=0 since 0.99 is taken down to 0 â so as you see float operations are required if you want to create some usefully counters... there is info how you can use them. I would only tell how to divide two variables, since other operations are similar...:

 

echo "scale=5; $A / $B" | bc

this would divide A by B and would show 5 places... This looks so strange, because bash can store only integer values, so floats are stored as string and we have to do some conversions... but it works...

 

Other thing I would mention at end are command line arguments so you can call you script not by:

 

command

but

 

command option1 oprion2 option3 ...

command line arguments can be accessed very easy â just use:

 

$0 gives you command name that was executed to run script (without arguments)

$1 stands for first argument

$2 stands for second argument

$# gives you number of arguments

$* gives you all arguments as list

 

they are considered usual variables, so you can for example do:

 

echo there are $# command line arguments...

So you know how to use command line arguments... was easy, right?

 

Probably you also want to communicate with user in other way â let user type something on keyboard and store it under variable... this is as easy as say â just type in script:

 

read VARIABLEecho $VARIABLE

and you would see how it works... Ok... so we have that... and other thing... if your script executed other script you probably want to know if itâs succeed... there with help comes exit status... Just add to end of file exit with status number (0 means everything went ok, number form 1 to 255 means error) and you would know what happened to script... for example in scriptA executed from scriptB there is:

 

if [ -e config ]then	exit 1fi...exit 0

and in scriptB we have:

 

if scriptAthen	echo config createdelse	echo error ocured â config file present...fi

And last thing that left to make you skilled shell programmer â functions... why use them? To make your script clean and easy to read, to not write same thing few times, there are many âbecauseâ so there is how you can define function:

 

somefunction () {	echo Argument number 1 $1	echo Argument number 2 $2}

then you can call it by typing:

 

somefunction arg1 arg2

Oh man â really long tutorial, don't you think? But say... you can now write almost all scripts that you may want! Wasn't it worth to spend some time and read it?

 

Any way â that is it â if you have some question just ask in this topic and I for sure would answer it... If you would find error (I'm just human...) also post it there I would check it again and PM moderator or admin to change it...

 

Hope you liked it

 

Regards

 

Giniu

 

-------------------------------------

 

PS.: If there is such need I can write another tutorial â essential Linux commands in which I would cover all topics that I said you have to start with it like using of commands like cp, mv, rm and basic Linux usage like changing mode to console and so on... everything that would fill the hole in wisdom... I would be happy to take a part in filling it Also if I made some mistake - just post fixed example/information so moderators/admins would be able to join it into this tutorial with note who fixed it

 

PPS.: Thanks to Nelle for checking language things - english isn't my primary language... keep this in mind...

[NoMore1405241533 0]

WOWman thats a hard worktnxthis gona help me alot :)NoMore

[iGuest 3]

thank youShould I “bash” Or “bash”?

I went thru your tutorial and wanted to say kudos for putting this all together.

great job!

-feedback by coolmn