Shell Scripting and Process Programming

Shells

A shell is a program that runs on top of the OS, and main task is to interface the use with the operating system (OS). The graphical env in Windows, MacOS are both examples of graphical shells, allowing user to interact with OS using graphical elements.

Bash

Bash - Bourne Again SHell, is a command-line shell, just like zsh in MacOS. Users interact with the OS by typing in commands. Here, we will look at two aspects of programming command-line shells.

• How to do shell script programming on Bash
• How to create processes, run programs, do input and output redirection and pipes in C.

Bash Scripting:

Bash scripting is an essential skill for those who work on servers running *nix operating systems like Linux. It can automate many tasks, e.g. write Bash script to automatically compile your code, run unit tests if compilation succeeds and then push code to a Github repository if tests pass, while capturing outputs of each stage to a file for later review.

Bash Basics

Creating a Hello World Script

#!/bin/bash
echo “Hello world!” # Echo is similar to printf in C.

touch hello.sh
vim hello.sh
chmod a+x ./hello.sh
./hello.sh

• Name this file hello.sh
• chmod a+x ./hello.sh sets ‘executable’ flag to ‘all’, thus everyone can execute it.
• #! is a shebang, a character sequence consisting of the characters number sign and exclamation mark (#!) at the beginning of a script. The loader executes specified interpreter program.
• To process escape sequencesm we need to do echo -e “\nHello world.\n”

Variables

#!/bin/bash
x=15
y=20
z=$(($x-$y)) #changed this section
echo "$x - $y = $z"

• The format for arithmetic expansion shall be as follows: $((expression))

Storing values in variables: Try date +%A, it prints out the current day!

day=$(date +%A)
echo “Today is $day.”

• Usage of Date: date [OPTION]… [+FORMAT]
  • Example: date "+%F"

Test statements

Bash can test for certain conditions using the [[.]] operator. Note the spaces after [[ and before ]]. They ARE important! These tests can be use within if..elif..else..fi statements. Some things to note:

• You can read from the keyboard using “read”. The syntax is “read < varname >”, where < varname > is the variable that we want to store the read data to.
• The “if” statement has an odd syntax; you need a semi-colon after the [[..]].

#!/bin/bash
echo "Enter the first number: "
read NUM1
echo "Enter the second number: "
read NUM2
if [[ NUM1 -eq NUM2 ]]; then
    echo "$NUM1 = $NUM2"
elif [[ NUM1 -gt NUM2 ]]; then
    echo "$NUM1 > $NUM2"
else
    echo "$NUM1 < $NUM2"
fi

Loops

Bash supports both for-loops and while-loops. The for-loop is similar to Python’s. For example, to list all the files in the /etc directory, we could do:

for i in /etc/*; do
    echo $i
done

for i in {1..5}; do
    echo $i
done

i=0
while [[ $i -le 5 ]]; do
    echo $i
let i=i+1
done;

cat file.txt | while read line; do
    echo $line
done

You can read text files using the while loop!

Functions

You can declare a function called “func” this way. Parameters are accessed using $1, $2, etc. for the first parameter, second parameter, etc.

• List and Explanation of (the 9) Special Variables in Bash:
  • $0 The filename of the current script.
  • $1…$9 Positional parameters storing the names of the first 9 arguments
  • $$ The process id of the current shell.
  • $# The number of arguments supplied to a script.
  • $* Stores all the command line arguments.
  • $@ Stores the list of arguments as an array.
  • $? Specifies the exit status of the last command or the most recent execution process.
  • $! Shows the id of the last background command.
  • $- Shows the options set used in the current bash shell.

Misc Topics

• Redirecting Output: You can redirect output to the screen (stdout) to a file.
  • ls > ls.out
  • You can append to an output file by using » instead of >.
• Redirecting Input: You can also redirect input from the keyboard to a file.
• Getting the Result Returned by a Program: Using $?
• Pipes: Assuming one program prints to the screen and another reads from the keyboard, you can channel the output of the first program to the input of the second using a mechanism called a “pipe”.
• Running Programs Sequentially and In Parallel: ; for sequential, & for parallel

POSIX Process Calls

How do we create processes, parallelize processes, redirect input and output, and how to set up pipes between processes.

Process Management in C

One can spawn a new process using “fork”:

• Parent and child processes are executing concurrently. We can see this because both parent and child processes are printing their output interleaved, without waiting for each other.

• The parent’s parent, in this case, is the shell process that invoked the program ./lab2p2a. When you run ps | grep "lab2p2a" in the second terminal, you’ll see the parent process ID (PPID) of the lab2p2a process. This PPID corresponds to the shell process that spawned the lab2p2a process.

Accessing Arguments and Environment Variables in C:

• ac stands for “argument count” and contains the number of command-line arguments passed to the program, including the program name itself.

• av stands for “argument vector” and is an array of strings (char *) containing the command-line arguments passed to the program. av[0] typically holds the name of the program, and subsequent elements (av[1] onwards) contain the actual command-line arguments.

• vp stands for “environment vector” and is an array of strings (char *) containing the environment variables passed to the program. Each element of vp represents an environment variable in the form “variable=value”. The last element of vp is typically NULL.

When you execute the program with different arguments, ac will change to reflect the number of arguments passed, av will contain the actual arguments passed, and vp will remain unchanged as it represents environment variables, which are not affected by command-line arguments.

Loading and Executing a program:

To use the exec* family of system calls. We must #include<unistd.h>.

• There are also execle and execve function calls which pass in environment variables, and we will ignore these for now.

• Since all the exec* functions replace the current process image with the process image of the program being run, and is conventionally run within a fork().

Redirecting Input and Output:

we can redirect input and output using “<” and “>” respectively.

• To change the permissions of a file using the chmod command, you need to specify the desired permissions and the filename. Here’s the basic syntax:

chmod permissions filename

Pipes

A “pipe” is a byte-orientated communication mechanism between two processes using two file handles; the first is for reading, the second is for writing.

• when a process is forked, the pipe is duplicated. In Unix-like operating systems, when a process calls fork(), a new child process is created which is an exact copy of the parent process, including its open file descriptors.

• This means that if a pipe is created before the fork, both the parent and the child processes will have a copy of the pipe. The pipe file descriptor will refer to the same underlying pipe object in the operating system, but each process will have its own file descriptor representing the read end and the write end of the pipe.

• Therefore, after forking, both the parent and the child processes can independently read from and write to the pipe using their respective file descriptors, and communication can occur between the parent and the child via the pipe.

man7:

pipe() creates a pipe, a unidirectional data channel that can be
       used for interprocess communication.  The array pipefd is used to
       return two file descriptors referring to the ends of the pipe.
       pipefd[0] refers to the read end of the pipe.  pipefd[1] refers
       to the write end of the pipe.  Data written to the write end of
       the pipe is buffered by the kernel until it is read from the read
       end of the pipe.

Piping between Commands

We can write a program that pipes the output of one program to the input of another, WITHOUT using | and > from the shell.

The essential thing is that you need to run the two programs as two processes, then redirect the output of one program to the writing end of the pipe, and the input of the other program to the reading end of the pipe.