This post is targeted to help individual read UNIX shell scripts. This will cover UNIX redirection operator used in shell scripts. one of the post you would like to read first before going through crazy symbols shown below. structure of unix script.
Target audience : Anyone who is going through UNIX SHELL SCRIPT first time. it will help them to read script. There is so much information available on internet on unix redirection operator. My efforts here is to make it simple so it can create explicit value. This is targeted for data warehosue, ETL, data management professionals.
Unix Redirection Operator – what is ?
There are standard input, output and error files that we always come across while working in a unix shell. You can imagine if you are invoking a unix script / program an input, an output and an error file are created and they are directly displayed on your screen to enable the user to read through. Unix Redirection operators pretty well help you in routing these standard displaying input, output and error to a desired file in a desired location for a desired purpose.
You can identify these standard input, output and error files with unique identifiers as below –
|0||Standard Input read from command prompt by key strokes.|
|1||Standard output displays on screen.|
|2||Standard Error displays on screen.|
Different Redirection Operators or Operations (How Many)?
There are different Unix redirection operators and their corresponding operations described here. Take a look –
- Input Redirection. To read input from a file to a command or a program. (<)
- Output Redirection. To write output of the command / job executed to a file. (> and >>)
- Error Redirection. To write error of the execution to a different file. (> and >>)
- Pipe Operator. Redirects ” The result ” of a UNIX program to another. (vertical bar or Pipe – |), Just to relate – Pipe is an instrument to connect to things ( literal meaning ).
Let’s look into those various situations where we can use redirection operators to make our job effective.
Different functions you can perform using Unix Redirection Operator:
Since now you are aware of UNIX redirection operator functions, it’s time to look at some real life uses and application areas of using Unix redirection operators at work.
NOTE: Below Commands are tested in K shell. Use notepad to copy the command before executing at UNIX prompt.
To create file. You can create a new file.
Date > job_process_date.txt
Ps -ef > process_running.txt
To append to a file. You can merge files
Cat file1 file2 > merge_file.txt
Cat file1 >> file2.txt\
Creating a job wise parameter file.
Imagine you have a huge parameter file in your project and you want maintain audit of the changes to your parameter file. You can design a small unix program with grep, cut and redirection operations. Happy audit.
Creating a subset of the file.
Head 100 > txt
Tail -100 > last100.txt
Creating a list file with all file names to be processed in a loop
Ls -lrt "project inbound folder" > list_files_to_process.txt
Reading a variable in unix program
grep -i "^sys" log_file.log | cut -d ":" -f 2 | read VARIABLE
Read sales invoice number of null valued lines in your CSV_FILE
Grep ",," CSV_FILE.csv || cut -d "," -f 1
Hope this was of some help , this post will be updated soon with few more Unix commands, Your comments are welcome 🙂