awk - a powerful data extraction and manipulation tool
Archive - Originally posted on "The Horse's Mouth" - 2007-09-25 11:53:31 - Graham Ellis
Unix and Linux come with a whole toolkit of utility programs for filtering, sorting and manipulating data. As a "one off" job, you can use them from the command line, and for repeated jobs you can save a series of instructions for these tools in a shell script. Yesterday, I ran a private shell scripting bootcamp course, and found myself enjoying working with
grep - the filtering tool
sort - the tool that re-orders data
awk - for pattern matching and data manipulation
sed - the stream editor
tr - for transforming characters
Awk is especially worthy of mention. With Awk, you can select rows of data based on what they contain, or what is in certain fields, and manipulate those lines. Instructions may be given to allow different instructions depending on how they're elected, and awk also includes integer and floating point arithmetic capabilities, and the ability to undertake actions before and after the processing of any of the data, in matching clauses called BEGIN and END.
I have added sample awk scripts here and here to our web site, and a slightly longer article on awk here. And, yes, I can deliver training on awk and shell scripting ;-) ... Linux/Unix Basics will get you started, and we can tailor to meet your needs from there!