In the landscape of programming languages dedicated to text-based data processing, AWK holds a prominent position due to its power and versatility. Originally designed for data manipulation and report generation, AWK has proven over time that it can extend its functionalities well beyond its original purpose, especially when combined with other Linux commands.
In this article, we aim to explore in detail the features and functionalities of AWK. We will start with an overview of the installation process of AWK, which is pre-installed on many Unix-based operating systems, such as Linux and MacOS. Then, we will delve into the basic syntax of AWK, illustrating the fundamental structure of AWK commands and the rules governing the use of patterns and actions.
We will continue with a series of practical examples of using AWK, demonstrating how this language can be used to manipulate data and generate reports. These examples will cover a wide range of scenarios, from pattern-based text filtering to tabular data processing.
Furthermore, we will explore the use of AWK in writing more complex scripts, highlighting how the power and flexibility of AWK can be leveraged to automate and simplify tasks of significant data processing complexity.
Finally, we will discuss how AWK can be combined with other Linux commands to perform advanced tasks, such as process termination based on name or data stream processing in pipelines. This section will also include a series of additional examples illustrating the wide range of AWK applications, from splitting a file into multiple files to converting uppercase characters to lowercase.
The goal of this article is to provide an overview of AWK, illustrating its capabilities through practical examples and providing the foundation for an effective and informed use of this powerful tool. Whether you are IT professionals looking for new skills or researchers in need of an effective means for data processing, we hope this article can serve as an introduction to your learning journey with AWK.
AWK Installation
AWK is pre-installed on many Unix-based operating systems, such as Linux and MacOS. You can check if AWK is installed on your system by typing `awk –version` in the terminal.
Basic Syntax of AWK
The basic syntax of an AWK command is as follows:
awk '/pattern/ {azione}' file
– `pattern` is a regular expression. AWK will perform the action on every line of the file that matches the pattern.
– `action` is an action that AWK will perform on every line that matches the pattern. If no action is specified, the default action is to print the line.
– `file` is the input file on which AWK will execute the command.
Example of Using AWK
Suppose we have a file called `students.txt` with the following content:
John Doe 23 Mathematics Jane Doe 22 Physics
If we wanted to print only the lines that contain “Doe”, we could use the following AWK command:
awk '/Doe/ {print $0}' students.txt
The result would be:
John Doe 23 Mathematics Jane Doe 22 Physics
If we wanted to print only the name and age of the students, we could use, for example:
awk '{print $1, $2}' students.txt
This command would print:
John 23 Jane 22
AWK Scripts
Beyond its use from the command line, AWK can also be used to write more complex scripts. An example of an AWK script is the following:
BEGIN {
print "Start of the script"
}
/Doe/ {
print $1, $2
}
END {
print "End of the script"
}
This script would print the name and age of each student with the surname “Doe,” in addition to printing a message at the beginning and end of the script.
It’s also possible to use awk in a pipeline with other shell commands. For example, if you wanted to kill all processes with the same name, such as “myprocess,” you could use a command that combines `ps`, `awk`, and `kill`:
ps aux | grep myprocess | awk '{print $2}' | xargs kill -9
In this command, `ps aux` lists all the processes running on your system, while `grep myprocess` filters the output to only include lines that contain “myprocess”. `awk ‘{print $2}’` extracts the second field of each line, which is the process ID (PID). Finally, `xargs kill -9` passes these PIDs to the `kill` command to terminate the processes.
The `-9` option forces the process to terminate and can cause unsaved data loss.
Example 2: How to terminate all processes of a specific user
Suppose now you want to terminate all processes of the user “myuser”. You can do something very similar to the previous example:
ps aux | grep myuser | awk '{print $2}' | xargs kill -9
Beyond managing processes, Awk can be used in a number of other ways. Here are some additional examples:
1. Counting lines in a file
awk 'END { print NR }' filename
This command counts the number of lines in the specified file and prints the result.
2. Printing lines from a file that match a pattern
awk '/pattern/ {print}' filename
This command prints only the lines that contain the specified pattern.
3. Splitting a file into multiple files
awk '{print > ("file" NR ".txt")}' filename
This command splits the input file into multiple files, with one line per file.
4. Sum of values in a column
awk '{sum+=$1} END {print sum}' filename
This command calculates the sum of the values in the first column of a file.
5. Converting uppercase characters to lowercase
echo "HELLO WORLD" | awk '{print tolower($0)}'
This command converts all uppercase letters to lowercase.
6. Printing the nth field of a line
echo "field1 field2 field3 field4" | awk '{print $3}'
This command prints the third field of a line.
In any case, it’s possible to delve deeper into all the capabilities of awk by using the reference manual: https://man7.org/linux/man-pages/man1/awk.1p.html
I am passionate about technology and the many nuances of the IT world. Since my early university years, I have participated in significant Internet-related projects. Over the years, I have been involved in the startup, development, and management of several companies. In the early stages of my career, I worked as a consultant in the Italian IT sector, actively participating in national and international projects for companies such as Ericsson, Telecom, Tin.it, Accenture, Tiscali, and CNR. Since 2010, I have been involved in startups through one of my companies, Techintouch S.r.l. Thanks to the collaboration with Digital Magics SpA, of which I am a partner in Campania, I support and accelerate local businesses.
Currently, I hold the positions of:
CTO at MareGroup
CTO at Innoida
Co-CEO at Techintouch s.r.l.
Board member at StepFund GP SA
A manager and entrepreneur since 2000, I have been:
CEO and founder of Eclettica S.r.l., a company specializing in software development and System Integration
Partner for Campania at Digital Magics S.p.A.
CTO and co-founder of Nexsoft S.p.A, a company specializing in IT service consulting and System Integration solution development
CTO of ITsys S.r.l., a company specializing in IT system management, where I actively participated in the startup phase.
I have always been a dreamer, curious about new things, and in search of “new worlds to explore.”
