Wrapping your head around the most used bash operatorsposted on 2020-07-26T15:06:55Z · view page on GitHub
This is mostly geared towards helping bash-beginners. Having a sound understanding of what each bash operator does might help you becoming a bash-wizard ;)
Let's start with the most simple of them all: the semicolon
; is used to execute multiple commands in sequence. It is in fact equivalent as putting a command on a new line:
!echo "This will be printed"; echo "This will also be printed"
This will be printed This will also be printed
Closely Related is the and-operator
&&. This operator will execute the second command if the first command succeeds. If something goes wrong with the first command, the second command will not be executed.
!echo "This will be printed" && echo "This will also be printed"
This will be printed This will also be printed
In the following, the second command will not be execuded however:
!echo"This message will not be printed" && echo "This will also not be printed"
/bin/sh: 1: echoThis message will not be printed: not found
In the above command, there is an error (there's no space between the message and the echo command). Although the error message gets printed, the intended message is not printed. Because the and-operator is used, the second message is not printed.
In the same ballpark as before is the or-operator
||. This operator works opposite of the and-operator: If something goes wrong int he first command, the second command will be executed:
!echo"This message will not be printed" || echo "This message will be printed!"
/bin/sh: 1: echoThis message will not be printed: not found This message will be printed!
However, if the first command succeeds, the second command will not be executed. In this sense the bash or-operator is actually more close to an xor-operator.
!echo "This message will be printed" || echo "This message will not be printed."
This message will be printed
! ! echo "This message will be printed" || echo "This message will also be printed."
This message will be printed This message will also be printed.
! ! echo "This message will be printed" && echo "This message will not be printed."
This message will be printed
The background operator
& is a totally different way to chain commands. Everything preceding this operator will be executed in the background. Any command following will be immediately executing without waiting for the output of the first commmand. Not the difference here with the semicolon, wher commands were executed in sequence, whereas here, they are executed in parallel:
!echo "This will be printed" & echo "This will also be printed"
This will also be printed This will be printed
Because of the parallel nature of this command, the order of execution is no longer guarenteed. This operator is also very commonly used to start a process without blocking the terminal: the following command will for example NOT block the terminal for 10 seconds:
!sleep 10 &
!echo "Hello" > test.txt
The above created a file named
test.txt with the content
The above operator, however, will only redirect output intended for
stdout: the 'normal' output. Conversely, one can redirect error messages (going to
stderr) with the
2> operator. The following command
!echo"Hello" 2> test.txt
will for example have the following content:
/bin/sh: 1: echoHello: not found, i.e. the complete error message resulting from using the echo command without a space between the command and the message.
If both the error messages and the normal messages need to be redirected at the same time, one uses the
!echo"Hello" &> test.txt
/bin/sh: 1: echoHello: not found
This is effectively equivalent with redirecting with
!echo"Hello" > test.txt 2> test.txt
An often used redirection destination is
null, which will just throw away any output sent to it:
!echo "hello" > /dev/null
Note that the previous command dit not produce any output.
Finally, it's worth mentioning that to send a message to stderr, the
>&2 redirection is often prepended to the command:
!>&2 echo "hello"
This marks the message with
2, the error mark. We can check that this is indeed an error message (in stead of a normal message) by redirecting all normal messages to
!(>&2 echo "hello") > /dev/null
The message is still visible, hence it is an error message.
Arguably one of the most important operators for bash scripting: the pipe. Consider it the glue that holds bash scripting together. The pipe
| operator transfers the output of the first command to the input of the second command.
A very simple example is for example the following, where one is interested in the contents of the folder
/usr/bin. One could use the command
ls /usr/bin to print out all the files in this folder, however, because there are quite a few, one could pipe this output to
less, which is a program that listens to piped input and allows to scroll through this input.
!ls /usr/bin | less
2to3 2to3-2.7 2to3-3.7 a2ping a2x a2x.py a52dec a5toa4 abw2html abw2raw abw2text acat acceleration_speed accept accessdb aclocal aclocal-1.16 aconnect acpi acpid acpi_listen addftinfo :
The above example might be a bit contrived however, a more practical approach might be to search for a specific occurance of a file in
/usr/bin. This is often done by the
grep command, which for the intents of this tutorial can be considered as a command to search for occurances of a string in the stream coming in through the pipe:
!ls /usr/bin | grep bash
Two executables were found here that contain the letter sequence 'bash'. Note that multiple pipes can also be cascaded:
!ls /usr/bin | grep bash | grep bug
such a chained command of pipes is often called a pipeline.
The double dash operator
-- is used to signify the end of a optional parameter list. Consider for example the difference in the following to commands:
!echo "hey! are you all right?\n-or is there a problem?" | grep -o
Usage: grep [OPTION]... PATTERNS [FILE]... Try 'grep --help' for more information.
!echo "hey! are you all right?\n-or is there a problem?" | grep -- -o
-or is there a problem?
In the first command, the
-o is considered an option to
grep, resulting in an error because no search query is given. In the second command
-o is considered the search query, as the
-- operator signifies the end of the optional flags.
Round brackets are used to group multiple commands together. Consider the differences between the following commands:
!echo "hello" || echo "hey" && echo "bye"
!(echo "hello" || echo "hey") && echo "bye"
!echo "hello" || (echo "hey" && echo "bye")
![ 1 = 1 ] && echo "hey"
![ 1 = 2 ] && echo "hey"
Note the spaces between the square brackets and inside the conditional. These are necessary because
[ is actually a bash executable (and thus not a real bracket) everything following the
[ executable thus acts as an argument to
[. Consequently they need to be spaced properly apart with spaces. The program
[ expects the last argument to always be equal to
], giving the impression that that the
[ ... ] sequence look like braces. However, we could just as well have called
[ whatever we want:
!alias conditional="[" !conditional 1 = 1 ] && echo "hello"
!conditional 1 = 2 ] && echo "hello"
[ is on its own an alternative for
test, with the added requirement that it's arguments need to end with
]. Indeed, the following statements are equivalent to the previous ones:
!test 1 = 1 && echo "hey"
!test 1 = 2 && echo "hey"
Conditionals can be combined with the
|| operators defined above.
![ 1 = 1 ] && [ 2 = 1 ] && echo "hello"
![ 1 = 1 ] || [ 2 = 1 ] && echo "hello"
Note that these operators are placed outside of the brackets, as we'd expect since we know now that
[ is just a program that sends an exit code corresponding to the conditional statement. Note that this means you get an exit code of 0 for true; 1 for false. This is opposite of what you'd expect from more traditional programming languages. The reason for this is that programs can have many ways to fail (often each resulting in a different exit code), but can only have one preprogrammed way to succeed, hence the single exit code 0 for success, anything else for failure.
In contrast to the double square brackets, a newer alternative for the single square brackets was introduced: the double square brackets
]]. These newer variants are intended for the same purpose as the single square brackets, but they are introduced as keywords, adding several parsing benefits arising from the fact that the conditional following
[[ will not be considered arguments of a bash function. This has the advantage that one does not have to be afraid of spaces inside string variables and such. The downside of using these keywords is that they are not posix-compliant: they do not work on all shell versions, only on 'newer' variants like
![[ 1 = 1 ]] && echo "hello again!"
![[ 1 = 2 ]] && echo "hello again!"
Here is an example highlighting the differences between
!string="hello again!" ![[ $string = "hello again!" ]] && echo $string
Note that this won't work with single brackets, as the space inside the string will make the
$string variable be parsed as two arguments, resulting in a "too many arguments" error:
![ $string = "hello again!" ] && echo $string
bash: [: too many arguments
To make the above work with single brackets, one might have to wrap the string variable in double quotes:
![ "$string" = "hello again!" ] && echo $string
Other benefits are that operators don't need to be escaped:
![[ 2 < 3 ]] && echo "2 is smaller than 3"
2 is smaller than 3
Whereas the same statement with single brackets will fail (can you figure out why?):
![ 2 < 3 ] && echo "2 is smaller than 3"
bash: 3: No such file or directory
The following (escaped) conditional works, however
![ 2 \< 3 ] && echo "2 is smaller than 3"
2 is smaller than 3
Combining conditionals can now be done in two ways: outside the brackets, much like with single brackets and inside the brackets, an option that becomes available due to the more liberal parsing requirements:
![[ 1 = 1 ]] || [[ 2 = 1 ]] && echo "hello"
![[ 1 = 1 || 2 = 1 ]] && echo "hello"
In general, double brackets are just nicer to work with. However, if posix-complient bash scripting is important to you, single brackets are often sufficient. Just be aware of the pitfalls highlighted above.
The (double) square bracket syntax can be used inside control flow statements. For example:
if [ 1 = 1 ]; then echo "hello" else echo "hey" fi
Note that this is a very simplified example, so simple even that it is in fact equivalent to one of those oneliners we have been using so far:
![ 1 = 1 ] && echo "hello" || echo "hoi"
However, these control flow statements obviously allow for much more complex cases.
That's it for now.
If you like this post, consider leaving a comment or star it on GitHub.