6.3 KiB
Shell glue
When you run something in the terminal, then you are interacting with the so called shell.
The default shell on almost all Linux systems is bash. (We will learn about the fish shell later)
The shell has the power to glue commands together to make the impossible possible! Lets use some gluing magic!
Piping
We will start with pipes. In the last task of the last day, you did see the usage of the symbol | between two separate commands.
By entering cowsay "Hello" | lolcat, the output of the first command cowsay is sent to the second command lolcat.
lolcat takes the input, colors it and outputs it again!
Many Linux commands support handeling input of another command.
You might have seen in the manual of wc in day 1 that the file as an argument is only optional. How could you use wc without arguments?
You might have guessed it now, make some wc pipes.
OK, I admit that the naming is not the best 😂
Lets get some data to work with. To do so, we will use the command curl which graps content from the internet.
Lets count the number of lines of the html file of the homepage of this book:
$ curl -s https://how-to-linux.mo8it.xyz | wc -l
201
The option -s tells curl to be silent and not show progress information.
You can see that wc did count the number of lines. We did just combine to completely different tools with some pipes glue!
How about counting the number times the word "Linux" was mentioned on the homepage?
To do so, we will add a new pipe inbetween!
grep is a command that searches for matches of a specified pattern. Each match is printed in a new line.
To demonstrate grep, here is an usage example:
$ curl --help | grep "silent"
-f, --fail Fail silently (no output at all) on HTTP errors
-s, --silent Silent mode
We did just filter the output of the help of a command. This way, you can also search quickly for command options!
Back to the main example:
$ curl -s https://how-to-linux.mo8it.xyz | grep "Linux" | wc -l
6
You can see that you can use multiple pipes. This allows for almost infinite combinations!
Being able to combine commands is the reason why many commands are simple. They do one thing and do it well! To do more, combine them!
This is much more flexibel and powerful that a program that tries to do a lot of things.
Input, output
Before going any further, we need to understand an important concept in Linux.
A command accepts input and generates two types of output. The input is called standard input. The output is split to standard output and standard error.
The standard output has the number 1 while the standard error has the number 2.
Normal output is sent to the standard output. Errors (and sometimes output that is not very important) are sent to the standard error.
You can redirect the standard output or the standard error to a stream. Normally you will redirect the output to a file. Other stream forms are not relevant for now.
Redirections
Lets see how you can redirect the output of commands to a file.
If you just run curl -s https://how-to-linux.mo8it.xyz, you will the html file printed in the terminal. Lets redirect the output to a html file on your disk:
$ curl -s https://how-to-linux.mo8it.xyz > how-to-linux.html
Now view the content of the new file how-to-linux.html. You will be able to see the same output from the terminal without redirection.
Now try this command:
$ curl https://not-existent-site.mo8it.xyz > test.html
curl: (60) SSL certificate problem: self-signed certificate
More details here: https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
$ cat test.html
You will see that the file is empty since curl did not find a page to show as normal output.
If you are using this command in a script, then it might be wise to redirect the error to a log file:
$ curl https://not-existent-site.mo8it.xyz 2> curl.log
$ cat curl.log
curl: (60) SSL certificate problem: self-signed certificate
More details here: https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
You can see that the error is now not shown after running the curl command. It was redirected to the file curl.log.
Did you notice the number 2 before the redirection symbol 2>?
The last section did mention that the number of the standard error is 2. Therefore, 2 has to be specified to redirect the errors.
If you don't specify a number, then it is equivalent to 1 which stands for the standard output. This means that > is equivalent to 1>.
Bash scripts
Lets write our first Bash script (also for some more demonstration of the two outputs):
#!/usr/bin/bash
echo "What is your favorite operating system after reading this book?"
echo "1. Linux"
echo "2. Windows"
echo "3. Mac"
echo -n "Enter a number: "
read ANSWER
if [ "$ANSWER" == "1" ]
then
echo "Good choice!"
else
echo "Nah, that can't be right! It must be an error!" > /dev/stderr
fi
Copy this code into a file called which-os.sh.
Now run chmod +x which-os.sh. Then run ./which-os.sh.
I am really sorry, but I can not continue this book during the course. It was not only meant for the couse but also like a personal project that I wanted to keep on the internet even after the course.
But I did underestimate the huge amount of time it takes to write such a book/script.
Continuing the book during the course would mean that I can not cover a lot of topics because I can not write enough. It would also harm my health. I need sleep :')
I will continue it after the course, not only for the participants as output, but also for the purpose of a personal project. So please don't be disappointed. You will find all contents on this website some time after the course. I hope that it would then help you as a resource for looking things up.
I do especially apologize to the online participants. I did give my best.
Again, it is not lost. You will get it, but after the course :)