mo8it/How_To_Linux
1
0
Fork 0
mirror of https://codeberg.org/Mo8it/How_To_Linux.git synced 2024-11-22 01:48:03 +00:00
Code Releases Activity

Gotta sleep :/

Browse source
This commit is contained in:
Mo 2023-08-17 05:15:27 +02:00
parent 9a29ee859e
commit ba80482b59
4 changed files with 234 additions and 180 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
src/day_4/clis_of_the_day.md
View file

@ -43,6 +43,7 @@ $ sed 's/values/strings/g' demo.txt
here,are,some here,are,some
comma,separated,strings comma,separated,strings
de mo,file,t x t de mo,file,t x t
$ sed '/separated/d' demo.txt $ sed '/separated/d' demo.txt
here,are,some here,are,some
de mo,file,t x t de mo,file,t x t
@ -54,11 +55,14 @@ When you specify `FILE`, you can use the option `-i` to operate _inplace_. This
## find ## find
Find everything ending with `.sh` of type file (`f`) using globbing:
```bash ```bash
# Find everything ending with `.sh` of type file `f` using globbing
find . -type f -name '*.sh' find . -type f -name '*.sh'
```
Using regex:
# Using regex ```bash
find . -type f -regex '.+\.sh' find . -type f -regex '.+\.sh'
``` ```

51
src/day_4/python_scripting.md
View file

@ -1,54 +1,63 @@
# Python scripting # Python scripting
Bash scripts are fine if your script is not complicated.
But Bash is ugly and if you try for example to use arrays/lists in Bash, you probably should just use Python for what are trying to accomplish with your Bash script.
You can use what you have learned about running programs and feed Python with commands that it can run:
```python-repl ```python-repl
>>> import subprocess >>> import subprocess
>>> proc = subprocess.run(["ls", "student"]) >>> proc = subprocess.run(["ls", "student"])
(...) (…)
CompletedProcess(args=['ls', '/home/student'], returncode=0) CompletedProcess(args=['ls', '/home/student'], returncode=0)
>>> proc.returncode >>> proc.returncode
0 0
>>> proc = subprocess.run("ls /home/student", shell=True) >>> proc = subprocess.run("ls /home/student", shell=True)
(...) (…)
CompletedProcess(args='ls /home/student', returncode=0) CompletedProcess(args='ls /home/student', returncode=0)
>>> proc = subprocess.run("ls /home/student", shell=True, capture_output=True, text=True) >>> proc = subprocess.run("ls /home/student", shell=True, capture_output=True, text=True)
CompletedProcess(args='ls /home/student', returncode=0, stdout='(...)', stderr='') CompletedProcess(args='ls /home/student', returncode=0, stdout='(…)', stderr='')
>>> proc.stdout >>> proc.stdout
(...) (…)
>>> proc.stderr >>> proc.stderr
>>> proc = subprocess.run("ls /home/nonexistent", shell=True, capture_output=True, text=True) >>> proc = subprocess.run("ls /home/nonexistent", shell=True, capture_output=True, text=True)
CompletedProcess(args='ls /home/nonexistent', returncode=2, stdout='', stderr="ls: cannot access '/home/nonexistent': No such file or directory\n") CompletedProcess(args='ls /home/nonexistent', returncode=2, stdout='', stderr="ls: cannot access '/home/nonexistent': No such file or directory\n")
>>> proc = subprocess.run("ls /home/nonexistent", shell=True, capture_output=True, text=True, check=True) >>> proc = subprocess.run("ls /home/nonexistent", shell=True, capture_output=True, text=True, check=True)
--------------------------------------------------------------------------- ---------------------------------------------------------------------------
CalledProcessError Traceback (most recent call last) CalledProcessError Traceback (most recent call last)
Input In [8], in <cell line: 1>() (…)
----> 1 subprocess.run("ls /home/nonexistent", shell=True, capture_output=True, text=True, check=True)
File /usr/lib64/python3.10/subprocess.py:524, in run(input, capture_output, timeout, check, *popenargs, **kwargs)
522 retcode = process.poll()
523 if check and retcode:
--> 524 raise CalledProcessError(retcode, process.args,
525 output=stdout, stderr=stderr)
526 return CompletedProcess(process.args, retcode, stdout, stderr)
CalledProcessError: Command 'ls /home/nonexistent' returned non-zero exit status 2.
``` ```
Most used modules when writing system scripts with Python: `pathlib`, `os`, `shutil`. The most used modules when writing system scripts with Python are `pathlib`, `os`, `shutil`.
```python-repl ```python-repl
>>> from pathlib import Path >>> from pathlib import Path
>>> p = Path.home() / "day_5" >>> p = Path.home() / "day_5"
PosixPath('/home/student/day_5') PosixPath('/home/student/day_5')
>>> p.is_dir() >>> p.is_dir()
(...) (…)
>>> p.is_file() >>> p.is_file()
(...) (…)
>>> p.exists() >>> p.exists()
(...) (…)
>>> p.chmod(0o700) >>> p.chmod(0o700)
(...) (…)
>>> rel = Path(".") >>> rel = Path(".")
PosixPath('.') PosixPath('.')
>>> rel.resolve() >>> rel.resolve()
PosixPath('/home/student/(...)') PosixPath('/home/student/(…)')
``` ```

161
src/day_4/shell_scripting.md
View file

@ -1,10 +1,18 @@
# Shell scripting # Shell scripting
Task automation requires multiple instructions that have to run on demand. To combine multiple instructions, we need to write a shell script. Task automation requires multiple instructions that have to run on demand.
To combine multiple commands together for automation, we need to write a shell script.
Since `bash` is the default shell on most Linux distributions, we learn bash scripting. Since Bash is the default shell on most Linux distributions, we will learn Bash scripting.
## First bash script Although I recommend using the Fish shell when using the terminal and although Fish supports scripting too, I would not recommend using it for scripting since you would not be able to share and run these scripts anywhere.
You need to have Fish installed which is not always the case.
I use Fish for my personal scripts.
But if I write a script that will be shared with others, then I write it in Bash.
Learn to write Bash scripts first before considering using Fish scripts, even for your personal scripts.
## First Bash script
Let's write our first Bash script: Let's write our first Bash script:
@ -33,32 +41,36 @@ fi
Copy this code into a file called `which-os.sh`. Copy this code into a file called `which-os.sh`.
Now run `chmod u+x which-os.sh`. Then run `./which-os.sh`. Don't worry, everything will be explained afterwards. Now, run `chmod u+x which-os.sh`.
Then run `./which-os.sh`.
Don't worry, everything will be explained afterwards.
After running the script, you will see a prompt asking you to enter a number corresponding to an operating system. If you choose Linux, you get the output "Good choice". This output is in the standard output. After running the script, you will see a prompt asking you to enter a number corresponding to an operating system.
If you choose Linux, you get the output "Good choice".
This output is in the standard output.
If you don't choose Linux, you get the output "Nah, that can't be right! (...)". This output is redirected to the standard error. If you don't choose Linux, you get the output "Nah, that can't be right! (...)".
This output is redirected to the standard error.
We did learn that we can redirect to a file using `>`. The syntax `1>&2` redirects the standard output to the standard error. On the other hand. `2>&1` redirects the standard error to the standard output. The building blocks of the script above will be explained in the next sections.
If you want to redirect both the standard output and error to a file, you can use this syntax: `COMMAND >FILE 2>&1`.
The redirection **order** is important in this case! `COMMAND 2>&1 >FILE` will redirect the standard output **only** to the file `FILE`!
`COMMAND >FILE 2>&1` has a useful shortcut: `COMMAND &>FILE`. You can also use `&>>` to append.
All other aspects of the script above will be explained in the next sections.
## Shebang ## Shebang
The first line of our first script starts with `#!` which is called the _shebang_. The shebang is followed by the program that runs the script. Since the script is a bash script, we use the program `bash` to run it. But writing `bash` after the shebang is not enough. We have to specify the path to the program. We can find out the path of the program by using the command `which`: The first line of our first script starts with `#!` which is called the _shebang_.
The shebang is followed by the program that runs the script.
Since the script is a Bash script, we use the program `bash` to run it.
But writing `bash` after the shebang is not enough.
We have to specify the full path to the program.
We can find out the path of a program by using the command `which`:
```console ```console
$ which bash $ which bash
/usr/bin/bash /usr/bin/bash
``` ```
You can also write a Python script and add a shebang at its beginning. We can find out the path to the Python program: You can also write a Python script and add a shebang at its beginning.
We can find out the path to the Python interpreter by running the following:
```console ```console
$ which python3 $ which python3
@ -77,31 +89,37 @@ Let's save this tiny Python script as `hello_world.py`, make it executable with
```console ```console
$ chmod u+x hello_world.py $ chmod u+x hello_world.py
$ ./hello_world.py $ ./hello_world.py
Hello world! Hello world!
``` ```
We could have written the Python script without the shebang, but then we would have to run with `python3 hello_world.py`. Adding the shebang lets you see a script as a program and ignore what language it is written in when running it. We could have written the Python script without the shebang, but then we would have to run with `python3 hello_world.py`.
Adding the shebang lets you see a script as a program and ignore what language it is written in when running it.
You can use the shebang with any program that runs a script (interpreter). You can use the shebang with any program that can run a script.
## Variables ## Variables
In our first bash script, we have the line `RIGHT_ANSWER=1`. This line defines a variable with the name `RIGHT_ANSWER` and the value `1`. In our first Bash script, we have the line `RIGHT_ANSWER=1`.
This line defines a variable with the name `RIGHT_ANSWER` and the value `1`.
To define a variable in bash, you have to write the name of the variable _directly_ followed by an equal sign `=`. The equal sign has to be _directly_ followed by the value of the variable. To define a variable in Bash, you have to write the name of the variable **directly** followed by an equal sign `=`.
The equal sign has to be **directly** followed by the value of the variable.
_directly followed_ means that spaces between the variable name, the equal sign `=` and the value are not allowed! _directly followed by_ means that spaces between the variable name, the equal sign `=` and the value are not allowed!
But what if you want to set a variable equal to a string that contains spaces? But what if you want to set a variable equal to a string that contains spaces?
In this case, you have to use quotation marks `"`. For example: In that case, you have to use quotation marks `"`.
For example:
```bash ```bash
HELLO="Hello world!" HELLO="Hello world!"
``` ```
To use a defined variable, we use a dollar sign `$` before the variable name. For example: To read the value of a defined variable, we use a dollar sign `$` before the variable name.
For example:
```bash ```bash
echo $HELLO echo $HELLO
@ -118,9 +136,10 @@ echo $MESSAGE
The two lines above would lead to the output `Tux says: Hello world!`. The two lines above would lead to the output `Tux says: Hello world!`.
## Capturing command's output ## Capturing command output
You can capture the standard output of a command using the the syntax `$(COMMAND)`. Example: You can capture the standard output of a command using the the syntax `$(COMMAND)`.
Example:
```bash ```bash
BASH_VERSION=$(bash --version) BASH_VERSION=$(bash --version)
@ -132,8 +151,8 @@ Let's run the command in the terminal first to see its output:
```console ```console
$ bash --version $ bash --version
GNU bash, version 5.1.16(1)-release (x86_64-redhat-linux-gnu) GNU bash, version 5.2.15(1)-release (x86_64-redhat-linux-gnu)
Copyright (C) 2020 Free Software Foundation, Inc. Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it. This is free software; you are free to change and redistribute it.
@ -144,10 +163,11 @@ Now, let's output the variable that we did define above:
```console ```console
$ echo $BASH_VERSION $ echo $BASH_VERSION
GNU bash, version 5.1.16(1)-release (x86_64-redhat-linux-gnu) Copyright (C) 2020 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software; you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. GNU bash, version 5.2.15(1)-release (x86_64-redhat-linux-gnu) Copyright (C) 2022 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software; you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
``` ```
You can see that the lines are squashed into one line! If you want to output the lines without them being squashed, you have to use quotation marks `"`: You can see that the lines are squashed into one line!
If you want to output the lines without them being squashed, you have to use quotation marks `"`:
```console ```console
$ echo "$BASH_VERSION" $ echo "$BASH_VERSION"
@ -159,9 +179,9 @@ This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. There is NO WARRANTY, to the extent permitted by law.
``` ```
This is the output that we did expect 😃 This is the output that we expect 😃
## Temporary variables ## Environment variables
Let's write the following tiny script `hello.sh`: Let's write the following tiny script `hello.sh`:
@ -171,15 +191,20 @@ Let's write the following tiny script `hello.sh`:
echo "Hello $USER!" echo "Hello $USER!"
``` ```
Now we run the script. The output on my machine is: Now, we run the script.
The output on my machine is:
```console ```console
$ chmod u+x hello.sh $ chmod u+x hello.sh
$ ./hello.sh $ ./hello.sh
Hello mo! Hello mo8it!
``` ```
We see `Hello mo!` as output. This is because my user name on my machine is `mo`. `USER` is a so called environment variable that is defined for all programs. If you run the script on your machine, you will get your user name instead of `mo`. There are more environment variables like `PATH` which we will learn about later. We see `Hello mo8it!` as output.
This is because my user name on my machine is `mo8it`.
`USER` is a so called **environment variable** that is defined for all programs.
If you run the script on your machine, you will get your username instead of `mo8it`.
Now, let's run the following: Now, let's run the following:
@ -188,27 +213,56 @@ $ USER=Tux ./hello.sh
Hello Tux! Hello Tux!
``` ```
We did define a temporary variable `USER` with the value `Tux`. This temporary variable overwrites the environment variable `USER` in our script. Therefore we get the output `Hello Tux!` and not our user name. We defined an environment variable `USER` with the value `Tux` just before running the script.
This variable overwrites the global value of the variable `USER` in our script.
Therefore we get the output `Hello Tux!` and not our user name.
You can use temporary variables not only to overwrite environment variables. These are basically variables that can be used by the program that you specify after their definition. You can use environment variables not only to overwrite existing ones variables.
These are basically variables that can be used by the program that you specify after their definition.
## Comments ## Comments
In our first bash script, you can find three lines starting with a hashtag `#`. Two lines are comments, the shebang `#!` is an exception as a special comment at the beginning of a file that is not ignored while running the script. All other lines that start with `#` are comments that are ignored by the computer. Comments are only for humans to explain things. In our first Bash script, you can find three lines starting with a hashtag `#`.
Two lines are comments, the shebang `#!` is an exception as a special comment at the beginning of a file that is not ignored while running the script.
All other lines that start with `#` are comments that are ignored by the computer.
Comments are only for humans to explain things.
Especially in long scripts, you should write comments to explain what the script and its "non trivial sections" do. Especially in long scripts, you should write comments to explain what the script itself and its "non trivial sections" do.
## User input ## User input
In a script, you can ask for user input. To do so, you can use the command `read`. In a script, you can ask for user input.
To do so, you can use the command `read`.
In our first bash script, we use `read` to take the answer of the user. The input is then saved in the variable `ANSWER`. You can choose a different name for this variable. After the line with `read`, you can use the variable storing the input as a normal variable. In our first Bash script, we use `read` to ask the user for his answer.
The input is then saved in the variable `ANSWER`.
You can choose a different name for this variable.
After the line with `read`, you can use the variable storing the input as a normal variable.
## Arguments
To read the `n`-th argument that is provided to a script, we can use `$n`.
Take a look at the following example script called `arg.sh`:
```bash
#!/usr/bin/bash
echo "The first argument is: $1"
```
When you run this script with an argument, you get the following output:
```console
$ ./arg.sh "Hello"
The first argument is: Hello
```
## Conditions ## Conditions
### `if` block ### `if` block
Our first bash script checks if the user input which is stored in the variable `ANSWER` equals the variable `RIGHT_ANSWER` which stores the value `1`. Our first Bash script checks if the user input which is stored in the variable `ANSWER` equals the variable `RIGHT_ANSWER` which stores the value `1`.
To check for a condition in Bash, we use the following syntax: To check for a condition in Bash, we use the following syntax:
@ -221,11 +275,14 @@ fi
Here, `(...)` stands for the commands that we want to run if the condition is true. Here, `(...)` stands for the commands that we want to run if the condition is true.
In our first bash script, we check for equality of two variables with a double equal sign `==`. In our first Bash script, we check for equality of two variables with a double equal sign `==`.
`fi` is not a typo! It is just `if` reversed to indicate the end of the `if` block. Although the syntax is not the best, you have to sadly accept it. Bash does not have the best syntax... `fi` is not a typo! It is just `if` reversed to indicate the end of the `if` block.
Although the syntax is not the best, you have to sadly accept it.
Bash does not have the best syntax...
Speaking about syntax: You have to take spaces seriously with conditions. Speaking about syntax:
You have to take spaces seriously with conditions.
For example, if we define the variable `VAR=1`, the following snippets **do not work** (or have an unexpected behavior): For example, if we define the variable `VAR=1`, the following snippets **do not work** (or have an unexpected behavior):
@ -236,7 +293,6 @@ For example, if we define the variable `VAR=1`, the following snippets **do not
echo "VAR has the value 1" echo "VAR has the value 1"
fi fi
``` ```
1. No space before `]` 1. No space before `]`
```bash ```bash
if [ $VAR == 1] if [ $VAR == 1]
@ -266,7 +322,7 @@ For example, if we define the variable `VAR=1`, the following snippets **do not
fi fi
``` ```
But the following snippet **work**: But the following snippet **works**:
- Space after `[`, before `]`, before `==` and after `==` - Space after `[`, before `]`, before `==` and after `==`
```bash ```bash
@ -278,7 +334,8 @@ But the following snippet **work**:
### `else` block ### `else` block
The `else` block runs commands inside it only if the condition is not true. The syntax is: The `else` block runs commands inside it only if the `if` condition is false.
The syntax is:
```bash ```bash
if [ CONDITION ] if [ CONDITION ]
@ -302,24 +359,18 @@ else
fi fi
``` ```
<!-- TODO: else -->
<!-- TODO: else if --> <!-- TODO: else if -->
<!-- TODO: test --> <!-- TODO: test -->
<!-- TODO: if [ ! -f ] --> <!-- TODO: if [ ! -f ] -->
<!-- TODO: case -->
<!-- TODO: for --> <!-- TODO: for -->
<!-- TODO: while --> <!-- TODO: while -->
<!-- TODO: command | while read --> <!-- TODO: command | while read -->
<!-- TODO: math (( )) -->
<!-- TODO: Long command on multiple lines \ --> <!-- TODO: Long command on multiple lines \ -->
<!-- Why ./SCRIPT_NAME --> <!-- Why ./SCRIPT_NAME -->
@ -333,5 +384,3 @@ fi
<!-- TODO: chmod codes --> <!-- TODO: chmod codes -->
<!-- TODO: https://chmodcommand.com --> <!-- TODO: https://chmodcommand.com -->
Rest coming soon...

194
src/day_4/tasks.md
View file

@ -1,150 +1,142 @@
# Tasks # Tasks
## Task: Vim macros Don't forget to add a shebang to your scripts and make them executable!
In this task, we will learn about the power of macros in Vim. ## Task: Essential tools
1. Visit [this URL](https://www.randomlists.com/random-names?qty=500), select the generated 500 names with the mouse and copy them. Write a Bash script that installs the following _essential_ Linux packages for you:
1. Create a new text file and open it with `vim`.
1. Paste the names into the file. You should see 500 lines with a name in each line.
1. Record a macro that changes a line in the form `FIRST_NAME LAST_NAME` to `("FIRST_NAME", "LAST_NAME"),`.
1. Run the macro on all lines.
1. Now, go to the beginning of the file (`gg`) and add this line as a first line: `names = [`
1. Go to the end of the file (`G`) and add this line as last line: `]`
Congratulations, you did just convert the names into a form that could be directly used by a Python program! It is a list of tuples now. - `cmatrix`
- `cowsay`
- `lolcat`
- `fortune`
After the installation is done, the script should let `cowsay` inform the user that the insallation was successful.
Test your script!
> **Fun**: Run the command `cbonsai -i -l -t 0.01 -w 1` 🌳
## Task: Current directory
Write a Bash script that calculates the total number of files and directories (together) in the current directory.
For example, if the current directory has two directories and one file, then the number would be `3`.
Store the result into a variable.
The output should be the full path to the current working directory with a message telling about the calculated number.
Example:
```console
$ whereami.sh
You are in the directory /home/student/scripts
There are 3 files and directories in this directory.
```
### Hints
- `ls -l` shows every file or directory in the current directory on a separate line.
## Task: Backup then update
Write a Bash script that does the following:
1. Create the directory `~/installed_packages` if it does not exist. If it exists, it should not through any errors.
1. Stores the output of the command `dnf list --installed` into the file `~/installed_packages/dnf.txt`.
1. Stores the output of the command `cargo install --list` into the file `~/installed_packages/cargo.txt`.
1. Installs system upgrades using `dnf`.
1. Installs Cargo updates.
1. Check if `cowsay` is installed. Only if not, install it first. After that, `cowsay` should inform you that installed packages where logged into the directory `~/installed_packages` and updates were successfully installed.
If any of the steps above fails, the script should stop and not continue.
This can be achieved by adding the following line to the beginning of the script (after the shebang): `set -e`.
## Task: Scripted "Gimme that PDF" 😈
On day 2, you downloaded a PDF given some web page link.
We want to write a script now that is generic over the web page link.
It should ask the user for a web page link and then download the PDF if a PDF link was found on that web page.
The script should exit with the status code 1 while printing an error message to stderr in case no PDF link was found on the web page ❌
Otherwise, it should print a message to stdout that the download was sucessful ✔️
## Task: Job scheduler ## Task: Job scheduler
> Warning ⚠️ : This task is not an easy task. Don't give up quickly and ask for help if you don't get further! > **Warning** ⚠️ : This task is not an easy one.
> Don't give up quickly and ask for help if you don't get further!
In this task, we want to write our own job scheduler. In this task, we want to write our own job scheduler.
Understanding how job schedulers work is important when you are working on a computer cluster. Understanding how job schedulers work is important when you are working on a computer cluster.
Computer clusters are shared by many users. Therefore, running jobs on a cluster has to be scheduled to make sure that the resources are shared probably. Computer clusters are shared by many users.
Therefore, running jobs on a cluster has to be scheduled to make sure that the resources are shared probably.
In this task, we will keep it simple. No aspects of multiple users or any optimizations. In this task, we will keep it simple.
No aspects of multiple users or any optimizations.
We want to be able to submit a job as a single script (without any dependencies). The submitted scripts should run one after the another to save CPU usage for example. We want to be able to submit a job as a single script (without any dependencies).
The submitted scripts should run one after the another.
We will use the program `inotifywait`. This program can monitor a directory and notify on changes within this directory. We will use the program `inotifywait`.
This program can monitor a directory and notify on changes within this directory.
1. Find out which package installs `inotifywait` and install it. 1. Start Zellij if you are not already using it.
1. Find out which package porvides the program `inotifywait` and install it.
1. Read the manual of `inotifywait` for a better understanding of what it does. 1. Read the manual of `inotifywait` for a better understanding of what it does.
1. Find out how to tell `inotifywait` to keep monitoring a directory and not exit after the first event.
1. Find out what events mean in the context of `inotifywait`. 1. Find out what events mean in the context of `inotifywait`.
1. Find out how to tell `inotifywait` to keep monitoring a directory and not exit after the first event.
1. Create a new directory called `jobs` to be monitored. 1. Create a new directory called `jobs` to be monitored.
1. Create a new directory called `logs` that will be used later. 1. Create a new directory called `logs` that will be used later.
1. Run `inotifywait` while telling it to monitor the directory `jobs`. Leave the command running in a terminal and open a second terminal (tab) to continue the work in. 1. Run `inotifywait` while telling it to monitor the directory `jobs`. Leave the command running in one Zellij pane and open a second pane to continue the work in.
1. Create a file **outside** of the directory `jobs` and then copy it to the directory `jobs`. 1. Create a file **outside** of the directory `jobs` and then copy it to the directory `jobs`.
1. Go back to the first terminal and see the output of `inotifywait` was. 1. Look at the output of `inotifywait` in the first pane after copying the file in to the `jobs` directory.
1. Based on the output, choose an event that you want to listen to with `inotifywait` that tells you when a file is _completely_ added to the directory `jobs`. Use the manual to read more about specific events. 1. Based on that output, choose an event that you want to listen to with `inotifywait` that tells you when a file is **completely written** to the directory `jobs`. Use the manual to read more about specific events.
1. Find an option that lets you tell `inotifywait` to only notify when the chosen event occurs. 1. Find an option that lets you tell `inotifywait` to only notify when the chosen event occurs.
1. Find an option that lets you format the output of the notification of `inotifywait`. Since we only listen on one event and monitor only one directory, an output that shows only the name of the new file should be enough. 1. Find an option that lets you format the output of the notification of `inotifywait`. Since we only listen on one event and monitor only one directory, an output that shows **only the name of the new file** should be enough.
1. Enter the command that you have until now in a script. Now, extend it by using a `while` loop that continuously listens on the notifications of `inotifywait`. Use the following snippet while replacing the sections with `(...)`: 1. Enter the `inotifywait` command that you have until now in a script. Now, extend it by using a `while` loop that continuously listens on the notifications of `inotifywait`. Use the following snippet after replacing `…`:
```bash ```bash
inotifywait (...) | while read FILENAME inotifywait … | while read NEW_SCRIPT_NAME
do do
(...) …
done done
``` ```
1. After a notification, the body of the `while` loop should first print the name of the script that was added. From now on, we only want to add scripts to the `jobs` directory. 1. After a notification, the body of the `while` loop should first print the name of the script that was added (`NEW_SCRIPT_NAME`). Then, the new script should be run.
1. After printing the script name, run the script!
1. Save the standard output and standard error of the script into two separate files in the `logs` directory. If the name of the script is `job.sh` for example, then the output should be in the files `logs/job.sh.out` and `logs/job.sh.err`. 1. Save the standard output and standard error of the script into two separate files in the `logs` directory. If the name of the script is `job.sh` for example, then the output should be in the files `logs/job.sh.out` and `logs/job.sh.err`.
### Hints ### Hints
- Take a look at the examples from the sections of this day. - Take care of file permissions.
- Take care of permissions.
If you have extra time, read about the command `screen` in the internet. `screen` allows you to run commands in the background. This way, you don't need two terminals.
## Task: Job submitter ## Task: Job submitter
In this task we will write a small script that lets us submit a job script to the scheduler from the last task. In this task, we will write a small script called submitter that lets us submit a job script to the scheduler from the last task.
The script should take the path to the job script as a single required argument. The submitter should take the path to the job script as a single required argument.
The script should then copy the job script to the directory `jobs` while adding the time and date to the beginning of the name of the job script in the `jobs` directory. The submitter should then copy the job script to the directory `jobs` while adding the current time and date as a prefix to the new job script name.
Read the manual of the command `date` to know how to get the time and date in the following format: `2022-08-22T20:00:00+00:00`. Read the manual of the command `date` to know how to get the current time and date in the following format: `2022-08-22T20:00:00+00:00`.
If the name of the job script is `job.sh` for example, the job script should be named `2022-08-22T20:00:00+00:00_job.sh` in the `jobs` directory. If the name of the job script is `job.sh` for example, the job script should be named `2022-08-22T20:00:00+00:00_job.sh` in the `jobs` directory.
Use variables to write the script to make it more understandable. Use variables while writing the script to make it more understandable.
#### Help ### Hints
To save the output of a command into a variable, use you have to use the following syntax: - To save the output of a command into a variable, you have to use the following syntax after replacing `…`:
```bash
```bash DATETIME=$(date …)
DATE=$(date ...) ```
```
Replace `...` with your code.
To read the `n`-th argument that is provided to a script you write, you have to use `$n`.
Example script called `arg.sh`:
```bash
#!/usr/bin/bash
echo "The first argument is: $1"
```
When you run this script with an argument:
```console
$ ./arg.sh "Hello"
The first argument is: Hello
```
## Task: Submit a job ## Task: Submit a job
Write a small scripts of your choice that require a long time to run and submit them using the script from the last task. Make sure that the scheduler is running in the background. Write a small job script that requires at least 10 seconds to run (can be simulated with the command `sleep 10`).
Submit that job script by using the submitter from the last task.
You can use the command `sleep` to simulate a job that needs long time to run. Make sure that the scheduler is running in one Zellij pane before submitting.
Submit your job script multiple times and take a look at the terminal that is running the scheduler to make sure that the job scripts are running one after the other. Submit your job script multiple times and take a look at the pane where scheduler is running to make sure that the job scripts are running one after the other.
Verify the redirection of the standard output and standard error in the directory `logs`. Verify the redirection of the standard output and standard error in the directory `logs`.
## Task: Parsing a CSV file
1. Use `curl` to take a look at the file with the following link: [https://gitlab.rlp.net/mobitar/julia\_course/-/raw/main/Day\_3/resources/fitting\_task\_data.csv](https://gitlab.rlp.net/mobitar/julia_course/-/raw/main/Day_3/resources/fitting_task_data.csv). The file contains measurement of a (fake) free fall experiment.
1. Now that you know what the file contains, pipe the output to tools that let you remove the first 6 and last 2 lines. Afterwards, extract the first (measured height) and third column (measured time).
1. Write a small Python that processes the output of the command from the last step. Since this book is not about programming in Python or plotting, the simple code to process the output of the variable `h_t` is given below:
```
#!/usr/bin/env python3
import matplotlib.pyplot as plt
(...)
h_values = []
t_values = []
for line in h_t.strip().split("\n"):
h, t = line.strip().split(",")
h_values.append(h)
t_values.append(t)
plt.plot(h_values, t_values)
plt.xlabel("h")
plt.ylabel("t")
plt.savefig("h_t.pdf")
```
Use the command that you did write in the last step to extract the two columns and save the output in a variable called `h_t` in the `(...)` block.
1. Save the Python script into a file in an empty directory called `bash_python_harmony`.
1. Use `poetry` to initialize an environment in the directory `bash_python_harmony`.
1. Use `poetry` to add the package `matplotlib` to the environment.
1. Use `poetry shell` to enter the environment.
1. Run the script and check the generated PDF file `h_t.pdf`.
## Task: Python rewrite
Choose one of the scripts that you have written during the course and rewrite it in Python!