The other day, I stumbled upon a little curiosity while creating a shell script. At first I was weird about the thing, but then I managed to solve the problem with a little follow-up, and finally it became clear. I thought I'd share it to see if it could benefit someone else and save you a little searching.

First, let's take a simple working example while cycle, before which we set a counter to 0, then in the cycle we move this counter and clear the value, and finally after the cycle we write the value of the variable.

1
2
3
4
5
6
7
#!/bin/bash
c=0                                 # Változó kezdőérték
while [[ $c < 5 ]] ; do             # ciklus indul
    (( c++ ))                       # számláló növelése
    echo "$c. futás"
done 
echo "Számláló értéke: $c"         # változó kiiratása a cikluson kívül.

The example does exactly what we expect. And the output is:

1. futás
2. futás
3. futás
4. futás
5. futás
Számláló értéke: 5

So there’s nothing weird about it here, it works the way it should. But what if, for example, you want to go through the files in a directory that you want to perform operations on and then post-process or display the variables used and modified in the loop

The problem

So there is a directory that now contains four files during our test. We would like to go through these and perform various operations and then display the data collected in the cycle at the end of the cycle.

For the sake of simplicity, here again we just increment a counter in the loop, the value of which is written out step by step, then the name of the found file. Finally, outside the cycle, we clear the counter value in the same way:

1
2
3
4
5
6
7
8
#!/bin/bash
c=0                                 # változó kezdőérték
find . -type f |                    # fájlok keresése a könyvtárban és csővezetéken a ciklusba továbbítása
while read fn; do                   # ciklus indul
    (( c++ ))                       # számláló növelése
    echo "$c. futás: $fn"          # a számláló és a fájlnév kiiratása
done
echo "Számláló értéke: $c"         # változó kiiratása a cikluson kívül.

Here, our output is more interesting, because after the cycle the expired counter value is 0:

1. futás: ./probafile_3
2. futás: ./valtozok_elerese_ciklusokban.sh
3. futás: ./probafile_2
4. futás: ./probafile_1
Számláló értéke: 0

The cycle did its job properly, only after the counter printout did weird things happen.

Solution

At first, when a person stumbles, this phenomenon can cause a bit of a headache, but once you have the solution, the thing is completely clear.

The interesting thing about this is that a bash in the pipelines it executes commands on separate threads, that is, in subshells. Quote from bash man: "Each command in a pipeline is executed as a separate process (ie, in a subshell).". And the variables in the subshells are destroyed after the commands have run in it. By default, there would be no problem with the subshell, but in this case it's in a bad situation because it only covers our while loop, from which we want to extract the value of our variables later. Thus, at the end of the loop, our variables used in it are lost, because then the subshell is destroyed.

One does not necessarily think of this while writing cycles until one day he needs the value of the variables in it even after the cycle. But this way the situation is completely different, the solution is now simple: Organize the loop and the part after it where the value of the internal variables is needed:

1
2
3
4
5
6
7
8
9
#!/bin/bash
c=0                                 # változó kezdőérték
find . -type f | {                  # fájlok keresése a könyvtárban és csővezetéken a ciklusba továbbítása
    while read fn; do               # ciklus indul
        (( c++ ))                   # számláló növelése
        echo "$c. futás: $fn"      # a számláló és a fájlnév kiiratása
    done
    echo "Számláló értéke: $c"     # változó kiiratása a cikluson kívül.
}

So here we have changed so much that we have placed the part after the pipeline in a separate code block {}, which also includes the counting of the counter. Thus, the whole thing is a single command for the pipeline - and thus the subshell. Our output is as follows:

1. futás: ./probafile_3
2. futás: ./valtozok_elerese_ciklusokban.sh
3. futás: ./probafile_2
4. futás: ./probafile_1
Számláló értéke: 4

This works fine as well, after the loop we can access the variables we created or modified while we are still in the same code block.

But what if we have a long or complex script that requires variables long after the cycle, so we don't want to embed additional parts in this code block? There is a solution to this.

Alternative solution

If we do not want to upset the structure and logic of our code, we can do this differently by avoiding the use of a pipeline. This will be our help here stringimplementation, the essence of which is that we do not use a pipeline, so we can transfer the elements to the cycle in the same thread:

1
2
3
4
5
6
7
8
9
10
11
#!/bin/bash
c=0                                 # változó kezdőérték
while read fn; do                   # ciklus indul
    (( c++ ))                       # számláló növelése
    echo "$c. futás: $fn"          # a számláló és a fájlnév kiiratása
done <<< "$(find . -type f)"        # ciklus elemek átadása a herestring-el
 
# további kódrészek ugyanebben a kódblokkban
# ...
 
echo "Számláló értéke: $c"         # változó kiiratása jóval a ciklus után.

Although we have now solved the file search with a subshell query, it does not interfere here, as the loop itself and the subsequent code snippets are in the same shell running, so variables will remain available later in the script. And when you get a list of files, it doesn't matter that the find command was not executed on the same thread.

And the output is exactly the same:

1. futás: ./probafile_3
2. futás: ./valtozok_elerese_ciklusokban.sh
3. futás: ./probafile_2
4. futás: ./probafile_1
Számláló értéke: 4

So we can see how to avoid the use of the pipeline when we run into an unwanted side effect that would make our code unnecessarily complicated.

This way, after all, it makes perfect sense, and next time you will surely remember how you can immediately avoid the inconvenience caused by pipelines.