[Bash Challenge 8] Can You Solve This Bash Script Puzzle?

rm -rf ItsFOSS
mkdir -p ItsFOSS
cd ItsFOSS
cat > HEADER << EOT
# Month, Year, Est.Value
EOT
cat > DATA01 << EOT
Dec, 2015, 15000
Jan, 2016, 12540
Feb, 2016, 11970
EOT
clear
head HEADER DATA01
cat HEADER DATA01 | tee DATA01

cat HEADER DATA01 | ( sleep 1; tee DATA01 )
# Month, Year, Est.Value
Dec, 2015, 15000
Jan, 2016, 12540
Feb, 2016, 11970

(sleep 1 ; cat HEADER DATA01 ) | tee DATA01
# Month, Year, Est.Value

cat HEADER DATA01 > DATA01

Do I really need to comment that? Well, while being rudimentary, this solution has a nice feature: since rm will use the system call rename, which itself is atomic in that sense that referencing the DATA01 file, other processes will either see the old content or the new content — but neither an half-written content.

A somewhat similar solution, but avoiding to create a temporary file visible on the filesystem would obtain first a file descriptor to read from the original file before overwriting it:

• Open the file DATA1 for reading using the file descriptor 3;

• Unlink the original file (i.e.: remove its directory entry, but not the data as the file is still open);
• Use cat to read the header first, followed by a stdin read from file descriptor 3 and write to a new DATA01 file;

• Close the file descriptor 3 This will effectively delete the old DATA01 content.

Please note this solution is no longer atomic in the sense used above. Anyways, Kudos to Adithya Kiran Gangu for having proposed that solution!

While encountering similar problems for the first time, my idea was to use sed. It is quite easy to insert a “header” after the first line using sed. But it’s more difficult to insert something before the first line. In fact, to achieve that, we will need a little bit of magic:

To fully understand, you need to know the (r)ead command inserts the content of a file in the destination stream, but only once the current line processing has ended. That’s why I used the (N)ext command: it will end the line 1 processing early (i.e.: before normal line output). So, when encountering that command, sed ends processing of line 1. Which triggers output of the content of the HEADER file. But the line 1 itself is not sent to the output. It is kept in the sed buffer.

Then sed reads the next line of input, append it to the buffer, and as we do not have any rule for line 2, process it as usual by sending its buffer to the output (remember at that stage, the buffer contains both line 1 and line 2).

This solution has a major drawback: it assumes there is a line 2. If the data file contains only one line, this will fail miserably.

We have very few occasions of using ed or its cousin ex. Both are line oriented editors. Their behavior is very similar to vi in that sense you load file into memory, and send commands to the editor to modify that file. The only difference here is we will script the commands instead of sending them interactively.

This works great, but as we have to load the whole file into memory which could be an issue for very large files.

As always, those are probably only a subset of all possible solutions. So don’t hesitate to use the comment section to share your own ideas.

And stay tuned for more fun!