Fixed: A Named Pipe's Reader Never Finishes in a Shell Script
My script fed commands to a program through a named pipe. It closed the pipe when it was done, and waited for the program to finish, and that wait never ended. Another process had a copy of the pipe open, and I'd given it that copy without noticing.
What I Was Doing
A script kept a mysql session open by writing to it through a named pipe, and started another program in between. Here it is in its smallest form, with cat as the reader and sleep as the other program:
mkfifo demo.pipe
cat demo.pipe > fifo.out &
READER=$!
exec 3> demo.pipe
print -u3 "first line"
sleep 8 &
exec 3>&-
wait $READER
exec 3> demo.pipeopens the pipe for writing, as file descriptor 3, and keeps it open.print -u3writes to it.exec 3>&-closes it, and the reader should then see the end of the file.
What I Expected
cat to finish right after exec 3>&-, and wait to return.
What I Got
wait didn't return until sleep had ended. In my real script, the other program was a server that kept running, so it never returned at all. With a check added after a second:
After 1 second, the reader is still running
COMMAND FD TYPE NAME
sleep 3w FIFO demo.pipe
Once sleep ended, the reader finished
What I Checked
Who has it open. lsof -p lists a process's open files. It's in the output above: sleep has descriptor 3, open for writing (3w), on the pipe.
What ends a read. POSIX's read says: "If no process has the pipe open for writing, read() shall return 0 to indicate end-of-file." No process at all. So while any writer exists, the reader keeps waiting.
Why It Happens
A child process inherits its parent's open file descriptors. sleep started while descriptor 3 was open, so it got its own copy. exec 3>&- closed only the script's copy.
The Fix
Close the descriptor for the child, as you start it:
sleep 8 3>&- &
Or start other programs before you open the pipe. Do this for every program started while it's open.
After the Fix
The reader finished
first line
Common Mistakes
| Mistake | What happens | Do this instead |
|---|---|---|
| Closing it only in the script | Children keep their copies | Add 3>&- to each child |
| Starting a server while it's open | It holds the pipe for as long as it runs | Start it first, or close it for the child |
| Hiding the hang with a timeout | The real holder stays unknown | Find it with lsof |
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
exec 3> pipe itself hangs | No reader has opened the pipe yet | Start the reader first |
| The reader gets no data | The writer's output is buffered | Flush it, or write less often |
| The script ends and the reader stays | A child outlived it, with the descriptor | Check with lsof |
Conclusion
A pipe's reader sees the end only when every writer has closed it, and child processes inherit a copy. Use 3>&- on anything you start while it's open.
Additional Resources
- read in POSIX — what ends a read from a pipe.
