pipelining and redirection are broken because gosh does not close file descriptors

[lersek]

Versions

• go version: upstream 656b5b3abe25 ("internal/poll: don't skip empty writes on Windows", 2025-03-28)
• gosh version: upstream bf4faa20c0ea ("interp: clarify handler docs a bit", 2025-03-28)

Description

For constructing a pipeline of N commands, a shell performs something like this:

1. call pipe() N-1 times
2. call fork() N times
3. in each child process:
   1. call dup2() to duplicate the read end of the pipe to the left (if any) over fd#0
   2. call dup2() to duplicate the write end of the pipe to the right (if any) over fd#1
   3. close all pipe fds from step 1
   4. exec()
4. in the parent, close all pipe fds from step 1

The bug is that gosh omits the last step, and it breaks pipeline behavior.

Reproducer

gosh
$ dash -c 'exec >/dev/null; exec sleep 3600' | cat >/dev/null &

$ ls -g -o /proc/$(pidof sleep)/fd/1
l-wx------. 1 64 Mar 28 19:19 /proc/781245/fd/1 -> /dev/null

$ ls -g -o /proc/$(pidof cat)/fd/0
lr-x------. 1 64 Mar 28 19:19 /proc/781246/fd/0 -> 'pipe:[592425]'

$ ls -g -o /proc/$$/fd | grep -F 'pipe:[592425]'
lr-x------. 1 64 Mar 28 19:20 3 -> pipe:[592425]
l-wx------. 1 64 Mar 28 19:20 4 -> pipe:[592425]

Explanation:

• The first command in the pipeline drops its fd#1 reference to the pipe. At this point, there should be no writers left for the pipe.
• However, the second command in the pipeline does not see EOF on the pipe; it blocks on reading the pipe.
• Turns out that gosh itself keeps the pipe open for writing (fd#4), which is what prevents cat from seeing EOF.
• cat hangs uselessly, as a result, instead of exiting.
• The problem occurs because gosh does not close file descriptors 3 and 4 right after the pipeline is constructed (step 4 at the top).

Expected behavior

dash
$ dash -c 'exec >/dev/null; exec sleep 3600' | cat >/dev/null &

$ ls -g -o /proc/$(pidof sleep)/fd/1
l-wx------. 1 64 Mar 28 19:25 /proc/781363/fd/1 -> /dev/null

$ ls -g -o /proc/$(pidof cat)/fd/0
ls: cannot access '/proc//fd/0': No such file or directory

$ ls -g -o /proc/$$/fd | grep -F 'pipe:'
(nothing)

Dash doesn't keep the pipe open (at all); thus, cat reads EOF (and exits), after the first command in the pipeline drops its reference -- the only reference -- to the write end of the pipe. This is the proper behavior.

Alternative reproducer

This one uses redirection operators with a named pipe (FIFO), instead of a pipeline.

Erroneous behavior

gosh
$ rm -f fifo
$ mkfifo fifo
$ dash -c 'exec >/dev/null; exec sleep 3600' >fifo &
$ cat <fifo >/dev/null &

$ ls -g -o /proc/$(pidof sleep)/fd/1
l-wx------. 1 64 Mar 28 19:35 /proc/781581/fd/1 -> /dev/null

$ ls -g -o /proc/$(pidof cat)/fd/0
lr-x------. 1 64 Mar 28 19:35 /proc/781580/fd/0 -> .../fifo

$ ls -g -o /proc/$$/fd | grep -F 'fifo'
l-wx------. 1 64 Mar 28 19:36 3 -> .../fifo
lr-x------. 1 64 Mar 28 19:36 4 -> .../fifo

Expected behavior

dash
$ rm -f fifo
$ mkfifo fifo
$ dash -c 'exec >/dev/null; exec sleep 3600' >fifo &
$ cat <fifo >/dev/null &

$ ls -g -o /proc/$(pidof sleep)/fd/1
l-wx------. 1 64 Mar 28 19:35 /proc/781581/fd/1 -> /dev/null

$ ls -g -o /proc/$(pidof cat)/fd/0
ls: cannot access '/proc//fd/0': No such file or directory

$ ls -g -o /proc/$$/fd | grep -F 'fifo'
(nothing)

[lersek]

The bug is that gosh omits the last step, and it breaks pipeline behavior.

More precisely: gosh does close the pipe fds, but only after waiting for / reaping the child. That's what's wrong; the child shouldn't run with unrelated dangling references existing.

AFAICT, this dates back to commit dacbf6b:

+			go func() {
+				r2.node(x.X)
+				pw.Close()
+			}()
+			r.node(x.Y)
+			pr.Close()

The pipe ends are only closed after the commands return; IOW, the commands run while the wrong references exist.

[lersek]

IOW, in the parent (= the shell), the pipe fds need to be closed (step 4) immediately after the forks (step 2).

[lersek]

Somewhere between exec.Cmd.Start() and exec.Cmd.Wait().

[lersek]

The following are broken, too:

cat /dev/zero | dash -c 'exec </dev/null; sleep 3600' &

and

rm -f fifo
mkfifo fifo
cat /dev/zero >fifo &
dash -c 'exec </dev/null; sleep 3600' <fifo &

In both cases, cat should get a SIGPIPE, but it doesn't, because gosh keeps the read end of the pipe open / keeps the fifo open for reading.

I've tried my hand at a patch, but it breaks a test case in the interpreter test suite,

echo foo_interp_missing | { $ENV_PROG >/dev/null; cat; }

which highlighted a design decision (?) in gosh that is inconsistent with both bash and dash. Namely, in the example pipeline

echo a | read X

gosh sets the X variable, while dash and bash don't.

In other words, gosh runs the rightmost command of a multi-command pipeline that is not an asynchronous list (&) in the current shell execution environment, not in a subshell. While POSIX does permit this as an extension, it seems to interfere badly with my attempt to close the read-end of the pipe, as the current shell execution environment seems to lose its own stdin with that.