subprocess.Popen on windows cannot use default handles for stdin/stdout/stderr when some of them are redirected

[panda-34]

Bug report

Bug description:

suppose I want to run a subprocess with piped stdin/stdout but let that process print to console using stderr (which is a common use pattern)
currently in Popen there's no way to override stdin=PIPE, stdout=PIPE but let stderr use default handle

if I have a child.py script:

import sys, time
try:
    print('console message', file=sys.stderr)
except Exception as e:
    open('CONOUT$', 'w').write(str(e))
while True:
    time.sleep(1)

and parent.py script:

import subprocess
with subprocess.Popen(('python.exe', 'child.py'), stdin=subprocess.PIPE, stdout=subprocess.PIPE, creationflags=subprocess.CREATE_NEW_CONSOLE):
    pass

then it works if parent script is run in a console. But if it is a GUI script (e.g. run via pythonw.exe) then child script attempting to print to stderr gets [Errno 22] Invalid argument
this happens because this line:

cpython/Lib/subprocess.py

Line 1398 in 0671451

errwrite = _winapi.GetStdHandle(_winapi.STD_ERROR_HANDLE)

tries to inherit stderr, but it returns None in a GUI app, so then a fake Pipe is created and ultimately passed to CreateProcess as stderr instead of NULL handle.

Here's a hack which allows me to selectively pipe only stdin and stdout and leave stderr as default:

import subprocess

class HackedSTARTUPINFO(subprocess.STARTUPINFO):
    def __setattr__(self, attr, value):
        if attr == 'hStdError':
            value = None
        super().__setattr__(attr, value)

    def copy(self):
        return self

si = HackedSTARTUPINFO()
with subprocess.Popen(('python.exe', 'child.py'), stdin=subprocess.PIPE, stdout=subprocess.PIPE, startupinfo=si, creationflags=subprocess.CREATE_NEW_CONSOLE):
    pass

This way child.py can print to stderr even if run from a GUI script.

There should be a way to selectively set STARTUPINFO.hStdInput/hStdOutput/hStdError to None using Popen arguments.

CPython versions tested on:

3.10

Operating systems tested on:

Windows

[zooba]

You might be able to pass stderr=2 to bypass most of our handling here and directly pass the C Runtime's stderr stream. It's not a fix, but if you're on 3.10 then it might be the best you're going to get (if it works).

I wonder whether the _get_handles function is working too hard? If the caller has passed None, then CreateProcess should pass along the default stream handles anyway. Not sure if we have anyone active who would've implemented this, so might require some archaeology to find out why it is implemented this way.

[panda-34]

You might be able to pass stderr=2 to bypass most of our handling here

if one of the handles is overridden in Popen arguments then all three are inherited from the current process, and inheriting handle=2 from a GUI process doesn't produce errors but doesn't print anything either.