gh-156920, gh-156698: fix ProactorEventLoop datagram transport hangs on close() and after write errors

[graingert]

• Issue: ProactorEventLoop datagram transports drop buffered datagrams on close() and never call connection_lost() #156920

• Issue: _ProactorDatagramTransport never resumes a paused protocol after a write error #156698

[kumaraditya303]

I think you need to add closing check otherwise any _loop_reading() that runs after close() now arms a fresh recvfrom on a closing transport.

    def _loop_reading(self, fut=None):
        data = None
        try:
            if self._closing:
                # close() no longer bumps _conn_lost while a write is still
                # being flushed, so gate reads on _closing: after close()
                # no more data will be received, and arming a recvfrom()
                # here would leave an overlapped read to be aborted when
                # connection_lost() closes the socket.
                return

[graingert]

@kumaraditya303 thanks for spotting that!

[miss-islington-app]

Thanks @graingert for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

[bedevere-app]

GH-157607 is a backport of this pull request to the 3.15 branch.

[bedevere-app]

GH-157608 is a backport of this pull request to the 3.14 branch.

[bedevere-app]

GH-157609 is a backport of this pull request to the 3.13 branch.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot ARM64 Windows Non-Debug 3.x (tier-2) has failed when building commit 7f52444.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/730/builds/14583) and take a look at the build logs.
4. Check if the failure is related to this commit (7f52444) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/730/builds/14583

Failed tests:

• test.test_asyncio.test_sslproto

Failed subtests:

• test_shutdown_corrupted_ssl_sends_close_notify - test.test_asyncio.test_sslproto.SelectorStartTLSTests.test_shutdown_corrupted_ssl_sends_close_notify

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "C:\Users\buildworker\buildarea\3.x.ware-win11-arm64.nondebug\build\Lib\test\test_asyncio\functional.py", line 255, in _run
    self._handle_client(conn)
    ~~~~~~~~~~~~~~~~~~~^^^^^^
  File "C:\Users\buildworker\buildarea\3.x.ware-win11-arm64.nondebug\build\Lib\test\test_asyncio\functional.py", line 264, in _handle_client
    self._prog(TestSocketWrapper(sock))
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\buildworker\buildarea\3.x.ware-win11-arm64.nondebug\build\Lib\test\test_asyncio\test_sslproto.py", line 920, in server
    sock.unwrap()
    ~~~~~~~~~~~^^
  File "C:\Users\buildworker\buildarea\3.x.ware-win11-arm64.nondebug\build\Lib\ssl.py", line 1402, in unwrap
    s = self._sslobj.shutdown()
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host


Traceback (most recent call last):
  File "C:\Users\buildworker\buildarea\3.x.ware-win11-arm64.nondebug\build\Lib\threading.py", line 1218, in _bootstrap_inner
    self._context.run(self.run)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "C:\Users\buildworker\buildarea\3.x.ware-win11-arm64.nondebug\build\Lib\test\test_asyncio\functional.py", line 227, in run
    self._run()
    ~~~~~~~~~^^
  File "C:\Users\buildworker\buildarea\3.x.ware-win11-arm64.nondebug\build\Lib\test\test_asyncio\functional.py", line 261, in _run
    self._test._abort_socket_test(ex)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^
  File "C:\Users\buildworker\buildarea\3.x.ware-win11-arm64.nondebug\build\Lib\test\test_asyncio\functional.py", line 110, in _abort_socket_test
    self.fail(ex)
    ~~~~~~~~~^^^^
  File "C:\Users\buildworker\buildarea\3.x.ware-win11-arm64.nondebug\build\Lib\unittest\case.py", line 772, in fail
    raise self.failureException(msg)
AssertionError: [WinError 10054] An existing connection was forcibly closed by the remote host
k


Traceback (most recent call last):
  File "C:\Users\buildworker\buildarea\3.x.ware-win11-arm64.nondebug\build\Lib\test\test_asyncio\test_sslproto.py", line 950, in test_shutdown_corrupted_ssl_sends_close_notify
    self.loop.run_until_complete(client(srv.addr))
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "C:\Users\buildworker\buildarea\3.x.ware-win11-arm64.nondebug\build\Lib\asyncio\base_events.py", line 723, in run_until_complete
    raise RuntimeError('Event loop stopped before Future completed.')
RuntimeError: Event loop stopped before Future completed.


Traceback (most recent call last):
  File "C:\Users\buildworker\buildarea\3.x.ware-win11-arm64.nondebug\build\Lib\threading.py", line 1218, in _bootstrap_inner
    self._context.run(self.run)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "C:\Users\buildworker\buildarea\3.x.ware-win11-arm64.nondebug\build\Lib\test\test_asyncio\functional.py", line 227, in run
    self._run()
    ~~~~~~~~~^^
  File "C:\Users\buildworker\buildarea\3.x.ware-win11-arm64.nondebug\build\Lib\test\test_asyncio\functional.py", line 261, in _run
    self._test._abort_socket_test(ex)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^
  File "C:\Users\buildworker\buildarea\3.x.ware-win11-arm64.nondebug\build\Lib\test\test_asyncio\functional.py", line 110, in _abort_socket_test
    self.fail(ex)
    ~~~~~~~~~^^^^
  File "C:\Users\buildworker\buildarea\3.x.ware-win11-arm64.nondebug\build\Lib\unittest\case.py", line 772, in fail
    raise self.failureException(msg)
AssertionError: [WinError 10054] An existing connection was forcibly closed by the remote host
ERROR