Node support

[prescod]

Please document whether Node is supported and if so, please provide an example. This example failed for me:

const { loadPyodide } = require("pyodide");

async function hello_python() {
  let pyodide = await loadPyodide();
  await pyodide.loadPackage("micropip");
  const micropip = pyodide.pyimport("micropip");
  await micropip.install("requests");
  return pyodide.runPythonAsync(`
  # 1. Install this package
  import micropip
  await micropip.install('pyodide-http')
  
  # 2. Patch requests
  import pyodide_http
  pyodide_http.patch_all()  # Patch all libraries
  
  # 3. Use requests
  import requests
  response = requests.get('http://www.google.com/')
  `);
}

hello_python().then((result) => {
  console.log("Python says that the lineup is", result);
});

Node.js v18.12.1

Error:

: Failed to establish a new connection: [Errno 50] Protocol not available'))

Or with https:

Can't connect to HTTPS URL because the SSL module is not available.

Versions:

        "fs": "^0.0.1-security",
        "node-fetch": "^3.3.0",
        "pyodide": "^0.21.3"

[koenvo]

Thanks for your report. I didn't test on node and during setting up tests I had some issues with node-fetch. Let me setup a test and see where it fails.

[prescod]

Thank you! Requests support for Pyodide and Node is the hard-blocker for my project.

[LijieZhang1998]

Hi, is there update on this issue? I also encountered the same issue. Thanks.

BTW, @prescod for the https error Can't connect to HTTPS URL because the SSL module is not available., you can await pyodide.loadPackage("ssl"); in the js code to address it.

@koenvo how would you debug it? I'm interested in debugging it myself and may contribute to it, but have no idea about how to start it. Any guidance could be helpful. Thanks.

Also I think this issue is related to #28

PythonError: Traceback (most recent call last):
  File "/lib/python3.11/site-packages/urllib3/connection.py", line 203, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/lib/python3.11/site-packages/urllib3/util/connection.py", line 67, in create_connection
    _set_socket_options(sock, socket_options)
  File "/lib/python3.11/site-packages/urllib3/util/connection.py", line 100, in _set_socket_options
    sock.setsockopt(*opt)
OSError: [Errno 50] Protocol not available

[davidmreed]

I'm pretty sure I'm working on the same thing @prescod was and can report a few more details.

pyodide_http works by patching requests and urllib to use XMLHttpRequest instead of their original networking primitives. Node doesn't provide XMLHttpRequest. This results in pyodide_http.should_patch() returning False, because the import from js fails. That's why you get the "Protocol not available" messages: your requests or urllib isn't actually patched!

One can install the xmlhttprequest Node package to provide a version of the browser's API. This does allow pyodide_http to complete patching of requests, but it still doesn't work. The mock-XMLHttpRequest object provided by that Node package doesn't provide multiple attributes that pyodide_http's patches require, including overrideMimeType.

I suspect the fix is to build a different set of patches to be applied in a Node environment, using Node's HTTP classes.