🔖 Release 3.4.0 (#59)

3.4.0 (2024-01-01)
------------------

**Added**
- Support for specifying a custom DNS resolver in `Session`.
- Support for passing the source address in `Session`.
- Support for disabling either IPv4 or IPv6 within a `Session`.

**Changed**
- PySocks is no longer used for SOCKS proxies. Replaced by
**python-socks** instead.
- urllib3.future minimal version raised to 2.4+ to leverage newly added
features.
- Improve compatibility when end-user inadvertently pass a `Timeout` or
`Retry` instance from the legacy `urllib3` instead of `urllib3_future`.

**Fixed**
- Blocking the event loop when closing the `AsyncSession` using `with`.
- Rare exception on older PyPy interpreters due to Generic having
unsupported type variable in `extensions._sync_to_async` module.

**Misc**
- Project extras aligned with **urllib3.future**.
- Using nox instead of tox.
- Switch to ruff instead of black and isort.

.github/workflows/lint.yml

@@ -11,7 +11,7 @@ permissions:
 
 jobs:
   lint:
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
     timeout-minutes: 10
 
     steps:
@@ -20,5 +20,7 @@ jobs:
       uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5.0.0
       with:
         python-version: "3.x"
-    - name: Run pre-commit
-      uses: pre-commit/action@646c83fcd040023954eafda54b4db0192ce70507 # v3.0.0
+    - name: Install nox
+      run: python -m pip install nox
+    - name: run pre-commit
+      run: nox -s lint

.github/workflows/run-tests.yml

@@ -33,7 +33,7 @@ jobs:
         cache: 'pip'
     - name: Install dependencies
       run: |
-        make
+        pip install nox
     - name: Run tests
       run: |
-        make ci
+        nox -s "test-${{ startsWith(matrix.python-version, 'pypy') && 'pypy' || matrix.python-version }}"

.gitignore

@@ -21,6 +21,8 @@ env/
 .pytest_cache/
 .vscode/
 .eggs/
+.nox/
+.ruff_cache/
 
 .workon
 

.pre-commit-config.yaml

@@ -8,28 +8,24 @@ repos:
   - id: debug-statements
   - id: end-of-file-fixer
   - id: trailing-whitespace
-- repo: https://github.com/PyCQA/isort
-  rev: 5.12.0
-  hooks:
-    - id: isort
-- repo: https://github.com/psf/black
-  rev: 23.7.0
-  hooks:
-    - id: black
-      exclude: tests/test_lowlevel.py
 - repo: https://github.com/asottile/pyupgrade
   rev: v3.10.1
   hooks:
     - id: pyupgrade
       args: [--py37-plus]
-- repo: https://github.com/PyCQA/flake8
-  rev: 6.1.0
+- repo: https://github.com/astral-sh/ruff-pre-commit
+  # Ruff version.
+  rev: v0.1.7
   hooks:
-    - id: flake8
+    # Run the linter.
+    - id: ruff
+      args: [ --fix ]
+    # Run the formatter.
+    - id: ruff-format
 - repo: https://github.com/pre-commit/mirrors-mypy
   rev: v1.5.1
   hooks:
   -   id: mypy
       args: [--check-untyped-defs]
-      exclude: 'tests/'
-      additional_dependencies: ['charset_normalizer', 'urllib3.future>=2.3.900', 'wassima>=1.0.1', 'idna', 'kiss_headers']
+      exclude: 'tests/|noxfile.py'
+      additional_dependencies: ['charset_normalizer', 'urllib3.future>=2.4.900', 'wassima>=1.0.1', 'idna', 'kiss_headers']

HISTORY.md

@@ -1,6 +1,28 @@
 Release History
 ===============
 
+3.4.0 (2024-01-01)
+------------------
+
+**Added**
+- Support for specifying a custom DNS resolver in `Session`.
+- Support for passing the source address in `Session`.
+- Support for disabling either IPv4 or IPv6 within a `Session`.
+
+**Changed**
+- PySocks is no longer used for SOCKS proxies. Replaced by **python-socks** instead.
+- urllib3.future minimal version raised to 2.4+ to leverage newly added features.
+- Improve compatibility when end-user inadvertently pass a `Timeout` or `Retry` instance from the legacy `urllib3` instead of `urllib3_future`.
+
+**Fixed**
+- Blocking the event loop when closing the `AsyncSession` using `with`.
+- Rare exception on older PyPy interpreters due to Generic having unsupported type variable in `extensions._sync_to_async` module.
+
+**Misc**
+- Project extras aligned with **urllib3.future**.
+- Using nox instead of tox.
+- Switch to ruff instead of black and isort.
+
 3.3.4 (2023-12-03)
 ------------------
 

Makefile

@@ -1,15 +1,15 @@
 .PHONY: docs
 init:
-	python -m pip install -r requirements-dev.txt
+	python -m pip install nox coverage
 test:
 	# This runs all of the tests on all supported Python versions.
-	tox -p
+	nox -s test
 ci:
-	python -m pytest tests --verbose --junitxml=report.xml
+	nox -s test
 
 coverage:
-	python -m pytest --cov-config .coveragerc --verbose --cov-report term --cov-report xml --cov=niquests tests
+	python -m coverage combine && python -m coverage report --ignore-errors --show-missing
 
 docs:
-	cd docs && make html
+	nox -s docs
 	@echo "\033[95m\n\nBuild successful! View the docs homepage at docs/_build/html/index.html.\n\033[0m"

README.md

@@ -12,21 +12,26 @@ Niquests, is the “**Safest**, **Fastest<sup>*</sup>**, **Easiest**, and **Most
 
 ```python
 >>> import niquests
->>> r = niquests.get('https://httpbin.org/basic-auth/user/pass', auth=('user', 'pass'))
+>>> s = niquests.Session(resolver="doh+google://", multiplexed=True)
+>>> r = s.get('https://pie.dev/basic-auth/user/pass', auth=('user', 'pass'))
 >>> r.status_code
 200
 >>> r.headers['content-type']
 'application/json; charset=utf8'
+>>> r.oheaders.content_type.charset
+'utf8'
 >>> r.encoding
 'utf-8'
 >>> r.text
 '{"authenticated": true, ...'
 >>> r.json()
 {'authenticated': True, ...}
 >>> r
-<Response HTTP/2 [200]>
+<Response HTTP/3 [200]>
 >>> r.ocsp_verified
 True
+>>> r.conn_info.established_latency
+datetime.timedelta(microseconds=38)
 ```
 
 Niquests allows you to send HTTP requests extremely easily. There’s no need to manually add query strings to your URLs, or to form-encode your `PUT` & `POST` data — but nowadays, just use the `json` method!
@@ -48,9 +53,11 @@ Niquests officially supports Python or PyPy 3.7+.
 
 Niquests is ready for the demands of building robust and reliable HTTP–speaking applications, for the needs of today.
 
+- DNS over HTTPS, DNS over QUIC, DNS over TLS, and DNS over UDP
 - Automatic Content Decompression and Decoding
 - OS truststore by default, no more certifi!
 - OCSP Certificate Revocation Verification
+- Advanced connection timings inspection
 - In-memory certificates (CAs, and mTLS)
 - Browser-style TLS/SSL Verification
 - Sessions with Cookie Persistence
@@ -59,6 +66,7 @@ Niquests is ready for the demands of building robust and reliable HTTP–speakin
 - Automatic honoring of `.netrc`
 - Basic & Digest Authentication
 - Familiar `dict`–like Cookies
+- Network settings fine-tuning
 - Object-oriented headers
 - Multi-part File Uploads
 - Chunked HTTP Requests
@@ -69,8 +77,12 @@ Niquests is ready for the demands of building robust and reliable HTTP–speakin
 - HTTP/2 by default
 - HTTP/3 over QUIC
 - Multiplexed!
+- Thread-safe!
+- DNSSEC!
 - Async!
 
+Need something more? Create an issue, we listen.
+
 ## Why did we pursue this?
 
 For many years now, **Requests** has been frozen and blocked millions of developers, left in a vegetative state

docs/api.rst

@@ -142,7 +142,7 @@ Removed
 * Dependency check at runtime for ``urllib3``. There's no more check and warnings at runtime for that subject. Ever.
 
 Behavioural Changes
-~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~
 
 * Niquests negotiate for a HTTP/2 connection by default, fallback to HTTP/1.1 if not available.
 * Support for HTTP/3 can be present by default if your platform support the pre-built wheel for qh3.

docs/index.rst

@@ -28,21 +28,27 @@ is designed to be a drop-in replacement for **Requests** that is no longer under
 
 **Behold, the power of Niquests**::
 
-    >>> r = niquests.get('https://api.github.com/user', auth=('user', 'pass'))
-    >>> r.status_code
-    200
-    >>> r.headers['content-type']
-    'application/json; charset=utf8'
-    >>> r.encoding
-    'utf-8'
-    >>> r.text
-    '{"type":"User"...'
-    >>> r.json()
-    {'private_gists': 419, 'total_private_repos': 77, ...}
-    >>> r
-    <Response HTTP/2 [200]>
-    >>> r.ocsp_verified
-    True
+   >>> import niquests
+   >>> s = niquests.Session(resolver="doh+google://", multiplexed=True)
+   >>> r = s.get('https://pie.dev/basic-auth/user/pass', auth=('user', 'pass'))
+   >>> r.status_code
+   200
+   >>> r.headers['content-type']
+   'application/json; charset=utf8'
+   >>> r.oheaders.content_type.charset
+   'utf8'
+   >>> r.encoding
+   'utf-8'
+   >>> r.text
+   '{"authenticated": true, ...'
+   >>> r.json()
+   {'authenticated': True, ...}
+   >>> r
+   <Response HTTP/3 [200]>
+   >>> r.ocsp_verified
+   True
+   >>> r.conn_info.established_latency
+   datetime.timedelta(microseconds=38)
 
 See `similar code, sans Niquests <https://gist.github.com/973705>`_.
 
@@ -57,16 +63,19 @@ Beloved Features
 
 Niquests is ready for today's web.
 
+- DNS over HTTPS, DNS over QUIC, DNS over TLS, and DNS over UDP
 - Automatic Content Decompression and Decoding
 - OS truststore by default, no more certifi!
 - OCSP Certificate Revocation Verification
+- Advanced connection timings inspection
+- In-memory certificates (CAs, and mTLS)
 - Browser-style TLS/SSL Verification
 - Sessions with Cookie Persistence
 - Keep-Alive & Connection Pooling
 - International Domains and URLs
-- Automatic honoring of ``.netrc``
+- Automatic honoring of `.netrc`
 - Basic & Digest Authentication
-- Familiar ``dict``–like Cookies
+- Familiar `dict`–like Cookies
 - Object-oriented headers
 - Multi-part File Uploads
 - Chunked HTTP Requests
@@ -77,6 +86,8 @@ Niquests is ready for today's web.
 - HTTP/2 by default
 - HTTP/3 over QUIC
 - Multiplexed!
+- Thread-safe!
+- DNSSEC!
 - Async!
 
 Niquests officially supports Python 3.7+, and runs great on PyPy.

docs/user/advanced.rst

@@ -286,6 +286,27 @@ You may specify the private key passphrase using the following example::
     >>> niquests.get('https://kennethreitz.org', cert=('/path/client.cert', '/path/client.key', 'my_key_password'))
     <Response HTTP/2 [200]>
 
+DNS with mTLS
+~~~~~~~~~~~~~
+
+You can pass your client side certificate to authenticate yourself against the given resolver.
+To do so, you will have to do as follow::
+
+    from urllib3 import ResolverDescription
+    from niquests import Session
+
+    rd = ResolverDescription.from_url("doq://my-resolver.tld")
+    rd["cert_data"] = in_memory_cert  # not a path, it should contain your cert content PEM format directly
+    rd["cert_key"] = ...
+    rd["key_password"] = ...
+
+    with Session(resolver=rd) as s:
+        ...
+
+.. note:: Instead of in-memory cert, you can pass file path instead with ``cert_file``, ``key_file``.
+
+This method of authentication is broadly used with DNS over TLS, QUIC, and HTTPS.
+
 In-memory Certificates
 ----------------------
 
@@ -345,7 +366,7 @@ make the request within a ``with`` statement to ensure it's always closed::
 Keep-Alive
 ----------
 
-Excellent news — thanks to urllib3, keep-alive is 100% automatic within a session!
+Excellent news — thanks to urllib3.future, keep-alive is 100% automatic within a session!
 Any requests that you make within a session will automatically reuse the appropriate
 connection!
 
@@ -365,9 +386,7 @@ file-like object for your body::
     with open('massive-body', 'rb') as f:
         niquests.post('http://some.url/streamed', data=f)
 
-.. warning:: It is recommended that you open files in :ref:`binary
-             mode <tut-files>`. Errors may occur if you open the file in *text mode*.
-             due to the fact that this will be re-encoded later in the process.
+.. warning:: It is recommended that you open files in binary mode.
 
 
 .. _chunk-encoding:
@@ -500,11 +519,18 @@ You can find a example of how to retrieve the connection information just before
 Here, ``r`` is the ``PreparedRequest`` and ``conn_info`` contains a ``ConnectionInfo``.
 You can explore the following data in it.
 
-- **certificate_der**: The certificate in DER format (binary)
-- **certificate_dict**: The certificate as a dictionary like ``ssl.SSLSocket.getpeercert(binary_from=False)`` output it.
+- **certificate_der**: The peer certificate in DER format (binary)
+- **certificate_dict**: The peer certificate as a dictionary like ``ssl.SSLSocket.getpeercert(binary_from=False)`` output it.
 - **tls_version**: TLS version.
 - **cipher**: Cipher used.
 - **http_version**: Http version that is about to be used.
+- **destination_address**: The remote peer address given to us by the DNS resolver.
+- **issuer_certificate_der**: Immediate issuer (in the TLS certificate chain) in DER format (binary)
+- **issuer_certificate_dict**: Immediate issuer (in the TLS certificate chain) as a dictionary
+- **established_latency**: The amount of time consumed to get an ESTABLISHED network link.
+- **resolution_latency**: The amount of time consumed for the hostname resolution.
+- **tls_handshake_latency**: The amount of time consumed for the TLS handshake completion.
+- **request_sent_latency**: The amount of time consumed to encode and send the whole request through the socket.
 
 .. warning:: Depending on your platform and interpreter, some key element might not be available and be assigned ``None`` everytime. Like **certificate_dict** on MacOS.
 
@@ -1290,4 +1316,50 @@ Thread Safety
 Niquests is meant to be thread-safe. Any error or unattended behaviors are covered by our support for bug policy.
 Both main scenarios are eligible, meaning Thread and Async, with Thread and Sync.
 
-Support include notable performance issues like abusive lock.
+Support include notable performance issues like abusive lock.
+
+Use a custom CA without loosing the official ones
+-------------------------------------------------
+
+There's an interesting use-case where a user may want to be able to request both private
+and public HTTP endpoints without doing some gymnastic with ``verify=...``.
+
+Thanks to our underlying library ``wassima`` you can register globally your own set
+of certificate authorities like so::
+
+    import wassima
+
+    wassima.register_ca(my_own_ca_pem_str)
+
+That's it! Niquests will now automatically recognize it and use it to verify your secure endpoints.
+You'll have to register it prior to your HTTP requests.
+
+.. note:: While doing local development with HTTPS, we recommend using tool like ``mkcert`` that will register the CA into your local machine trust store. Niquests is natively capable of picking them up.
+
+Disable either IPv4 or IPv6
+---------------------------
+
+You may be interested in controlling what kind of address you would accept connecting to.
+Since Niquests 3.4+, you can configure that aspect per ``Session`` instance.
+
+Having a session without IPv6 enabled should be done that way::
+
+    import niquests
+
+    session = niquests.Session(disable_ipv6=True)
+
+.. warning:: You cannot set both ``disable_ipv4`` and ``disable_ipv6`` at the cost of receiving a RuntimeError exception.
+
+Setting the source network adapter
+----------------------------------
+
+In a complex scenario, you could face the following: "I have multiple network adapters, some can access this and other that.."
+Since Niquests 3.4+, you can configure that aspect per ``Session`` instance.
+
+Having a session without IPv6 enabled should be done that way::
+
+    import niquests
+
+    session = niquests.Session(source_address=(10.10.4.1, 4444))
+
+It will be passed down the the lower stack. No effort required.