Expose --num-workers and --native-parser (#21387)

Three things here:
* Add the docs for `--num-workers` and `--native-parser`
* Remove `argparse.SUPPRESS` from them
* Add couple checks for `num_workers`

Author: ilevkivskyi

docs/source/command_line.rst

@@ -1032,6 +1032,40 @@ beyond what incremental mode can offer, try running mypy in
     Skip cache internal consistency checks based on mtime.
 
 
+.. _parallel:
+
+Parallel type-checking
+**********************
+
+By default, mypy checks all modules in the same Python process. This can be slow
+for large code bases. Mypy offers experimental parallel type-checking mode using
+multiple worker processes. In parallel mode, modules that do not depend om each
+other are type-checked in parallel. :ref:`Incremental cache <incremental>` is
+used to manage most of the shared state. Parallel type-checking also requires
+:option:`--local-partial-types <mypy --no-local-partial-types>`, which is
+enabled by default starting from mypy 2.0.
+
+.. option:: -n NUMBER, --num-workers NUMBER
+
+    Use ``NUMBER`` parallel worker processes (in addition to the coordinator
+    process) to perform type-checking. Specifying ``--num-workers 0`` (default)
+    disables parallel checking. Automatic detection of the optimal number
+    of workers is not supported yet.
+
+Notes:
+
+* An import cycle is always processed as a whole by a worker process. Thus,
+  avoiding large import cycles in your code will *significantly* improve
+  type-checking speed.
+
+* Specifying a number of workers that is larger than the number of *physical*
+  CPU cores is not beneficial, since mypy is usually CPU bound. Best way to
+  tune the number of workers on a given machine is to start from 3-4 workers
+  and increase the number while you see a performance improvement.
+
+* Parallel mode requires and automatically enables :option:`--native-parser`.
+
+
 Advanced options
 ****************
 
@@ -1105,6 +1139,11 @@ in developing or debugging mypy internals.
     cause mypy to type check the contents of ``temp.py`` instead of  ``original.py``,
     but error messages will still reference ``original.py``.
 
+.. option:: --native-parser
+
+    This enables fast Rust-based parser that parses directly to mypy AST.
+    It will become the default parser in one of the next mypy releases.
+
 
 Report generation
 *****************

docs/source/config_file.rst

@@ -1006,6 +1006,20 @@ These options may only be set in the global section (``[mypy]``).
     Skip cache internal consistency checks based on mtime.
 
 
+Parallel type-checking configuration
+************************************
+
+These options may only be set in the global section (``[mypy]``).
+
+.. confval:: num_workers
+
+    :type: integer
+    :default: 0
+
+    Use specific number of parallel worker processes for type-checking, see
+    :ref:`parallel type-checking <parallel>` for more details.
+
+
 Advanced options
 ****************
 
@@ -1067,6 +1081,14 @@ These options may only be set in the global section (``[mypy]``).
     Warns about missing type annotations in typeshed.  This is only relevant
     in combination with :confval:`disallow_untyped_defs` or :confval:`disallow_incomplete_defs`.
 
+.. confval:: native_parser
+
+    :type: boolean
+    :default: False
+
+    This enables fast Rust-based parser that parses directly to mypy AST.
+    It will become the default parser in one of the next mypy releases.
+
 
 Report generation
 *****************

mypy/main.py

@@ -100,8 +100,12 @@ def main(
     if options.num_workers:
         # Supporting both parsers would be really tricky, so just support the new one.
         options.native_parser = True
+        if options.num_workers < 0:
+            fail("error: Number of workers cannot be negative", stderr, options)
         if options.cache_dir == os.devnull:
             fail("error: Cache must be enabled in parallel mode", stderr, options)
+        if not options.local_partial_types:
+            fail("error: --local-partial-types must be enabled in parallel mode", stderr, options)
         if options.report_dirs:
             fail(
                 "error: Reports are not supported in parallel mode yet\n"
@@ -1175,7 +1179,11 @@ def add_invertible_flag(
 
     # Experimental parallel type-checking support.
     internals_group.add_argument(
-        "-n", "--num-workers", type=int, default=0, help=argparse.SUPPRESS
+        "-n",
+        "--num-workers",
+        type=int,
+        default=0,
+        help="Number of separate mypy worker processes (experimental)",
     )
 
     report_group = parser.add_argument_group(
@@ -1288,7 +1296,11 @@ def add_invertible_flag(
         help=argparse.SUPPRESS,
     )
     # --native-parser enables the native parser (experimental)
-    add_invertible_flag("--native-parser", default=False, help=argparse.SUPPRESS)
+    add_invertible_flag(
+        "--native-parser",
+        default=False,
+        help="Enable faster parser that parses directly to mypy AST",
+    )
     # --logical-deps adds some more dependencies that are not semantically needed, but
     # may be helpful to determine relative importance of classes and functions for overall
     # type precision in a code base. It also _removes_ some deps, so this flag should be never