TYP: Enhance static typing

[n-takumasa]

Fixes # NA

Summary/Motivation:

To provide better autocompletion in vscode+pylance.
Complex dynamic patterns (e.g. Var() can return ScalarVar or IndexedVar) are out of scope as they are difficult to model accurately.

Changes proposed in this PR:

• TYP: Use typing.overload during TYPE_CHECKING
• TYP: Model.__new__ should return an instance of the subclass
• TYP: better typing for SolverFactorys and pyomo.future.solver_factory
• TYP: reorder @overloads, overlapped overloads never be used

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

1. I agree my contributions are submitted under the BSD license.
2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

[jsiirola]

Thanks for this! I have several questions (see below). Some are less about this PR specifically, and more about generally how we want to better support typing in Pyomo. It would be useful to talk through that sometime in a Tuesday Dev Call.

[jsiirola]

Why do you need to disable the overload?

[n-takumasa]

A function calling typing.overload is not recognized as an overload by the type checker.
We need to replace it with typing.overload during type checking.

[jsiirola]

OK. I went digging and it turns out that logic is primarily needed by a dependent project -- but only for Python versions through 3.10. I think we should actually document that better in the code, with something like:

if sys.version_info[:2] <= (3, 10) and not TYPE_CHECKING:
    def overload(func: typing.Callable):
        """Wrap typing.overload that remembers the overloaded signatures

        This provides a custom implementation of typing.overload that
        remembers the overloaded signatures so that they are available for
        runtime inspection (backporting `get_overloads` from Python 3.11+).

        """
        _overloads.setdefault(_get_fullqual_name(func), []).append(func)
        return typing.overload(func)

    def get_overloads_for(func: typing.Callable):
        return _overloads.get(_get_fullqual_name(func), [])
else:
    from typing import overload, get_overloads as get_overloads_for

[n-takumasa]

typing_extensions might be helpful.

[jsiirola]

We haven't really settled on a typing convention for naming types, but appending T seems confusing.

• would standardizing on appendint Type be more clear?
• should local TypeVar objects be private by default (i.e., _modelType here)?

[n-takumasa]

_ModelType is better.

[jsiirola]

Agreed.

[jsiirola]

We shouldn't need these annotations, should we?

[n-takumasa]

Because we cannot add attributes to FunctionType.

[jsiirola]

You can't annotate the function, plus the attribute whereit is first instantiated at the bottom of the file?

def solver_factory(version: int | None = None) -> int:
# ...
solver_factory._active_version: int = solver_factory()

[n-takumasa]

It is allowed at runtime, but it's a type violation.
ref: microsoft/pyright#8838

[jsiirola]

The Python documentation indicates that you should only use the Ellipsis notation in .pyi files. I think this should be:

Suggested change

	@overload
	def __call__(self, _name: None = None, **kwds) -> "SolverFactoryClass": ...
	@overload
	def __call__(self, _name, **kwds) -> "OptSolver": ...
	@overload
	def __call__(self, _name: None = None, **kwds) -> "SolverFactoryClass":
	pass
	@overload
	def __call__(self, _name, **kwds) -> "OptSolver":
	pass

[n-takumasa]

I do not believe the documentation states that you must not use Ellipsis in .py files.
In fact, it is uncommon to see pass used as the body for an @overloaded function.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.39%. Comparing base (74cd43e) to head (58ec35d).
⚠️ Report is 103 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3771      +/-   ##
==========================================
+ Coverage   89.23%   89.39%   +0.16%     
==========================================
  Files         907      907              
  Lines      104807   104815       +8     
==========================================
+ Hits        93522    93697     +175     
+ Misses      11285    11118     -167     

Flag	Coverage Δ
builders	29.16% <50.00%> (+<0.01%)	⬆️
default	85.98% <50.00%> (?)
expensive	35.86% <50.00%> (?)
linux	86.69% <100.00%> (-2.31%)	⬇️
linux_other	86.69% <100.00%> (+<0.01%)	⬆️
osx	82.82% <100.00%> (+<0.01%)	⬆️
win	84.94% <100.00%> (+<0.01%)	⬆️
win_other	84.94% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.