Skip to content

tests/test_incremental_pca.py::test_whitening[auto] failing #1009

@TomAugspurger

Description

@TomAugspurger
❯ pytest -vs tests/test_incremental_pca.py::test_whitening[auto]                                                                                                                                                                                                                                                                                                                                            (base) 
=============================================================================================================================================================================================== test session starts ===============================================================================================================================================================================================
platform darwin -- Python 3.12.8, pytest-8.3.4, pluggy-1.5.0 -- /Users/toaugspurger/gh/dask/.direnv/python-3.12/bin/python3
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase(PosixPath('/Users/toaugspurger/gh/dask/dask-ml/.hypothesis/examples'))
rootdir: /Users/toaugspurger/gh/dask/dask-ml
configfile: pyproject.toml
plugins: cov-6.0.0, hypothesis-6.124.7, anyio-4.8.0, rerunfailures-15.0, timeout-2.3.1, mock-3.14.0, xdist-3.6.1
collected 1 item                                                                                                                                                                                                                                                                                                                                                                                                  

tests/test_incremental_pca.py::test_whitening[auto] FAILED

==================================================================================================================================================================================================== FAILURES =====================================================================================================================================================================================================
______________________________________________________________________________________________________________________________________________________________________________________________ test_whitening[auto] _______________________________________________________________________________________________________________________________________________________________________________________________

svd_solver = 'auto'

    @pytest.mark.parametrize("svd_solver", ["full", "auto", "randomized"])
    @pytest.mark.filterwarnings("ignore:invalid value:RuntimeWarning")
    def test_whitening(svd_solver):
        # Test that PCA and IncrementalPCA transforms match to sign flip.
        X = datasets.make_low_rank_matrix(
            1000, 10, tail_strength=0.0, effective_rank=2, random_state=1999
        )
        X = da.from_array(X, chunks=[200, -1])
        prec = 3
        n_samples, n_features = X.shape
        for nc in [None, 9]:
            pca = PCA(whiten=True, n_components=nc, svd_solver=svd_solver).fit(X.compute())
            ipca = IncrementalPCA(
                whiten=True, n_components=nc, batch_size=250, svd_solver=svd_solver
            ).fit(X)
    
            Xt_pca = pca.transform(X)
            Xt_ipca = ipca.transform(X)
>           assert_almost_equal(np.abs(Xt_pca), np.abs(Xt_ipca), decimal=prec)

tests/test_incremental_pca.py:454: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../miniforge3/lib/python3.12/contextlib.py:81: in inner
    return func(*args, **kwds)
../../../miniforge3/lib/python3.12/contextlib.py:81: in inner
    return func(*args, **kwds)
../.direnv/python-3.12/lib/python3.12/site-packages/numpy/_utils/__init__.py:85: in wrapper
    return fun(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

args = (<function assert_array_almost_equal.<locals>.compare at 0x131d1ac00>, array([[0.34637469, 0.64285423, 1.28803667, ......    0.13500149]]), dask.array<absolute, shape=(1000, 10), dtype=float64, chunksize=(200, 10), chunktype=numpy.ndarray>), kwds = {'err_msg': '', 'header': 'Arrays are not almost equal to 3 decimals', 'precision': 3, 'verbose': True}

    @wraps(func)
    def inner(*args, **kwds):
        with self._recreate_cm():
>           return func(*args, **kwds)
E           AssertionError: 
E           Arrays are not almost equal to 3 decimals
E           
E           Mismatched elements: 1510 / 10000 (15.1%)
E           Max absolute difference among violations: 3.30337443
E           Max relative difference among violations: 17.32129949
E            ACTUAL: array([[0.346, 0.643, 1.288, ..., 0.853, 0.466, 0.18 ],
E                  [0.092, 0.656, 1.029, ..., 0.886, 0.37 , 0.011],
E                  [0.092, 1.329, 1.784, ..., 0.104, 0.396, 0.077],...
E            DESIRED: array([[0.346, 0.643, 1.288, ..., 0.853, 0.464, 1.238],
E                  [0.092, 0.656, 1.029, ..., 0.886, 0.369, 0.19 ],
E                  [0.092, 1.329, 1.784, ..., 0.104, 0.395, 0.606],...

../../../miniforge3/lib/python3.12/contextlib.py:81: AssertionError
============================================================================================================================================================================================= short test summary info =============================================================================================================================================================================================
FAILED tests/test_incremental_pca.py::test_whitening[auto] - AssertionError: 
================================================================================================================================================================================================ 1 failed in 1.02s ================================================================================================================================================================================================

xref scikit-learn/scikit-learn#29534

Skipping for now.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions