Skip to content

Fix TypeError for nonsquare linear systems with QR factorization #675

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ChrisRackauckas-Claude
Copy link
Contributor

Summary

Problem

When solving nonsquare linear systems (A*u = b where A is m×n with m≠n), some platforms return QRPivoted with Vector{Int32} for pivot indices from Julia's qr function with ColumnNorm pivot, while LinearSolve's DefaultLinearSolverInit struct expects Vector{Int64}.

Solution

Added a helper function _ensure_int64_pivot that converts QRPivoted pivot vectors from Int32 to Int64 when necessary, ensuring compatibility across all platforms.

Changes

  • Added _ensure_int64_pivot helper function to convert pivot indices to Int64
  • Applied conversion in do_factorization for QRFactorization
  • Applied conversion in all init_cacheval functions that return QRPivoted
  • Ensured PREALLOCATED_QR_ColumnNorm uses Int64 pivot indices

Test Plan

  • Created test case reproducing the original issue from TypeError: in setfield! for nonsquare linear system #674
  • Added test for Int32 to Int64 pivot conversion
  • Tested with various nonsquare matrix sizes (m×n where mn)
  • All new tests pass locally
  • Formatted code with JuliaFormatter using SciMLStyle

Test Results

Test Summary:                                   | Pass  Total  Time
Issue #674 - NonSquare QR Int32/Int64 pivot fix |   13     13  0.2s
All tests passed\!

🤖 Generated with Claude Code

Fixes SciML#674

The issue occurred when solving nonsquare linear systems (A*u = b where A is m×n with m≠n).
On some platforms, Julia's qr function with ColumnNorm pivot returns QRPivoted with
Vector{Int32} for pivot indices, while LinearSolve's DefaultLinearSolverInit struct
expects Vector{Int64}.

This fix adds a helper function _ensure_int64_pivot that converts QRPivoted pivot
vectors from Int32 to Int64 when necessary, ensuring compatibility across all platforms.

Changes:
- Added _ensure_int64_pivot helper function to convert pivot indices to Int64
- Applied conversion in do_factorization for QRFactorization
- Applied conversion in all init_cacheval functions that return QRPivoted
- Ensured PREALLOCATED_QR_ColumnNorm uses Int64 pivot indices

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

TypeError: in setfield! for nonsquare linear system
2 participants