GH-49158: [Python] Allow type conversion in JSON parser when explicit schema is provided

[shashbha14]

Fixes #49158

The issue: when you provide an explicit schema to the JSON parser, it errors if JSON types don't exactly match schema types, even when conversion is straightforward.

For example, if you have:n
{"_id": "152934"}
{"_id": 152934}And your schema says _id should be string, it fails on row 1 with "Column changed from string to number" instead of converting 152934 to "152934".

I fixed this by making the parser attempt type conversion when an explicit schema is provided. Before erroring on a type mismatch, it checks if we have an explicit schema and tries to convert the value to match the expected type.

Changes:

• Store explicit_schema in HandlerBase so we can access it during parsing
• Modified AppendScalar() to try conversion before erroring when explicit schema exists
• Added TryConvertAndAppend() helper that handles the conversion logic
• Updated Bool() handler to also support conversion
• Added tests for number->string and string->number cases

Conversions that work now:

• Number -> String (152934 -> "152934")
• String -> Number (when the string is numeric)
• Boolean conversions to/from string and number
• Number -> Boolean (0 is false, non-zero is true)

This only happens when explicit schema is provided, so it's backward compatible. All existing tests still pass.

Fixes #49158

• GitHub Issue: [Python] PyArrow cannot read from a newline-delimited JSON file with inconsistent column types, even if parse_options specifies a schema #49158

[rok]

Thanks for jumping on this @shashbha14. However please note we might not want to change the json parser right now depending of the outcome of discussion on the issue.

[rok]

Out of curiosity, why is personal_data needed?

[raulcd]

I've seen this ARROW:personal_data field on other PRs too like:

• #48994

This PR seems con contain 6 commits, several of them of unrelated PRs to this one. Please could you remove the unnecessary commits.

[rok]

Since @shashbha14 seems inactive we best close this and leave the issue unassigned for now.

[raulcd]

I agree with you. I've removed the assignee from the issue. Happy to close this PR.

[rok]

@shashbha14 closing this now for inactivity. Feel free to reopen this PR and engage us in review.

[shashbha14]

 I had an initial PR (#49177) for this but it mixed in unrelated commits and the
 JSON parser behavior is still being discussed. I’m happy to revisit this once
 there’s consensus on how coercion with explicit schema should behave.

[rok]

Thanks for replying @shashbha14! Feel free to propose a PR when we have consensus in the discussion.