fix(spark): improve port-forward stability for local development

[KartikSuryavanshi]

What this PR does / why we need it:

Fixes #492 - Spark Connect local example fails during SDK-managed port-forward.

The SDK-managed port-forward fails in local/Kind cluster environments because:

1. Pod-based port-forward via kube-apiserver proxy is unstable (drops connections)
2. Service name guessing may not match actual service names
3. No stabilization delay after restart causes TCP port conflicts
4. Missing stderr logging makes debugging impossible

Changes:

1. Reorder port-forward candidates (backend.py): Prefer service-based port-forward (via kube-proxy, more stable) over pod-based (via kube-apiserver proxy, unstable in Kind clusters). Try info.service_name first, then {name}-server, then {name}-svc, with pod as fallback.

2. Add --address 127.0.0.1 (backend.py): Force IPv4-only binding on kubectl port-forward to prevent IPv6 localhost resolution conflicts on WSL and other platforms.

3. Add stabilization delay (backend.py): Add 2s delay after port-forward restart to allow TCP port release before reconnecting.

4. Log stderr on death (backend.py): Read and log stderr from dead port-forward process for better diagnostics.

5. Fix test mock (backend_test.py): Add service_name to mock response and add service-first ordering assertion.

Validation:

• All 263 Spark unit tests pass
• ruff check and ruff format pass cleanly

Fixes #492

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign andreyvelich for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[github-actions]

🎉 Welcome to the Kubeflow SDK! 🎉

Thanks for opening your first PR! We're happy to have you as part of our community 🚀

Here's what happens next:

• If you haven't already, please check out our Contributing Guide for repo-specific guidelines and the Kubeflow Contributor Guide for general community standards
• Our team will review your PR soon! cc @kubeflow/kubeflow-sdk-team

Join the community:

• Slack: Join our #kubeflow-ml-experience and #kubeflow-trainer Slack channels
• Meetings: Attend the Kubeflow SDK and ML Experience bi-weekly meetings

Feel free to ask questions in the comments if you need any help or clarification!
Thanks again for contributing to Kubeflow! 🙏