Skip to content

Comments

Cythonize get_coinc_indices#5180

Open
spxiwh wants to merge 5 commits intogwastro:masterfrom
spxiwh:pr_cythonize_grb_coinc
Open

Cythonize get_coinc_indices#5180
spxiwh wants to merge 5 commits intogwastro:masterfrom
spxiwh:pr_cythonize_grb_coinc

Conversation

@spxiwh
Copy link
Contributor

@spxiwh spxiwh commented Sep 10, 2025

This continues work in #5148 by Cythonizing the get_coinc_indices function. For now this is still draft, and I put it here to demonstrate how this might work (especially if anyone in the GRB team is interested in learning some Cython, it would be great to have more people with that knowledge).

Particular things that need doing

  • Writing Cython functions for other common use cases
  • Making the one-detector case a no-op if (as I think) it should be
  • Creating a new .pyx file for GRB Cython functions and moving the new functions there (expecting this will not be the only time we Cythonize bits).
  • Testing (I ran Tito's test case 2 and obtained the same number of triggers (within 1)), I've also ran the code on the same test case generating indices with both the old and new methods and set to raise an error if they differ ... It did not fail (after I fixed the bugs that test uncovered).

Standard information about the request

This is a: efficiency update

This change affects: PyGRB

This change changes: Core scientific code, but only optimization.

This change: has appropriate unit tests, follows style guidelines (See e.g. PEP8), has been proposed using the contribution guidelines

Motivation

#5148 has all the motivation. PyGRB can be slow in some cases. This function was written in "nice" python, but not "fast" python.

Contents

For the 2-detector case this is just finding duplicated values in 2 sorted arrays. There's a well known algorithm for that problem, and I implement it here.

Additional notes

image

Profile (for test case 2) with this PR. It's not as great as I would like, but the dominant cost of get_coinc_indexes remains outside of the Cython function (even though it's doing very little!)

  • The author of this pull request confirms they will adhere to the code of conduct

@titodalcanton titodalcanton added the PyGRB PyGRB development label Sep 10, 2025
@github-project-automation github-project-automation bot moved this to In Progress in PyGRB Development Sep 10, 2025
@spxiwh
Copy link
Contributor Author

spxiwh commented Sep 10, 2025

... I also wonder if caching the output array might fix the 10% of time spent in get_coinc_indexes and not in the Cython function??

@titodalcanton
Copy link
Contributor

Please rebase on master, we would like to get this on master soon to include it in the latest analyses.

Copilot AI review requested due to automatic review settings January 29, 2026 09:14
@spxiwh spxiwh force-pushed the pr_cythonize_grb_coinc branch from 334415c to c373940 Compare January 29, 2026 09:14
@spxiwh
Copy link
Contributor Author

spxiwh commented Jan 29, 2026

This has been rebased now. I note that there were a bunch of additional potential things here listed at the top that would be good for the GRB group to look at (especially in terms of getting more people touching our Cython code).

However, the patch will work fine on it's own especially if the 2-detector case is the main place where slowdown is observed.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request optimizes the get_coinc_indexes function by implementing a Cythonized version for the common two-detector, two-coincidence case. This addresses performance issues identified in issue #5148, where PyGRB workflows were experiencing long run times due to inefficient trigger coincidence finding.

Changes:

  • Added a Cythonized implementation using a merge algorithm to find coincident triggers between two sorted detector arrays
  • Introduced a fast path in get_coinc_indexes for the two-detector case when min_nifos == 2
  • Maintained backward compatibility by falling back to the original implementation for other cases

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 8 comments.

File Description
pycbc/events/eventmgr_cython.pyx Adds the Cython implementation get_coinc_indexes_cython_twodet_twocoinc using a fused type to support both int and long arrays, implementing a standard merge algorithm for finding array intersections
pycbc/events/coherent.py Adds a Python wrapper and integrates the fast path into get_coinc_indexes, allocating output arrays and calling the Cython function when applicable

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@spxiwh
Copy link
Contributor Author

spxiwh commented Jan 29, 2026

Thanks copilot. There were some useful comments there (and it's worth reading through how you analysed the code to see how you are checking everything). I've resolved the comments you raised now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

PyGRB PyGRB development

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

2 participants