Conversation
|
... I also wonder if caching the output array might fix the 10% of time spent in get_coinc_indexes and not in the Cython function?? |
|
Please rebase on master, we would like to get this on master soon to include it in the latest analyses. |
334415c to
c373940
Compare
|
This has been rebased now. I note that there were a bunch of additional potential things here listed at the top that would be good for the GRB group to look at (especially in terms of getting more people touching our Cython code). However, the patch will work fine on it's own especially if the 2-detector case is the main place where slowdown is observed. |
There was a problem hiding this comment.
Pull request overview
This pull request optimizes the get_coinc_indexes function by implementing a Cythonized version for the common two-detector, two-coincidence case. This addresses performance issues identified in issue #5148, where PyGRB workflows were experiencing long run times due to inefficient trigger coincidence finding.
Changes:
- Added a Cythonized implementation using a merge algorithm to find coincident triggers between two sorted detector arrays
- Introduced a fast path in
get_coinc_indexesfor the two-detector case whenmin_nifos == 2 - Maintained backward compatibility by falling back to the original implementation for other cases
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 8 comments.
| File | Description |
|---|---|
| pycbc/events/eventmgr_cython.pyx | Adds the Cython implementation get_coinc_indexes_cython_twodet_twocoinc using a fused type to support both int and long arrays, implementing a standard merge algorithm for finding array intersections |
| pycbc/events/coherent.py | Adds a Python wrapper and integrates the fast path into get_coinc_indexes, allocating output arrays and calling the Cython function when applicable |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
Thanks copilot. There were some useful comments there (and it's worth reading through how you analysed the code to see how you are checking everything). I've resolved the comments you raised now. |
This continues work in #5148 by Cythonizing the get_coinc_indices function. For now this is still draft, and I put it here to demonstrate how this might work (especially if anyone in the GRB team is interested in learning some Cython, it would be great to have more people with that knowledge).
Particular things that need doing
Standard information about the request
This is a: efficiency update
This change affects: PyGRB
This change changes: Core scientific code, but only optimization.
This change: has appropriate unit tests, follows style guidelines (See e.g. PEP8), has been proposed using the contribution guidelines
Motivation
#5148 has all the motivation. PyGRB can be slow in some cases. This function was written in "nice" python, but not "fast" python.
Contents
For the 2-detector case this is just finding duplicated values in 2 sorted arrays. There's a well known algorithm for that problem, and I implement it here.
Additional notes
Profile (for test case 2) with this PR. It's not as great as I would like, but the dominant cost of get_coinc_indexes remains outside of the Cython function (even though it's doing very little!)