Rename multiplier to frames_per_event and move to first dim of shape

[thomashopkins32]

This PR does the following:

• Renames multiplier -> frames_per_event
• Add the frames_per_event as the first dimension of DataKey.shape
• Ensure that the index provided by DetectorWriter.get_indices_written() and DetectorWriter.observe_indices_written() is divided by frames_per_event so that it actually captures the correct amount of exposures in each index (except for PandA which explicitly says it only has 1 "frame" per event)
• Add unit tests showing that describe() works as intended
• Add unit tests showing that stream resources are actually batches of exposures
• Re-order self._writer.open() and self._writer.get_indices_written(). The writer needs to be opened in order to get the indices written. Otherwise, it has no idea what frames_per_event to use when returning the index last written.

I could not actually add tests using bluesky plans and inspecting the data afterword because TriggerInfo is hardcoded in StandardDetector. I think it is a separate issue that should be raised since it would enhance the scope of this PR. I will open an issue for this soon and mention it below.

Otherwise, I have a few open questions regarding my understanding of ophyd-async as well as the implementation which I will also leave as review comments. Please see below.

Closes #576
Closes #431

[thomashopkins32]

@jwlodek, @jennmald , and I did some testing on actual devices and found a few more issues that need to be resolved. We didn't get through all of the testing we planned for so we will continue next week most likely.

For ophyd-async (completed in a5b1f27) :

• PandA needs to be able to handle frames_per_event > 1. @coretl do you know why this was limited to only being 1 for PandA?
• The computed total_number_of_triggers needs to be multiplied by frames_per_event

For bluesky:

• ConsolidatorBase needs to be reworked based on the new assumption that frames_per_event is the first dim of datum_shape.

For tiled:

• TBD

That covers pretty much everything that we tested a debugged on the devices so far. We will see if changes to tiled are necessary in further testing.

[coretl]

• PandA needs to be able to handle frames_per_event > 1. @coretl do you know why this was limited to only being 1 for PandA?

I'm not sure, I don't think that's a real restriction. If you remove it, what breaks?

[jwlodek]

• PandA needs to be able to handle frames_per_event > 1. @coretl do you know why this was limited to only being 1 for PandA?

I'm not sure, I don't think that's a real restriction. If you remove it, what breaks?

Nothing actually, we removed it and got everything to work as expected, just into separate streams. We're going to make sure they can fit into the same stream this week

[jwlodek]

We've now tested this and it is working for us, pending the Consolidator PR.

[thomashopkins32]

Actually we decided that it does not work well with tiled just yet. We want tiled to have the frames_per_event explicitly in the shape which is causing issues with reading the data back from the files (due to how chunking works).

Basically, ophyd-async has the descriptor shape with the first dim as frames_per_event. bluesky's consolidators uses this to figure out the proper chunking of the data prior to writing it to the hdf5 file. Then tiled needs to also understand this chunking in order to read the data back from the file and unpack it properly.

The shape of the data from the user perspective should always be (num_events, frames_per_event, ...)

[jwlodek]

We tested this again, and we think we are happy to merge this pending the sister PR into bluesky being ready / tested as well.

[coretl]

Either we rename number_of_triggers to number_of_events and this logic is valid, or we remove the * self.frames_per_event.
At the moment we have:

• number_of_triggers
• livetime and deadtime of each trigger
• multiplier (which is effectively triggers_per_event)

We could move to:

• number_of_events
• livetime and deadtime of each frame
• frames_per_event

@jwlodek do you have an opinion?

[jwlodek]

I think the naming you suggest makes sense. number_of_events is a more accurate way of describing what this value is. I agree that livetime/deadtime should be on a per-frame basis, that's how most people will interpret it.

[thomashopkins32]

I think it should be:

TriggerInfo.number_of_triggers -> TriggerInfo.number_of_events
TriggerInfo.frames_per_event -> TriggerInfo.triggers_per_event

This would make the total_number_of_triggers property meaningful. The word "frame" is really only relevant to detectors that capture 2-D data.

Then we can add that the word "event" here relates to a single StreamDatum index being emitted.

[coretl]

I'm happy with that

[thomashopkins32]

@coretl @jwlodek

This might seem like overkill but please help me check my understanding of the terms. Here is what I understand:

Level	Term	Meaning	Example
EPICS (or some other control system)	"capture"	One single data point collected from a detector	A singular frame is collected from an area detector
ophyd-async	"trigger"	One or many "captures" that returns a status	5 frames are collected from an area detector
bluesky & beyond	"event"	Occurrence of a single "trigger" that finishes with successful status	5 frames are collected and a document is emitted

The TriggerInfo schema is actually pretty confusing when you try to do the following:

trigger_info = TriggerInfo(number_of_triggers=4, frames_per_event=2)
RE(bps.prepare(det, trigger_info))
RE(bp.count([det], num=10))

This example currently fails to run without error.

Is this something that should be possible? I can't even tell what the expected behavior here should be...

This may be an unintended consequence of doing a step scan versus a fly scan but I need to dig a bit further.

Either way, if there is a real issue here, it should probably be handled in a separate PR.

[coretl]

This might seem like overkill but please help me check my understanding of the terms.

My take:

Level	Term	Meaning	Example
EPICS (or some other control system)	"exposure"	One single data point collected from a detector	A singular frame is collected from an area detector
ophyd-async	"trigger"	One software or hardware trigger that is sent to the detector to trigger an exposure (with the given livetime)	A singular frame is collected from an area detector
bluesky & beyond	"event"	An atomic row of data in a stream within a scan, synchronous data points from multiple detectors	5 frames are collected from one detector as a single stream datum index, 1 frame from another as a single stream datum index, and a document is emitted with the motor position at this scan point

The TriggerInfo schema is actually pretty confusing when you try to do the following:

trigger_info = TriggerInfo(number_of_triggers=4, frames_per_event=2)
RE(bps.prepare(det, trigger_info))
RE(bp.count([det], num=10))

Definitely. There are 3 terms here:

• number_of_triggers -> number_of_events which is the number of events to produce for each kickoff in a flyscan, it should always be set to 1 for a step scan as each step is an event
• frames_per_event -> triggers_per_event which is the number of triggers and therefore the number of frames the detector will produce in each event
• num argument to count plan which is the number of events to produce

So number_of_events and num are both specifying the same thing, but in a step scan the number has to be controlled from the plan, so number_of_events must equal 1

This example currently fails to run without error.

This should error with something like "Can only trigger() to contribute to a single event, but detector was prepared with number_of_events=4"

[thomashopkins32]

@coretl Okay I understand now. It makes sense that a step scan can only have 1 trigger, by your definition of trigger.

However, the term "trigger" seems overloaded with bluesky's definition in the sense that a StandardDetector is Triggerable and a bluesky trigger happens once per event (in the case of a step scan, not sure of a fly scan). I suggest we use exposures_per_event to replace frames_per_event.

I will do the renaming part in this PR and add checks + expected functionality (with tests) in another PR.

[jwlodek]

I agree with @coretl 's understanding. One thing I thought of however when mulling this over, is that currently we are missing one thing when operating in step scan mode - namely that if I set num_images to say 5 in my AD, and run count with num=3, I'd expect 3 events w/ 5 frames per event, but currently we don't actually have a mechanism for setting frames_per_event in a step scan with trigger - you'd end up with three events of 5 images, but the shape would be incorrect with a frames_per_event of 1.

I think the easiest solution to this would be to add a property to StandardDetector that basically just returns the currently configured frames_per_event for a step scan, with it just returning 1 by default, with the expectation that the implementation should override this. So then for AD based detectors this would get the value of num_images for example, and would set frames_per_event=THE_VALUE_OF_THIS_PROPERTY in the implicit prepare in the trigger method.

Thoughts?

[coretl]

I think the easiest solution to this would be to add a property to StandardDetector that basically just returns the currently configured frames_per_event for a step scan, with it just returning 1 by default, with the expectation that the implementation should override this. So then for AD based detectors this would get the value of num_images for example, and would set frames_per_event=THE_VALUE_OF_THIS_PROPERTY in the implicit prepare in the trigger method.

How about we make frames_per_event optional? If it isn't set then the detector should read num_images and use it as the value of frames_per_event, if it isn't then it should set num_images to be frames_per_event.

I suggest we use exposures_per_event to replace frames_per_event.

I'm not sure whether frames or exposures is better, @jwlodek thoughts?

[thomashopkins32]

How about we make frames_per_event optional? If it isn't set then the detector should read num_images and use it as the value of frames_per_event, if it isn't then it should set num_images to be frames_per_event.

I think this should be handled in a different PR. There is some complexity in doing this. Something like this is already done for the TriggerInfo.deadtime but I'm not sure we want to do it the same way for frames_per_event.

We would basically be doing the following:

frames_per_event <- num_images
num_images <- number_of_events * frames_per_event

So if someone re-uses a TriggerInfo that has frames_per_event=None, then the num_images could increase over-time if number_of_events > 1. I think this user would have to call prepare twice in the same bluesky run though?

[coretl]

I think this should be handled in a different PR. There is some complexity in doing this. Something like this is already done for the TriggerInfo.deadtime but I'm not sure we want to do it the same way for frames_per_event.

Agreed.

We would basically be doing the following:

frames_per_event <- num_images
num_images <- number_of_events * frames_per_event

I think I've got some terminology confused. In my mind we're mapping:

• TriggerInfo.livetime -> ADAcquireTime
• TriggerInfo.livetime + TriggerInfo.deadtime -> ADAcquirePeriod
• TriggerInfo.number_of_events -> ADNumImages
• TriggerInfo.frames_per_event -> ADNumExposures

As defined in https://areadetector.github.io/areaDetector/ADCore/ADDriver.html

All of those fields in TriggerInfo should be optional apart from number_of_events. If there is an optional field, it should take its value from the currently set areaDetector value. number_of_events should always be 1 in a step scan.

So if someone re-uses a TriggerInfo that has frames_per_event=None, then the num_images could increase over-time if number_of_events > 1. I think this user would have to call prepare twice in the same bluesky run though?

I think I made a mistake in the current implementation, the passed in TriggerInfo should be immutable and we make a copy with a different deadtime. That would stop any surprises when a TriggerInfo was reused.

[thomashopkins32]

I think I've got some terminology confused. In my mind we're mapping:

* `TriggerInfo.livetime` -> `ADAcquireTime`

* `TriggerInfo.livetime + TriggerInfo.deadtime` -> `ADAcquirePeriod`

* `TriggerInfo.number_of_events` -> `ADNumImages`

* `TriggerInfo.frames_per_event` -> `ADNumExposures`

As defined in https://areadetector.github.io/areaDetector/ADCore/ADDriver.html

My understanding of the ADDriver is not great, so I will defer to @jwlodek on this one. My understanding was that we want multiple images (or data points) per event. So maybe "exposure" is not the right word and we just use "point" instead.

All of those fields in TriggerInfo should be optional apart from number_of_events. If there is an optional field, it should take its value from the currently set areaDetector value. number_of_events should always be 1 in a step scan.

Agreed! I think this would be great.

I think I made a mistake in the current implementation, the passed in TriggerInfo should be immutable and we make a copy with a different deadtime. That would stop any surprises when a TriggerInfo was reused.

Yes currently it modifies the existing TriggerInfo