Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output metadata #162

Closed
wants to merge 29 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
1e227c4
First implementation of metadata file
andreab1997 Nov 10, 2022
7172b48
Fix write_text
andreab1997 Nov 14, 2022
cf7d4c2
Add method to access metadata
andreab1997 Nov 14, 2022
fc43644
Add metadata to detached
andreab1997 Nov 14, 2022
960b681
Add metadata argument
andreab1997 Nov 14, 2022
5925d95
Fix names of objects inside metadata dict
andreab1997 Nov 14, 2022
c43ff92
Drop if in detached
andreab1997 Nov 14, 2022
99ee17f
Add rotations key to metadata and implement function to update metadata
andreab1997 Nov 14, 2022
a85fd19
Update metadata after manipulation
andreab1997 Nov 14, 2022
230b79d
Fix stream.len
andreab1997 Nov 14, 2022
27d2f05
Fix stream.tell()
andreab1997 Nov 14, 2022
61c0abe
Add docs
andreab1997 Nov 15, 2022
d6a922b
Cast to numpy array
andreab1997 Nov 15, 2022
dcb40cb
Move casting to np array in detached
andreab1997 Nov 15, 2022
9a43a41
First draft of solution for metadata file
andreab1997 Nov 15, 2022
76b66c9
Use __post_init__ even when rotations is called
andreab1997 Nov 15, 2022
d1449e9
Fix test_output and remove dump of update_metadata
andreab1997 Nov 16, 2022
fa7eacf
Update pids after to_evol
andreab1997 Nov 16, 2022
b47561e
Fix test_output_struct
andreab1997 Nov 16, 2022
050addd
Move update_metadata after
andreab1997 Nov 16, 2022
c354d56
Store matrices for pids and fix test
andreab1997 Nov 16, 2022
d8386b6
Fix test_runner
andreab1997 Nov 16, 2022
b65dc37
Fix ekobox and tests
andreab1997 Nov 16, 2022
fbf9290
Explicitly cast pids to list in op card generation
andreab1997 Nov 16, 2022
f6dbc2f
Fix test_apply
andreab1997 Nov 16, 2022
174b3f7
Create operator_card class
andreab1997 Nov 29, 2022
34ddcde
Fix benchmark of ekobox
andreab1997 Nov 29, 2022
bfc6c32
Update metadata file after reshape
andreab1997 Nov 29, 2022
f01fc19
Update src/eko/output/struct.py
andreab1997 Dec 1, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update metadata after manipulation
andreab1997 committed Nov 14, 2022
commit a85fd194824bc4f195a4250682dca0b18acdec3f
15 changes: 15 additions & 0 deletions src/eko/output/manipulate.py
Original file line number Diff line number Diff line change
@@ -61,6 +61,8 @@ def xgrid_reshape(
)
target_rot = b.get_interpolation(targetgrid.raw)
eko.rotations._targetgrid = targetgrid
# update metadata
eko.update_metadata({"rotations": {"targetgrid": targetgrid}})
if inputgrid is not None:
b = interpolation.InterpolatorDispatcher(
inputgrid,
@@ -69,6 +71,8 @@ def xgrid_reshape(
)
input_rot = b.get_interpolation(eko.rotations.inputgrid.raw)
eko.rotations._inputgrid = inputgrid
# update metadata
eko.update_metadata({"rotations": {"inputgrid": inputgrid}})

# build new grid
for q2, elem in eko.items():
@@ -134,12 +138,19 @@ def flavor_reshape(
ops = elem.operator
errs = elem.error
if targetpids is not None and inputpids is None:
# update metadata
eko.update_metadata({"rotations": {"targetpids": targetpids}})
ops = np.einsum("ca,ajbk->cjbk", targetpids, ops)
errs = np.einsum("ca,ajbk->cjbk", targetpids, errs)
elif inputpids is not None and targetpids is None:
# update metadata
eko.update_metadata({"rotations": {"inputpids": inputpids}})
ops = np.einsum("ajbk,bd->ajdk", ops, inv_inputpids)
errs = np.einsum("ajbk,bd->ajdk", errs, inv_inputpids)
else:
# update metadata
eko.update_metadata({"rotations": {"inputpids": inputpids}})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the call should move down to were the write is happening (else we get out of sync) .. note that we're casting to list (see comment below )

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This made sense: we want operations to be as atomic as possible (in the ACID sense)

eko.update_metadata({"rotations": {"targetpids": targetpids}})
ops = np.einsum("ca,ajbk,bd->cjdk", targetpids, ops, inv_inputpids)
errs = np.einsum("ca,ajbk,bd->cjdk", targetpids, errs, inv_inputpids)
elem.operator = ops
@@ -175,5 +186,9 @@ def to_evol(eko: EKO, source: bool = True, target: bool = False):
# assign pids
if source:
eko.rotations._inputpids = br.evol_basis_pids
# update metadata
eko.update_metadata({"rotations": {"inputpids": inputpids}})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is in contradiction with the line above ... actually the problem is more complicated: because for to_evol we were putting the saved inputpids back to a list - instead the inputpids for for flavor_reshape is a matrix as it should. I think we can retain that behaviour but let's think for a moment ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this point I believe that in any case rotations.inputpids and rotations.targetpids should be matrices, not only for to_evol. The question is, do we want to store the matrix itself inside the file metadata or do we want to store the vector and then internally change the object (for example rotations.targetpids)? In the second case we can add something in __post__init__: Since we have both the vectors rotations.pids and rotations.targetpids as readed from metadata we can just compute the matrix and assign it to rotations.targetgrid in place of the vector. In this way, we will only store vectors, which I believe are more clear to read for an user, but then internally we have the matrices we need. Do you agree?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we discussed yesterday, we want the matrix: it is uniform, also for non-standard bases, and always reliable.
The less special cases we have, the easier will be the code, and maintainability will improve.

if target:
eko.rotations._targetpids = br.evol_basis_pids
# update metadata
eko.update_metadata({"rotations": {"targetpids": targetpids}})