Mnt prep for demo by tacaswell · Pull Request #23 · NSLS2/pdf-profile-collection

tacaswell · 2021-02-26T23:42:51Z

Work to get queueserver to start up against the PDF beamline profile.

So far this starts against spoof beamline, but can not run scans yet due to not having a good enough fake shutter.

~/.config/acq/pdf.yml

archive_base_dir_name: .userbeamtimearchive
archive_root_dir: /mnt/data/bnl/xpdacq_special/archive
base_dir: /mnt/data/bnl/xpdacq_special/data
beamline_host_name: [jupiter]
beamline_id: 28-ID-1
blconfig_dir_name: xpdConfig
blconfig_name: xpd_beamline_config.yml
calib_config_name: xpdAcq_calib_info.yml
dark_window: 0.1
facility: NSLS-II
frame_acquire_time: 0.1
glbl_yaml_name: glbl.yml
group: XPD
home_dir_name: user_data
image_field: pe1c_image
owner: PDF
simulation: false
shutter_conf: {close: 0, open: 1}
mask_kwargs:
  edge: 30
  lower_thresh: 0.0
  bs_width: 13
  tri_offset: 13
  v_asym: 0
  alpha: 3.0
dark_field_key : sc_dk_field_uid
det_image_field : pe1_image
exp_broker_name: pdf
outbound_proxy_address: 'xf28id1-ca1:5578'
inbound_proxy_address: 'xf28id1-ca1:5577'
shutter_sleep: 0
diffraction_dets: ['pe1c', 'pe1', 'pe2c', 'pe2','dexela_image', 'dexela']
radiograph_names : ['blackfly_det']
radiogram_dets: ['blackfly_det']
image_fields: ['pe1c_image', 'pe2c_image', 'dexela_image', 'dexela']

tacaswell · 2021-03-03T17:35:52Z

total 0
drwxr-xr-x 4 tcaswell tcaswell 40 Feb 26 16:21 .
drwxr-xr-x 3 tcaswell tcaswell 18 Feb 26 16:21 ..
drwxr-xr-x 7 tcaswell tcaswell 95 Feb 26 16:21 user_data
drwxr-xr-x 2 tcaswell tcaswell 37 Feb 26 16:27 xpdConfig

./user_data:
total 0
drwxr-xr-x 7 tcaswell tcaswell 95 Feb 26 16:21 .
drwxr-xr-x 4 tcaswell tcaswell 40 Feb 26 16:21 ..
drwxr-xr-x 2 tcaswell tcaswell  6 Feb 26 16:21 Import
drwxr-xr-x 3 tcaswell tcaswell 42 Feb 27 16:20 config_base
drwxr-xr-x 2 tcaswell tcaswell  6 Feb 26 16:21 tiff_base
drwxr-xr-x 2 tcaswell tcaswell 20 Feb 26 16:21 userAnalysis
drwxr-xr-x 2 tcaswell tcaswell  6 Feb 26 16:21 userScripts

./user_data/Import:
total 0
drwxr-xr-x 2 tcaswell tcaswell  6 Feb 26 16:21 .
drwxr-xr-x 7 tcaswell tcaswell 95 Feb 26 16:21 ..

./user_data/config_base:
total 4
drwxr-xr-x 3 tcaswell tcaswell 42 Feb 27 16:20 .
drwxr-xr-x 7 tcaswell tcaswell 95 Feb 26 16:21 ..
-rw-r--r-- 1 tcaswell tcaswell 23 Feb 27 16:23 .sample_order.yml
drwxr-xr-x 4 tcaswell tcaswell 71 Feb 26 16:26 yml

./user_data/config_base/yml:
total 12
drwxr-xr-x 4 tcaswell tcaswell   71 Feb 26 16:26 .
drwxr-xr-x 3 tcaswell tcaswell   42 Feb 27 16:20 ..
-rw-r--r-- 1 tcaswell tcaswell   95 Feb 27 17:25 bt_bt.yml
-rw-r--r-- 1 tcaswell tcaswell 4896 Feb 27 17:25 glbl.yml
drwxr-xr-x 2 tcaswell tcaswell    6 Feb 26 16:21 samples
drwxr-xr-x 2 tcaswell tcaswell    6 Feb 26 16:21 scanplans

./user_data/config_base/yml/samples:
total 0
drwxr-xr-x 2 tcaswell tcaswell  6 Feb 26 16:21 .
drwxr-xr-x 4 tcaswell tcaswell 71 Feb 26 16:26 ..

./user_data/config_base/yml/scanplans:
total 0
drwxr-xr-x 2 tcaswell tcaswell  6 Feb 26 16:21 .
drwxr-xr-x 4 tcaswell tcaswell 71 Feb 26 16:26 ..

./user_data/tiff_base:
total 0
drwxr-xr-x 2 tcaswell tcaswell  6 Feb 26 16:21 .
drwxr-xr-x 7 tcaswell tcaswell 95 Feb 26 16:21 ..

./user_data/userAnalysis:
total 4
drwxr-xr-x 2 tcaswell tcaswell   20 Feb 26 16:21 .
drwxr-xr-x 7 tcaswell tcaswell   95 Feb 26 16:21 ..
-rw-r--r-- 1 tcaswell tcaswell 1426 Feb 26 16:21 Ni24.D

./user_data/userScripts:
total 0
drwxr-xr-x 2 tcaswell tcaswell  6 Feb 26 16:21 .
drwxr-xr-x 7 tcaswell tcaswell 95 Feb 26 16:21 ..

./xpdConfig:
total 4
drwxr-xr-x 2 tcaswell tcaswell 37 Feb 26 16:27 .
drwxr-xr-x 4 tcaswell tcaswell 40 Feb 26 16:21 ..
-rw-r--r-- 1 tcaswell tcaswell 58 Feb 26 16:47 xpd_beamline_config.yml

(sys39) /m/d/b/x/data 
✔ 12:31:01 $ cat user_data/config_base/.sample_order.yml
0: bob.yml
1: bill.yml

(sys39) /m/d/b/x/data 
✔ 12:31:40 $ cat user_data/config_base/yml/bt_bt.yml
bt_experimenters: []
bt_piLast: Caswell
bt_safN: '123456'
bt_uid: 76f3b555
bt_wavelength: null

(sys39) /m/d/b/x/data 
✔ 12:32:19 $ cat user_data/config_base/yml/glbl.yml
_dark_dict_list:
- acq_time: 5
  exposure: 5
  timestamp: 1614460063.2800422
  uid: 37d192fb-df2c-44a9-b41b-2a05c9751247
- acq_time: 5
  exposure: 5
  timestamp: 1614460242.3911414
  uid: cf374ae7-2ed5-4e5d-9563-f4841a3759e9
- acq_time: 5
  exposure: 5
  timestamp: 1614460464.3035522
  uid: 5fc90d2e-ef96-4ca7-9207-ae72121923d0
- acq_time: 5
  exposure: !!python/object/apply:numpy.core.multiarray.scalar
  - &id001 !!python/object/apply:numpy.dtype
    args:
    - f8
    - false
    - true
    state: !!python/tuple
    - 3
    - <
    - null
    - null
    - null
    - -1
    - -1
    - 0
  - !!binary |
    AAAAAAAAFEA=
  timestamp: 1614460566.1485345
  uid: 12c2f3ca-b459-4180-a12e-2b2d4b6d5aa8
- acq_time: 5
  exposure: !!python/object/apply:numpy.core.multiarray.scalar
  - *id001
  - !!binary |
    AAAAAAAAFEA=
  timestamp: 1614464054.0055833
  uid: 5bf51ebe-6439-4815-a9bb-0c824fd3afe8
- acq_time: 5
  exposure: !!python/object/apply:numpy.core.multiarray.scalar
  - *id001
  - !!binary |
    AAAAAAAAFEA=
  timestamp: 1614464263.5876548
  uid: 25d95820-b20f-4d37-901d-6c96c86462a1
- acq_time: 5
  exposure: !!python/object/apply:numpy.core.multiarray.scalar
  - *id001
  - !!binary |
    AAAAAAAAFEA=
  timestamp: 1614464433.3411603
  uid: bb969c1c-2e41-4914-91db-3440b576c752
- acq_time: 5
  exposure: !!python/object/apply:numpy.core.multiarray.scalar
  - *id001
  - !!binary |
    AAAAAAAAFEA=
  timestamp: 1614464443.6444564
  uid: 319ccd2f-7483-459c-96ac-e308b150006e
_exclude_dir:
- /mnt/data/bnl/xpdacq_special/data/user_data
- /mnt/data/bnl/xpdacq_special/data/xpdConfig
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml
_export_tar_dir:
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base
- /mnt/data/bnl/xpdacq_special/data/user_data/userScripts
all_folders: &id002
- /mnt/data/bnl/xpdacq_special/data/user_data
- /mnt/data/bnl/xpdacq_special/data/xpdConfig
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml/samples
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml/scanplans
- /mnt/data/bnl/xpdacq_special/data/user_data/tiff_base
- /mnt/data/bnl/xpdacq_special/data/user_data/userScripts
- /mnt/data/bnl/xpdacq_special/data/user_data/Import
- /mnt/data/bnl/xpdacq_special/data/user_data/userAnalysis
allfolders: *id002
archive_base_dir: /mnt/data/bnl/xpdacq_special/archive/.userbeamtimearchive
archive_base_dir_name: .userbeamtimearchive
archive_dir: /mnt/data/bnl/xpdacq_special/archive/.userbeamtimearchive/2021
archive_root_dir: /mnt/data/bnl/xpdacq_special/archive
auto_dark: true
auto_load_calib: true
base: /mnt/data/bnl/xpdacq_special/data
base_dir: /mnt/data/bnl/xpdacq_special/data
beamline_host_name:
- jupiter
beamline_id: 28-ID-1
blconfig_dir: /mnt/data/bnl/xpdacq_special/data/xpdConfig
blconfig_dir_name: xpdConfig
blconfig_name: xpd_beamline_config.yml
blconfig_path: /mnt/data/bnl/xpdacq_special/data/xpdConfig/xpd_beamline_config.yml
bt_dir: /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml
calib_config_name: xpdAcq_calib_info.yml
config_base: /mnt/data/bnl/xpdacq_special/data/user_data/config_base
dark_field_key: sc_dk_field_uid
dark_window: 0.1
det_image_field: &id003
- pe1c_image
- pe2c_image
- dexela_image
- dexela
diffraction_dets:
- pe1c
- pe1
- pe2c
- pe2
- dexela_image
- dexela
dk_window: 0.1
exp_broker_name: pdf
exp_hash_uid: f4be92ca-79c1-49ae-9bcd-59c7a8185aa3
facility: NSLS-II
frame_acq_time: 0.1
frame_acquire_time: 0.1
glbl_yaml_name: glbl.yml
glbl_yaml_path: /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml/glbl.yml
group: XPD
home: /mnt/data/bnl/xpdacq_special/data/user_data
home_dir: /mnt/data/bnl/xpdacq_special/data/user_data
home_dir_name: user_data
image_field: pe1c_image
image_fields: *id003
import_dir: /mnt/data/bnl/xpdacq_special/data/user_data/Import
inbound_proxy_address: xf28id1-ca1:5577
is_simulation: false
mask_kwargs:
  alpha: 3.0
  bs_width: 13
  edge: 30
  lower_thresh: 0.0
  tri_offset: 13
  v_asym: 0
name: glbl
outbound_proxy_address: xf28id1-ca1:5578
owner: PDF
radiogram_dets:
- blackfly_det
radiograph_names:
- blackfly_det
sample_dir: /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml/samples
scanplan_dir: /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml/scanplans
shutter_conf:
  close: 0
  open: 1
shutter_control: true
shutter_sleep: 0
simulation: false
tiff_base: /mnt/data/bnl/xpdacq_special/data/user_data/tiff_base
user_backup_dir_name: '2021'
userscript_dir: /mnt/data/bnl/xpdacq_special/data/user_data/userScripts
usrAnalysis_dir: /mnt/data/bnl/xpdacq_special/data/user_data/userAnalysis
usrScript_dir: /mnt/data/bnl/xpdacq_special/data/user_data/userScripts
xpdconfig: /mnt/data/bnl/xpdacq_special/data/xpdConfig
yaml_dir: /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml

(sys39) /m/d/b/x/data 
✔ 12:32:52 $ cat xpdConfig/xpd_beamline_config.yml
Verification time: '2021-02-26 16:47:03'
Verified by: TAC

✔ 12:33:57 $ cat user_data/userAnalysis/Ni24.D
#!/usr/bin/env python
##############################################################################
#
# xpdacq            by Billinge Group
#                   Simon J. L. Billinge sb2896@columbia.edu
#                   (c) 2016 trustees of Columbia University in the City of
#                        New York.
#                   All rights reserved
#
# File coded by:    Soham Banerjee
#
# See AUTHORS.txt for a list of people who contributed.
# See LICENSE.txt for license information.
#
##############################################################################
# Calibrant: Nickel (Ni)
# Face centered cubic cell a=3.5239 b=3.5239 c=3.5239 alpha=90.000 beta=90.000 gamma=90.000
2.034582348620  # (1, 1, 1) 8
1.761935000000  # (2, 0, 0) 6
1.245922148450  # (2, 2, 0) 12
1.062525978290  # (3, 1, 1) 24
1.017291174310  # (2, 2, 2) 8
0.881000000000  # (4, 0, 0) 6
0.808461046160  # (3, 3, 1) 24
0.787990355271  # (4, 2, 0) 24
0.719333487797  # (4, 2, 2) 24
0.678194116208  # (3, 3, 3) 8
0.622961074225  # (5, 1, 1) 24
0.595664718733  # (4, 4, 0) 12
0.587333333333  # (5, 3, 1) 48
0.557193323722  # (4, 4, 2) 24
0.537404961852  # (6, 0, 0) 6
0.531262989146  # (6, 2, 0) 24
0.508645587156  # (5, 3, 3) 24
0.493458701611  # (6, 2, 2) 24
0.488690872874  # (4, 4, 4) 8
0.470914308250  # (5, 5, 1) 24
0.458785722296  # (7, 1, 1) 24
0.440500000000  # (6, 4, 0) 24
0.430525121912  # (6, 4, 2) 48
0.427347771314  # (5, 5, 3) 24

tacaswell · 2021-03-19T14:08:48Z

rebased on #24

mrakitin · 2021-03-19T14:37:16Z

@tacaswell, is there something missing?

[E 2021-03-19 14:15:18,504.504 nslsii.ipython  logutils:56] 'NoneType' object has no attribute 'md'
Traceback (most recent call last):
  File "/home/vsts/miniconda/envs/collection-2021-1.0/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3418, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-1-19fca0ca65e8>", line 13, in <module>
    ip.parent._exec_file(f)
  File "/home/vsts/miniconda/envs/collection-2021-1.0/lib/python3.7/site-packages/IPython/core/shellapp.py", line 380, in _exec_file
    raise_exceptions=True)
  File "/home/vsts/miniconda/envs/collection-2021-1.0/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2742, in safe_execfile
    self.compile if shell_futures else None)
  File "/home/vsts/miniconda/envs/collection-2021-1.0/lib/python3.7/site-packages/IPython/utils/py3compat.py", line 168, in execfile
    exec(compiler(f.read(), fname, 'exec'), glob, loc)
  File "/home/vsts/work/1/s/startup/94-load.py", line 107, in <module>
    RE.beamtime = bt
  File "/home/vsts/miniconda/envs/collection-2021-1.0/lib/python3.7/site-packages/xpdacq/xpdacq.py", line 161, in beamtime
    self.md.update(bt_obj.md)
AttributeError: 'NoneType' object has no attribute 'md'
==============================================================

Assume pe1c is the detector we are using

mrakitin assigned tacaswell, MikeHart85, mrakitin and dmgav Mar 3, 2021

mrakitin mentioned this pull request Mar 3, 2021

5LS Demo NSLS-II/project-management#39

Closed

17 tasks

AbbyGi mentioned this pull request Mar 9, 2021

Update CI #24

Merged

tacaswell added 18 commits March 19, 2021 10:07

MNT: avoid bare except

21c7868

MNT: lift missing value from _py file

669832d

MNT: add missing import to keep the linter happy

ad96016

MNT: put back in an RE API compatibly wrapper of xrun

172036d

MNT: files for queueserver startup

da32e93

ENH: add plan that looks up sample metadata from xpdacq

ace7731

ENH: update and curate published plans

fbabb22

rapid development of sorensen power supply

0ae9e28

ENH: add caproto IOC for ramp plan

64dfab5

tweak controlled ramp plan

b50c7d4

MNT: make writing the CSV optional

cf77222

FIX: the grid objects are in globals not locals

bb6a421

BUG: add ability to optionally write out a CSV of temperatures

4b937bf

Collection of on-the-fly changes to thermo control

5daa121

formatting + changes db insertion

db73d88

add flag to IOC to control xrd collection

ff3ac6b

HACK: overnight script to stop experiment after it melts

21860b9

ignore vscode files

2934cd8

tacaswell force-pushed the mnt_prep_for_demo branch from ca0accb to 2934cd8 Compare March 19, 2021 14:08

tacaswell added 2 commits April 7, 2021 13:38

remove suspenders

7473d51

Add plans for demo

b3fff96

tacaswell added 5 commits April 7, 2021 13:38

Regenerate list of known plans + use existing filtering

d8ebcde

STY: fix indentation of comments

bc08329

ENH: skeletons of the plans we need for the demo

e30ad53

FIX: make positions actually optional

daa3f2c

MNT: simplify signature of sample_aware_count

755714f

Assume pe1c is the detector we are using

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mnt prep for demo#23

Mnt prep for demo#23
tacaswell wants to merge 25 commits intodeprecated-masterfrom
mnt_prep_for_demo

tacaswell commented Feb 26, 2021

Uh oh!

tacaswell commented Mar 3, 2021

Uh oh!

tacaswell commented Mar 19, 2021

Uh oh!

mrakitin commented Mar 19, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

tacaswell commented Feb 26, 2021

Uh oh!

tacaswell commented Mar 3, 2021

Uh oh!

tacaswell commented Mar 19, 2021

Uh oh!

mrakitin commented Mar 19, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants