uvcmetrics fails with cdscan error

[mcenerney1]

Running the following diags with uvcdat 2.8 gets a cdscan error.

diags --package AMWG --set 8 --var T --seasons ANN --no-antialiasing --model path=/Users/mcenerney1/uvcmetrics/test/data/cam_output,climos=no --obs path=/Users/mcenerney1/uvcmetrics/test/data/obs,filter="f_contains('NCEP')",climos=yes --outputdir /Users/mcenerney1/diagout

UV-CDAT Diagnostics, command-line version
/Users/mcenerney1/anaconda/envs/2.8/bin/diags --package AMWG --set 8 --var T --seasons ANN --no-antialiasing --model path=/Users/mcenerney1/uvcmetrics/test/data/cam_output,climos=no --obs path=/Users/mcenerney1/uvcmetrics/test/data/obs,filter=f_contains('NCEP'),climos=yes --outputdir /Users/mcenerney1/diagout
ERROR: ERROR: cdscan terminated. This is usually fatal. The arguments were:['cdscan', '-q', '-x', '/tmp/mcenerney1/uvcmetrics/NCEP_csafcc95446a2cb4d2d592c51969023f4d.xml', '-e', 'time.units=months since 1979-01-01 00:00:00', '/Users/mcenerney1/uvcmetrics/test/data/obs/NCEP_01_climo.nc', '/Users/mcenerney1/uvcmetrics/test/data/obs/NCEP_02_climo.nc', '/Users/mcenerney1/uvcmetrics/test/data/obs/NCEP_03_climo.nc', '/Users/mcenerney1/uvcmetrics/test/data/obs/NCEP_04_climo.nc', '/Users/mcenerney1/uvcmetrics/test/data/obs/NCEP_05_climo.nc', '/Users/mcenerney1/uvcmetrics/test/data/obs/NCEP_06_climo.nc', '/Users/mcenerney1/uvcmetrics/test/data/obs/NCEP_07_climo.nc', '/Users/mcenerney1/uvcmetrics/test/data/obs/NCEP_08_climo.nc', '/Users/mcenerney1/uvcmetrics/test/data/obs/NCEP_09_climo.nc', '/Users/mcenerney1/uvcmetrics/test/data/obs/NCEP_10_climo.nc', '/Users/mcenerney1/uvcmetrics/test/data/obs/NCEP_11_climo.nc', '/Users/mcenerney1/uvcmetrics/test/data/obs/NCEP_12_climo.nc']

[painter1]

I reproduced the problem with
cdscan ~/tmp/diagout/set8_ANN_T.xml -e time.units=months since 1979-01-01 00:00:00 ~/metrics_data/obs_data_20140804/NCEP_??_climo.nc
But the problem isn't in cdscan. It's in dataset.py or associated files such as CDMLParser.py

The error occurs where cdscan tries to open the output file in read-only mode. I don't know why it does that, but it's a legitimate operation. In openDataset() (dataset.py), this boils down to the simple line
dataset=load(path)
Load() successfully opens the file. It creates and a CDMLParser object and returns its root attribute. Thus that root attribute becomes the variable 'dataset' in openDataset(). Later on in openDataset(), something is done with 'dataset'. The problem is that load() had never done anything to set the root attribute of its CDMLParser object, hence 'dataset' is None, which causes a failure when it is used.

[painter1]

Forget about that last message. I had left out the '-x' which cdscan needs. I'm concerned about the problem with opening an xml file, but that isn't the real cause of this issue.

[mcenerney1]

Note the diags command line calls cdscan as indicated and is the one invoked by running ctest as in
ctest diags_test_08

[mcenerney1]

Do you have an example of diags set 8 or 15 running with 2.8?

[painter1]

Yes, this one from my lotsadiags.sh.  I added the --no-antialiasing to make it more like yours.  It ran on my machine, but I bet that, with appropriate path changes, it won't run on yours: diags --outputdir ~/tmp/diagout/ --no-antialiasing --model path=~/metrics_data/cam35_data/,climos=yes --obs path=/Users/painter1/metrics_data/obs_data_5.6/,filter="f_startswith('NCEP')",climos=yes --package AMWG --set 8 --seasons ANN --vars T I have to admit that I was running that in my multidiags branch.  But I don't think that that fact matters. On 12/8/16 2:26 PM, James McEnerney wrote: Do you have an example of diags set 8 or 15 running with 2.8? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread. {"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/UV-CDAT/uvcdat","title":"UV-CDAT/uvcdat","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/UV-CDAT/uvcdat"}},"updates":{"snippets":[{"icon":"PERSON","message":"@mcenerney1 in #2147: Do you have an example of diags set 8 or 15 running with 2.8?"}],"action":{"name":"View Issue","url":"#2147 (comment)"}}}

[painter1]

On another issue, how does "python setup.py install" work for you with UV-CDAT 2.8?   I have to disable compilation of C code. On 12/8/16 2:26 PM, James McEnerney wrote: Do you have an example of diags set 8 or 15 running with 2.8? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread. {"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/UV-CDAT/uvcdat","title":"UV-CDAT/uvcdat","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/UV-CDAT/uvcdat"}},"updates":{"snippets":[{"icon":"PERSON","message":"@mcenerney1 in #2147: Do you have an example of diags set 8 or 15 running with 2.8?"}],"action":{"name":"View Issue","url":"#2147 (comment)"}}}

[mcenerney1]

The obs data, obs_data_5.6 doesn't have T. Did this run for you?

I haven't had issues with install under 2.8. I almost always use
rm -rf build; python setup.py install

[painter1]

My NCEP obs data has T. I wonder whether you're using a cut-down obs set made for automated tests.

Anyway, cdscan shouldn't crash whether T is in there or not.