hissw + CHIANTI not working in Jupyter notebook?

[jwreep]

The following code snippet works fine in a terminal, but not a Jupyter notebook. Is there anything special about Jupyter that would prevent hissw from working properly?

>>> import hissw
>>> env = hissw.Environment(ssw_packages=['chianti'])
>>> script="""
... print,max_temp('Fe X')
... """
>>> env.run(script)

Error in Jupyter:

---------------------------------------------------------------------------
SSWIDLError                               Traceback (most recent call last)
Cell In[23], line 6
      2 env = hissw.Environment(ssw_packages=['chianti'])
      3 script="""
      4 print,max_temp('Fe X')
      5 """
----> 6 env.run(script)

File ~/anaconda3/lib/python3.11/site-packages/hissw/environment.py:185, in Environment.run(self, script, args, save_vars, verbose, **kwargs)
    182     subprocess.call(['chmod', 'u+x', shell_filename])
    183     cmd_output = subprocess.run([shell_filename], shell=True, stderr=subprocess.PIPE,
    184                                 stdout=subprocess.PIPE)
--> 185     self._check_for_errors(cmd_output, verbose, **kwargs)
    186     results = readsav(save_filename)
    188 return results

File ~/anaconda3/lib/python3.11/site-packages/hissw/environment.py:200, in Environment._check_for_errors(self, output, verbose, **kwargs)
    198 if kwargs.get('raise_exceptions', True):
    199     if 'execution halted' in stderr.lower():
--> 200         raise SSWIDLError(stderr)
    201     if 'failed to acquire license' in stderr.lower():
    202         raise IDLLicenseError(stderr)

SSWIDLError: IDL 8.9.0 (darwin x86_64 m64).
(c) 2023, L3Harris Geospatial Solutions, Inc.

Licensed for use by: University of Hawaii (MAIN)
License: 828837
A new version is available: IDL 9.0
https://www.nv5geospatialsoftware.com/Product-Downloads/
 
% Program caused arithmetic error: Floating overflow
% Compiled module: USE_CHIANTI.
% Compiled module: DIR_EXIST.
% OPENR: Error opening file. Unit: 1
         File: /Users/reep/ssw/packages/chianti/dbase/VERSION
  No such file or directory
% Execution halted at: USE_CHIANTI       155
   /Users/reep/ssw/packages/chianti/idl/setup/use_chianti.pro
%                      $MAIN$          
% Compiled module: HISSW_PROCEDURE.
% Compiled module: MAX_TEMP.
% Compiled module: PICKFILE.
% Compiled module: READ_IONEQ.

The VERSION file it's failing to read does in fact exist:

(base) reep@atrcw9 dbase % more /Users/reep/ssw/packages/chianti/dbase/VERSION
8.0.7

A hello-world with hissw does work in Jupyter:

import hissw
env = hissw.Environment()
script = """
print,'hello world!'
"""
env.run(script)

so this leads me to suspect that CHIANTI is not being properly instantiated in a Jupyter notebook. Have you encountered this?

[wtbarnes]

Yes, I've recently run into this and I'm stumped as to what the issue is. I've come across it in a slightly different context and I think the common issue is any IDL function that is also reading from a file. There is something about running in a Jupyter notebook that doe not like a subprocess reading from an external file. The real test of this would be if you had a (non SSW) script that tried to read from a file.

This has not always been an issue and I suspect is either Python-version dependent or Jupyter notebook version dependent. What version of both of those are you running?

[jwreep]

You can read files in hissw directly using the IDL functions:

import hissw
env = hissw.Environment()
script="""
filename = '/Users/reep/ssw/packages/chianti/dbase/VERSION'
openr,lun,filename,/get_lun
line= ''
readf,lun,line
close,lun
print,line
"""
env.run(script)


IDL 8.9.0 (darwin x86_64 m64).
(c) 2023, L3Harris Geospatial Solutions, Inc.

Licensed for use by: University of Hawaii (MAIN)
License: 828837
A new version is available: IDL 9.0
https://www.nv5geospatialsoftware.com/Product-Downloads/
 
% Program caused arithmetic error: Floating overflow
% Compiled module: HISSW_PROCEDURE.

SSW  setup will include: <gen>

Type <sswidl> to start SSW IDL
--------------------------------------------------------
Running SSW, Last Updated: Wed Nov 2 22:22:51 GMT 2005
 
PROBLEMS? - e-mail TO: freeland@penumbra.nascom.nasa.gov
--------------------------------------------------------
Executing SSW IDL_STARTUP for: GEN
Executing SSW IDL_STARTUP for: SITE
8.0.7

{'filename': b'/Users/reep/ssw/packages/chianti/dbase/VERSION',
 'line': b'8.0.7',
 'lun': 100}

... but not with the CHIANTI functions. I don't know what the difference is!

Versions:

(base) reep@atrcw9 read % conda list jupyter
# packages in environment at /Users/reep/anaconda3:
#
# Name                    Version                   Build  Channel
jupyter                   1.0.0           py311hca03da5_8  
jupyter-lsp               2.2.0           py311hca03da5_0  
jupyter_client            8.6.0           py311hca03da5_0  
jupyter_console           6.6.3           py311hca03da5_0  
jupyter_core              5.5.0           py311hca03da5_0  
jupyter_events            0.8.0           py311hca03da5_0  
jupyter_server            2.10.0          py311hca03da5_0  
jupyter_server_terminals  0.4.4           py311hca03da5_1  
jupyterlab                4.0.8           py311hca03da5_0  
jupyterlab_pygments       0.1.2                      py_0  
jupyterlab_server         2.25.1          py311hca03da5_0  
jupyterlab_widgets        3.0.9           py311hca03da5_0  
(base) reep@atrcw9 read % python --version
Python 3.11.7

[wtbarnes]

Alright I'm still confused as to what is actually going on, but I've managed to create an environment where this problem goes away and it seems to be down to the version of jupyter_core and/or jupyter_client that is installed, but is independent of Python version (I tried on both 3.10 and 3.11).

Creating a fresh environment and running,

$ pip install jupyter_core==5.0.0 jupyter_client==7.4.1 ipykernel==6.17.1 hissw

I can then successfully run the following in a Jupyter notebook,

import hissw
hissw.Environment(ssw_packages=['chianti']).run("use_chianti,'/Users/wtbarnes/ssw/packages/chianti/dbase/'")

Ah, but it seems like ipykernel is the culprit here. Using ipykernel==6.27.1 brings the problem back (leaving all other packages the same). Unfortunately, the problem also persists on the latest version, 6.29.2.

Still not sure what the root cause is here. I'd be interested to see if we can reproduce this problem outside of hissw. Without doing that, it would be hard to report this upstream.