Maximum recursion level reached on input_process

[ideznaby]

I am getting this error when I run input_process.py on physionet data, the error is happening on this line:
rec = json.dumps(rec)
do you have any solution for this?
Thanks

[Ubikas]

Hi @ideznaby ,
I've been experimenting with BRITS model recently. In my opinion data format output is incorrect in input_process.py. I've changed parse_rec to

def parse_rec(values, masks, evals, eval_masks, dir_):
    deltas = parse_delta(masks, dir_)
    ts_list = []
    for r in range(values.shape[0]):
        rec = {}
        rec['values'] = np.nan_to_num(values[r]).tolist()
        rec['masks'] = masks[r].astype('int32').tolist()
        # imputation ground-truth
        rec['evals'] = np.nan_to_num(evals[r]).tolist()
        rec['eval_masks'] = eval_masks[r].astype('int32').tolist()
        rec['deltas'] = deltas[r].tolist()
        ts_list.append(rec)

    return ts_list

[ideznaby]

Hi @Ubikas thank you for your answer, I tried this but I am still getting the same error, were you able to run the code on Physionet data?

Thanks

[Ubikas]

Hi @ideznaby ,

as far as I remember I was getting the same error with physionet data, but I adopted algorithm to my dataset and it has 19 attributes instead of 35. In my opinion to use json is not the best idea, I'd rather use numpy array with strict ts type order like [sample, ts_type, ts_length, attribute]. So in this case would be [2000,5,49,35]. Unfortunately I didn't have time to code it and as json format worked fine for me I left it as it was.

[CavaJ]

Instead of using usjon, just use python's original json API as follows in data_loader.py and input_process.py files:

# import ujson as json
import json

dumps() and loads() functions are the same. In input_process.py, define your own JSON encoder and dump rec as follows:

class NpEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, np.integer):
            return int(obj)
        elif isinstance(obj, np.floating):
            return float(obj)
        elif isinstance(obj, np.ndarray):
            return obj.tolist()
        else:
            return super(NpEncoder, self).default(obj)

rec = json.dumps(rec,  cls=NpEncoder)

This effectively mitigated Maximum recursion level reached error in my code.

[TanmDL]

cls=NpEncoder

Thank you for your suggestions. It solved the problem of maximum recursion level issues. Btw have you got exact paper claimed results?

[CavaJ]

No, because PyTorch itself is non-deterministic. Please check: https://pytorch.org/docs/stable/notes/randomness.html