oboe not releasing memory using oboe.drop

[leonidasthelab]

First, thank you so much for working on this awesome library!

Exactly what I have been looking for, but am unable to use it due to the memory issues I am seeing...I hope this can be fixed!

Am working on an angular app and using oboe.js to consume data that is sent in json streams by the server. Read through documentation and following to a T, but am not seeing that memory is being freed, leading to internet explorer 11 (no choice in browser) crashing with out of memory error.

I am able to grab the data streams and process the data perfectly, but just need oboejs to release the memory.

I have the latest oboe.js (v 2.1.5).

//sample code
const config = { 'url': "serverurlhere"}; // and other configs such as headers, withCredentials
oboeService = oboe(config);

oboeService.node('Chunk.*', function (node) {
// send in node to be processed
return oboe.drop
});

Even if I comment out all the processing code and use oboejs in the most basic form and just have 'return oboe.drop' in the .node, I still see the memory climbing up until the browser crashes. I do not understand why this happening...

Any help is much appreciated!

[Aigeec]

hi @leonidasthelab could you please provide some details on where the code is running, is it node or a browser, could you specific which version please.

Thanks for raising the issue, will look into it.

[leonidasthelab]

@Aigeec, the code is running clientside on IE 11, using latest oboe.js (v 2.1.5). Thanks!

[Aigeec]

Sorry, you mentioned that didn't you, the heap check is running for chrome I think only at the moment. Is difficult enough to get a headless IE going in Travis.

[leonidasthelab]

Any plan to support both IE 11 and Edge browsers for releasing memory? Really want to use this library...exactly what we are looking for!

[Aigeec]

It should do that already, will see what the issue is.

[Aigeec]

Hi @leonidasthelab, I can't recreate this issue. I have a sample array that is being returned from the source. When I call oboe.drop the nodes are dropped from the array as expected.

Can you provide a more complete sample.

Here is a JsFiddle where you can see that the final array has each element replaced with null:

https://jsfiddle.net/014fr8ac/10/embedded/result,css,html,js

[leonidasthelab]

Thanks for the help. Is there any importance in the fact that the json in your sample is an array versus what I am using, a nested object containing an array?

example:

• going to listen on the largeObj node, while server streams 1 nestedObj at a time, containing an array of size 1000..

structure of overall json once all completed after streaming

{
largeObj: {
nestedObj: {
[]
}
}
}

[mscottnelson]

I can't help but notice that your final structure is:
{ largeObj: { nestedObj: { [] } } }
If that's the case, the inner array is not an array, but instead an object index with no value on the other side, ie. I'm just looking over oboe.js without having actually used it, glancing at issues, but thought that might be relevant.

{ largeObj: { nestedObj: { ['some','Huge','Amount,'Of','Data','Here'] } } }
                                  ^
// SyntaxError: Unexpected token ,

(you can quickly make an example repo to demonstrate if there's a real error here: https://my-json-server.typicode.com/ )