Initial gc handling implementation

[KaruroChori]

Initial implementation. Not to be merged!
It required me patching quickjs to add a getter for the threshold, as that is currently only exposed via a setter.

There is something I don't particularly like: this is the formula used to update it:

        rt->malloc_gc_threshold = rt->malloc_state.malloc_size +
            (rt->malloc_state.malloc_size >> 1);

Basically we have no control over its rescaling after a GC event is run. I think there should be a configurable flag to determine if such threshold can be rescaled. This would allow for a wider range of control strategies to be implemented.

From a high level perspective these would be set with tjs.gc.strategy. Proposed scaling heuristics for now:

• default the self scaling mechanism used above
• always forcing GC events at each allocation
• fixed the threshold will not be rescaled after the event

I am going to open the relevant issues on the quickjs repo for this as well as the callbacks.

[KaruroChori]

By the way, how is linting handled in this project? It seems it does not automatically integrate with my current setup.

[saghul]

I think it's better to just set the threshold at the beginning ourselves, so we don't need a getter.

If / when there is a getter we can change that.

Also, please drop the not yet implemented handlers, we'll add them once QuickJS has them.

Regarding strategy, right now there is not much control from outside QuickJS, so I wouldn't make promises on our end just yet!

[saghul]

By the way, how is linting handled in this project? It seems it does not automatically integrate with my current setup.

It's a manual step, npm run lint

[KaruroChori]

Sorry for the misunderstanding, this was not meant as code to be merged at this stage.
I just wanted to identify what was needed in terms of features and reach consensus upon an interface, so that we don't have to go back and forth changing it too much later :D.

Since I was in the mood I finished implementing everything on both txiki and quickjs. So there are now proper callbacks, fixable thresholds, getters and from the brief tests I made it seems to be working fine.
At least this is running as expected:

const v = []
let i = 0;
tjs.gc.onBefore = ()=>{console.log("BEFORE!", i);return true;}
//tjs.gc.onAfter = ()=>{console.log("AFTER!", i);}
tjs.gc.threshold = 0;
for(; i < 50000000; i++){
    if(i% 1000000 === 0)console.log(`[${tjs.gc.threshold}]`)
    v.push(i);
}
let p = 0;
for(; i < 50000000; i++){
    p+=v[i]
}

Tests are missing, but I will only write the once these or similar features are agreed upon in both repositories.

Clearly a separate branch for quickjs was made for that here.

Now that the code is there I will create a separate branch and pr specific for the kind of integrations which can be made using mainline quickjs. Ideally as things progresses on their repo I will be moving over more and more features from the "complete" branch based on your reviews and with the additional tests and documentation needed.

[KaruroChori]

By the way, how is linting handled in this project? It seems it does not automatically integrate with my current setup.

It's a manual step, npm run lint

For future reference, npm run lint -- --fix to also apply them. I never used lint this way, so there was a bit of confusion :D.

[KaruroChori]

As for what I can bring to txiki right now based on the current quickjs master branch:

• The enable flag
• The threshold variable, knowning that when it will be changed by the dynamic scaling formula after GC is called, it will end up out of sync. The getter was needed to ensure consistency.

If you agree, I will open a new (not draft) PR for with these 1.5 features.

[saghul]

As for what I can bring to txiki right now based on the current quickjs master branch:

• The enable flag

• The threshold variable, knowning that when it will be changed by the dynamic scaling formula after GC is called, it will end up out of sync. The getter was needed to ensure consistency.

If you agree, I will open a new (not draft) PR for with these 1.5 features.

Yeah let's do the 1.5 :-)

[KaruroChori]

It looks like github does not like me renaming branches.

[saghul]

Since it looks like it will take a bit to get the changes into QuickJS, would you be willing to adjust this PR to what we can do today and extend it later?

[KaruroChori]

Actually the plan was to open a second one, and leave this in the background until those features were merged/adapted in quickjs. This branch reflects changes on my fork of quickjs where everything is already up and running.

Ignoring the callback & and the other extras I needed, the real problem is that the getter for the threshold is not implemented in master.

After a more careful evaluation, I considered the partial implementation without even the getter worse than the current lack of features, as it breaks the intuitive behaviour the threshold should have for anyone using it from the js side. And without a decent way to document

[saghul]

Ah wait, you mean because the threshold is changed by QuickJS but there is no way to get it?

I'd land a PR just adding the getter, maybe your orional effort had too many things in it, let's break it down ;-)

[KaruroChori]

Ah wait, you mean because the threshold is changed by QuickJS but there is no way to get it?

Exactly! And this messes up the entire logic behind the high level interface to disable or set the gc threshold.

[KaruroChori]

I'd land a PR just adding the getter, maybe your orional effort had too many things in it, let's break it down ;-)

I am not against that, but I am a bit short on time since I am finishing working on the serial module :D. It might take a while.

[saghul]

Sure thing, no problem. I'll try and see if I have the time to add the getter.

[KaruroChori]

For my future reference: quickjs-ng/quickjs#424