New Public API

[Razican]

Related to #440.

Current situation
The current public API of Boa, that can be found here for Boa 0.8.0 and can be seen offline running cargo doc --open for the master branch, is far from optimal. We have 3 global functions:

• exec(): That creates a new lexer, parser, Realm and interpreter, and isolates the execution of a given JavaScript string, and return a string with the result.
• forward(): That will receive an already initialized interpreter and a JavaScript string and will lex, parse and execute this code in the initialized interpreter, and return the result as a string.
• forward_val(): That will work the same way as forward() but will return a ResultValue, which will indicate if the execution was successful or not.

Then, we have the following modules public:

• syntax: which includes:
  • ast: the AST implementation (nodes, constants, keywords, builtin types, positions, spans, tokens, operations, punctuators...), with some re-exports to make life easier.
  • lexer: The tokenizer/lexer implementation, that turns a string into a vector of tokens. (this is changing in Started with the new lexer implementation #432)
  • parser: The parser implementation, with its error and a couple of parsers (Script and ScriptBody), which turns a vector of tokens into a StatementList, that can then be executed with an interpreter.
• realm, which includes the implementation of a Realm, an interesting helper function to register a global function, and makes public the global object, environment, and the lexical environment. This in theory allows anyone to add stuff to the global environment, but it's a bit cumbersome.
• builtins: Where we have the builtin objects such as Array, Number, JSON and more, and also the implementations of Value and object properties. And the initialization function.
• environment, which has modules for each of the environments and the global trait, everything exposed because the realm has public members so that they can be modified.
• exec: We only contain the Interpreter, with just two public functions: new() and to_string(). The first creates a new interpreter with a given Realm, and the second one converts a Value to a string. It also contains the Executable trait, that allows calling run() on any AST node passing the interpreter so that it runs it.

Finally, we also have some re-exports to make things easier to use.

Problem description
When a user wants to use Boa for their projects, I can imagine the following use cases:

• They want to interpret JavaScript from a file or a stored String and get the result.
• They want to interpret JavaScript that is coming from a socket with a Read interface and get the result by streaming the bytes.
• They want to extend our engine by providing new global functions and objects, so that with the JavaScript they can either perform more rich things in their setup (for example by providing an API for a browser window), or because they want to add JavaScript helpers for common actions in their setup. This also includes things as being the scripting language of a complex platform.
• They want to change the global object, for example, because they want it to be the window or a frame.

I cannot think of anything else, but there might be others. In this context, our current API is very poor. It does allow most of this (except for the Read interface, that is being worked on in #432). But, it is very difficult to do it if you are not an expert in Boa.

Proposed solution
I think we should go for a global Boa structure. This structure should have a default() function (implementing the default() trait, that would setup a clean environment for the execution. Then a interpret() function that would receive either a &str or a u8 (Read) stream (we might need two functions) and return a ResultValue. We could then call to_string() on this if we needed to show it, for example, or on the Ok/Err variants.

It should also have a new() or with_global_obj() function that would require a custom global object. Note that we don't need to expose all the Gc stuff here, as we would be owning the global object.

One option is for it to receive the full Realm, but I think it's best if we didn't expose the complexity of a Realm. In any case, that global object should be wrapped, to provide an easy to use API:

• It should have a register_global_object() function, that would just receive a name and a NativeObject. The current register_global_func() should be kept (maybe renamining to register_global_function().

The NativeObject should just implement a trait, and should easily be boxable, or we could use dynamic dispatch. This trait should allow adding native methods to the object, and should implement a way of retrieving and setting fields. I want to be vague about this, because #419 should give us a clear path.

So, what do you think? What could we improve?

[jasonwilliams]

One option is for it to receive the full Realm, but I think it's best if we didn't expose the complexity of a Realm. In any case, that global object should be wrapped, to provide an easy to use API:

yeah we don't need to expose that really.
For ergonomics we should make it easy to add your own native function to the global object.

Are there any existing standards/interfaces we should be following?

[HalidOdat]

Are there any existing standards/interfaces we should be following?

Speaking of standard/interfaces we should follow the Rust API Guidelines

[Razican]

Are there any existing standards/interfaces we should be following?

Not that I know of. We could check how V8 and SpiderMonkey interface with browsers, for example.

[HalidOdat]

Not that I know of. We could check how V8 and SpiderMonkey interface with browsers, for example.

Neon crate has a nice API. that's were i got the idea of ctx.throw_range_error(msg)

The ctx.throw_range_error(msg) and ctx.throw_type_error(msg) should be public to the user

[HalidOdat]

In the future we might want to have some kind of procedural macro to make it easy to interoperate from native to javascript

wasm-bindgen has this and it's awesome, for example:

#[wasm_bindgen]
#[derive(Copy, Clone)]
pub struct Answer(u32);

#[wasm_bindgen]
impl Answer {
    pub fn new() -> Answer {
        Answer(41)
    }
    #[wasm_bindgen(getter)]
    pub fn the_answer(self) -> u32 {
        self.0 + 1
    }
    pub fn foo(self) -> u32 {
        self.0 + 1
    }
}

We could have something like this, instead of wasm_bindgen we could have to be boa, for example:

#[boa]
#[derive(Copy, Clone)]
pub struct Answer(u32);

#[boa]
impl Answer {
    pub fn new() -> Answer { // this will be ignored
        Answer(Self::helper_function())
    }

	fn helper_function() -> u32 { // this too
		42
	}

    #[boa(name = "theAnswer", getter)] // we could have the proc macro convert the name of the function to camelCase implicitly, so we don't have to supply it, when we don't want to rename it.
    pub fn the_answer(&mut self, args: &[Value], ctx: &mut Interpreter) -> u32 {
        self.0 + 1
    }
	
	#[boa(name = "bar", prototype)] // Reaname function to 'bar' and add it to the prototype.
    pub fn foo(&mut self, args: &[Value], ctx: &mut Interpreter) -> u32 {
        self.0 + 1
    }
}

And the boa proc macro would automatically generate an init function, so the user only has to call it (Answer::init(global)).

For the function:

#[boa(name = "theAnswer", getter)]
pub fn the_answer(&mut self, args: &[Value], ctx: &mut Interpreter) -> u32 {
    self.0 + 1
}

Where it takes Self instead of Value we could have proc macro downcast it to Self and use it as if it was a native type, this could be done with internal state so Answer would require InternalState trait, we could also implement this with trough boa proc marco.

This is for objects and methods but we can have something similar for functions.

BTW: I'm just brainstorming.

What do you think?

[Razican]

What do you think?

This would be awesome. I would automatically rename to camelCase, though, since that's the usual pattern in JavaScript, and maybe allow to specify the name if needed.

[jasonwilliams]

It should have a register_global_object() function, that would just receive a name and a NativeObject. The current register_global_func() should be kept (maybe renamining to register_global_function().

This might need to be register_global_property(), as global object usually refers to the top level global object itself window or globalThis etc.

I like how in the random crate you have an option to just use it directly without much setup https://rust-random.github.io/book/guide-start.html i think using Default() helps with this, we should be able to get users up and running pretty quickly.

Whats a simple use case? You pass in a string (or path to file) then get a value back.
we could have a fast-path function which gives you back a ValueData enum.

Then a interpret() function that would receive either a &str or a u8 (Read) stream (we might need two functions) and return a ResultValue.

I think fromString() and fromFile() would be the most explicit and easiest to understand right?

How much of this do we want to follow:
https://tc39.es/ecma262/#sec-execution-contexts

Embedding V8 example
For some inspiration this is V8
https://chromium.googlesource.com/v8/v8/+/branch-heads/5.8/samples/hello-world.cc

[HalidOdat]

Here is V8 API https://v8.dev/docs/embed (it also has some examples)

[jasonwilliams]

#461 implements Default for Realm which right now includes all the builtins.

Here's a Q
Should the Default Realm be empty or have all the builtIns Boa provides?
I don't see why anyone would want the empty realm.