Add 'optional' imports/exports

Author: lukewagner

design/mvp/Binary.md

@@ -217,13 +217,20 @@ labellit      ::= len:<u32> l:<label>                      => "l"  (if len = |l|
 valtype       ::= i:<typeidx>                              => i
                 | pvt:<primvaltype>                        => pvt
 resourcetype  ::= 0x3f t:<core:valtype> f?:<core:funcidx>? => (resource (rep t) (dtor f)?)
-functype      ::= 0x40 ps:<paramlist> rs:<resultlist>      => (func ps rs)
-                | 0x43 ps:<paramlist> rs:<resultlist>      => (func async ps rs)
+functype      ::= 0x40 pr:<paramsresult>                   => (func pr)
+                | 0x43 pr:<paramsresult>                   => (func async pr)
+                | 0x44 i*:<funcimms> pr:<paramsresult>     => (func i* pr)
+funcimms      ::= 0x00                                     =>
+                | 0x01                                     => optional
+                | 0x02                                     => async
+                | 0x03                                     => optional async
+paramsresult  ::= p*:<paramlist> r?:<result?>              => p* r?
 paramlist     ::= lt*:vec(<labelvaltype>)                  => (param lt)*
-resultlist    ::= 0x00 t:<valtype>                         => (result t)
+result?       ::= 0x00 t:<valtype>                         => (result t)
                 | 0x01 0x00                                =>
 componenttype ::= 0x41 cd*:vec(<componentdecl>)            => (component cd*)
 instancetype  ::= 0x42 id*:vec(<instancedecl>)             => (instance id*)
+                | 0x45 0x01 id*:vec(<instancedecl>)        => (instance optional id*)
 componentdecl ::= 0x03 id:<importdecl>                     => id
                 | id:<instancedecl>                        => id
 instancedecl  ::= 0x00 t:<core:type>                       => t
@@ -240,6 +247,7 @@ externtype    ::= 0x00 0x11 i:<core:typeidx>               => (core module (type
                 | 0x05 i:<typeidx>                         => (instance (type i))
 typebound     ::= 0x00 i:<typeidx>                         => (eq i)
                 | 0x01                                     => (sub resource)
+                | 0x02 0x01                                => (sub optional resource)
 valuebound    ::= 0x00 i:<valueidx>                        => (eq i) 🪙
                 | 0x01 t:<valtype>                         => t 🪙
 ```
@@ -301,6 +309,7 @@ canon    ::= 0x00 0x00 f:<core:funcidx> opts:<opts> ft:<typeidx> => (canon lift
            | 0x02 rt:<typeidx>                                   => (canon resource.new rt (core func))
            | 0x03 rt:<typeidx>                                   => (canon resource.drop rt (core func))
            | 0x04 rt:<typeidx>                                   => (canon resource.rep rt (core func))
+           | 0x07 si:<sortidx>                                   => (canon optional.present si (core global)) ❓
            | 0x24                                                => (canon backpressure.inc (core func)) 🔀
            | 0x25                                                => (canon backpressure.dec (core func)) 🔀
            | 0x09 rs:<resultlist> opts:<opts>                    => (canon task.return rs opts (core func)) 🔀
@@ -564,7 +573,7 @@ named once.
 
 * The opcodes (for types, canon built-ins, etc) should be re-sorted
 * The two `list` type codes should be merged into one with an optional immediate
-  and similarly for `func`.
+  and similarly for `func`, `instance` and `typebound`.
 * The redundant `0x00` and `0x01` opcodes of `nameattributes` will be merged.
 * The vestigial `0x00` in `thread.*` and `waitable-set.*` built-ins may be
   removed.

design/mvp/CanonicalABI.md

@@ -40,6 +40,7 @@ specified here.
   * [`canon resource.new`](#canon-resourcenew)
   * [`canon resource.drop`](#canon-resourcedrop)
   * [`canon resource.rep`](#canon-resourcerep)
+  * [`canon optional.present`](#-canon-optionalpresent) ❓
   * [`canon context.get`](#-canon-contextget) 🔀
   * [`canon context.set`](#-canon-contextset) 🔀
   * [`canon backpressure.{inc,dec}`](#-canon-backpressureincdec) 🔀
@@ -3367,6 +3368,10 @@ validation is performed:
 * if `len(flatten_types(ft.param_types())) > MAX_FLAT_PARAMS`, `realloc` is required
 * if `len(flatten_types(ft.result_type())) > max` (where `max = MAX_FLAT_RESULTS` for sync lifts, and `max = MAX_FLAT_PARAMS` for async lifts), `memory` is required
 
+TODO: if `$ft` is optional and one of its referenced resource types is imported and
+"none", the `canon lift`ed function becomes "none"
+TODO: define some `refers_to_non_existent_resource`, `assert(not refers_to_non_existent_resource(ft))`
+
 Note that an `async`-lifted function whose result type requires a memory to lift
 (either because it contains lists or strings or because the number of flattened
 types exceeds `MAX_FLAT_PARAMS`) must include a `memory` option, and that option
@@ -3612,10 +3617,13 @@ runtime Core WebAssembly arguments.
 Based on this, `canon_lower` is defined in chunks as follows. First, like most
 Canonical ABI functions callable from Core WebAssembly, lowered imports may not
 be called during `post-return` or `realloc`:
+TODO: mention optional
 ```python
 def canon_lower(callee, ft, opts, flat_args: list[CoreValType]) -> list[CoreValType]:
   thread = current_thread()
   trap_if(not thread.task.inst.may_leave)
+  assert(ft.optional or callee is not None)
+  trap_if(callee is None)
 ```
 
 The component-level function type maps to a Core WebAssembly function type using
@@ -3830,6 +3838,24 @@ Note that the "locally-defined" requirement above ensures that only the
 component instance defining a resource can access its representation.
 
 
+### ❓ `canon optional.present`
+
+For a canonical definition:
+```wat
+(canon optional.present $sortidx (core global $present))
+```
+validation specifies:
+* `$sortidx` must refer to a `(func optional ...)`, `(instance optional ...)` or
+  `(type (sub optional ...))`
+* `$present` is given type `(global i32)`
+
+The new immutable global `$present` is appended to the core global index space
+and is either `1` or `0` depending on whether the [`optional`] `$sortidx` is
+present or not, resp. Core wasm code can then import this synthesized global and
+use it to conditionalize calls to any `optional` function (which would otherwise
+trap if called).
+
+
 ### 🔀 `canon context.get`
 
 For a canonical definition:
@@ -4963,6 +4989,7 @@ def canon_thread_available_parallelism():
 [`canonopt`]: Explainer.md#canonical-definitions
 [`canon`]: Explainer.md#canonical-definitions
 [Type Definitions]: Explainer.md#type-definitions
+[`optional`]: Explainer.md#optionality
 [Component Invariant]: Explainer.md#component-invariants
 [JavaScript Embedding]: Explainer.md#JavaScript-embedding
 [ESM-integration]: Explainer.md#esm-integration

design/mvp/Explainer.md

@@ -20,12 +20,14 @@ more user-focused explanation, take a look at the
       * [Asynchronous value types](#asynchronous-value-types)
     * [Specialized value types](#specialized-value-types)
     * [Definition types](#definition-types)
+    * [Optionality](#optionality)
     * [Declarators](#declarators)
     * [Type checking](#type-checking)
   * [Canonical definitions](#canonical-definitions)
     * [Canonical ABI](#canonical-abi)
     * [Canonical built-ins](#canonical-built-ins)
       * [Resource built-ins](#resource-built-ins)
+      * [Optionality built-ins](#-optionality-built-ins)
       * [Concurrency built-ins](#-concurrency-built-ins)
       * [Error Context built-ins](#-error-context-built-ins)
   * [Value definitions](#-value-definitions)
@@ -77,6 +79,7 @@ shipped as part of a future WASI Developer Preview release:
 * 🐘: [memory64]
 * 📡: getters and setters
 * ➡️: `stream.forward` and `future.forward` built-ins
+* ❓: `optional` imports and exports
 
 
 ## Grammar
@@ -323,6 +326,7 @@ instanceexpr   ::= (instantiate <componentidx> <instantiatearg>*)
                  | <inlineexport>*
 instantiatearg ::= (with <name> <externidx>)
                  | (with <name> (instance <inlineexport>*))
+                 | (with <name>) ❓
 name           ::= <core:name>
 inlineexport   ::= (export <externnamelit> <externidx>)
                  | (export <externnamelit> <versionsuffix> <externidx>) 🔗
@@ -377,6 +381,8 @@ the `foo` function of its child component `$C` and re-export it directly from
 )
 ```
 
+TODO: mention `alias export` of `optional` `instance` ~~> optional X (disallow X=component for now)
+
 Additional syntactic sugar is added for allowing export aliases to be defined
 *inline* as a syntactic generalization of the `{X}sortidx` grammar rules
 defined [above](#index-spaces) for each core- and component-level sort `X`:
@@ -453,6 +459,8 @@ aliased by `$E`:
 )
 ```
 
+TODO: anything interesting here when reaching out of optional scope?
+
 For `outer` aliases, there is also inline syntactic sugar, which is simply to
 use the identifier of the outer definition, resolved using normal lexical
 scoping rules. For example, the following component:
@@ -636,9 +644,13 @@ valtype       ::= <typeidx>
 keytype       ::= bool | s8 | u8 | s16 | u16 | s32 | u32 | s64 | u64 | char | string 🗺️
 resourcetype  ::= (resource (rep i32) (dtor core-prefix(<core:funcidx>))?)
                 | (resource (rep i64) (dtor core-prefix(<core:funcidx>))?) 🐘
-functype      ::= (func async? (param <labellit> <valtype>)* (result <valtype>)?)
+functype      ::= (func <optional?> <async?> (param <labellit> <valtype>)* (result <valtype>)?)
+optional?     ::= ϵ
+                | optional ❓
+async?        ::= ϵ
+                | async 🔀
 componenttype ::= (component <componentdecl>*)
-instancetype  ::= (instance <instancedecl>*)
+instancetype  ::= (instance <optional?> <instancedecl>*)
 componentdecl ::= <importdecl>
                 | <instancedecl>
 instancedecl  ::= core-prefix(<core:type>)
@@ -655,7 +667,7 @@ externtype    ::= (<sort> (type <idx>) )
                 | (value <valuebound>) 🪙
                 | (type <typebound>)
 typebound     ::= (eq <typeidx>)
-                | (sub resource)
+                | (sub <optional?> resource)
 valuebound    ::= (eq <valueidx>) 🪙
                 | <valtype> 🪙
 
@@ -914,6 +926,12 @@ declarators. The meanings of these declarators is basically the same as the
 core module declarators introduced above, but expanded to cover the additional
 capabilities of the component model.
 
+#### Optionality
+
+TODO: describe
+
+See [`optional.test`](#-optionality-built-ins) below for more details.
+
 #### Declarators
 
 The `importdecl` and `exportdecl` declarators correspond to component `import`
@@ -1332,6 +1350,9 @@ replaced by `$R` when validating the instantiations of `$c1` and `$c2`. These
 type-checking rules for instantiating type imports mirror the *elimination*
 rule of [universal types]  (∀T).
 
+TODO: if don't provide type, substitute incompatible type; prevent functions
+from being supplied...
+
 Importantly, this type substitution performed by the parent is not visible to
 the child at validation- or run-time. In particular, there are no runtime
 casts that can "see through" to the original type parameter, avoiding
@@ -1555,6 +1576,7 @@ canon ::= ...
         | (canon resource.new <typeidx> (core func <id>?))
         | (canon resource.drop <typeidx> (core func <id>?))
         | (canon resource.rep <typeidx> (core func <id>?))
+        | (canon optional.present <sortidx> (core global <id>?)) ❓
         | (canon context.get <core:valtype> <u32> (core func <id>?)) 🔀
         | (canon context.set <core:valtype> <u32> (core func <id>?)) 🔀
         | (canon backpressure.inc (core func <id>?)) 🔀
@@ -1681,6 +1703,12 @@ transferring ownership of the newly-created resource to the export's caller.
 
 For details, see [`canon_resource_rep`] in the Canonical ABI explainer.
 
+##### ❓ Optionality built-ins
+
+###### ❓ `optional.present`
+
+TODO
+
 ##### 🔀🧵 Concurrency built-ins
 
 See the [concurrency explainer] for background.

design/mvp/WIT.md

@@ -308,6 +308,9 @@ world union-my-world {
 }
 ```
 
+TODO: add `optional` to some of the `include` examples above (unions *both* imports and exports)
+TODO: add a note that `include` does not produce subtype or supertype, but other ways to combine worlds (`meet` and `join`) could be added in the future
+
 ### De-duplication of interfaces
 
 If two worlds share an imported or exported [interface name], then the union of
@@ -339,6 +342,8 @@ world union-my-world-b {
 }
 ```
 
+TODO: mention merging of `optional` and non-`optional` ~~> non-`optional`
+
 ### Name Conflicts and `with`
 
 When two or more included Worlds have the same name for an import or export
@@ -436,12 +441,6 @@ world invalid-union-world {
 
 ```
 
-### A Note on Subtyping
-
-In the future, when `optional` export is supported, the world author may explicitly mark exports as optional to make a component targeting an included World a subtype of the union World.
-
-For now, we are not following the subtyping rules for the `include` statement. That is, the `include` statement does not imply any subtyping relationship between the included worlds and the union world.
-
 ## WIT Packages and `use`
 [use]: #wit-packages-and-use
 
@@ -563,6 +562,8 @@ interface my-interface {
 }
 ```
 
+TODO: `use` resolution when only used from `optional`y-imported interfaces --> `optional`
+
 #### Top-level `use`
 
 If a package being referred to has a version number, then using the above syntax
@@ -1075,6 +1076,7 @@ keyword ::= 'as'
           | 'list'
           | 'map'
           | 'option'
+          | 'optional'
           | 'own'
           | 'package'
           | 'record'
@@ -1441,10 +1443,10 @@ world-definition ::= export-item
                    | typedef-item
                    | include-item
 
-export-item ::= external-id? 'export' id ':' extern-type
-              | 'export' use-path ';'
-import-item ::= external-id? 'import' id ':' extern-type
-              | 'import' use-path ';'
+export-item ::= external-id? 'export' id ':' 'optional'? extern-type
+              | 'export' 'optional'? use-path ';'
+import-item ::= external-id? 'import' id ':' 'optional'? extern-type
+              | 'import' 'optional'? use-path ';'
 
 external-id ::= '@external-id' '(' string-literal ')' 🏷️
 
@@ -1508,6 +1510,8 @@ world my-world {
 }
 ```
 
+TODO: note `optional` and what it means
+
 [`componenttype`]: Explainer.md#type-definitions
 
 ## Item: `include`