#Writing Wren's JavaScript glue in Odin

Wren's JavaScript engine is the Odin engine (src/engine/jsrt, through the QuickJS C API of src/engine/jscapi) compiled to one WebAssembly module (build/js/wren_js.wasm), run by Wasmtime. Around the engine sits the glue: the code that implements the web platform's objects as native functions of that API and wires them to the browser's wr_* imports. All of it is Odin (native/js/odin/; the binding tables are generated into bindings_gen.odin). No C of ours is compiled any more: the module is linked by the wasm-ld that ships with Odin, with no C library (wj/libc.odin).

This document is the rule book: where things go, how to declare them, how to add a domain. The generic ABI that will replace the hand-written transmission layer is in odin-abi.md.

#The layout

native/js/odin/            package odin_glue: the ported domains, one file per former .c (web_url.odin, dom_style.odin...)
  ifaces.odin              interface indices several domains share (resolved by name, cached)
  bindings_gen.odin        GENERATED by src/tools/bindgen (-emit:glue): the trampolines (wjg_*), the dictionaries (wjd_*), the member
                           and interface tables, wj_install_bindings
  jsb/                     package jsb: members bound to shared declarations (native/js/odin_abi/api_decl) and the
                           wr_* import that carries each, by comptime bridges (odin-abi.md)
  wj/                      package wj: the one helper layer every domain imports
    wj.odin                core types (JSValue = u64, Context, Runtime, Native/Getter/Setter)
    quickjs.odin           the static inlines of quickjs.h reimplemented on the NAN box (the API itself: backend_jsrt.odin)
    engine.odin            the engine's structs mirrored (Obj, Realm, Interface, Listener, Buf), its globals
                           (wj_the_realm, wj_scratch, wj_interfaces, wj_class_*), obj_of / web_set_impl / realm_*
    host.odin              what a domain calls of the engine and of the runtime that is not a value or a string helper
    wr_imports.odin        GENERATED (src/tools/bindgen, from api/*.widl): every browser import wr_*
    strings.odin           Str, WJ_FETCH_STRING (fetch_string), scratch helpers, JS-string constructors
    values.odin            promises as values, blobs, FormData, Headers, streams, signals, structured clone
    errors.odin            throwers (wj_throw_*) and error values (web_dom_exception...)
    dom.odin               dom_internal.h: Dom_Impl (mirrored) and the dom_core.c helpers
    libc.odin              libc / libm (libc_malloc / libc_free, strlen, memcmp, snprintf, fmod...)
    convert.odin           Web IDL conversions (was engine/qjs_convert.c): wj_to_str, wj_to_int32, wj_to_handle...
    buffer.odin            BufferSource helpers (was runtime/buffer.c): wj_to_buffer_source, wj_*_from_host...
    promise.odin           browser promises (was runtime/promise.c): wj_promise_from_host, wj_host_promise_text

The C that used to sit beside the glue is gone: gen/bindings.c (replaced by bindings_gen.odin), wj_shim.c (struct layout asserts) and QuickJS-ng itself (native/js/{engine,jit,patches}). Two rules came out of the merge:

• the layer (wj/) cannot call the glue (odin_glue imports wj) and a symbol cannot be declared foreign where the same build defines it in Odin (one LLVM module: "Non-unique linking name"). A procedure of the glue that the layer calls is stored in wj.hooks (wj/hooks.odin) by wj_web_preinit; a variable the layer reads (wj_dispatch_trusted) is defined in wj/. A domain of the glue calls another domain unqualified, never wj.<name>;
• jsb.host takes up to 14 arguments (jsb/hostn.odin); jsb.host_n is the same overload set.

Every symbol is declared once. odin check native/js/odin -target:freestanding_wasm32 -no-entry-point fails with "Redeclaration of ..." the moment a second declaration appears; that is the point.

The WebAssembly JavaScript API (a page's own modules) is hand-written: webassembly-api.md.

#The engine (WJ_JSRT)

wj.WJ_JSRT is a constant (true): the layer sits on the Odin runtime. wj/backend_jsrt.odin aliases the names of the QuickJS C API to the procedures of src/engine/jscapi (the API implemented on src/engine/jsrt). The API types (Context, Native, Class_Def, ...) are defined once in jscapi/types.odin and aliased in wj. -define:JSCAPI_EXPORT=true also exports the API under its C names, which a few foreign declarations of wj/ resolve to. Details and contracts: ../engine/jscapi.md.

#Where a helper goes

A domain calls everything of the layer qualified: wj.wj_to_str, wj.JS_NewStringLen, wj.wr_URL_get. Nothing of wj is redeclared in a domain.

#How to declare a browser import

You do not. src/tools/bindgen emits native/js/odin/wj/wr_imports.odin from api/*.widl on every build.bat. If an import is missing, it is missing from the API description: add it to the .widl (and the host side in src/page), run build.bat, and it appears as wj.wr_<Name>.

Better still, do not call it by hand: declare the function in native/js/odin_abi/api_decl/<module>.odin and call jsb.host(&out, api_decl.X, wj.wr_X, args...) (or bind the member with jsb.getter / jsb.method / jsb.function): the strings, the result buffer, the status and the wire shape are then the compiler's (odin-abi.md, "How to migrate a domain"). web_url.odin is written that way.

The generated conventions, which a hand-written call follows:

• handles are u64, booleans i32 (relative ? 1 : 0), enums i32, strings (ptr: [^]u8, len: u32), string results (_buf: [^]u8, _cap: u32) -> i32 (the full length, or a negative wr_Error);
• callbacks are a function-table index i32: wj.table_index(rawptr(my_export));
• the calling convention is "contextless": a proc-typed parameter that receives a wr_* import must be proc "contextless" (...), not proc "c" (wj.fetch_string(wj.wr_Window_origin) works as is).

#How to add a domain

1. Write native/js/odin/<x>.odin, package odin_glue, import "../odin/wj". For what only carries a call to the host, declare it in api_decl and bind it with jsb (odin-abi.md, the 10 steps).
2. Name a hand-written member as the generated table (bindings_gen.odin) expects: wjr_<Interface>_<member>, proc "c", the wj types. Error messages matter: WPT and the test pages compare them.
3. Interface indices by name, once: wj.iface("Element") cached in a file-private variable (or ifaces.odin).
4. odin check native/js/odin -target:freestanding_wasm32 -no-entry-point -default-to-nil-allocator, then build.bat, then the tests of the domain (the js, jsdom, jsweb web tests, the WPT folders; see the end).

#Pitfalls (each one was hit)

• A global needs a link-time constant initializer. The glue is an object with no Odin startup code: a file-scope variable whose initializer is not a constant is left zero. Numbers, enums, structs of numbers, [N]string{...} and [N]cstring{"a", "b"} are constant; a nil among cstring literals is not (popover_states read as all-zero: HTMLElement.popover returned ""). Use "" instead of nil, or build the value in the domain's preinit.
• C varargs take C types. wj.wj_throw_type, wj.JS_ThrowTypeError, wj.snprintf are #c_vararg: pass cstring, [^]u8, i32, f64. An Odin string (a kind := cond ? "a" : "b" is one) goes as a two-word struct and prints garbage: write kind: cstring = .... %.*s takes an i32 length then a pointer.
• A C string stops at the first NUL. A JS string may contain one; when the value matters (custom element names, keys), keep the JSValue (JS_ToString) instead of a cstring round trip.
• Str.ptr is [^]u8 (may be nil for a nullable). Pass it to an import as wj.cstr_bytes(s.ptr); compare with wj.str_eq(p, "literal") (accepts [^]u8 or cstring).
• wj.wj_this / wj.wj_unwrap return ^wj.Obj: read o.handle, o.impl directly (or wj.handle_of(o) / wj.impl_of(o) when o may be nil). wj.obj_handle(v) / wj.obj_impl(v) take a JSValue.
• Do not reimplement the NAN-box float encoding: wj.new_float64(ctx, d) goes through the engine's JS_NewNumber (an Odin copy of __JS_NewFloat64 miscompiled a 64-bit wrapping subtraction). JS_IsString includes ropes, JS_IsBigInt short big ints: use wj.is_string / wj.is_big_int, never a single-tag compare.
• Two heaps. What the C freed with free() is freed with wj.libc_free; engine-owned data made for wj_new_object uses wj.malloc / wj.free (the engine's allocator); JS-owned bytes (JS_ToCStringLen, js_malloc) go back to the engine (JS_FreeCString, js_free).

#Struct mirrors and the shim

The engine's structs (wj_obj, wj_realm, wj_interface, wj_listener, wj_buf, wj_str, dom_impl, web_extracted) are plain wasm32 C (natural alignment, no bitfields): Odin's default layout reproduces them, so the glue reads wj.wj_the_realm.protos[i] or o.impl in place, without a call. The first ports went through one-line C accessors per field (three shims, 13 accessors); they are gone.

The mirrors were once checked against the C structs by wj_shim.c (_Static_assert(offsetof(...) == N)); the C structs are gone with the C, and engine.odin still asserts the layout on the Odin side (#assert(offset_of(Obj, impl) == 36)), so a change of a mirrored struct stops the build.

#Building

• src/tools/buildjs compiles native/js/odin (the glue, with the engine of src/engine) to one wasm32 object (odin build -build-mode:obj -target:freestanding_wasm32 -no-entry-point -default-to-nil-allocator -o:size) and links it with the wasm-ld of Odin's bin folder (--no-entry --gc-sections). There is no C library: wj/libc.odin hasmalloc/free (base:runtime's emmalloc over memory.grow), the string functions, strtod, an Odin snprintf over any values and ldexp; sin, cos and pow are imports of the host (module libm, src/wasm/wasi.odin); exp is fdlibm's, in src/engine/jsrt/b_number.odin. A symbol that is declared foreign and not defined is silently an import env.* of the module: check the imports of wren_js_raw.wasm after a change (only wasi_snapshot_preview1.{clock_time_get,random_get}, libm.* and the wr_* modules are legitimate).
• build.bat runs bindgen first, so wr_imports.odin is always current.

#Testing a ported domain

Build: cmd /c build.bat. Then, as relevant to the domain, the subjects of the web tests: js, jsdom, jsweb, jsload, workers, netapi, media (build\e2e_test.exe, docs/testing.md), build/test262 -quick, and src/tools/wpt -Only /<folder>/ -TimeoutMs 30000 for the WPT folders of the domain (the folders must be in wpt/suites.txt). A WPT "regression" is relative to wpt/expectations, which may be older than the code: compare with a build of the commit before the port before concluding (url's worker exclude= variants and webstorage's storage_session_window_noopener were already different before any port).

Edit this page on GitHub