Forewords: if you have basic familiarity with the rust language, WebAssembly, and the Node.js ecosystem, you will have no problems reading this post.
Today I decided to try wasm-pack from scratch.
What does it mean?
I’ll start with a minimal set of text files, each file having minimal content. I’ll run wasm-pack build, expecting that I’ll get a compiler error every step of the way. Each time, I’ll add some more logic to a file, or add a new file, until the compiler gives me a proper output. Then, I’ll start adding real business logic & keep compiling, until I end up with a useful output.
Every step of the way, I’ll commit the change and push to a GitHub repo. This way, I (or anyone interested) can backtrack and “re-learn” the process anytime.
I’ll be updating this blog post as I actually proceed with the code.
Before we begin, here is the system setup that I use:
- macOS Catalina
- Node.js v18.1.0
- rustc 1.60.0 (7737e0b5c 2022-04-04)
- wasm-pack 0.10.2
- binary target: Node.js
- my goal is to write the logic in
rust, then compile into a.wasmfile usable in a Node.js process
- my goal is to write the logic in
If your setup is different, your mileage might vary.
If you wish to check out the code, please click on the headings (each step below has a corresponding, clickable, heading) and you’ll be redirected to the GitHub project state that matches the step.
Step 0: Initial commit
With the bare minimum, we got this compiler error:
make build
wasm-pack build --target nodejs
Error: Error during execution of `cargo metadata`: error: failed to parse manifest at `/cosmos-tools/Cargo.toml`
Caused by:
virtual manifests must be configured with [workspace]
make: *** [build] Error 1
Step 1: Update Cargo.toml
With the minimal additions in Cargo.toml, we got this compiler error:
make build
wasm-pack build --target nodejs
Error: Error during execution of `cargo metadata`: error: failed to parse manifest at `/cosmos-tools/Cargo.toml`
Caused by:
no targets specified in the manifest
either src/lib.rs, src/main.rs, a [lib] section, or [[bin]] section must be present
make: *** [build] Error 1
Step 2: Add src/lib.rs
With an empty src/lib.rs file, we got this compiler error:
make build
wasm-pack build --target nodejs
Error: crate-type must be cdylib to compile to wasm32-unknown-unknown. Add the following to your Cargo.toml file:
[lib]
crate-type = ["cdylib", "rlib"]
make: *** [build] Error 1
Step 3: Add crate-type lib config
With the crate-type lib configuration added to Cargo.toml, we got this compiler error:
make build
wasm-pack build --target nodejs
[INFO]: 🎯 Checking for the Wasm target...
[INFO]: 🌀 Compiling to Wasm...
Finished release [optimized] target(s) in 0.01s
Error: Ensure that you have "wasm-bindgen" as a dependency in your Cargo.toml file:
[dependencies]
wasm-bindgen = "0.2"
make: *** [build] Error 1
Step 4: Add wasm_bindgen dependency
With the wasm_bindgen dependency added to Cargo.toml, finally we got the 1st successful compilation ( still, for an empty lib.rs file, for now 😉 )
make build
wasm-pack build --target nodejs
[INFO]: 🎯 Checking for the Wasm target...
[INFO]: 🌀 Compiling to Wasm...
Compiling proc-macro2 v1.0.37
Compiling unicode-xid v0.2.3
Compiling syn v1.0.92
Compiling wasm-bindgen-shared v0.2.80
Compiling log v0.4.17
Compiling cfg-if v1.0.0
Compiling lazy_static v1.4.0
Compiling bumpalo v3.9.1
Compiling wasm-bindgen v0.2.80
Compiling quote v1.0.18
Compiling wasm-bindgen-backend v0.2.80
Compiling wasm-bindgen-macro-support v0.2.80
Compiling wasm-bindgen-macro v0.2.80
Compiling cosmos_tools v0.1.0 (/cosmos-tools)
Finished release [optimized] target(s) in 24.15s
[INFO]: ⬇️ Installing wasm-bindgen...
[INFO]: Optimizing wasm binaries with `wasm-opt`...
[INFO]: Optional fields missing from Cargo.toml: 'description', 'repository', and 'license'. These are not necessary, but recommended
[INFO]: ✨ Done in 33.01s
[INFO]: 📦 Your wasm pkg is ready to publish at /cosmos-tools/pkg.
Note that the 1st time after wasm_bindgen was added, it may take quite a long time for the compilation process to finish (also depending on your network speed).
Step 5: Invoke the wasm lib
Now that the wasm lib is built successfully, we can import it in a standard Node.js script or module. Since we’re not doing anything in lib.rs yet, let’s just inspect the value of the imported wasm module:
make start
node --experimental-wasm-modules userland/main.mjs
(node:17470) ExperimentalWarning: Importing WebAssembly modules is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
[Module: null prototype] { memory: Memory [WebAssembly.Memory] {} }
Step 6: Add minimal application logic
Our end goal is to write the logic in rust, but the binary is executable in JavaScript environments, so we add some very basic Hello World example, in this case, a function that invokes console.log. Just as we added the logic, we got this compiler error ( you probably didn’t expect things to be so easy, did you? 😉 )
make build
wasm-pack build --target nodejs
[INFO]: 🎯 Checking for the Wasm target...
[INFO]: 🌀 Compiling to Wasm...
Compiling cosmos_tools v0.1.0 (/cosmos-tools)
error[E0433]: failed to resolve: maybe a missing crate `wasm_bindgen`?
--> src/lib.rs:1:5
|
1 | use wasm_bindgen::prelude::*;
| ^^^^^^^^^^^^ maybe a missing crate `wasm_bindgen`?
error: cannot find attribute `wasm_bindgen` in this scope
--> src/lib.rs:9:3
|
9 | #[wasm_bindgen]
| ^^^^^^^^^^^^
|
= note: `wasm_bindgen` is in scope, but it is a crate, not an attribute
error: cannot find attribute `wasm_bindgen` in this scope
--> src/lib.rs:3:3
|
3 | #[wasm_bindgen]
| ^^^^^^^^^^^^
|
= note: `wasm_bindgen` is in scope, but it is a crate, not an attribute
error: cannot find attribute `wasm_bindgen` in this scope
--> src/lib.rs:5:7
|
5 | #[wasm_bindgen(js_namespace = console)]
| ^^^^^^^^^^^^
|
= note: `wasm_bindgen` is in scope, but it is a crate, not an attribute
For more information about this error, try `rustc --explain E0433`.
error: could not compile `cosmos_tools` due to 4 previous errors
Error: Compiling your crate to WebAssembly failed
Caused by: failed to execute `cargo build`: exited with exit status: 101
full command: "cargo" "build" "--lib" "--release" "--target" "wasm32-unknown-unknown"
make: *** [build] Error 1
The “fix” for the issue above was thankfully mentioned by these cool folks in this thread. Therefore…
Step 7: Add edition in Cargo.toml
As “odd” as it may seem, once the correct edition config value is in the Cargo.toml file, our build process succeeds again. I’ve tested the edition values of 2018 and 2021, both work for me:
make build
wasm-pack build --target nodejs
[INFO]: 🎯 Checking for the Wasm target...
[INFO]: 🌀 Compiling to Wasm...
Compiling cosmos_tools v0.1.0 (/cosmos-tools)
Finished release [optimized] target(s) in 2.44s
[INFO]: ⬇️ Installing wasm-bindgen...
[INFO]: Optimizing wasm binaries with `wasm-opt`...
[INFO]: Optional fields missing from Cargo.toml: 'description', 'repository', and 'license'. These are not necessary, but recommended
[INFO]: ✨ Done in 4.09s
[INFO]: 📦 Your wasm pkg is ready to publish at /cosmos-tools/pkg.
Yet, now that we have some actual application logic inside lib.rs, it seems we can no longer invoke the JS module that imports the .wasm binary directly:
make start
node --experimental-wasm-modules userland/main.mjs
(node:28409) ExperimentalWarning: Importing WebAssembly modules is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
node:internal/errors:466
ErrorCaptureStackTrace(err);
^
Error [ERR_MODULE_NOT_FOUND]: Cannot find package '__wbindgen_placeholder__' imported from /cosmos-tools/pkg/cosmos_tools_bg.wasm
at new NodeError (node:internal/errors:377:5)
at packageResolve (node:internal/modules/esm/resolve:910:9)
at moduleResolve (node:internal/modules/esm/resolve:959:20)
at defaultResolve (node:internal/modules/esm/resolve:1174:11)
at ESMLoader.resolve (node:internal/modules/esm/loader:605:30)
at ESMLoader.getModuleJob (node:internal/modules/esm/loader:318:18)
at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:80:40)
at link (node:internal/modules/esm/module_job:78:36) {
code: 'ERR_MODULE_NOT_FOUND'
}
Node.js v18.1.0
make: *** [start] Error 1
Going too deep into the why is beyond the scope of this post. Here I’ll simply switched to importing the JavaScript module that wasm-pack created for us instead. TL;DR:
Step 8: Standard JS module import instead of direct wasm import
As before, with the “incorrect” import, we get a throw:
make throw
cd examples && npm run throw
> cosmos_tools_examples@0.1.0 throw
> node --experimental-wasm-modules main.direct.mjs
(node:7972) ExperimentalWarning: Importing WebAssembly modules is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
node:internal/errors:466
ErrorCaptureStackTrace(err);
^
Error [ERR_MODULE_NOT_FOUND]: Cannot find package '__wbindgen_placeholder__' imported from /cosmos-tools/pkg/cosmos_tools_bg.wasm
at new NodeError (node:internal/errors:377:5)
at packageResolve (node:internal/modules/esm/resolve:910:9)
at moduleResolve (node:internal/modules/esm/resolve:959:20)
at defaultResolve (node:internal/modules/esm/resolve:1174:11)
at ESMLoader.resolve (node:internal/modules/esm/loader:605:30)
at ESMLoader.getModuleJob (node:internal/modules/esm/loader:318:18)
at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:80:40)
at link (node:internal/modules/esm/module_job:78:36) {
code: 'ERR_MODULE_NOT_FOUND'
}
Node.js v18.1.0
make: *** [throw] Error 1
With the correct import, we finally achieve what we want:
make start
cd examples && npm start
> cosmos_tools_examples@0.1.0 start
> node main.mjs
[Module: null prototype] {
__wasm: [Object: null prototype] {
memory: Memory [WebAssembly.Memory] {},
bark_at: [Function: 17],
__wbindgen_malloc: [Function: 31],
__wbindgen_realloc: [Function: 35]
},
__wbg_log_502590c76efd9b3e: [Function (anonymous)],
bark_at: [Function (anonymous)],
default: {
bark_at: [Function (anonymous)],
__wbg_log_502590c76efd9b3e: [Function (anonymous)],
__wasm: [Object: null prototype] {
memory: Memory [WebAssembly.Memory] {},
bark_at: [Function: 17],
__wbindgen_malloc: [Function: 31],
__wbindgen_realloc: [Function: 35]
}
}
}
wuf, Cosmos
That was the rough journey trying to compile & run something with wasm-pack. The final code is still minimal enough (so we still manage to filter out all the noises that would otherwise come with e.g. a template / boilerplate project). Obviously, the example so far is not optimized for production uses.
If you check out the latest state of this project, you may notice things are very different from the content you’ve read so far. This should be expected, because I was showing an example based on a real project. All real projects evolve. Nevertheless, all the links shared above point to the exact commit state in the project history (thanks to git tag), so by clicking on the links above, you can check out the exact state of the project that matches every step of the journey shown above.
I might update this blog post in the future should the project evolve to a point that makes it worthy of being shared here.
I hope this post helps.
Cheers,
somewhere between Dev & Ops 