Wordsmith
An AI-native 3D world director: one sentence becomes a composed, editable 3D world you can walk through in the browser.
David Bueno · TypeScript · React · Three.js · Fastify · Tripo 3D · World Labs · Jupiter SR
The productFrom one sentence to a world you can edit
- The problem: composing a 3D scene means several DCC tools, hunting for assets and hand-placing them in an engine.
- Wordsmith acts as a spatial director. An LLM turns the prompt into a typed
WorldSpec. - World Labs builds the environment (Gaussian splat); Tripo builds every prop, character and building (GLB).
- The placement engine sizes, rotates and snaps each object to the real floor.
- You then edit, save and present the result, also on a glasses-free Jupiter SR display.
Example prompt
“Abandoned pirate island with sunken galleon and treasure chest.”
→ the LLM plans scenery + up to 5 objects
→ World Labs generates the beach and cliffs
→ Tripo generates captain, cook, sailor, two ships
→ everything is composed and ready to fly through
What it can doGenerate, compose, edit, present
1 · Generate
Three modes in the prompt bar: Everything (world + objects), Scenery only (keeps your objects) and Objects only (adds Tripo models to the current scene, e.g. “5 different pirates with swords”). Live progress by Server-Sent Events.
2 · Compose
Bounding boxes, real size, rotation and floor snapping using the World Labs collision mesh. New objects appear in a row in front of the camera, resting on the ground.
3 · Edit
Outliner (select, focus, rename, hide, lock, delete), numeric inspector, gizmo W E R, undo / redo, duplicate, floor calibration. Shortcuts follow Unreal Engine.
4 · Save & reuse
Scenes are plain JSON files (world, models, transforms, camera, lighting). Imported maps and GLB models are stored by content hash; pins keep favourites on top.
5 · Present
H hides the UI and orbits the camera. PNG capture. Lighting presets: Day, Sunset, Night, Foggy.
6 · Glasses-free 3D
Native Jupiter SR support: 1–30 interlaced views, optical calibration, profile import / export, fullscreen 1:1 subpixel alignment.
In the appPick an object, move it, it stays on the floor
- Click an entry in the Outliner (right) or in the scene: the gizmo and the numeric inspector appear.
- F flies the camera to the object; with nothing selected it frames the whole scene.
- The base of every model rests exactly on the surface, taking scale and rotation into account.
- Every edit is undoable: Ctrl+Z / Y.
ArchitectureSystem at a glance
ModulesOne monorepo, one typed contract
One schema, two runtimes
WorldSpec, GenerationJob and SceneFile are Zod contracts imported by both server and client. LLM output is validated before anything is generated.
Spec-driven
Built with the SDD framework dbv-specs-ops: spec, plan, build, test, ship; decisions logged as ADRs.
Tested
103 tests (Vitest) across the three packages, prompt evals, and an end-to-end run in real Chrome driven over CDP.
The serverHexagonal: the orchestrator never knows which provider it talks to
Provider cascade
LLM priority: OpenAI, then Anthropic, then Gemini, then a deterministic mock. Chosen at start-up from the keys that are present.
Never stuck
If Tripo or World Labs fails or credits run out, the provider falls back to models already stored locally.
Reproducible
Mock providers make the UI, the composition and the tests deterministic and free.
Data flowOne generation job
Tripo 3DFrom a prompt line to a GLB on the sand
Prompts written by the LLM
Each object gets a prompt optimised for Tripo (style, pose, material) plus a category: prop, character, architecture, nature or vehicle.
Credits-aware
Object count is chosen by the user (1–5). Models are cached with unique per-job ids so a new job never overwrites a model used by a saved scene.
A growing library
Every generated model stays in storage/models: 42 GLB models today, reusable from the 3D Models menu in any scene.
In the appScenes made from Tripo models
A scene is a small JSON file: world + models[] (position, rotation, scale, hidden, locked) + camera + lighting. Restoring it needs no AI call and no credits.
Spatial compositionPutting things on the floor of a Gaussian splat
Two bugs that taught us something
- Models floated half their height: the pivot is the model centre, but the full height was added. Now computed from the bounding box.
- A splat has no triangles: raycasting it gave random hits. The collision mesh from World Labs fixed it.
- The ray starts at the object, not 150 m up, so it no longer lands on roofs.
DisplayGlasses-free 3D on the Jupiter SR
- OFF: direct WebGL at native resolution (sharpest 2D; best for splats).
- ON: 1–30 synthetic views interlaced to the lens array through
jupiter-interlace-sdk. - Live tuning: views, baseline, zero-parallax plane, pitch, tan, offset, subpixel order, rotation.
- Profiles export / import as JSON; draco and KTX2 decoders served locally.
Why it matters
Tripo meshes gain real depth: a pirate ship pops out of the screen without VR or glasses.
EngineeringHow we kept it reliable
- Spec-driven: spec, plan, build, test, ship; every decision an ADR.
- 103 tests, plus prompt evals and a manual end-to-end in real Chrome (CDP).
- Validate at the edge: Zod on LLM output, prompts and scene files.
- Honest debt:
WorldViewer3D.tsis over 2,000 lines; the split is planned and documented.
TakeawaysWhat I learned
1 · The LLM plans, code decides
A strict WorldSpec contract stops a creative model from breaking the scene.
2 · Generate once, reuse forever
Saved scenes and cached Tripo models make the demo free and repeatable.
3 · Geometry beats guessing
Real floor from the collision mesh, pivot-aware placement: objects look planted, not pasted.
4 · Ports make it testable
Mocks for the LLM, World Labs and Tripo give deterministic tests and a no-credit demo.
5 · Editing is the product
Generation gets 80 %; Outliner, gizmo, undo and presentation mode close the rest.
6 · Test in the real browser
A headless Chrome run caught bugs unit tests could not.
Live demo
- Load the saved pirate scene (no credits)
- Select a character, F to focus, move it
- Prompt: Objects only, “5 different pirates with swords”
- Lighting preset, then H presentation mode
- Turn on Jupiter 3D
Questions? · github.com/davidbuenov/Wordsmith · Built with dbv-specs-ops
AppendixHTTP API and contracts
| Endpoint | Purpose |
|---|---|
POST /api/generate | Start a job from WorldPromptInput (prompt, mode, style, maxAssets) |
GET /api/jobs/:id · /events | Job state; Server-Sent Events with phase, progress and message |
GET /api/storage/worlds · /models | Stored maps and GLB models, alphabetical, pinned first |
POST /api/storage/upload | Persist an imported map or GLB (content-hashed, up to 500 MB) |
GET/POST/DELETE /api/storage/scenes | Save, list, load and delete scenes (SceneFileSchema) |
GET/PUT /api/storage/favorites/:kind | Pins for worlds and models |
Job phases: QUEUED · ANALYZING · GENERATING_WORLD · GENERATING_ASSETS · COMPOSING · COMPLETED · FAILED.