Hmm… for now, based on what I could find, this seems to be roughly how things stand:
I think all six problems are real, but they sit at slightly different layers, and that changes what is worth trying.
The short version would be:
| Question |
What seems most useful in practice |
| 1. Suppression without negative prompting |
With a closed API, most attention/CFG-based papers are not directly actionable. Marble’s own panorama edit stage is probably the first practical route; for API-only use, disable_recaption is at least a cheap controlled A/B. |
| 2. Cross-scene consistency |
The literature seems to favor shared spatial structure/memory, not merely a shared random seed. Floorplans, coarse geometry, references, or continued generation look more promising than trying to make independent scenes agree after the fact. |
| 3. Outdoor/unbounded degradation |
Unbounded generation really is treated as a separate hard problem in recent work, but I would first locate which pipeline stage is degrading before blaming 3DGS in general. |
| 4. Inserted artwork looks like a decal |
First rule out a Three.js color-management/material mismatch. After that, I would keep your current lightweight grading as a baseline and try a local environment probe before attempting to recover physical illumination from SPZ SH coefficients. |
| 5. Data-driven hanging |
Your three-height test looks like a reasonable task-specific wall detector if it is already reliable. I would keep it, but make the output a stable occupancy/placement layer so monthly rehangs never need to understand the raw generated geometry again. True scale and data-only rotation are not necessarily mutually exclusive. |
| 6. Selective spotlighting in splats |
Your option (c) is very close to something Spark already implements: SplatEdit supports spatial RGBA edits, including an INFINITE_CONE explicitly documented as a spotlight-like primitive. This is probably the first thing I would test. |
If I were trying to minimize engineering cost, my order would be:
- verify the Three.js color pipeline for the paintings;
- inspect whether the actual SPZ has any SH beyond degree 0;
- prototype one Spark
SplatEdit spotlight in one representative room;
- separate Q3 into pano → Marble world/mesh → exported SPZ → Spark;
- leave the working wall-segmentation heuristic alone until it produces an actual failure case;
- only then look at heavier inverse-rendering / relighting methods.
The common pattern I see is that it may help to separate three representations:
- the generative representation used to create the space;
- the visual representation used to render the splats;
- the operational representation used by the exhibition system.
The monthly hanging system does not really need to understand Marble’s raw geometry every month, and the painting renderer does not necessarily need to recover the physically correct latent lighting of the splat if a stable local appearance approximation is enough.
1. Negative-free suppression: closed API changes the useful solution space
There is a substantial literature on negative guidance, attention manipulation, CFG modifications, and related techniques, but most of it assumes access to the diffusion process itself. That makes it useful as background evidence that negation/content suppression is genuinely non-trivial, but not necessarily useful implementation advice for a Marble text-only API.
For a closed model, I found two more relevant directions.
A. Fix the generated panorama before committing to the 3D world
World Labs’ current Create & edit workflow explicitly separates:
- panorama generation;
- optional panorama editing;
- draft 3D generation;
- final world generation.
At the panorama-edit stage, the docs describe targeted local edits and specifically list modifying objects, adding/removing details, and fixing issues in the initial generation.
For persistent things like:
- outlets,
- an unwanted framed painting,
- a religious figure,
- a wrong piece of furniture,
that looks more reliable to me than trying to encode increasingly complicated negations into the original positive prompt.
It also has a nice engineering property: the correction occurs before the 2D panorama is lifted into the final world, rather than trying to repair the splats later.
B. If the workflow must remain API-only, test recaptioning separately
The current World API exposes both a generation seed and a disable_recaption field for text prompts in the world generation API.
I would not assume recaptioning is causing the anachronisms — I found no evidence for that — but this gives a cheap controlled test:
same model
same seed
same text_prompt
only change disable_recaption
If the unwanted objects are unchanged, that hypothesis can be discarded quickly.
C. Black-box prompt refinement exists, but I would rank it below editing
There is also work such as Test-time Prompt Refinement for Text-to-Image Models, where a black-box generator produces an image, a multimodal model checks the result against the prompt, and the prompt is rewritten for another round.
Conceptually:
prompt
↓
generation
↓
detect unwanted/missing content
↓
rewrite prompt
↓
regenerate
This is much closer to your “I can only control text” constraint than attention manipulation is.
But I would still treat it as a third-line option here:
- the paper is about T2I, not Marble;
- every iteration costs another generation;
- you need a reliable evaluator for period errors;
- Marble already exposes an edit stage designed for local corrections.
So my default route would be targeted panorama editing where available, API A/B diagnostics if not, and only then automated black-box prompt optimization if the number of worlds makes that worthwhile.
2. Cross-scene consistency: seed consistency vs spatial consistency
This is the area where the literature seems most consistent.
A fixed random seed can make sampling reproducible, but it does not by itself provide a persistent representation of:
- the building’s proportions;
- room adjacency;
- wall/floor material identity;
- window grammar;
- architectural details;
- where previously generated structure physically exists.
Recent whole-scene / multi-room systems tend to add exactly that missing state.
A particularly close example is PanoWorld, which targets consistent whole-house panorama synthesis. Its design uses:
- a floorplan-derived 3D shell as global structural guidance;
- a dynamic 3DGS cache as persistent spatial/visual memory.
That is much closer to “the next room knows what building it belongs to” than sharing a seed.
I would use PanoWorld mostly as evidence for the design pattern, not as a drop-in implementation: the repository currently exposes the PanoWorld-LRM inference path, while several components of the full generation pipeline are still listed as forthcoming.
Marble already exposes some controls in the same general direction
Chisel lets you block out coarse 3D geometry first and use it as the foundation for the generated world.
Expand continues outward from an existing world and is explicitly intended to preserve visual style, architecture, scale, and continuity at the connection.
That seems much closer to the cross-scene problem than generating eleven independent rooms and hoping their latent identities coincide.
There is one current constraint worth noting: the docs say a world generated with Marble 1.1 Plus cannot currently be expanded.
Studio Compose is another option, but it is useful to distinguish composition from generation consistency. The docs describe a multi-room house as independently generated room worlds that you manually position, rotate, scale, align, and connect; they even include “match lighting” as a connection step. So Compose solves assembly, but does not magically make independently generated rooms share architectural identity.
So I would frame the options like this
If independent worlds are a hard product requirement:
Keep the worlds separate, but increase shared evidence: common reference imagery, common coarse geometry, repeated spatial constraints, or some canonical authored layout.
If one connected generation is acceptable:
Prefer generation that grows from an already established structure/world over independent regeneration.
If neither is possible:
Treat consistency as an explicit post-generation acceptance criterion, rather than expecting a seed to enforce it.
I would therefore be cautious about “single huge scene then segment” as the answer. It is one way to force a shared spatial state, but the broader principle seems to be persistent structure/memory, not necessarily one monolithic splat.
3. Outdoor / unbounded scenes: real research problem, but isolate the failing stage first
There is good evidence that unbounded 3D generation is not just “indoor generation, but larger”.
For example, VideoRFSplat explicitly targets unbounded real-world scenes and jointly models multi-view imagery and camera pose.
GaussianCity similarly treats scaling 3DGS from finite scenes to unbounded city-scale environments as non-trivial, requiring a dedicated compact representation rather than simply allowing the point set to grow without bound.
Those papers do not establish the cause of your four bad outdoor scenes — their training setups and objectives are different — but they do support the narrower statement that large/unbounded generation has its own geometry, memory, pose-consistency, and representation problems.
For your pipeline, I think the higher-value test is to ask where the failure first appears.
The World API exposes the panorama, multiple mesh assets, splat assets, and hosted Marble world separately, so the stages can be compared.
Is the horizon/geometry already wrong in the panorama?
↓ yes
generation / panorama-side problem
Panorama looks correct, but Marble 3D world is wrong?
↓
3D lifting / world reconstruction side
Marble world looks correct, but exported SPZ/mesh is wrong?
↓
export / representation side
Exported asset looks correct elsewhere, but Spark is wrong?
↓
runtime / renderer side
That distinction matters because “3DGS cannot do outdoor scenes well” would be much too broad if the pano itself is already inconsistent, and equally misleading if the asset is good until it reaches the browser renderer.
World Labs currently describes Marble 1.1 Plus as its model for the largest worlds, automatically expanding 3D coverage where possible. So if your exterior scenes were generated with another model, a small 1.1 vs 1.1 Plus comparison is a reasonable branch.
I still would not assume Plus fixes horizon coherence; the documentation only establishes that it is intended for larger coverage.
One Spark-specific branch is also worth keeping in reserve: current SparkRenderer documentation includes pagedExtSplats, described as useful for avoiding quantization artifacts when splat scenes have very large internal position coordinates. That is a renderer-side precision tool, not a general fix for bad generated geometry, so I would only investigate it if the degradation appears specifically after loading into Spark and looks coordinate/precision-related.
4. Relighting the paintings: I think there are several cheaper steps before inverse rendering
Your current fix does not look unreasonable to me.
You effectively have a baked visual field, then insert an object that was never present when the field’s appearance was generated. The object does not inherit the wall’s baked exposure, color cast, local softness, or frequency characteristics, so some kind of appearance harmonization is expected.
There are papers treating the heavier version of this exact problem. GauUpdate explicitly observes that inserting new Gaussian objects into an existing Gaussian field gives inconsistent appearance when the source and target lighting differ, and solves it through inverse rendering of materials/environment illumination.
D3DR instead uses diffusion priors to harmonize inserted 3DGS objects, including lighting and shadows.
Those are useful evidence that the mismatch is a real research problem, but they are much heavier than your flat artwork use case.
I would try the following ladder first.
Step 0: verify this is not partly a Three.js color-pipeline mismatch
Before estimating illumination, I would check the ordinary renderer plumbing.
Three.js’ Color Management guide distinguishes:
- sRGB input color textures;
- Linear-sRGB working/rendering space;
- output color conversion / tone mapping.
For normal PNG/JPEG artwork textures, the color texture should generally be tagged appropriately as sRGB color data. If the painting quad, splat renderer, and post-processing path are going through different color transforms, a brightness/color mismatch can look deceptively like a lighting mismatch.
I would check at least:
texture.colorSpace;renderer.outputColorSpace;- tone-mapping settings/exposure;
- whether post-processing performs the final output transform;
- whether the artwork uses
MeshBasicMaterial, MeshStandardMaterial, a custom shader, etc.
In particular, if the painting is effectively unlit (MeshBasicMaterial or equivalent), a physical Three.js light will never make it inherit the same illumination behavior as the room.
If the color pipeline is correct and the mismatch remains, then I would call it the baked-vs-inserted appearance problem.
Step 1: keep your current empirical grading as the baseline
Mean luminance, saturation, channel ratio, and a little blur are all very cheap and directly optimize what you actually care about: visual integration.
I would keep that baseline even if you later add something more principled, because it gives you an easy A/B:
does the more complicated method visibly outperform the simple wall-statistics transform?
There is also one museum-specific constraint I would keep separate from generic “object harmonization”: you probably do not want the harmonizer to materially change the painting itself.
For an ordinary inserted chair, changing the object’s chroma or tonal range to fit the room may be fine. For an artwork, aggressive harmonization can defeat the point of showing the artwork accurately.
So I would separate:
- faithful painting image;
- frame/glass/carrier geometry;
- local wall response.
That gives you more freedom to harmonize the frame and surroundings while preserving the artwork pixels.
Step 2: Spark can render a local environment probe directly
This was the most interesting practical thing I found for Q4.
The current SparkRenderer documentation exposes:
renderCubeMap(...);readCubeTargets();renderEnvMap(...).
renderEnvMap() renders the splat scene from a supplied world position, builds the six cube faces, prefilters them using Three.js PMREMGenerator, and returns a texture that can be assigned directly to MeshStandardMaterial.envMap.
So instead of:
decode SPZ SH → infer the original environment lighting
you can potentially do:
render the actual baked scene appearance around the painting position → use that as a local image-based-lighting probe for the inserted frame/carrier.
That is not a reconstruction of the true physical light field. It is a local appearance probe derived from what the viewer actually sees, which may be exactly the useful quantity for this application.
readCubeTargets() also means you can retrieve the six rendered faces as RGBA buffers. If you wanted to extend your current luminance / R:B heuristic, you could estimate low-frequency local exposure/tint from a 360° neighborhood rather than from only the wall patch.
I would personally try this before attempting to interpret SPZ SH as irradiance.
Why I would be cautious about the SH route
The current Niantic SPZ implementation allows sh_degree from 0 to 4.
Degree 0 means there are no additional SH coefficients at all, and the source describes the SH field as coefficients for view-dependent colors.
So there are two separate questions:
- does the particular Marble SPZ actually contain non-zero-degree SH?
- if so, what physical quantity can legitimately be inferred from it?
The standard SPZ structure gives you Gaussian geometry, alpha, base color and optional view-dependent color coefficients. It does not give you a clean, separately identified BRDF + surface normal + incident illumination decomposition.
Relightable-3DGS papers generally have to estimate or learn those additional factors precisely because they are not already handed to you as a standard splat asset.
So I would inspect the file’s sh_degree, but I would not design the production system around “SH = environment light”.
Step 3: if local probes + grading are not enough, use a proxy-mesh relighting approach
There is a very relevant production precedent in PlayCanvas’ Gaussian Splat Relighting.
Their method is roughly:
- use a simplified mesh approximating the splat;
- light that mesh with ordinary dynamic lights;
- render the mesh lighting to an offscreen texture from the active camera;
- modulate the splat fragments using that lighting texture.
That is a clever bridge between ordinary real-time lighting and a baked splat.
You already have a Marble collision mesh, so it might be worth checking whether it is geometrically close enough to the visible wall surface to serve as that sort of proxy.
I would treat that as a test, not an assumption: PlayCanvas notes that proxy/splat alignment is an important quality factor.
So for Q4 my preferred escalation path would be:
Three.js color sanity
↓
your current appearance grading
↓
local Spark env-map probe
↓
proxy-mesh lighting transfer
↓
inverse rendering / diffusion harmonization
That seems much cheaper than jumping directly from wall RGB statistics to full inverse rendering.
5. Artwork placement: I would preserve the current heuristic, but formalize its output
For the first question — whether the three-height coplanarity test is reasonable — I think yes, as a task-specific heuristic, provided the observation that it reliably rejects windows/recesses/fixtures holds across your rooms.
I could not find anything saying that exactly:
- 1.10 m;
- 1.50 m;
- 1.90 m;
- ±10 cm
is a standard gallery-wall algorithm.
So I would not present those particular constants as generally established.
But the broader idea is well connected to indoor-geometry processing: using evidence across multiple horizontal slices/heights is a normal way to make wall detection less sensitive to clutter, occlusion, and local protrusions.
Given that your purpose is not “reconstruct the mathematically perfect wall plane”, but “find conservative regions where a painting will not intersect architectural clutter”, your heuristic is arguably solving the right problem.
I would not replace it with RANSAC unless you have an actual failure
If the current procedure works, replacing it with semantic segmentation, a full mesh classifier, or global plane fitting may increase complexity without increasing useful information.
A reasonable escalation would be only when a bad segment is observed:
three-height agreement
↓ failure case?
local plane residual
↓ still ambiguous?
surface-normal coherence
↓
test the whole artwork rectangle for clearance
That keeps the cheap detector in the common case.
The more important architectural step is what happens after detection
I think your “compute once per scene, rotate data monthly” idea is the right boundary.
A useful comparison is OpenVGAL, an open-source virtual gallery project.
Its authored gallery templates contain Occupancy_* planes: simple strips saying, effectively, “art can be placed here”. The gallery generator extracts those strips and then performs width-aware artwork packing from data.
Your case is harder because the room was generated, so a human did not author the occupancy planes.
But the architecture maps almost perfectly:
your generated collider
↓
three-height / plane tests
↓
stable usable-wall strips
↓
monthly artwork metadata
↓
packing / slots
In other words, your current raycast step can be viewed as an automatic front-end that generates the occupancy abstraction an authored gallery would normally supply manually.
Once those strips exist, I would try hard to keep the monthly system completely independent from the raw collider.
True physical scale does not have to conflict with data-only rotation
This is where I would slightly modify the current design choice.
Your fixed square slots are a very defensible production compromise, especially if the full-screen overlay gives the true dimensions.
But data-only operation does not require sacrificing scale.
OpenVGAL’s current layout stores artwork width/height in real-world centimetres and packs them into available occupancy strips by width.
There is also academic work on automated virtual-gallery layout, for example Space-adaptive Artwork Placement Based on Content Similarities for Curating Thematic Spaces in a Virtual Museum, which treats artwork placement as an optimization over spatial constraints rather than hard-coded manual transforms.
So you could support three modes without changing the scene authoring boundary:
Fixed-slot mode
- what you have now;
- simplest;
- very predictable.
Physical-scale mode
- store real dimensions in metadata;
- place at true scale if the strip has capacity;
- overflow to another strip when necessary.
Hybrid mode
- preserve true scale for works where scale is curatorially important;
- slot-fit the rest.
That turns “true scale or data-driven operation” into a policy choice rather than a technical limitation.
For a monthly rotation system I would probably still retain fixed slots as the default, because predictability is valuable. But the stable occupancy layer gives you room to add scale-aware packing later without reauthoring 11 scenes.
6. Selective illumination: Spark already has a lightweight version of option (c)
This one seems the most directly actionable.
Spark’s current Splat Editing documentation says that SplatEdit applies RGBA/XYZ fields to splats as part of the normal SplatMesh pipeline.
For color it currently exposes:
MULTIPLY;SET_RGB;ADD_RGBA.
And the available spatial SDFs include:
- sphere;
- box;
- ellipsoid;
- cylinder;
- capsule;
INFINITE_CONE.
The documentation explicitly describes spheres as useful for point-light-like effects and infinite cones as useful for spotlight-like effects.
So your option (c) is not only tractable in principle; Spark already exposes the basic building blocks for it.
There is also a first-party Dynamic Lighting example that loads an SPZ scene, creates SplatEdit lighting layers, applies spherical ADD_RGBA regions, and changes the colors over time in the animation loop.
For your use case, conceptually I would start with something like:
// sketch only — tune position/orientation/falloff for each wall
const spill = new SplatEdit({
rgbaBlendMode: SplatEditRgbaBlendMode.ADD_RGBA,
softEdge: 0.4,
});
const cone = new SplatEditSdf({
type: SplatEditSdfType.INFINITE_CONE,
color: warmLightColor,
opacity: 0, // avoid changing splat opacity
radius: coneWidth,
});
spill.add(cone);
// Prefer attaching/scoping the edit to the room SplatMesh
// rather than unintentionally editing every splat in the scene.
roomSplat.add(spill);
I would keep the RGB contribution fairly restrained. Spark’s documentation explicitly warns that ADD_RGBA can become hyper-saturated, and adding non-zero alpha can make previously low-opacity splats more opaque.
That matters here because the goal is not to make the wall glow; it is to create just enough local response that the painting no longer reads as a self-illuminated display.
Performance: 500k is not enough information by itself
Spark’s performance guide gives broad platform budgets in the millions of splats, but also makes an interesting warning: even about 500k splats concentrated in a small screen area can become a GPU rendering/blending bottleneck.
So I would not estimate this from splat count alone.
The lowest-cost benchmark is probably just:
same room
same camera poses
same pixel ratio
A: SplatEdit disabled
B: one soft cone enabled
and compare frame/GPU time at:
- normal walking distance;
- close to the painting;
- an oblique angle where many splats overlap in screen space.
That will tell you more than a theoretical “500k should be fine”.
One small current-version trap
The Spark repository’s Dynamic Lighting example currently contains an ambient layer using SplatEditRgbaBlendMode.DARKEN, while the current SplatEditRgbaBlendMode source and documentation list only:
MULTIPLY;SET_RGB;ADD_RGBA.
So I would use the current documented enum as the API contract rather than blindly copying every line of that example. The ADD_RGBA lighting part itself matches the current API.
If the fake spotlight is visually sufficient, I would stop there
This seems important.
Your interaction requirement is:
one artwork brightens as the visitor approaches, with a believable hint that its surrounding wall responds.
That does not necessarily require recovering a physically correct time-varying radiance field.
A subtle SplatEdit cone plus controlled artwork brightening may satisfy the perceptual requirement at a tiny fraction of the complexity of relightable 3DGS.
If it does not, then I would move up one level to the proxy-mesh approach mentioned under Q4.
Only if the exhibition genuinely needs:
- correct cast shadows;
- consistent normals/material response;
- large lighting changes;
- arbitrary moving light sources;
would I start looking seriously at methods such as GauUpdate, GS inverse rendering, or diffusion-based 3DGS harmonization.
Putting it together
I think the useful design boundary is roughly this:
GENERATION TIME
│
prompt / references / Chisel / Expand
│
▼
Marble world + pano
│
┌───────────┴───────────┐
▼ ▼
visible SPZ collision mesh
│ │
│ ▼
│ usable wall strips
│ │
│ stable slot/packing
│ │
└──────────┐ monthly artwork data
│ │
▼ ▼
RUNTIME EXHIBITION
│
┌────────────┼─────────────┐
▼ ▼ ▼
artwork texture local probe splat spill
/ frame material / grading via SplatEdit
The part I like about this separation is that none of the work you have already done needs to be thrown away.
- The three-height test can remain the cheap front-end for deriving wall occupancy.
- The current artwork grading remains a useful baseline.
- Fixed slots can remain the default while leaving room for true-scale packing later.
- Your interaction model can be restored approximately with existing Spark machinery.
- Cross-room generation consistency can be attacked upstream with more shared spatial evidence rather than by complicating the exhibition runtime.
If I had to pick only three experiments before doing anything more ambitious, I would choose:
- Q4: make sure the painting texture/material is in the same Three.js color-management/tone-mapping pipeline as the rest of the render;
- Q6: try one soft
INFINITE_CONE SplatEdit on one representative 500k room and measure it;
- Q2: compare one pair of independent rooms against one pair produced with genuinely shared structural context (Chisel/reference/continuation), rather than another fixed-seed prompt variant.
Those three should answer fairly quickly whether the remaining problems require research-grade methods or whether the production-friendly approximations are already enough.
