Maelstrom data support - #33
Open
ivan-ushakov wants to merge 32 commits into
Open
Conversation
ivan-ushakov
force-pushed
the
feature/maelstrom-data-support
branch
from
August 7, 2026 17:10
bd422fa to
ea8767e
Compare
…re caches Maelstrom (KD Lab, 2007) runs on VistaEngine, but not on this VistaEngine: its content was authored before ~August 2007 and this tree is a 2008+ snapshot. The gap is dated in our own source -- the "|name|first" / "|type|second" aliases carry CONVERSION 31.07.07, TextDB and TerToolCtrl carry 2008-1-18 and 2008-1-24 -- and it was enough to abort in loadAllLibraries before a window ever opened. XPrmIArchive already absorbs most of that drift on its own: unknown field names are skipped, missing ones keep defaults, unregistered enum values and classes warn and carry on. What it cannot absorb is a field whose C++ type changed under the name, because strtol parks on the '.' of a float literal and closeNode then demands the ';' and aborts. tools/maelstrom_convert.py rewrites exactly that. The rules were not guessed from the data -- diffing by field name conflates unrelated structs, and did report two false positives -- but derived by pairing every ar.serialize(member, "wire", ...) site with the member's declaration in both trees. Across the modules this build compiles that resolves ~1500 wire names a side and yields two real changes, both in Scripts/Engine/RigidBodyPrmLibrary: steering_duration is float there and int here, and groundPass/waterPass are PassabilityFlags rather than bool. It works on bytes, since the files are CP1251 with CRLF and the engine reads them verbatim. Fonts needed a third, generated file. Maelstrom rasterised its fonts offline and ships no .ttf at all; by 2008 the engine read a TTF named by UI_FontAttributes, a different file under a different shape. Without it the default font never creates and FT::Font::size faults on null inside UI_TextParser. The engine side is three fallbacks, each reached only when the normal path finds nothing, so this engine's own data behaves exactly as before: - Models. Maelstrom ships no .3DX for units or buildings -- 32 in the whole distribution, all terrain decals and sky helpers -- only 1896 baked .dat. That cache turns out to be the same chunked container as a .3DX written with the same C3DX_* ids, and cSkinVertex's layout never changed, so the vertex and index bytes go to initBuffersInPlace verbatim. LoadInternal can be reused for the chains block, nodes, materials and lights; LoadChainData cannot, because baking flattened the animation chunks and C3DX_ANIMATION_GROUP holds a plain record where a .3DX nests sub-chunks under the same id. - Textures. Shipped the same way, and simpler: ordinary DDS under the detail level the engine already selects. Without this everything renders white. - Terrain colour. loadVMP handled S5L2 already, but that revision stores one palette index per cell where clrBuf holds RGB565, and the existing conversion needs a true-colour buffer that is not allocated at run time -- so the branch fell through a release-mode no-op assert and left the terrain black. Read the indices through the world's own inDam.act palette instead. Camera::DrawSilhouetteObject is guarded on the null device and registered as Render-PORTING.md #22: stencil work that was never ported, unreached on retail Perimeter 2 and so unnoticed until data that uses it arrived. Documents/Maelstrom-PORTING.md is the register -- the drift, the converter, the cache formats, and what is still approximate. The visibility sets are the part to know about: Maelstrom's per-LOD group indices address a structure this tree replaced, so each set is collapsed to one catch-all group and parts the original hid per animation state are all shown. Perimeter 2 still loads and exits clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tion Maelstrom's interface is drawn from data whose schema predates this tree by a year, and two groups of fields it still writes are ones we stopped reading. The in-game HUD chrome is a 3D model, not UI sprites: UI_BackgroundScene draws one per race through a camera of its own, with the buttons, minimap and text laid over it as 2D controls. That camera used to be described in the data -- position, focusx, perspective -- and the read had been left commented out. By 2008 the focus had become the per-model "scale" (x scale2focus), the camera had gone orthographic, and a new modelPosition_ pushed the model 1024 units off the origin, so Maelstrom's model was drawn with Perimeter 2's camera: too large, wrong projection, ~130 units too high. The focus is useless without the placement, and only the translation is new -- modelAngles_ is the same constant in both engines, and zeroing it makes the model vanish edge-on. The minimap needed three fields that drifted apart. minimapAngle moved from Environment to Universe, so an old world writes it in the wrong block; getAngleFromWorld became exclusive with rotateByCamera and is now unreachable for data that sets both; and rotationScale did not exist, because rotating the map always rescaled it to fit. Maelstrom's worlds are 2048x4096 and turn the minimap 90 degrees, which needs all three. Perimeter 2 writes none of these fields, so it keeps its constructed defaults throughout -- measured, its minimap parameters are unchanged. Documents/Maelstrom-PORTING.md carries both, plus two new open items: the main menu draws no UI at all off a normal start, and OPTION_SCREEN_SIZE is an index into a C++ list, so Maelstrom's saved value picks a resolution its data never meant. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
GameShell::init loads exactly one path, Scripts\Content\Triggers\GlobalTrigger.scr. Maelstrom's sits a directory up, in Scripts\Content\, and its Triggers\ holds only AI scripts -- so the chain loaded empty. That chain is what starts the game. Its START trigger runs a Hide Cursor / Start Main Menu sequence, and Start Main Menu carries the ActionStartMission that loads Resource\Worlds\Menu.spg -- Maelstrom's main menu, like Perimeter 2's, is a running mission. With the file unread nothing fired: no mission, no screen selected, not one control reaching UI_ControlBase::redraw, and a black window that reads as a renderer fault and is not one. All eight classes the chain names still exist here, so it runs as written once found, and the intro reels skip cleanly under DisableVideo. Handled in the converter rather than with an engine fallback: it is a file location, not a format. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…nested
Pre-2008 a control state wrote its show modes flat, one field per
UI_ControlShowModeID name, inside a transparent openBlock(""). By 2008 the same
table had moved under a named "showModes" block. The contents are identical
either way -- EnumTable::serialize is what wrote them then and what writes them
inside the block now -- but our reader asks for the block, openStruct fails, and
it moves on.
Every control in Maelstrom's data therefore loaded with an empty show-mode table,
and a control with no show mode draws no sprite at all. That is what made the
interface look like a texture-loading fault: a blank main menu, white boxes where
the HUD's resource icons belong, a white panel where the minimap belongs. No
texture ever failed to load -- none was ever asked for. Measured before and after
on the same controls: tex=<no-mode> throughout, then real names
(1024_START_SCREEN.DDS, 1024_LOAD_MISSION.TGA, ...).
Perimeter 2 writes the block and takes the existing path unchanged; verified its
menu still resolves its own sprites.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four fields of UI_ControlBase drifted between Maelstrom's engine and this one,
and with the show modes now read they were what was left of the blank start
screen:
locText -> text a caption was a localization key the engine
resolved at load, not a literal; unread, every
button, title and prompt came up untitled.
borderColor -> borderClr a rename *and* a type change, sColor4f to
borderOutlineColor -> ...Clr Color4c. Pointing Color4c::serialize at a
float aborts the load mid-number, so read a
Color4f and convert.
borderEnabled -> gone the third flag that gated drawing at all.
screenZ_ -> screenZ depth stayed 0 everywhere and each screen drew
in list order rather than in depth order.
The white sheet was the second of these: every screen has a full-screen
"background" control whose only job is a borderFill dimming the world behind it
to 30% black. The flag kept its name, the colour did not, so the fill drew at
the constructed opaque white.
All four are input-only and ask for the old name only after the new one is
missing -- a name the data does not carry costs openNode a rescan of the whole
control, children and all. borderEnabled has no such guard, so it is asked for
only where a border is actually switched on.
Verified: Maelstrom reaches its start screen with captions, the dim behind them
and the right draw order; Perimeter 2 loads C2_M08 unchanged, in the same 2s.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The sun, sky, fog and shadow gradients, the sky models, the latitude and slant of the sun and the time of day used to be written flat in a world's "environment" block. 2008 moved all of it under an "environmentTime" sub-block, so openStruct fails on a pre-2008 world and it lights itself entirely from constructed defaults: midday where Menu.spg asks for 9:12, latitude 36 where it asks for 11, one default cloud layer instead of its own three sky models, and the built-in sun ramp instead of its 8-key one. The visible half of that is ambient_maximal -- 0.2 against the 0.5 the world asks for. Terrain survives it, its colour being baked per fine cell in the height map, but an object is lit from the sun alone, so every surface facing away from it comes out near black. Measured over matching patches of the same shot against the original game: terrain at 0.42x its brightness, the shaded side of a tower block at 0.06x. objectShadowing rides along. One ShadowingOptions used to light the ground and the objects standing on it alike; the 2008 split leaves the objects on a default the world never chose, so give them the ground's. The global_<name>_color flags beside each gradient are deliberately not read: the writer resolved them before saving, so the copy in the world is already the global gradient -- c1_m1.spg sets all six and carries Scripts\Content\GlobalAttributes' 11-key sun_color verbatim. Perimeter 2 writes the block and takes the existing path unchanged; verified that -world C2_M08 loads without entering the conversion. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ources
Drift between Maelstrom's data and this tree comes in two shapes. Where a
field's type changed under its name, rewriting the value is the whole fix and
tools/maelstrom_convert.py does it offline. Where a field changed nesting or
owner there is nothing to rewrite and nothing for the archive to skip: the
reader asks for a block the old writer never wrote and the entire subtree goes
unread -- a control's show modes, a world's whole lighting, all of its sources.
Those sites were being detected at runtime: ask for the 2008 name, fall back to
the old one when it is missing. That works, and for one or two fields it is
tidy, but it does not scale. A name the data does not carry costs
XPrmIArchive::openNode a rescan of the whole enclosing block -- three passes,
children and all -- so Perimeter 2 pays for every fallback, on every control in
the library. And the sources below cannot be detected that way at all: they
need a different call graph, not a different field name.
So they are written twice and chosen by MAELSTROM_DATA, off by default:
cmake -S . -B build-mael -DMAELSTROM_DATA=ON
The rule at each site is #ifdef MAELSTROM_DATA / #else / #endif with the 2008
layout in the #else, so a stock build is the code that was already there -- no
Maelstrom branches, no fallback lookups, and nothing a bug in one can reach.
The cost is that a binary reads one game's data or the other's. Converted:
the environment's lighting, the minimap's rotation and its three drifted
flags, control show modes, captions, border colours, depth, and the background
scene's camera.
The sources are new here. A world's SourceZones -- the zones that hold its
standing effects, its damage and its unit generators -- were written in the
environment block and 2008 moved them into Universe's sourceManager. Unread,
every placed effect in the world is absent: in Maelstrom's menu that is each
building fire, every smoke column and the green outflow from the pipe. This
looked like a renderer fault for a long time, because the .effect files all
load, the world builds 205 cEffects, and the particle renderer is reached
thousands of times a frame by the sun, the moon and the coast foam -- every
measurement short of which texture reached SetMaterial said the sprites were
being submitted and dropped. They were never created.
They also force the load order back: camera, universe, environment, where 2008
reads the environment first. A source refers to units -- its owner, its
targets, the squads a generator fills -- and SourceManager::serialize switches
each one on as it finishes reading it, so the players have to be in place
first. Maelstrom's own engine read the environment last for this reason.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
One object used to own it: EnvironmentAttributes, written as the world's "environmentColors" node. 2008 dissolved that object -- its fields became Environment's own, its fog-of-war colours FogOfWar's, two of its constants cWater's -- and moved the group out of the world into a preset file. Maelstrom ships no Scripts\Content\Presets\ at all, so loadPreset() opened nothing and the whole SERIALIZE_PRESET_DATA branch never ran, on any world. Menu.spg asks for fog at 900-1200 and a 2-1300 camera frustum and got the constructed 1000-1400 and 30-4000; the weather, the shore sprites, the lens flare and the underwater effect were not read at all. The group sits at two depths, so it takes two things. Most of it -- fog_enable, the underwater, bloom and DOF settings, outside, lensFlare_, fallLeaves, the ice and chaos textures, the cloud shadow -- is written flat in the environment block where this already reads it, and only needed the branch to run under the world filter as well. The rest needs a real descent: openBlock is a no-op in an XPrm archive, editor-only grouping that does not nest, and openNode's rescan skips a nested block whole, so from the environment's level not one name inside environmentColors is visible. openStruct descends for real. ownAttributes picks the copy. 11 of the 51 worlds carry their own node; the other 40 take the global one, which Maelstrom kept in Scripts\Content\ GlobalAttributes -- that file is its preset file, and loadPreset() now reads its environmentColors for exactly the worlds that ask for it. timeColors_ is deliberately not read. It is the node's own copy of the six sky gradients, and a world is not lit from it: Maelstrom lit from EnvironmentTime's gradients, written flat in the environment block, and reached into a timeColors_ only for the global set (ReplaceGlobal(GlobalAttributes::instance(). environmentAttributes_.timeColors_)). What the editor saved beside them in the world is that global set, a 9-key ramp against Menu.spg's own 8-key one, so reading it would overwrite a world's own lighting with the global default. Two names drifted rather than moved: outside was capitalised to Outside, taken with the archive's own |a|b alias so Perimeter 2 matches on the first name and pays nothing for the second, and FogOfWar's fogColor/scoutAreaAlpha were fogOfWarColor/scout_area_alpha. Verified against Maelstrom's data: fog 900-1200, height_fog_circle 500, frustum 2-1300, hideSmoothly true, effects 0/0/1e6, outside = ENVIRONMENT_WATER, and the underwater and ice textures resolving to real paths instead of empty strings. Perimeter 2 compiles the #else throughout and still loads to Universe created. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Perimeter 2's content sets is_reflect_sky on nothing, so the cubemap was registered as having no consumer and skipped. That is true of Perimeter 2 alone: 116 of Maelstrom's 1031 models name their reflection map sky.*, among them the city buildings, whose glass material carries a diffuse averaging RGB(45,42,31). Nearly all of that glass's colour is the reflection, so without it the towers render black -- which is what they did. cRenderSky draws again, from Environment::graphQuant. All six faces on the first frame, then one face per frame, and only the sky scene goes into each, so the steady cost is about one extra sky render a frame -- the amortization the original was built around. The faces are not rendered into directly. Every renderer here builds its own SDL_GPUColorTargetInfo, so aiming a pass at a single cube layer would mean threading a layer argument through all ten of them. Instead a face is drawn into an ordinary offscreen 2D target -- the path the planar water reflection already proves -- and copyToCubeFace moves it into the layer afterwards: one 256x256 copy per frame. createCubeTexture pre-fills all six faces with the sky fone colour, because a face is only written when a pass opens on it and the first sky draw of a world records nothing; sampled before then, a face read as uninitialised memory -- magenta on every reflective surface for the first frames. The shader is a permutation of the reflection shader we already had, because that is how the original did it too: one vsSkinReflection/psSkinReflection with an is_cube flag taken off the bound texture's TEXTURE_CUBEMAP. REFLECT_CUBE=1 carries the original's world reflection vector, dir + (2*dot(n,dir))*n, and a TextureCube sample; the renderer picks the permutation the same way the original picks the flag. The other half of the fix stands on its own: cObject3dx::Draw required !mat.is_reflect_sky in BOTH the reflection and the bump branch, so these materials fell through to the plain lit path even when a cubemap existed. The original tests pReflectTexture || is_reflect_sky together, with bump as the else. Restored. Verified against Maelstrom's data: the cube holds real sky (a dumped face measures teal 56,130,151), the windows material binds it with TEXTURE_CUBEMAP set, the cube pipeline is selected, and the glass is visible again. Perimeter 2 builds and runs unchanged -- it has no material that reaches any of this. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Maelstrom's .3DX record carries two boxes. cStaticLogicBound::bound is an optional collision box whose constructor zeroes it and which almost no model fills; cStatic3dx::bound_box is the real extent, guarded by is_inialized_bound_box. This tree merged the two into one boundBox, so the reader has to choose -- and it chose the logic one, leaving boundBox empty for nearly every model. The original's cObject3dx::GetBoundBox returns bound_box, so that is the one the merged member has to hold. Nothing repaired it downstream: cObject3dx's constructor only recomputes a box when isBoundBoxInited is false, and these files store that flag true. The damage surfaced far from the cause. UnitEnvironmentBuilding::setModel does `scale_ = radius()/max(boundBox.radius2D(), 0.001f)`, where an empty box turns the floor into a x1000 multiplier: corpses and other decor were built at scale ~10^4 (radius 12.99 gave scale 12986.40). Drawn into the shadow map, whose caster pipeline clamps depth rather than clipping it, that geometry pinned all 2048x2048 texels to 0.0 -- so every receiver compared as occluded and the whole terrain rendered at shadow intensity. Measured before and after with a validated readback of the map: 100% of texels exactly 0.0 before, min 0.060 / max 1.0 / no zeros after, with the object casters left enabled. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The D3D9 CreateTexture scanned a freshly loaded texture's alpha and recorded what it found: all-or-nothing alpha set TEXTURE_ALPHA_TEST, fully opaque cleared both alpha flags. cObject3dx::Draw reads that back through isAlphaTest() to pick ALPHA_TEST over ALPHA_NONE. Nothing replaced the scan when the D3D9 backend went, so every material stayed opaque and cut-out texels drew as solid colour -- the broken windows of Maelstrom's towers are about a third of the facade texture at alpha 0, and came out as black panes. Graded alpha is left alone: a material that wants a real blend gets there through its opacity map (is_opacity_texture), as the original does. The DDS loaders also had to stop premultiplying. It looked harmless -- it pairs with a (ONE, ONE_MINUS_SRC_ALPHA) blend and hides the grey halo around cutout decals -- but it is destructive: it zeroes the RGB of every fully transparent texel, and a material drawn ALPHA_NONE never looks at alpha, so it reads that RGB as colour. Nothing downstream can undo it. loadDDS, the base-cache path every Maelstrom texture comes through, also premultiplied while leaving isPremultiplied() reporting false, telling its consumers the opposite of what it handed them. Both loaders now flag the texture TEXTURE_ALPHA_TEST optimistically, as createFileImage does for every other format, so the scan above actually runs on a DDS -- it is gated on that flag, and without it a DDS never reached it. Specular and bump maps are excluded: their alpha is power and nothing. Affects both games, not just Maelstrom. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two formulas in EnvironmentTime::SetTime changed under the same field names, so Maelstrom's worlds are read correctly and then lit by the wrong arithmetic. Objects took the sun at double strength. Before 2008 one ShadowingOptions lit the ground and the objects on it alike, and SetTime wrote tilemap_color.a*2 and tilemap_color.rgb*2 into SetSun, which clamped to 1. Splitting a separate objectShadowing out in 2008 dropped the factor, because a world can now write the doubled numbers itself -- Maelstrom's cannot, and every one of them asks for ambient_factor 0.5, so the shaded side of a building was lit at half what the original gave it. The terrain is untouched: it never had the factor. shadow_color means something different in each engine. 2008 divides by the gradient's own darkest channel, treating the colour as a hue and taking the depth of the shadow from shadow_intensity alone -- hence the editor caption asking for "normal grey, about 0.5". Before 2008 the colour was the shadow, doubled and faded by the sun's height, and Maelstrom's worlds are authored that way: c1_m1 asks for (0.23,0.27,0.47) at noon, which the 2008 reading normalises to (0.48,0.54,0.90) -- barely a shadow, and the blue gone with it. time_shadow_off and speed_shadow_off were constructor constants, never serialized; shadowDecay, which replaced them, has no counterpart in the file. Both under MAELSTROM_DATA; the 2008 path is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The renderer built every pipeline CULLMODE_NONE, on the reasoning that the scene passes flip culling per node type and the device does not track D3DRS_CULLMODE. Reading the original the other way round: setCamera culls back faces throughout (CurrentCullMode = D3DCULL_CW) and flips the winding for the reflection camera, whose mirror matrix reverses every triangle (D3DCULL_CCW); Camera::DrawObject restores that with RS_CULLMODE -1 and DrawSortObject inherits it, so in the main scene only DrawObjectSpecial turns culling off. Drawing double-sided what the original culled shows a model's far wall from the inside, which is most visible through a hole in the near one. With SDL's counter-clockwise front face, D3DCULL_CW is CULLMODE_BACK and D3DCULL_CCW is CULLMODE_FRONT, so the reflection case keeps the FRONT it already had. The shadow caster stays double-sided: what the light sees is a separate question from what the camera does, and a one-sided caster changes which surface writes the depth receivers are compared against. One divergence needs MAELSTROM_DATA. Before 2008 DrawSortObject saved the cull mode, forced D3DCULL_NONE around the sorted pass and put it back; 2008 dropped the call. Maelstrom's transparent materials -- glass, foliage cards -- were authored against the first of those and expect both sides. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
logicRNDinterval is the *integer* macro -- it expands to logicRNDii(int, int) in the debug build and logicRnd(int, int) otherwise, and RandomGenerator offers only int operator()(int, int). Two callers hand it a float range, which truncates on the way in. Bird::create scaled every flock model by logicRNDinterval over a Rangef, so a size range of 0.01..0.04 collapsed to logicRnd(0, 0) == 0: gulls, eagles and fish were created, animated and attached at scale zero, and drew as nothing. Maelstrom wrote `logicRndInterval(owner->modelSize())`, which resolved to an `inline float logicRndInterval(const Rangef&)` overload; 2008 dropped that overload and rewrote the call site to the int one, taking the truncation with it. SourceBase::environmentAnalysis had the same slip: a wind sensitivity of 0.9..1.1 became 0..0, so move_by_wind multiplied the wind vector away to nothing. The line immediately below it already uses the float macro. Both are invisible in retail Perimeter 2 -- no shipped world contains a SourceFlock, and nothing there sets move_by_wind -- so this only shows up on Maelstrom's data. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
cObject3dx::PreDraw attaches a unit carrying ATTRUNKOBJ_SHOW_FLAT_SILHOUETTE to SCENENODE_FLAT_SILHOUETTE *instead of* SCENENODE_OBJECT, so that list is the only route those units have into the frame. DrawSilhouetteObject returned as soon as it saw a null gb_RenderDevice3D, which dropped them from the colour pass altogether rather than merely dropping their outline. The shadow attach happens earlier in the same PreDraw, and independently of the camera's own TestVisible, so the units kept casting shadows while being invisible -- the signature that pinned this down. Retail Perimeter 2 never fills the list, which is why it went unnoticed; Maelstrom's menu world fills it with the guard tower, the legionaries and the warship. Draw the list plainly, exactly as the child-camera branch below already does. The stencil outline itself is still unported (Render-PORTING #22): the whole effect is built out of stencil state the SDL GPU backend does not expose yet, and the body below it would fault on gb_RenderDevice3D's null. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
cStatic3dx::prepareMesh guarded BuildMeshes/CreateDebrises on gb_RenderDevice3D,
dating from a point in the port where the SDL backend could not raise model vertex
and index buffers at all. It can now, and gb_RenderDevice3D is null on every
platform since the D3D9 backend was retired, so the guard had quietly become
"never build a mesh" for everything reaching this path.
BuildMeshesLod and cStaticSimply3dx::BuildFromNode allocate through
gb_RenderDevice's Create/Lock{Vertex,Index}Buffer, which cSDLRenderDevice
implements, so the honest test is whether a render device exists at all.
Models that arrive as an in-place cache image are unaffected either way -- both
Perimeter 2's .3dxG and Maelstrom's baked .dat carry their buffers already built
and deliberately bypass prepareMesh.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every one of these sites indexes a chain list with an index validated against a different, longer list, or dereferences a pointer that a release build never checks. None of it mattered while no animation chain ever started: chainIndex stayed 0 and the visibility group stayed whatever the constructor set. The moment a unit animates, the first model whose lights, materials or nodes carry fewer chains than the model itself walks off the end. cObject3dxAnimation::SetAnimationGroupChain checks the index against the model's animationChains_, but lights, materials and nodes each keep their own list. UpdateMatrix already bounds the node case and skips; the light loop in Update, the two visibility-track readers and ProcessEffect did not, and the material index is bounded where it is stored rather than at the six places that read it. UpdateVisibilityGroup took &groups[index] blind. That is undefined for an out-of-range index, and for an empty list it quietly hands back null, because an empty vector's data() is null -- the crash was a null dereference in the light loop's visibleNodes lookup. SetVisibilityGroup only screens the index through an xassert, so nothing catches it outside a debug build, and a visibility set that exists but carries no groups slips past DummyVisibilitySet, which only fires when there is no set at all. Hand out null on purpose and let both readers treat "no group" as "not visible". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Maelstrom's units loaded, drew and moved, but stood perfectly still while doing it -- legionaries slid across the ground with their legs frozen. Two separate pieces of drift, both silent because these files serialize by name: a key that no longer exists reads as nothing, and a name that changed meaning reads as the wrong thing. Neither produces a parse error. The list itself was never read. 2008 rewrote it and gave it a new key, "animationChainsNew", so Maelstrom's "animationChains" matched nothing and every unit came up with an empty chain list: nothing to start, so ChainController::quant returned at finished() and no phase ever advanced. The ids inside it name chains this tree never asks for. Maelstrom kept the gait in the chain id -- CHAIN_STAND / WALK / RUN / TURN, plus the walking and running variants of fire and aim -- where 2008 collapsed each family onto one id and tells the members apart by MovementState's pose and movement bits instead. StateBuildingStand and UnitLegionary::setMovementChain both look up CHAIN_MOVEMENTS, findAnimationChainInterval compares chainID for equality, and nothing matched. CHAIN_STAND survives in this tree but means something else now (projectiles and items), and CHAIN_RUN and CHAIN_TURN are gone entirely -- their old values belong to CHAIN_BUILDING_STAND and CHAIN_PAD_STAND today, so the pre-2008 ids are given fresh values rather than their original ones, and the descriptor maps the old names onto those. CONVERSION 15.02.08 already turns Maelstrom's flat movementState into the split state/terrainType pair, but it can only recover what was written: the pose and movement bits were never in the file, because they were the chain id. The fold puts them back. findAnimationChainInterval's test is a superset one, and getMovementState always names both a pose and a movement, so the sets are deliberately generous -- a standing unit still reports whichever gait it would walk with, so the stand chain has to accept all of them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The converter rewrites a data root; nothing in this tree creates one, and the document never said so. Record the two routes -- unzip the distribution, or lay a symlink farm over a read-only pristine copy -- and show both passes of an actual run, dry and applied, with the counts they print. Also say where the .ttf comes from, since Maelstrom ships none: copy one out of Perimeter 2's own Resource\UI\Fonts. Two things there are silent. --font is engine-relative with backslashes, not a host path, and build_font_attributes only interpolates the string -- a wrong path converts cleanly and then faults in FT::Font::size(this=0x0) at run time, which reads as a font bug and is not one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The silhouette section said the draw list was guarded off. It has not been since the units started drawing; what is still missing is only the outline. Say what guarding the whole function off actually cost, because the symptom is worth recognising -- PreDraw routes those objects to SCENENODE_FLAT_SILHOUETTE instead of SCENENODE_OBJECT, so the units left the frame while their shadows, attached earlier in the same call, kept drawing. Render-PORTING.md #22 carried the same stale claim in its status column. The animation chains had no section at all, though the drift is the largest found since the show modes. Record both halves -- the renamed list key and the gait moving out of the chain id -- and the three parts of the fold that are not mechanical. Two items in "Still open" were wrong rather than stale. The frame interval really is absent from the cache, but nothing consumes it: intervalSize() is its only reader and has no callers, so it is a Models trap, not an open problem. The basement is not silently ignored on the path that matters -- the cache reader has to parse it to stay in step, and then drops it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A model's meshes are split into visibility groups and the object shows one at a time -- a transformer's robo/transform/tank forms, a building's build stages, a wreck's debris. The unit names the one it wants, and the name is resolved against the model's group list. Only the set's head record was being read, so that list was always DummyVisibilityGroup()'s single catch-all and every name in the data fell back to group 0. Of the 3436 VisibilityGroup requests in the shipped data, 3294 name a group that exists in some cache. The groups are in the cache in full: C3DX_AVS_ONE_RAW carries each one, and C3DX_AVS_ONE_LODS follows with three index lists into them, one per LOD. Better, StaticVisibilityGroup::Load already reads Maelstrom's record byte for byte -- it kept the throwaway lod field through the rewrite. The reader survived; nothing was calling it here. Take the first LOD's list. The other two name the same groups in the same order in all 836 sets that split them, and each LOD renumbers its visible_shift from bit 0, so LOD 0 matches the bunch masks of every LOD. The node flags come from the file too -- they cannot be derived from the mesh names the old code used, because temp_visible_object is empty in all 4235 cached groups; baking had already resolved it into visible_nodes. This did not show the wrong parts, it showed too few. isVisible tests the bunch mask against the group's visibility, and the dummy's is 1 << 0, so only bunches carrying bit 0 ever passed: 2151 of 3475 LOD-0 bunches, leaving 1324 in 491 models undrawable in any state. An object that never switches groups renders identically either way, which is why it went unnoticed -- the fault is invisible until a unit transforms, finishes building or dies. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RigidBodyPrm's wire list is 23 names longer on the old side, and the missing names read like the whole pre-2008 flight model: flying_stiffness, rudder_speed, steering_acceleration_max, the lot. Nineteen of the twenty-three are dead in Maelstrom too -- declared, defaulted, saved, loaded, never read -- so none of them can explain any behaviour. Only four are live, and they are named here so the next person starts from those. hoverMode and alwaysMoving are the mirror case: fields we read that the old schema cannot write, tempting to set for aircraft, and already correct at their defaults. hoverMode means riding on top of the unit beneath, not hovering in place, and the original has neither that machinery nor canRotate_/canMoveBack_ -- so false reproduces it and setting them would diverge. Missile knockback is unreachable twice over: absent from the original engine, and never enabled by the data. The method note is the part worth keeping. A wire name the old data writes and this tree does not read proves nothing on its own; check that the original consumed it before calling it a regression. Also record what the menu actually is -- a scripted set-piece whose clock is frozen per screen -- since that decides whether its lighting is a bug, and note that ActionSetCoastSprites is the one unresolvable class the menu reaches. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Helicopters flew a clean path while spinning about their own axis -- heading swinging +-40 to 57 degrees per quant, the model rolling through ~140, reversing whenever the turn rate changed sign. The position was never wrong: traced, step held 4.99-5.10 per quant throughout, which is why it reads as an animation fault and is not one. The bank is clamp(rotSpeed()/G2R(pathTrackingAngle), -1, 1) * additionalHorizontalRot. Pre-2008 divided by pathTrackingAngle as written, in degrees; 2008 converted the denominator to radians. G2R(30) = 0.524 against 30 is a 57x larger ratio, so the clamp saturates and the bank becomes the whole of additionalHorizontalRot -- an angle in radians, and Maelstrom's aircraft prms carry 20 of them. Our own PathTracking.cpp:507 still has the old formula untouched, so the tree already disagreed with itself; only the FormationController path converts, and that is the one a flying unit in a formation takes. Not a port regression. git log -L shows the line unchanged since the initial commit, and Perimeter 2 reads the same 20 and 45 in three of its own flying prms, so stock 2008 does this too. Hence MAELSTROM_DATA, with the #else byte-identical. Measured after: peak heading swing 118 -> 11 degrees, peak roll 0.94 -> 0.18. The reversal *rate* barely moves (11-24%), because what was wrong was the amplitude of each correction, not how often they happen. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ivan-ushakov
force-pushed
the
feature/maelstrom-data-support
branch
from
August 8, 2026 17:08
ea8767e to
85a1288
Compare
…g class cObject3dx's destructor asserted gb_RenderDevice3D, meaning "the device must still exist while this object releases its GPU resources". That pointer is the D3D9 device and has been permanently null on every platform since the retirement, so the assert could only fail -- it fired for every object destroyed in a build with assertions on, which is how it turned up: buried in a Windows 10 user's log under a pile of unrelated ones. It was the last assert of its kind in the tree. Not re-pointed at gb_RenderDevice. Destruction legitimately outlives the device at shutdown, which is exactly what the sPtr buffer destructors already guard for, so the same assert on the live pointer would fail there instead. The same log named UI_ACTION_EXPAND_TEMPLATE, a Maelstrom UI action with no counterpart in UI_Enums.h -- an eleventh class in the register's list, and the second known to be actually reached. Non-fatal: XPrmIArchive reports, skips the block and carries on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ulting Loading every world in turn with -world found four that crash: Maelstrom's TEST_Effects, TEST_Environment_Buildings, TEST_Environment_Trees and TEST_World_Tutorial. All four die in vrtMap::getAlt (VMAP.H:122) on a null vxaBuf -- the "water vxaBuf null crash" that has been open and unreproduced. None of it is schema drift. Those four are .spg files with no world directory beside them: no world.cls to open, so allocMem4Buf never runs and vxaBuf stays null. vrtMap::load gets that right and returns false. GameShell::GameLoad called it as a statement and dropped the result, so the mission went on to build a universe over a heightfield that was never allocated. Unconditional, because Perimeter 2 ships orphan worlds of its own -- cs_c1_open and intro_01 have no directory either, so retail can reach the same fault. The name goes out through dprintf as well as ErrH.Abort. Abort prints to stderr, and stderr stops reaching the log after renderer init, so the abort on its own turned a crash into a silent exit(1) -- which is barely better, since nothing then said which world was at fault. The rest of the pass is the good news: 47 of 51 worlds build their universe with no assertions at all, across all four campaigns, the multiplayer maps, the cutscene worlds and the menu. The register claimed two. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
GameOptions stores an option as `number = <index>` into a list that lives in C++ (GameOptionsSerialization.cpp), not in the data. The `comment` beside it looks like the list but is only a label the engine never reads, so an index written by one revision silently selects a different entry in the other. Three lists changed shape. OPTION_SCREEN_SIZE 25 is 1920*1080 there and 1680*945 here, because we added 858*484 and 1680*945; OPTION_SHADOW 3 is High there and we dropped Maelstrom's Circle; OPTION_LANGUAGE 0 is "Use Steam Language", which we have no equivalent for. The other four indexed options have identical lists in both games and are left alone. Screen size is the one that shows. The wrong index changes the window aspect, and with it which branch of the letterbox code runs -- and it scales the UI font, since createFont computes fontSize_ * windowHeight / 768. A 16-point font at the 900-pixel window that follows asks FreeType for 19, which is the ":19" in a Windows 10 user's UI_Font.cpp:80 assert. The comment is Maelstrom's own list, so the conversion needs nothing external: resolve the index to a name against it, then look that name up in ours. The comment is rewritten too, and that is what keeps the pass idempotent -- leaving the old list would make a second run resolve the new index against the old names and convert twice. Verified: 25 -> 27, 3 -> 2, 0 -> 0, second run byte-identical, and the file keeps all 161 lines and all 161 CRs. A bad antialias index is a different fault and no offline pass can settle it: those are filtered at run time against what the GPU reports. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…Type Maelstrom ships no TrueType at all -- zero .ttf in the distribution -- so the UI ran on whatever face --font named, and the register recorded the lettering as a substitution we could not avoid. It pointed at the wrong file. The .xfont files beside the masters are only a cache: cFontInternal::CreateTexture tries Load(name) first and, failing, builds from the master and writes the .xfont back. There are six of those, at whatever sizes one machine asked for, against fifteen *.font masters that are the real source -- every face, any size. A master is a 1bpp bitmap, laid out as the original's LoadFontImage reads it: "font" | int32 real_height | uint16 char_min | uint16 char_max per char: int32 width | uint8 bits[((width+7)/8) * real_height] All fifteen parse with every byte consumed. Courier New.font reports one width for every glyph, which is a free check that the width field is read correctly. BitmapFont.cpp turns one into an ordinary FT::Font -- same atlas, same charTable_, same metrics -- so nothing downstream knows the difference, and createFont dispatches on the extension with the TrueType path untouched. All of it is #ifdef MAELSTROM_DATA: verified absent from the Perimeter 2 binary. Three things that are easy to get wrong. The masters are not one codepage: byte 0xC0 is a plain A in Russian/MAEL_small.font and an accented A-grave in the English one, so the LocData directory picks it and the reverse map is built with the same MultiByteToWideChar that built the char table. The whole cell is scaled, padding included, as CreateImage did -- trimming to the ink would give a larger font than Maelstrom shipped. And the greys are the downsample: the source is one bit deep. The converter now needs no --font, which retires the failure class it invited -- it never checked the named file existed, so a wrong path converted cleanly and faulted later in FT::Font::size(this=0x0). --font survives as an override. Verified: all three faces build from the masters at 23/45/34 px into 256x256, 1024x512 and 512x512 atlases, no assert; Perimeter 2 loads unaffected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
selfCameraRestriction, cameraBorder and the flat CAMERA_* run were Environment fields and are CameraManager ones now, so a pre-2008 world writes them in its environment block and CameraManager::serialize never sees a name of them. Read them there and hand them across, as the original did. The global set in Scripts\Content\GlobalAttributes was half-arriving already -- this tree opens that node, so the shared names were read and only the renamed ones defaulted: zoomMin 300 against the file's 50, heightMin 0 against 50, and zoomDefault 300 against 500, which is the distance every mission opens at. CAMERA_THETA_MIN is not a floor under the tilt. The tilt ceiling falls off with distance, and the old pair are the two ends of that ramp -- MAX zoomed in, MIN zoomed out -- so they pair with thetaMaxLow and thetaMaxHigh and the floor stays 0. The 2008 defaults for that pair are 60 and 18 degrees, which are Maelstrom's own global values. The zoom-speed trio is deliberately left unread: the dynamics were rewritten in 2008, not renamed, and no value of zoomKeyAcceleration reproduces a delta that was not scaled by the distance. aboveWater goes false, the original having tracked the ground with vMap.GetApproxAlt rather than the water surface. Also corrects the register: FarPlane and NearPlane are the depth-of-field pair, not camera fields, and Environment::serialize reads them already. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…t matter
XPrmIArchive reports an unregistered class, skips the block and carries on, so
none of these had been sorted by whether anything reaches them. Logging str at
the result == -1 branch turns that into a measurement: seven of the eleven live
in Scripts/Engine/AiActionChainList and AuxDictionary, two files no engine
opens -- and grepping origin/Maelstrom for AiAction/AiCondition/AttributeReal
finds no C++ at all, so they were dead in Maelstrom too. ActionSquadMove is
live there but its only call site is in MISSIA.scr, which no world and no other
chain names. The other three are reached, and are ported:
ActionSetCoastSprites -- 14 call sites, one per screen of the main menu. Our
cCoastSprites::serialize is Maelstrom's Init() with the read folded into it, so
the apply half is split back out for the action to call.
ConditionObjectNearObjectByLabel -- measures from a labelled unit where the
surviving sibling measures from an anchor. Ten of its twelve uses are in AI
chains that c1_m1, c1_m7, c2_m1 and c2_m7 load, where an unregistered condition
reads as one that is never true.
UI_ACTION_EXPAND_TEMPLATE -- the pre-2008 UI_ACTION_LOCALIZE_CONTROL, expanding
the control's own caption rather than a string of its own. The in-game clock is
one of its six users: L_ASTRO TIME resolves to "{time_h12} : {time_min}
{time_ampm}", drawn literally without it. The caption is kept unexpanded beside
text_, as the original kept it, or the first update consumes the template.
Tooltips come back with them. A control carried its hover text and cursor
itself, in a transparent openBlock("hover"); 2008 collected the pair into a
UI_ACTION_HOVER_INFO action. Reading the old fields back into that action lets
findAction's control-then-state lookup stand in for the old hint() fallback,
and leaves the delay, the type match and the template expansion untouched. 757
controls name a tooltip key and not one names the action, so the 2008 path was
dead end to end -- 727 of them on the three race HUDs, not in the menu.
Verified: removing one REGISTER_CLASS brings ActionSetCoastSprites back 14
times; with all three registered, a boot to the menu and a load of the four
campaign worlds report nothing unresolved.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… own copyToCubeFace acquired a command buffer of its own and submitted it on the spot. flushTarget above it only *records* the face's render pass into commandBuffer_, which is not submitted until EndScene -- so the copy ran first and read pFaceTarget before anything had been drawn into it. Every slot therefore received the previous face: the cube came out rotated by one, and each lookup returned a neighbour. On the first pass, when nothing had been rendered yet, all six faces took uninitialised magenta. It hides well. The six faces of a still sky differ little, so the reflection stays plausible -- it is simply the wrong direction. It surfaced only once the water started sampling the cube, asking for the sky overhead and getting the horizon band, whose lower half is the fone colour below the horizon: a flat sheet with no sun in it. The other consumer, the 116 Maelstrom models with is_reflect_sky, had been reading a neighbouring face all along. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two faults, both leaving the water a flat sheet of colour. The wave maps never loaded under MAELSTROM_DATA. waves.dds / waves1.dds are read in one place, cWater::serialize's SERIALIZE_PRESET_DATA branch. Perimeter 2 reaches it through loadPreset() and Presets\global.set; Maelstrom ships no Presets directory, its loadPreset() descends into GlobalAttributes' environmentColors alone, and the world is read under SERIALIZE_WORLD_DATA. The branch never ran, both handles stayed null, and the renderer bound its flat 1x1 stand-in -- a zero slope, which is no waves in any technique. The pre-2008 engine loaded them in the constructor (origin/Maelstrom:Water/Water.cpp:88); 2008 made them serialized fields and moved the load into the preset. Load them in the constructor again, with no #ifdef: GetElement3D is a cache, so Perimeter 2's later call only takes a reference, and no shipped preset names bumpTextureName in either game. WATER_EMPTY was standing in for the reflections-off look. It is not that. ca9aa43's setTechnique picks, on any PS2.0 card, IsReflection() ? WATER_LINEAR_REFLECTION : WATER_REFLECTION and WATER_EMPTY is the no-PS2.0 path. Maelstrom ships OPTION_REFLECTION = false, so its water ran a flat vPS11Color whose only variation is an alpha crest term that saturates past ~16% depth. Port WATER_REFLECTION as a third build of water.{vert,frag}.hlsl (CUBE=1, from water_cube.{vsl,psl}) with its own pipeline pair and a TextureCube on stage 2, and let setTechnique make the original's choice. Note there is no glint term in water_cube.psl: the sun sliding over the crests is the sun in the cubemap, displaced by cube.xy += (tex0.xy+tex1.xy)*0.3. uv_mirror is normalize(camPos - pos) with dir.z -= pos.z -- the eye vector mirrored about z=0, not a surface normal -- ported verbatim, so at a low camera the reflection is a horizon one rather than a mirror of the zenith. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
createFont's .font test called strcasecmp. That is the POSIX name; MSVC's CRT has only _stricmp, so the line does not compile with Visual Studio. It went the wrong way through the compat layer: Platform/WindowsAPI.h is force-included everywhere and maps the MSVC spellings onto POSIX in its non-Windows branch (#define stricmp strcasecmp), which makes stricmp the portable name here and strcasecmp the one that only builds off Windows. The rest of the tree already writes stricmp. CI did not catch it and could not have. The call sits inside #ifdef MAELSTROM_DATA, the option defaults to OFF, and none of the three workflows pass -DMAELSTROM_DATA=ON -- so the preprocessor deletes the block before any CI compiler sees it, on every platform. The whole Maelstrom path is uncovered that way, BitmapFont.cpp included: it is listed unconditionally in Render/CMakeLists.txt but its entire body is guarded, so CI builds it as an empty translation unit. A green Windows run is not evidence that Maelstrom code compiles under MSVC; only a local MAELSTROM_DATA=ON build is. strcasecmp was the only POSIX-only spelling in the ~30 guarded files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.