NGMP implementation

.github/instructions/ngmp.instructions.md

@@ -0,0 +1,70 @@
+---
+applyTo: '**/GeneralsOnline/**,**/NextGenMP/**'
+---
+
+# NGMP Subsystem Implementation Instructions
+
+These instructions govern the Next-Gen Multiplayer (NGMP) client protocol integration into **GeneralsX**. All changes in `GeneralsOnline` and `NextGenMP` components must adhere strictly to these guidelines.
+
+> [!IMPORTANT]
+> **GameSpy Online is being discontinued.** NGMP is the sole multiplayer backend going forward. Compatibility with the legacy GameSpy online flow is **NOT a goal** and must NOT block NGMP development. If making NGMP work requires changing or breaking legacy GameSpy online code paths, that is acceptable and expected. Only the following non-online modes must remain fully functional: **Singleplayer campaigns**, **Skirmish vs AI**, and **LAN multiplayer**.
+
+---
+
+## GameSpy Compatibility Policy
+
+| Mode | Required to keep working | Notes |
+|---|---|---|
+| Singleplayer / Campaign | ✅ Yes | Must not regress |
+| Skirmish (vs AI, offline) | ✅ Yes | Must not regress |
+| LAN Multiplayer | ✅ Yes | Must not regress |
+| GameSpy Online (WOL) | ❌ No | Being discontinued; may be broken by NGMP changes |
+| GameSpy P2P auth/serial | ❌ No | Superseded by NGMP auth |
+
+When a piece of legacy UI code is in the critical path for NGMP but also serves the legacy GameSpy online flow:
+- **Modify it** to work correctly with NGMP, even if that removes or changes the GameSpy online behavior.
+- If a function or callback is **only** called in the GameSpy online context (never touched in singleplayer, skirmish, or LAN), it may be stubbed out, simplified, or removed to unblock NGMP work.
+- Prefer **replacing** legacy `TheGameSpy*` queue calls with equivalent NGMP interface calls rather than maintaining both paths in parallel.
+- Dummy/blackhole implementations (like dummy message queues) are a **last resort** workaround — prefer proper NGMP replacements whenever feasible.
+
+---
+
+## Golden Constraints
+
+1. **Strict Prohibition of `<windows.h>` & Win32 APIs**:
+   - Never include `<windows.h>`, `<winsock2.h>`, `<ws2tcpip.h>`, `<wincred.h>`, or `<shellapi.h>` in NGMP sources.
+   - Use standard C++ (`<chrono>`, `<thread>`, `<mutex>`), POSIX networking (`<sys/socket.h>`, `<netinet/in.h>`, `<arpa/inet.h>`), or SDL3 primitives (`SDL_GetTicks()`, `SDL_Delay()`, `SDL_GetPrefPath()`).
+
+2. **Platform Layer Isolation**:
+   - Low-level network socket and OS calls must reside exclusively in `Core/GameEngineDevice/`.
+   - Higher-level network logic under `GeneralsMD/Code/GameEngine/Source/GameNetwork/GeneralsOnline/` must use pure abstract interfaces and cross-platform abstractions.
+
+3. **Thread Safety & UI Main Thread Separation**:
+   - Network callbacks from HTTP/WebSocket/P2P threads **MUST NOT** directly access or mutate UI controls (`GameWindow`, `WOLLobbyMenu`, etc.).
+   - Network events must be posted to a thread-safe thread/FIFO queue, consumed strictly on the main render thread during `Shell::update()`.
+
+4. **Credential & Token Storage**:
+   - Do NOT use Windows Credential Manager.
+   - Save session JWT tokens in an obfuscated local configuration file under the directory provided by `SDL_GetPrefPath("GeneralsX", "GeneralsOnline")`.
+
+5. **Server Endpoint Environment**:
+   - Development backend server target: `ws://192.168.1.120:9001/ws` (WebSocket) and `http://192.168.1.120:9001/api` (REST).
+   - Server URLs must be configurable via INI/JSON config rather than hardcoded string literals.
+
+6. **Cross-Platform Math & Endianness**:
+   - Ensure network packet data serialization handles network byte order (`htons`/`ntohs`, `htonl`/`ntohl`) explicitly to support cross-play between x86_64 Linux and ARM64 macOS.
+
+7. **Reference repositories**:
+   - There are two reference repositories for NGMP, take a look on these to understand the protocol and implementation details:
+     1. `references/GameClient`: The client-side implementation.
+     2. `references/GameServer`: The server-side implementation.
+
+8. **REST API & Routing Conventions (Lessons Learned)**:
+   - **Environment/Contract Prefix**: The C# Kestrel backend strictly requires environment and contract version routing. Do **NOT** use `GetServerRESTEndpoint()` directly to build URLs. Always use `NGMP::GetAPIEndpoint("EndpointName")` (e.g. `NGMP::GetAPIEndpoint("Lobbies")`), which appends the required `/env/dev/contract/1/` prefix automatically.
+   - **Authorization**: Almost all endpoints require authorization. You must inject the `Authorization: Bearer <token>` HTTP header in all requests (GET, POST, PUT, DELETE). Missing this will result in a `401 Unauthorized` response.
+   - **HTTP Methods & Payloads**: Backend endpoints have strict method bindings (`[HttpGet]`, `[HttpPut]`, `[HttpPost]`). For example, creating a lobby requires `PUT /Lobbies`, while joining a lobby requires `PUT /Lobby/{id}`. The JSON payloads must also contain all expected fields (even if default or empty), or the server will reject the request with `400 Bad Request`.
+
+9. **WebSocket Protocol Conventions (Lessons Learned)**:
+   - **Message ID Routing**: The backend `WebSocketController` strictly uses integer `msg_id` values to route actions, NOT string action types (e.g., `msg_id: 1` for sending chat, `msg_id: 2` for receiving chat, `msg_id: 4` for member list updates). Always include the correct `msg_id` in sent JSON payloads and use it to parse incoming messages.
+   - **Network Rooms (Lobby Registration)**: The backend requires the client to explicitly register into a network room to receive lobby and player updates. When initializing the global lobby chat, you must send `{"msg_id": 3, "room": 0}` (NETWORK_ROOM_CHANGE_ROOM). If you fail to do this, the server considers you in room `-1`, which causes the HTTP `GET /Lobbies` endpoint to return `0 lobbies` (as it filters by your current room) and prevents you from receiving player list updates (`msg_id: 4`).
+   - **Heartbeats**: The WebSocket server automatically drops connections if a PING is not received within a timeout window (often 20s). The client must periodically send `{"msg_id": 8}` (ping) at least every 10 seconds to keep the socket alive.

.github/workflows/build-linux-flatpak.yml

@@ -85,10 +85,17 @@ jobs:
             flatpak-builder-${{ runner.os }}-${{ inputs.preset }}-
 
       - name: Build Flatpak Bundle
+        env:
+          NGMP_DEFAULT_HOST: ${{ secrets.NGMP_DEFAULT_HOST }}
+          NGMP_SERVER_HOST: ${{ secrets.NGMP_DEFAULT_HOST }}
+          NGMP_SERVER_PORT: ${{ secrets.NGMP_SERVER_PORT }}
+          NGMP_USE_SSL: ${{ secrets.NGMP_USE_SSL }}
+          NGMP_WEB_PORTAL_URL: ${{ secrets.NGMP_WEB_PORTAL_URL }}
+          NGMP_MOTD_URL: ${{ secrets.NGMP_MOTD_URL }}
         run: |
           set -o pipefail
           mkdir -p logs
-          ./scripts/build/linux/build-linux-flatpak.sh "${{ inputs.preset }}" "${{ matrix.game.value }}" 2>&1 | tee "logs/build_flatpak_${{ matrix.game.log_suffix }}.log"
+          bash ./scripts/build/linux/build-linux-flatpak.sh "${{ inputs.preset }}" "${{ matrix.game.value }}" 2>&1 | tee "logs/build_flatpak_${{ matrix.game.log_suffix }}.log"
 
       - name: Locate generated Flatpak bundle
         id: bundle

.github/workflows/build-linux.yml

@@ -159,6 +159,13 @@ jobs:
           echo "✅ vcpkg ready at: /opt/vcpkg"
 
       - name: Configure CMake (Linux)
+        env:
+          NGMP_DEFAULT_HOST: ${{ secrets.NGMP_DEFAULT_HOST }}
+          NGMP_SERVER_HOST: ${{ secrets.NGMP_DEFAULT_HOST }}
+          NGMP_SERVER_PORT: ${{ secrets.NGMP_SERVER_PORT }}
+          NGMP_USE_SSL: ${{ secrets.NGMP_USE_SSL }}
+          NGMP_WEB_PORTAL_URL: ${{ secrets.NGMP_WEB_PORTAL_URL }}
+          NGMP_MOTD_URL: ${{ secrets.NGMP_MOTD_URL }}
         run: |
           mkdir -p logs
           cmake --preset ${{ inputs.preset }} 2>&1 | tee logs/configure_linux.log

.github/workflows/build-macos.yml

@@ -113,6 +113,9 @@ jobs:
 
           # Freetype + Fontconfig (required by font rendering)
           brew install freetype fontconfig
+
+          # Protobuf, Abseil & OpenSSL (required by GameNetworkingSockets / NGMP P2P transport)
+          brew install protobuf abseil openssl@3
 
           # NOTE: GLM is intentionally NOT installed via Homebrew.
           # It is fetched via CMake FetchContent in cmake/glm.cmake (tag 1.0.3).
@@ -356,6 +359,12 @@ jobs:
           CXX: clang++
           CMAKE_C_COMPILER_LAUNCHER: ccache
           CMAKE_CXX_COMPILER_LAUNCHER: ccache
+          NGMP_DEFAULT_HOST: ${{ secrets.NGMP_DEFAULT_HOST }}
+          NGMP_SERVER_HOST: ${{ secrets.NGMP_DEFAULT_HOST }}
+          NGMP_SERVER_PORT: ${{ secrets.NGMP_SERVER_PORT }}
+          NGMP_USE_SSL: ${{ secrets.NGMP_USE_SSL }}
+          NGMP_WEB_PORTAL_URL: ${{ secrets.NGMP_WEB_PORTAL_URL }}
+          NGMP_MOTD_URL: ${{ secrets.NGMP_MOTD_URL }}
         run: |
           mkdir -p logs
           echo "VULKAN_SDK=$VULKAN_SDK"

.github/workflows/build-windows.yml

@@ -62,7 +62,7 @@ jobs:
         uses: actions/cache/restore@v4
         with:
           path: ${{ github.workspace }}/vcpkg-bincache
-          key: vcpkg-bincache-${{ runner.os }}-x86-${{ hashFiles('vcpkg.json', 'vcpkg-lock.json') }}
+          key: vcpkg-bincache-${{ runner.os }}-x86-${{ hashFiles('vcpkg.json', 'vcpkg-lock.json', 'cmake/ports/**') }}
           restore-keys: |
             vcpkg-bincache-${{ runner.os }}-x86-
 
@@ -73,14 +73,23 @@ jobs:
           New-Item -ItemType Directory -Force -Path $cacheDir | Out-Null
           "VCPKG_DEFAULT_BINARY_CACHE=$cacheDir" >> $env:GITHUB_ENV
           "VCPKG_BINARY_SOURCES=clear;files,$cacheDir,readwrite" >> $env:GITHUB_ENV
-          "VCPKG_OVERLAY_TRIPLETS=${{ github.workspace }}/triplets" >> $env:GITHUB_ENV
+          "VCPKG_OVERLAY_TRIPLETS=${{ github.workspace }}/cmake/triplets" >> $env:GITHUB_ENV
+          "VCPKG_OVERLAY_PORTS=${{ github.workspace }}/cmake/ports" >> $env:GITHUB_ENV
           "VCPKG_INSTALL_OPTIONS=--x-abi-tools-use-exact-versions" >> $env:GITHUB_ENV
 
       - name: Configure
         shell: pwsh
+        env:
+          NGMP_DEFAULT_HOST: ${{ secrets.NGMP_DEFAULT_HOST }}
+          NGMP_SERVER_HOST: ${{ secrets.NGMP_DEFAULT_HOST }}
+          NGMP_SERVER_PORT: ${{ secrets.NGMP_SERVER_PORT }}
+          NGMP_USE_SSL: ${{ secrets.NGMP_USE_SSL }}
+          NGMP_WEB_PORTAL_URL: ${{ secrets.NGMP_WEB_PORTAL_URL }}
+          NGMP_MOTD_URL: ${{ secrets.NGMP_MOTD_URL }}
         run: |
           $env:VCPKG_ROOT = $env:VCPKG_INSTALLATION_ROOT
           cmake --preset win32-vcpkg `
+            -DVCPKG_OVERLAY_PORTS="${{ github.workspace }}/cmake/ports" `
             -DRTS_BUILD_GENERALS=${{ matrix.game.build_generals }} `
             -DRTS_BUILD_ZEROHOUR=${{ matrix.game.build_zh }} `
             -DRTS_BUILD_CORE_TOOLS=OFF `
@@ -95,7 +104,7 @@ jobs:
         uses: actions/cache/save@v4
         with:
           path: ${{ github.workspace }}/vcpkg-bincache
-          key: ${{ steps.vcpkg_cache.outputs.cache-primary-key || format('vcpkg-bincache-{0}-x86-{1}', runner.os, hashFiles('vcpkg.json', 'vcpkg-lock.json')) }}
+          key: ${{ steps.vcpkg_cache.outputs.cache-primary-key || format('vcpkg-bincache-{0}-x86-{1}', runner.os, hashFiles('vcpkg.json', 'vcpkg-lock.json', 'cmake/ports/**')) }}
 
       - name: Build
         shell: pwsh
@@ -108,9 +117,26 @@ jobs:
           New-Item -ItemType Directory -Force ci-artifacts | Out-Null
           Copy-Item build/win32-vcpkg/${{ matrix.game.folder }}/Release/${{ matrix.game.exe }} artifacts/
           Copy-Item build/win32-vcpkg/${{ matrix.game.folder }}/Release/${{ matrix.game.pdb }} artifacts/
-          Copy-Item build/win32-vcpkg/vcpkg_installed/x86-windows/bin/zlib1.dll artifacts/
           Copy-Item build/win32-vcpkg/${{ matrix.game.folder }}/Release/${{ matrix.game.exe }} ci-artifacts/
-          Copy-Item build/win32-vcpkg/vcpkg_installed/x86-windows/bin/zlib1.dll ci-artifacts/
+
+          Get-ChildItem -Path @("vcpkg_installed/x86-windows/bin/*.dll", "build/win32-vcpkg/vcpkg_installed/x86-windows/bin/*.dll") -ErrorAction SilentlyContinue | ForEach-Object {
+            Write-Host "Collecting vcpkg DLL: $($_.FullName)"
+            Copy-Item $_.FullName artifacts/
+            Copy-Item $_.FullName ci-artifacts/
+          }
+
+          # Collect Desktop Release CRT DLLs (msvcp140.dll, vcruntime140.dll, etc.)
+          $vcReleaseDir = Get-ChildItem "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Redist\MSVC" -Directory -Recurse |
+            Where-Object { $_.FullName -match "\\x86\\" -and $_.FullName -match "Microsoft\.VC143\.CRT$" -and $_.FullName -notmatch "onecore" } |
+            Select-Object -First 1
+
+          if ($vcReleaseDir) {
+            Get-ChildItem -Path (Join-Path $vcReleaseDir.FullName "*.dll") | ForEach-Object {
+              Write-Host "Collecting Desktop Release CRT DLL: $($_.FullName)"
+              Copy-Item $_.FullName artifacts/
+              Copy-Item $_.FullName ci-artifacts/
+            }
+          }
 
           foreach ($name in @("binkw32.dll", "mss32.dll")) {
             $stub = Get-ChildItem build/win32-vcpkg -Filter $name -Recurse |

.github/workflows/release.yml

@@ -70,19 +70,22 @@ jobs:
     with:
       preset: linux64-deploy
       checkout_ref: refs/tags/${{ inputs.release_version }}
+    secrets: inherit
 
   build-macos:
     needs: [create-tag]
     uses: ./.github/workflows/build-macos.yml
     with:
       preset: macos-vulkan
       checkout_ref: refs/tags/${{ inputs.release_version }}
+    secrets: inherit
 
   build-windows:
     needs: [create-tag]
     uses: ./.github/workflows/build-windows.yml
     with:
       checkout_ref: refs/tags/${{ inputs.release_version }}
+    secrets: inherit
 
   # GeneralsX @build BenderAI 21/04/2026 Roll back the pushed tag when anything
   # downstream of create-tag fails, so a re-run with the same version is possible.

.github/workflows/replay-tests-windows.yml

@@ -252,6 +252,10 @@ jobs:
             $output += $stderrOutput
             $output += $stdoutOutput
             $output | ForEach-Object { Write-Host $_ }
+            if (!$timedOut -and $exitCode -is [int] -and $exitCode -ne 0 -and $output.Count -eq 0) {
+              $hexCode = "0x" + ([int64]$exitCode -band 0xFFFFFFFF).ToString("X8")
+              Write-Host "Process exited with code $exitCode ($hexCode) and produced no output"
+            }
 
             $source = if ($replay.Name.StartsWith("linux_")) { "Linux" } elseif ($replay.Name.StartsWith("macos_")) { "macOS" } else { "other" }
             $completionLines = @($stdoutOutput |

.gitignore

@@ -91,5 +91,6 @@ flatpak/*.flatpak
 #tmp and log files on root folder
 logs_*.zip
 *.log
+/cmake/ngmp_env.cmake
 references/GameClient/
 references/GameServer/

.gitmodules

@@ -2,3 +2,9 @@
 	path = references/fbraz3-dxvk
 	url = https://github.com/fbraz3/dxvk.git
 	branch = generalsx-macos-v2.6
+[submodule "references/GameClient"]
+	path = references/GameClient
+	url = https://github.com/GeneralsOnlineDevelopmentTeam/GameClient.git
+[submodule "references/GameServer"]
+	path = references/GameServer
+	url = https://github.com/fbraz3/GeneralsOnlineGameServer.git

AGENTS.md

@@ -36,6 +36,7 @@ GeneralsX is a cross-platform port of Command & Conquer: Generals Zero Hour for
 9. **Update worklog** – Update `docs/WORKLOG/YYYY-MM-DIARY.md` before committing (see [.github/instructions/docs.instructions.md](.github/instructions/docs.instructions.md) for details)
 10. **Reference repos** – Study patterns, don't copy-paste
 11. **Backport to Generals** – Bugfixes and improvements must be backported to the Generals base game.
+12. **NGMP over GameSpy** – The legacy GameSpy online backend is being discontinued. NGMP (`GeneralsOnline`) is the sole online multiplayer target. Singleplayer, Skirmish (vs AI), and LAN multiplayer must remain fully functional. GameSpy online compatibility is explicitly NOT required and must NOT block NGMP work. See [ngmp.instructions.md](.github/instructions/ngmp.instructions.md) for the full compatibility policy.
 
 ## Cross-Platform Determinism (Mac vs Linux)
 To guarantee cross-play between macOS ARM64 and Linux x86_64 without SyncCrash desyncs, you must obey the following rules:
@@ -289,5 +290,6 @@ The `**` at applyTo means all files, you MUST load it everytime.
 | [.github/instructions/platform-macos.instructions.md](.github/instructions/platform-macos.instructions.md) | `scripts/build/macos/**,references/fbraz3-dxvk/**` | macOS/DXVK build notes |
 | [.github/instructions/docs.instructions.md](.github/instructions/docs.instructions.md) | `**/*.md` | Documentation structure and workflow |
 | [.github/instructions/scripts.instructions.md](.github/instructions/scripts.instructions.md) | `scripts/**` | Script organization and naming |
+| [.github/instructions/ngmp.instructions.md](.github/instructions/ngmp.instructions.md) | `**/GeneralsOnline/**,**/NextGenMP/**` | NGMP cross-platform multiplayer integration guidelines |
 
 Update this table when instruction files are added, removed, or renamed.

CMakeLists.txt

@@ -106,6 +106,9 @@ include(cmake/miniaudio.cmake)
 # curl.cmake is self-guarded with if(SAGE_UPDATE_CHECK), so always safe to include.
 include(cmake/curl.cmake)
 
+# GeneralsX @feature GeneralsOnline NGMP protocol dependency setup
+include(cmake/ngmp.cmake)
+
 # GeneralsX @feature fbraz 03/05/2026 Phase 4: Deterministic math library integration
 # gamemath.cmake integrates fdlibm-based GameMath for cross-platform replay validation.
 # Upstream reference: Okladnoj, PR #2670

CMakePresets.json

@@ -83,7 +83,8 @@
                 "CMAKE_EXPORT_COMPILE_COMMANDS": "ON",
                 "CMAKE_MSVC_DEBUG_INFORMATION_FORMAT": "$<$<CONFIG:Release,Debug,RelWithDebInfo>:Embedded>",
                 "CMAKE_MSVC_RUNTIME_LIBRARY": "MultiThreaded$<$<CONFIG:Debug>:Debug>DLL",
-                "CMAKE_TOOLCHAIN_FILE": "$env{VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake"
+                "CMAKE_TOOLCHAIN_FILE": "$env{VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake",
+                "VCPKG_OVERLAY_PORTS": "${sourceDir}/cmake/ports"
             }
         },
         {

Core/GameEngine/Include/Common/GameDefines.h

@@ -17,7 +17,6 @@
 */
 
 #pragma once
-
 #include "WWLib/WWDefines.h"
 
 // Note: Retail compatibility must not be broken before this project officially does.

Core/GameEngine/Include/Common/UpdateChecker.h

@@ -50,6 +50,15 @@ class UpdateChecker
     // Returns false if still in progress or no update found.
     static bool poll(const char** outLatestTag);
 
+    // Returns true if background update check is finished
+    static bool isDone();
+
+    // Returns true if a newer version is available
+    static bool hasUpdate();
+
+    // Returns latest tag string or empty
+    static const char* getLatestTag();
+
     // GitHub repository used for the API query.
     static const char* getReleasesUrl();
 };

Core/GameEngine/Include/GameNetwork/GameSpy/LobbyUtils.h

@@ -53,3 +53,15 @@ enum GameSortType CPP_11(: Int)
 
 Bool HandleSortButton( NameKeyType sortButton );
 void PopulateLobbyPlayerListbox();
+
+enum LobbyGameModeFilter CPP_11(: Int)
+{
+    LOBBY_FILTER_ALL = 0,
+    LOBBY_FILTER_1V1,
+    LOBBY_FILTER_TEAM,
+    LOBBY_FILTER_FFA,
+    LOBBY_FILTER_AOD,
+};
+
+extern LobbyGameModeFilter theLobbyFilter;
+LobbyGameModeFilter detectGameMode(const std::string& name);

Core/GameEngine/Include/GameNetwork/GameSpy/PersistentStorageThread.h

@@ -30,6 +30,13 @@
 
 #include "gamespy/gstats/gpersist.h"
 
+#if defined(min)
+#undef min
+#endif
+#if defined(max)
+#undef max
+#endif
+
 #define MAX_BUDDY_CHAT_LEN 128
 
 typedef std::map<Int, UnsignedInt> PerGeneralMap;

Core/GameEngine/Include/GameNetwork/GameSpyOverlay.h

@@ -39,6 +39,7 @@ void ClearGSMessageBoxes();	///< Tear down any GS message boxes (e.g. in case we
 void GSMessageBoxOk(UnicodeString titleString,UnicodeString bodyString, GameWinMsgBoxFunc okFunc = nullptr);	///< Display a Message box with Ok button and track it
 void GSMessageBoxOkCancel(UnicodeString title, UnicodeString message, GameWinMsgBoxFunc okFunc, GameWinMsgBoxFunc cancelFunc);	///< Display a Message box with Ok/Cancel buttons and track it
 void GSMessageBoxYesNo(UnicodeString title, UnicodeString message, GameWinMsgBoxFunc yesFunc, GameWinMsgBoxFunc noFunc);	///< Display a Message box with Yes/No buttons and track it
+void GSMessageBoxCancel(UnicodeString title, UnicodeString message, GameWinMsgBoxFunc cancelFunc = nullptr);	///< Display a Message box with Cancel button and track it
 void RaiseGSMessageBox();		///< Bring GS message box to the foreground (if we transition screens while a message box is up)
 
 enum GSOverlayType CPP_11(: Int)