[PR #133] [MERGED] core: Implement new memory manager #1322

New issue

Closed

opened 2026-02-27 21:12:05 +03:00 by kerem · 0 comments

kerem commented

2026-02-27 21:12:05 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/shadps4-emu/shadPS4/pull/133
Author: @raphaelthegreat
Created: 5/14/2024
Status: ✅ Merged
Merged: 5/16/2024
Merged by: @raphaelthegreat

Base: main ← Head: new-memory

📝 Commits (3)

b321d3c core: Implement new memory manager
0608dd9 ci: Attempt to fix linux build
937ea71 code: Fix a few build errors

📊 Changes

22 files changed (+792 additions, -239 deletions)

View changed files

📝 .github/linux-appimage-qt.sh (+1 -0)
📝 .github/workflows/linux-qt.yml (+4 -24)
📝 .github/workflows/linux.yml (+3 -10)
📝 CMakeLists.txt (+13 -6)
📝 externals/boost (+1 -1)
📝 src/common/assert.cpp (+1 -0)
➕ src/common/scope_exit.h (+79 -0)
➕ src/core/address_space.cpp (+274 -0)
➕ src/core/address_space.h (+59 -0)
📝 src/core/libraries/kernel/libkernel.cpp (+6 -3)
➖ src/core/libraries/kernel/memory/flexible_memory.cpp (+0 -22)
➖ src/core/libraries/kernel/memory/flexible_memory.h (+0 -33)
➖ src/core/libraries/kernel/memory/kernel_memory.cpp (+0 -78)
➖ src/core/libraries/kernel/memory/kernel_memory.h (+0 -18)
📝 src/core/libraries/kernel/memory_management.cpp (+39 -38)
📝 src/core/libraries/kernel/memory_management.h (+4 -0)
📝 src/core/libraries/libc_internal/libc_internal.cpp (+2 -2)
📝 src/core/libraries/libc_internal/libc_internal.h (+2 -2)
➕ src/core/memory.cpp (+174 -0)
➕ src/core/memory.h (+128 -0)

...and 2 more files

📄 Description

The previous memory tracker was kinda dodgy and didn't perform much necessary tracking needed to implement some later memory functions. This PR rewrites it to be more accurate to the console.

The biggest problem to solve is the allocation granularity difference between windows and posix. Windows enforces 64KB granularity on VirtualAlloc calls while PS4 can use 16KB granularity. To get around this we use memory placeholders, a relatively new feature in Windows kernel, that allows us to reserve a virtual region and map inside it with 4KB granularity.

Using placeholders has many other benefits. We can reserve the user area upfront which will prevent any emulator allocations from polluting it. ASLR is disabled for this reason, as with it upper regions of user area are polluted from existing mappings. Later games appear to use dmem aliasing which cannot be emulated with plain VirtualAlloc. To effectively support aliasing, a backing file mapping is created and mapped that represents physical memory the guest application can map. Thus when a physical address is provided this backing file is mapped to the placeholder virtual area.

Tracking is performed using a simple std::map that tracks user and system managed areas. In the future it should be expanded to include the code areas as well but those reside in system reserved area so that's for later. Each virtual memory area stores a bunch of properties about it so VirtualQuery can be implemented relatively easily later. Some special cases with FIXED mapping and sceKernelMunmap have also been implemented

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/shadps4-emu/shadPS4/pull/133 **Author:** [@raphaelthegreat](https://github.com/raphaelthegreat) **Created:** 5/14/2024 **Status:** ✅ Merged **Merged:** 5/16/2024 **Merged by:** [@raphaelthegreat](https://github.com/raphaelthegreat) **Base:** `main` ← **Head:** `new-memory` --- ### 📝 Commits (3) - [`b321d3c`](https://github.com/shadps4-emu/shadPS4/commit/b321d3cfa83af6b28e84b9a1d2799aa47f47bc7e) core: Implement new memory manager - [`0608dd9`](https://github.com/shadps4-emu/shadPS4/commit/0608dd947bff50598ed9dbe38e77fb665fe8d39e) ci: Attempt to fix linux build - [`937ea71`](https://github.com/shadps4-emu/shadPS4/commit/937ea71f705d05990e73f57e5e439b201b647b25) code: Fix a few build errors ### 📊 Changes **22 files changed** (+792 additions, -239 deletions) <details> <summary>View changed files</summary> 📝 `.github/linux-appimage-qt.sh` (+1 -0) 📝 `.github/workflows/linux-qt.yml` (+4 -24) 📝 `.github/workflows/linux.yml` (+3 -10) 📝 `CMakeLists.txt` (+13 -6) 📝 `externals/boost` (+1 -1) 📝 `src/common/assert.cpp` (+1 -0) ➕ `src/common/scope_exit.h` (+79 -0) ➕ `src/core/address_space.cpp` (+274 -0) ➕ `src/core/address_space.h` (+59 -0) 📝 `src/core/libraries/kernel/libkernel.cpp` (+6 -3) ➖ `src/core/libraries/kernel/memory/flexible_memory.cpp` (+0 -22) ➖ `src/core/libraries/kernel/memory/flexible_memory.h` (+0 -33) ➖ `src/core/libraries/kernel/memory/kernel_memory.cpp` (+0 -78) ➖ `src/core/libraries/kernel/memory/kernel_memory.h` (+0 -18) 📝 `src/core/libraries/kernel/memory_management.cpp` (+39 -38) 📝 `src/core/libraries/kernel/memory_management.h` (+4 -0) 📝 `src/core/libraries/libc_internal/libc_internal.cpp` (+2 -2) 📝 `src/core/libraries/libc_internal/libc_internal.h` (+2 -2) ➕ `src/core/memory.cpp` (+174 -0) ➕ `src/core/memory.h` (+128 -0) _...and 2 more files_ </details> ### 📄 Description The previous memory tracker was kinda dodgy and didn't perform much necessary tracking needed to implement some later memory functions. This PR rewrites it to be more accurate to the console. The biggest problem to solve is the allocation granularity difference between windows and posix. Windows enforces 64KB granularity on VirtualAlloc calls while PS4 can use 16KB granularity. To get around this we use memory placeholders, a relatively new feature in Windows kernel, that allows us to reserve a virtual region and map inside it with 4KB granularity. Using placeholders has many other benefits. We can reserve the user area upfront which will prevent any emulator allocations from polluting it. ASLR is disabled for this reason, as with it upper regions of user area are polluted from existing mappings. Later games appear to [use](https://github.com/red-prig/fpps4-game-compatibility/issues/1617) dmem aliasing which cannot be emulated with plain VirtualAlloc. To effectively support aliasing, a backing file mapping is created and mapped that represents physical memory the guest application can map. Thus when a physical address is provided this backing file is mapped to the placeholder virtual area. Tracking is performed using a simple std::map that tracks user and system managed areas. In the future it should be expanded to include the code areas as well but those reside in system reserved area so that's for later. Each virtual memory area stores a bunch of properties about it so VirtualQuery can be implemented relatively easily later. Some special cases with FIXED mapping and sceKernelMunmap have also been implemented --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>