[PR #575] [MERGED] video_core: Account of runtime state changes when compiling shaders #1641

New issue

Closed

opened 2026-02-27 21:13:23 +03:00 by kerem · 0 comments

kerem commented

2026-02-27 21:13:23 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/shadps4-emu/shadPS4/pull/575
Author: @raphaelthegreat
Created: 8/25/2024
Status: ✅ Merged
Merged: 8/29/2024
Merged by: @raphaelthegreat

Base: main ← Head: permut

📝 Commits (10+)

007147c video_core: Compile shader permutations
19d5ac1 spirv: Only specific storage image format for atomics
74a5d1e ir: Avoid cube coord patching for storage image
f13aa08 spirv: Fix default attributes
347096b data_share: Add more instructions
5c1de38 video_core: Query storage flag with runtime state
833a366 kernel: Use std::list for semaphore
f118dc7 video_core: Use texture buffers for untyped format load/store
97770e2 buffer_cache: Limit view usage
d362928 vk_pipeline_cache: Fix invalid iterator

📊 Changes

43 files changed (+1057 additions, -975 deletions)

View changed files

📝 CMakeLists.txt (+2 -0)
📝 src/core/libraries/avplayer/avplayer.cpp (+1 -1)
📝 src/core/libraries/kernel/threads/semaphore.cpp (+13 -20)
📝 src/shader_recompiler/backend/spirv/emit_spirv.cpp (+25 -16)
📝 src/shader_recompiler/backend/spirv/emit_spirv.h (+1 -1)
📝 src/shader_recompiler/backend/spirv/emit_spirv_context_get_set.cpp (+15 -254)
📝 src/shader_recompiler/backend/spirv/spirv_emit_context.cpp (+93 -63)
📝 src/shader_recompiler/backend/spirv/spirv_emit_context.h (+12 -5)
📝 src/shader_recompiler/frontend/translate/data_share.cpp (+21 -14)
📝 src/shader_recompiler/frontend/translate/export.cpp (+0 -2)
📝 src/shader_recompiler/frontend/translate/translate.cpp (+1 -3)
📝 src/shader_recompiler/frontend/translate/translate.h (+7 -5)
📝 src/shader_recompiler/frontend/translate/vector_alu.cpp (+10 -4)
📝 src/shader_recompiler/frontend/translate/vector_memory.cpp (+108 -55)
📝 src/shader_recompiler/ir/ir_emitter.cpp (+5 -32)
📝 src/shader_recompiler/ir/ir_emitter.h (+3 -3)
📝 src/shader_recompiler/ir/microinstruction.cpp (+0 -3)
📝 src/shader_recompiler/ir/opcodes.inc (+2 -8)
📝 src/shader_recompiler/ir/passes/resource_tracking_pass.cpp (+74 -116)
📝 src/shader_recompiler/ir/passes/shader_info_collection_pass.cpp (+9 -0)

...and 23 more files

📄 Description

A lot of state used during shader compilation is stored in the resource definitions (sharps). For buffers the stride, format (in case of untyped buffer load/store) as well as the size are relevant to compilation. For images the type is most relevant as that determines the layout of the address stream.

Due to this, it is possible for these parameters to be modified while the same shader program is bound. This can cause a variety of issues which are difficult to debug. For example a read-only buffer may get compiled as a UBO first time, but then a larger buffer may be bound which cannot fit in the 16KB size limit. Some games can also use multiple image types with the same program.

To solve this, each Program (which represents a unique guest shader) can now store a collection of vulkan shader modules. After its fetched the resource layout is determined and a resource stage key is calculated which takes into account the position of the shader in the pipeline (start binding) as well as resource information relevant to compilation. Might fix issues in some games, but can also expose issues that where hidden before. This will allow us to remove another hack in bb-hacks branch but is not a fix for bloodborne specifically.

In addition memory usage from the shader cache should be significantly reduced. We only cache Shader::Info object per shader as well as the vulkan shader modules. Before this change, the generated SPIRV code was kept around which could be dozens of KBs per object as well as the IR::Program which stored blocks and translated gcn instructions that aren't much useful

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/shadps4-emu/shadPS4/pull/575 **Author:** [@raphaelthegreat](https://github.com/raphaelthegreat) **Created:** 8/25/2024 **Status:** ✅ Merged **Merged:** 8/29/2024 **Merged by:** [@raphaelthegreat](https://github.com/raphaelthegreat) **Base:** `main` ← **Head:** `permut` --- ### 📝 Commits (10+) - [`007147c`](https://github.com/shadps4-emu/shadPS4/commit/007147cc60dc1451bd98a0353e98c2c0af979a5f) video_core: Compile shader permutations - [`19d5ac1`](https://github.com/shadps4-emu/shadPS4/commit/19d5ac1318832fef830ae771be4fb9b6dea79863) spirv: Only specific storage image format for atomics - [`74a5d1e`](https://github.com/shadps4-emu/shadPS4/commit/74a5d1e7e472692cdacaeda1e66f1a08a40c7e0d) ir: Avoid cube coord patching for storage image - [`f13aa08`](https://github.com/shadps4-emu/shadPS4/commit/f13aa08e5dac7b83a35a8a185190f17dfb2abda0) spirv: Fix default attributes - [`347096b`](https://github.com/shadps4-emu/shadPS4/commit/347096b78fa6a967583bee64d3971f3595ae5bd2) data_share: Add more instructions - [`5c1de38`](https://github.com/shadps4-emu/shadPS4/commit/5c1de381c45239d0968dcc422b57aa83ff57317d) video_core: Query storage flag with runtime state - [`833a366`](https://github.com/shadps4-emu/shadPS4/commit/833a366e3b243b2637adf9d099c9ae0a6d56f92a) kernel: Use std::list for semaphore - [`f118dc7`](https://github.com/shadps4-emu/shadPS4/commit/f118dc7eca0ea55f8e798792974316d1560104a5) video_core: Use texture buffers for untyped format load/store - [`97770e2`](https://github.com/shadps4-emu/shadPS4/commit/97770e258d614640a7a5ad62423361c813ef1068) buffer_cache: Limit view usage - [`d362928`](https://github.com/shadps4-emu/shadPS4/commit/d36292873bf9168202cb2eb05571ba11e59569d5) vk_pipeline_cache: Fix invalid iterator ### 📊 Changes **43 files changed** (+1057 additions, -975 deletions) <details> <summary>View changed files</summary> 📝 `CMakeLists.txt` (+2 -0) 📝 `src/core/libraries/avplayer/avplayer.cpp` (+1 -1) 📝 `src/core/libraries/kernel/threads/semaphore.cpp` (+13 -20) 📝 `src/shader_recompiler/backend/spirv/emit_spirv.cpp` (+25 -16) 📝 `src/shader_recompiler/backend/spirv/emit_spirv.h` (+1 -1) 📝 `src/shader_recompiler/backend/spirv/emit_spirv_context_get_set.cpp` (+15 -254) 📝 `src/shader_recompiler/backend/spirv/spirv_emit_context.cpp` (+93 -63) 📝 `src/shader_recompiler/backend/spirv/spirv_emit_context.h` (+12 -5) 📝 `src/shader_recompiler/frontend/translate/data_share.cpp` (+21 -14) 📝 `src/shader_recompiler/frontend/translate/export.cpp` (+0 -2) 📝 `src/shader_recompiler/frontend/translate/translate.cpp` (+1 -3) 📝 `src/shader_recompiler/frontend/translate/translate.h` (+7 -5) 📝 `src/shader_recompiler/frontend/translate/vector_alu.cpp` (+10 -4) 📝 `src/shader_recompiler/frontend/translate/vector_memory.cpp` (+108 -55) 📝 `src/shader_recompiler/ir/ir_emitter.cpp` (+5 -32) 📝 `src/shader_recompiler/ir/ir_emitter.h` (+3 -3) 📝 `src/shader_recompiler/ir/microinstruction.cpp` (+0 -3) 📝 `src/shader_recompiler/ir/opcodes.inc` (+2 -8) 📝 `src/shader_recompiler/ir/passes/resource_tracking_pass.cpp` (+74 -116) 📝 `src/shader_recompiler/ir/passes/shader_info_collection_pass.cpp` (+9 -0) _...and 23 more files_ </details> ### 📄 Description A lot of state used during shader compilation is stored in the resource definitions (sharps). For buffers the stride, format (in case of untyped buffer load/store) as well as the size are relevant to compilation. For images the type is most relevant as that determines the layout of the address stream. Due to this, it is possible for these parameters to be modified while the same shader program is bound. This can cause a variety of issues which are difficult to debug. For example a read-only buffer may get compiled as a UBO first time, but then a larger buffer may be bound which cannot fit in the 16KB size limit. Some games can also use multiple image types with the same program. To solve this, each Program (which represents a unique guest shader) can now store a collection of vulkan shader modules. After its fetched the resource layout is determined and a resource stage key is calculated which takes into account the position of the shader in the pipeline (start binding) as well as resource information relevant to compilation. Might fix issues in some games, but can also expose issues that where hidden before. This will allow us to remove another hack in bb-hacks branch but is not a fix for bloodborne specifically. In addition memory usage from the shader cache should be significantly reduced. We only cache Shader::Info object per shader as well as the vulkan shader modules. Before this change, the generated SPIRV code was kept around which could be dozens of KBs per object as well as the IR::Program which stored blocks and translated gcn instructions that aren't much useful --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>