[GH-ISSUE #3074] Feature Request: Implement xdelta softpatch support #1269

Open
opened 2026-03-17 21:26:35 +03:00 by kerem · 13 comments
Owner

Originally created by @i30817 on GitHub (Aug 7, 2013).
Original GitHub issue: https://github.com/hrydgard/ppsspp/issues/3074

Seems like it would be useful.

Originally created by @i30817 on GitHub (Aug 7, 2013). Original GitHub issue: https://github.com/hrydgard/ppsspp/issues/3074 Seems like it would be useful.
Author
Owner

@solarmystic commented on GitHub (Aug 7, 2013):

Is this a feature request? If so, please label is as such in your title.

E.g. Feature Request:- Implement xdelta softpatch support

Thought you're making a pull request or something similar.

<!-- gh-comment-id:22222649 --> @solarmystic commented on GitHub (Aug 7, 2013): Is this a feature request? If so, please label is as such in your title. E.g. Feature Request:- Implement xdelta softpatch support Thought you're making a pull request or something similar.
Author
Owner

@unknownbrackets commented on GitHub (Aug 7, 2013):

At the file or iso level? If on the iso level, I'm not sure there's a way to process xdelta using random access, so it might take a lot of ram or make more sense to hardpatch.

-[Unknown]

<!-- gh-comment-id:22224105 --> @unknownbrackets commented on GitHub (Aug 7, 2013): At the file or iso level? If on the iso level, I'm not sure there's a way to process xdelta using random access, so it might take a lot of ram or make more sense to hardpatch. -[Unknown]
Author
Owner

@i30817 commented on GitHub (Aug 7, 2013):

At the iso level obviously, otherwise it would be useless. Nowadays the computer with less than 4GB is rare, so puttting a max 1GB iso in memory, if the user enables a flag saying 'softpatch support (takes a lot of memory)' doesn't seem so bad.

If you prefer ips or whatever the format that bsnes had, your choice, but i suggested xdelta because it's what the current PSP translations mostly use (although there is something to be said for encouraging the best format, if that bsnes one is the best).

I'm not saying that if you can implement a IO layer that patches in bytes into random reads, without reading the whole ISO into memory, that that wouldn't be good either, if you want to do it like that.

<!-- gh-comment-id:22235776 --> @i30817 commented on GitHub (Aug 7, 2013): At the iso level obviously, otherwise it would be useless. Nowadays the computer with less than 4GB is rare, so puttting a max 1GB iso in memory, if the user enables a flag saying 'softpatch support (takes a lot of memory)' doesn't seem so bad. If you prefer ips or whatever the format that bsnes had, your choice, but i suggested xdelta because it's what the current PSP translations mostly use (although there is something to be said for encouraging the best format, if that bsnes one is the best). I'm not saying that if you can implement a IO layer that patches in bytes into random reads, without reading the whole ISO into memory, that that wouldn't be good either, if you want to do it like that.
Author
Owner

@unknownbrackets commented on GitHub (Jan 11, 2014):

What games use xdelta patches right now?

Well, reading the entire thing into memory is prohibitive on Android, for example. Just can't do it, the RAM doesn't exist for it. And, xdelta doesn't support random access afaik. The only option would be to create a patched iso file and use it - hardpatching.

Maybe the emulator could do this for you, but it would take non-trivial SD card space. It'd be better to just upload a pre-patched iso.

Another format might be better for this purpose. I'm not very familiar with bps, and I can't find the technical specs anymore.... there's ppf which is pretty limited, but it should be random-access friendly.

Here's a link with some discussion:
http://forums.ppsspp.org/showthread.php?tid=6450

-[Unknown]

<!-- gh-comment-id:32102938 --> @unknownbrackets commented on GitHub (Jan 11, 2014): What games use xdelta patches right now? Well, reading the entire thing into memory is prohibitive on Android, for example. Just can't do it, the RAM doesn't exist for it. And, xdelta doesn't support random access afaik. The only option would be to create a patched iso file and use it - hardpatching. Maybe the emulator could do this for you, but it would take non-trivial SD card space. It'd be better to just upload a pre-patched iso. Another format might be better for this purpose. I'm not very familiar with bps, and I can't find the technical specs anymore.... there's ppf which is pretty limited, but it should be random-access friendly. Here's a link with some discussion: http://forums.ppsspp.org/showthread.php?tid=6450 -[Unknown]
Author
Owner

@i30817 commented on GitHub (Jan 12, 2014):

If you click though this wiki: http://akatranslations.wikispaces.com/PSP+Translations

you can check the ones that are xdelta. i know that la pucelle Ragnarok and Way of the Samurai dou are, but probably more. Many of them seem to be 'translations' only that is a wiki/faq with translated dialogue, not a hacking project, which is disappointing.

I was thinking that the xdelta could be turned into a inmemory representation that could be amenable to soft patching by a mechanism similar to a inputstream that would layer the 'patch' parts over the 'file' parts, but that wouldn't be easy. If any other representation is implemented, i suppose it's easy to turn one into the other, and translation groups will start to provide it instead. Xdelta is only used anyway because it's the one that compresses the most (because it depends on game input as compression dictionary data i guess). I doubt the differences are huge - and xdelta, like all binary patchers, is horrible for DRM'd games anyway, since they have to be decrypted completely anyway to work (a xdelta version of the valkyria chronicles 3 patch is 400mb compressed apparently)

If you want to go the emulator hardpatch option, you can use the little trick i use, where after patching i do a 'reverse patch' from the patched iso to the original and delete the original. Then when i want to play/revert to the original i patch the patched one with that file.

<!-- gh-comment-id:32114253 --> @i30817 commented on GitHub (Jan 12, 2014): If you click though this wiki: http://akatranslations.wikispaces.com/PSP+Translations you can check the ones that are xdelta. i know that la pucelle Ragnarok and Way of the Samurai dou are, but probably more. Many of them seem to be 'translations' only that is a wiki/faq with translated dialogue, not a hacking project, which is disappointing. I was thinking that the xdelta could be turned into a inmemory representation that could be amenable to soft patching by a mechanism similar to a inputstream that would layer the 'patch' parts over the 'file' parts, but that wouldn't be easy. If any other representation is implemented, i suppose it's easy to turn one into the other, and translation groups will start to provide it instead. Xdelta is only used anyway because it's the one that compresses the most (because it depends on game input as compression dictionary data i guess). I doubt the differences are huge - and xdelta, like all binary patchers, is horrible for DRM'd games anyway, since they have to be decrypted completely anyway to work (a xdelta version of the valkyria chronicles 3 patch is 400mb compressed apparently) If you want to go the emulator hardpatch option, you can use the little trick i use, where after patching i do a 'reverse patch' from the patched iso to the original and delete the original. Then when i want to play/revert to the original i patch the patched one with that file.
Author
Owner

@papel commented on GitHub (Feb 4, 2014):

It would be nice to have a way to use patches on emulator without patching the ISO file. It is good for translation of Japanese games into English.

There is no need to support multiple patching methods. If PPSSPP chose a single method, the community of users would convert many existing patches to use the method that PPSSPP uses. There are multiple methods: xdelta, ppf, livepatch, wqsg-umd and others.

<!-- gh-comment-id:34021301 --> @papel commented on GitHub (Feb 4, 2014): It would be nice to have a way to use patches on emulator without patching the ISO file. It is good for translation of Japanese games into English. There is no need to support multiple patching methods. If PPSSPP chose a single method, the community of users would convert many existing patches to use the method that PPSSPP uses. There are multiple methods: xdelta, ppf, livepatch, wqsg-umd and others.
Author
Owner

@i30817 commented on GitHub (Feb 4, 2014):

I've recently realized that softpatching doesn't work too well with encrypted games. From my experience with the Valkyria Chronicles 3 patch, the reason they're bundling jpcsp is to decrypt the files is the whole game appears to be in a compressed (without -zync friendliness) or encrypted format, with a virtual filesystem compacted into a single file.
Therefore, small modifications affect almost all of the rest of the file and so most of the data is different. The one way around this is what they've done, a patcher that 'understands' the game data crate and decompresses/decrypts it, modify the small file changes it must and recompresses/encrypt it again.
Obviously this process can never be efficient, barring a inmemory filesystem and a beefy machine, but more importantly, it's not generic.

So i'm worried about the future of softpatching in future emulators (if ever another key gets leaked ahah)

<!-- gh-comment-id:34023043 --> @i30817 commented on GitHub (Feb 4, 2014): I've recently realized that softpatching doesn't work too well with encrypted games. From my experience with the Valkyria Chronicles 3 patch, the reason they're bundling jpcsp is to decrypt the files is the whole game appears to be in a compressed (without -zync friendliness) or encrypted format, with a virtual filesystem compacted into a single file. Therefore, small modifications affect almost all of the rest of the file and so most of the data is different. The one way around this is what they've done, a patcher that 'understands' the game data crate and decompresses/decrypts it, modify the small file changes it must and recompresses/encrypt it again. Obviously this process can never be efficient, barring a inmemory filesystem and a beefy machine, but more importantly, it's not generic. So i'm worried about the future of softpatching in future emulators (if ever another key gets leaked ahah)
Author
Owner

@unknownbrackets commented on GitHub (Feb 4, 2014):

Well, the released Valkyria Chronicles 3 patch is also bundling other unnecessary things. It should be possible to patch the CPK file rather than extract and etc. it, they just bundle files to replace inside it. Even to recompress things to fit in the same space as per the documentation I wrote: http://wrttn.in/04fb3f

Anyway, I think we have to use something that applies patches on the fly. PPF is capable of this. I thought that xdelta used xor to apply its patches though (so that you didn't risk distributing any copyrighted content.)

-[Unknown]

<!-- gh-comment-id:34025195 --> @unknownbrackets commented on GitHub (Feb 4, 2014): Well, the released Valkyria Chronicles 3 patch is also bundling other unnecessary things. It should be possible to patch the CPK file rather than extract and etc. it, they just bundle files to replace inside it. Even to recompress things to fit in the same space as per the documentation I wrote: http://wrttn.in/04fb3f Anyway, I think we have to use something that applies patches on the fly. PPF is capable of this. I thought that xdelta used xor to apply its patches though (so that you didn't risk distributing any copyrighted content.) -[Unknown]
Author
Owner

@papel commented on GitHub (Feb 4, 2014):

There is a patching method that is used on real PSP without changing the ISO.
Since PSP has small RAM, I think this way could be used on PC and Android.

Maybe a new special method could be created for PPSSPP to read the patch from the disc without loading everything into RAM. Non-linear file seeking is very slow, but UMD reading is also slow. I do not know if it is fast enough.

<!-- gh-comment-id:34074771 --> @papel commented on GitHub (Feb 4, 2014): There is a patching method that is used on real PSP without changing the ISO. Since PSP has small RAM, I think this way could be used on PC and Android. Maybe a new special method could be created for PPSSPP to read the patch from the disc without loading everything into RAM. Non-linear file seeking is very slow, but UMD reading is also slow. I do not know if it is fast enough.
Author
Owner

@unknownbrackets commented on GitHub (Feb 4, 2014):

Applying a patch at runtime should not be an issue. An index buffer could be used to know which sectors to map to which sectors. As you say, this would still not even approach the real slowness of IO on the device.

LivePatch does what you say, but so far it only supports patching memory (e.g. the main executable.) I suppose it could be extended (which was the intention I believe) to hook sceIo funcs.

-[Unknown]

<!-- gh-comment-id:34075121 --> @unknownbrackets commented on GitHub (Feb 4, 2014): Applying a patch at runtime should not be an issue. An index buffer could be used to know which sectors to map to which sectors. As you say, this would still not even approach the real slowness of IO on the device. LivePatch does what you say, but so far it only supports patching memory (e.g. the main executable.) I suppose it could be extended (which was the intention I believe) to hook sceIo funcs. -[Unknown]
Author
Owner

@i30817 commented on GitHub (Feb 4, 2014):

You mentioned hooking sceIO functions @unknownbrackets... that would be nice as a 'standard' for PSP emulator patching i guess - it seems like it would avoid the 'crate' problem, but would probably need a custom format/patch creator no?

<!-- gh-comment-id:34111935 --> @i30817 commented on GitHub (Feb 4, 2014): You mentioned hooking sceIO functions @unknownbrackets... that would be nice as a 'standard' for PSP emulator patching i guess - it seems like it would avoid the 'crate' problem, but would probably need a custom format/patch creator no?
Author
Owner

@unknownbrackets commented on GitHub (Feb 4, 2014):

It would need to be custom probably, yes. I assume that was the model that LivePatch was planning on.

There are concerns such as the way that games use lba offsets to access files, though.... it's not so simple.

-[Unknown]

<!-- gh-comment-id:34112635 --> @unknownbrackets commented on GitHub (Feb 4, 2014): It would need to be custom probably, yes. I assume that was the model that LivePatch was planning on. There are concerns such as the way that games use lba offsets to access files, though.... it's not so simple. -[Unknown]
Author
Owner

@unknownbrackets commented on GitHub (Dec 30, 2015):

I know we've already got a few people who do rom hacking and translation involved here, but does anyone want to do some outreach and find out from people working on PSP games (in various languages, not just English) what they would prefer?

I think we've seen from some translations that people are finding patching PSP games to be a logistical challenge. A toolset to simplify this as well as first-class support in PPSSPP might help. And I think there are benefits (such as clearer application of bugfixes) to soft patching in general.

-[Unknown]

<!-- gh-comment-id:168055398 --> @unknownbrackets commented on GitHub (Dec 30, 2015): I know we've already got a few people who do rom hacking and translation involved here, but does anyone want to do some outreach and find out from people working on PSP games (in various languages, not just English) what they would prefer? I think we've seen from some translations that people are finding patching PSP games to be a logistical challenge. A toolset to simplify this as well as first-class support in PPSSPP might help. And I think there are benefits (such as clearer application of bugfixes) to soft patching in general. -[Unknown]
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/ppsspp#1269
No description provided.