Reverse Engineering file formats

[Quaker762]

Going to open this here so everyone can see. So far, the file formats that we have reversed are as follows:

.anm (partial) [ANiMation]
.tex [TEXture]
.tbn2 [Exported from KONAMI's SOFTIMAGE/PhotoShop .pic plugin]
.mdl [MoDeL]
.map [MAP]
.cld (partial) [CoLlision Data]

The file formats that we currently do not have reversed (or any idea how they are used) are:

.ded Dynamic Effects Data (ded). Probably crap to do with material files.
.cam (how does the camera interpret these values??) [CAMera data]
.kg1 Character and weapon shadow maps
.kg2 Level Shadow Maps
.fcl [???]

So far, it seems that (to some extent) the trigger/interaction data is stored in the .cld, however, what is unclear to me is where the motion capture data for cutscenes is. For SILENT HILL 2, this is in the .anm files, so I would assume it is the same here (they are stored in the /demo/ folder in SH2), however the absolutely horrendous directory structure and file naming makes it near impossible to deduce without having animations working properly in-engine.

If anybody has any idea, leave you comments/suggestions in this issue.

[Bigmanjapan]

"motion capture data for cutscenes" for SH2 cutscenes is contained within .dds files. I was able to change characters positions in the cutscenes by editing the files.

.cam files store locations' camera angles and are quie easy to manipulate once you play around with SH2 E3 Trial Verison's debug menu and understand the logic behind location cameras.

https://www.youtube.com/watch?v=-4zfPwGOhAk

[Quaker762]

"motion capture data for cutscenes" for SH2 cutscenes is contained within .dds files. I was able to change characters positions in the cutscenes by editing the files.

So model location data is stored in .dds files and the actual animation for the cutscene in .anm files? Could you possibly upload an example video of a modified .dds file?

Any idea where this would be stored for SH3? From playing around I can't really change any of the cutscene stuff.

.cam files store locations' camera angles and are quie easy to manipulate once you play around with SH2 E3 Trial Verison's debug menu and understand the logic behind location cameras.

Ahhh, I see. Is it just angle data, or is it data for the bounding boxes/clipping planes for the camera too? If you feel comfortable enough RE'ing it a bit more and modifying the camera class, by all means do it.

Nice video btw!

[Bigmanjapan]

So model location data is stored in .dds files and the actual animation for the cutscene in .anm files? Could you possibly upload an example video of a modified .dds file?

Sorry, I made a mistake in my previous post. .dds files contain data about camera position in ingame cutscenes.

Let's take a simple game engine cutscene with one static camera angle in which James opens a can of lightbulbs. By modyfing kanzume.dds I lowered vertical position of the static camera angle in that scene: https://youtu.be/r3rCN6PTwyk

Any idea where this would be stored for SH3? From playing around I can't really change any of the cutscene stuff.

No idea on how SH3 stores game engine cutscenes data. But I know that .arc archives hold .cam (just like SH2 since it's the same engine at their cores) files in them and it is possible to modify location camera angles (did it myself here).

Ahhh, I see. Is it just angle data, or is it data for the bounding boxes/clipping planes for the camera too? If you feel comfortable enough RE'ing it a bit more and modifying the camera class, by all means do it.

.cam files hold camera angle direction for static camera angles and boundary boxes for dynamic camera angles. Dymamic camera angles are easier to express since their camera direction is simply tracking / aiming at James' model. .cam files also hold various camera parameters like area type (tiny, small, wide, outdoor) as seen in E3 Trial Version debug menu.

I tried nulling all data in certain .kg2 file for a specific location and it didn't change anything in that location. And game didn't even crash, considering that SH2 engine constantly polls data from the actual files and not a copy of them stored in RAM.

Edit: I had my shadows turned off when I tested kg2 files. belek666's input is probably also true for SH2.

[Bigmanjapan]

A user named Jokie managed to reactivate SH3 PC debug logging function: https://cdn.discordapp.com/attachments/138060014316290048/431901317867503618/unknown.png

Could it help you in reversing formats? If you are interested in this, you can find him in Silent Hill Speedrun Discord server: https://discord.gg/TkMQj8G

[belek666]

Nulling *.kg files in sh3 pc data removes shadows. I've replaced kg file from one weapon to another and shadow pattern changed to that weapon.

[Quaker762]

Sorry for the late reply, uni's been kicking my arse this week.

Sorry, I made a mistake in my previous post. .dds files contain data about camera position in ingame cutscenes.

Let's take a simple game engine cutscene with one static camera angle in which James opens a can of lightbulbs. By modyfing kanzume.dds I lowered vertical position of the static camera angle in that scene: https://youtu.be/r3rCN6PTwyk

Ahhhh I see, that makes a lot more sense now. It would seem SH2 had more file separation than SH3, as there are no .dds files to be found. There are .cam, but I'm yet to find one that changes cutscenes. Perhaps I'm editing the wrong ones? So far it's all just memory editing, as I'm yet to work on a tool to repack .arc sections (though when I have some time)

.cam files hold camera angle direction for static camera angles and boundary boxes for dynamic camera angles. Dymamic camera angles are easier to express since their camera direction is simply tracking / aiming at James' model. .cam files also hold various camera parameters like area type (tiny, small, wide, outdoor) as seen in E3 Trial Version debug menu.

So the .cam files contain the bounding volumes for the camera (as well as a predefined type [outdoors, indoors etc])?? are you possibly able to map out a rough layout of the file in general (I see you semi have in the video you uploaded). If you can, I'll plop it straight on the wiki we have here.

A user named Jokie managed to reactivate SH3 PC debug logging function: https://cdn.discordapp.com/attachments/138060014316290048/431901317867503618/unknown.png

Could it help you in reversing formats? If you are interested in this, you can find him in Silent Hill Speedrun Discord server: https://discord.gg/TkMQj8G

Yes that would be immensely helpful. I was trying to work out how to enable debug strings, but wasn't sure what function was calling it (though I did find some debug enable stuff in IDA). Thanks for the heads up.

Nulling *.kg files in sh3 pc data removes shadows. I've replaced kg file from one weapon to another and shadow pattern changed to that weapon.

Interesting. Are these perhaps shadow map files?? I've figured out that .ded files contain lighting information for the scene (brightness/intensity of RGB channels) as well as fog colour, but there are some areas of the file that are still a bit iffy. I think .ded stands for Dynamic Effects Data

[Quaker762]

Okay, so according to the debug info, the only file loaded during the intro cutscene, as well as the Happy Burger scene is chhaa_basic1_none.anm. That must contain heather cutscene data, though from the looks of it, it could just be idle animations... Perhaps the animation data is actually inside the executable.. Hopefully it isn't..

[Polymega]

Hi @Quaker762 ,

Can you tell me a little bit more about the .tbn2 file format? In regards to SH2 PC, whenever I edit one of these texture files the game itself does not reflect these changes. So I was thinking maybe these textures (for SH2 PC, anyways) are also stored somewhere in the main binary itself and it's pulling them from there?

But the thing is: If you remove the external .tbn2 files from the game's directory the game won't boot up. So the game is calling on them to some capacity... except any edits done don't reflect within the game itself. I'm fully perplexed by this...

[Quaker762]

Hi @Polymega

Sorry for such a late reply, I didn't see the notification or get an email for this... Strange...

Anyways, I'm not 100% sure about .tbn2 files. From what I remember they were textures, perhaps animated, but I've forgotten completely as it's been a while since I've had anything to do with Silent Hill. I also had an IDA instance of SH3 where I did most of my reversing, but I've since lost it as it ended up corrupt when the game crashed (yeah...). The easiest way to work out what the issue is would be to attach a debugger (like IDA) to sh3.exe, set a read breakpoint for the file that's been modified, and then follow the flow of execution to see why it's not being updated.

Let me know if you you need any more help.

[Polymega]

Sorry for the even later reply, @Quaker762. :)

I hate finding forums/threads where people ask a question and either never follow up with it or just say, "I figured it out," and don't explain how they did.

For those who are curious: Quaker762 is indeed correct that the .tbn2 format seems to be primarily meant for animated textures. (For SH2, think of the water texture that sits on top of the water that gently moves in a circular motion.)

In regards to SH2 PC: The reason why any edits I made to them didn't change anything in-game is because the .tbn2 files you see within the game's directory are red herrings. All .tbn2 files except for footmark.tbn2 and lens_flare.tbn2 located in /data/etc/effect/ are not called by the executable at all. This was tested/confirmed by @Bigmanjapan.

Instead, the .tbn2 files were located/used within .map files. So if you wanted to edit the .tbn2 water textures used throughout the labyrinth levels, you'd instead open ps.map and find the header/data for those particular .tbn2 texture files in there. The .tbn2 files are located in all non-numbered .map files (ap.map, ps.map, ru.map, etc and NOT ap22.map, ps37.map, ru61.map, etc).

To find where the .tbn2 data is held within these .map files, you could open the individual ("red herring") .tbn2 files, copy their headers, and search for the header data strings within the .map files to find their exact locations.

[SNMetamorph]

Hello everybody. Where I can found any information about reversed file formats?

[Quaker762]

Hello everybody. Where I can found any information about reversed file formats?

Hi,

Some info on the formats can be found on the XenTax wiki. Texture stuff can be found in the code here on github (it's mostly complete, there's probably some edge cases we don't take into account). Apart from that, you'd need to work it out yourself.

The SILENT HILL Engine is, unfortunately, very crude with a lot of hacked together formats. I don't think they had much of a concept of longevity or refactoring at Team SILENT.

[SNMetamorph]

Thank for information. And another question - maybe you know, do SH3 engine (on Windows) may work with modified .arc files or not? (which is bigger/smaller than original .arc file) I wanted to make software for SH2/3/4 that can provide ability to edit game content, like textures, models, sounds, etc. But I don't know that is possible at all.

[piratesephiroth]

Thank for information. And another question - maybe you know, do SH3 engine (on Windows) may work with modified .arc files or not? (which is bigger/smaller than original .arc file)

if you build proper arc files, sure

[SNMetamorph]

I have some doubts that SH3 engine is fully data-driven.