If you are still using the old URL (datacrystal.romhacking.net), please update your bookmarks! The old URL may stop working soon.
The current URL is datacrystal.tcrf.net.

Strike Witches: Silver Wing/Notes: Difference between revisions

From Data Crystal
Jump to navigation Jump to search
(Compression command research)
(Updated research on reinsert2 failing. Update research on GGXArchiver format)
(2 intermediate revisions by the same user not shown)
Line 12: Line 12:
| Magic || "GGXArchiver1.00" + 0x00
| Magic || "GGXArchiver1.00" + 0x00
|-
|-
| Unknown (number of files?  more info?) || 16-bytes, ex: '''0x02 0x00 0x00 0x00 0x02 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00''' (/media/data/sc_logo.bin)
| Archive Header || 16-bytes, ex: '''0x02 0x00 0x00 0x00 0x02 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00''' (/media/data/sc_logo.bin)
First four bytes are number of entries in the archive's file table<sup>(LE)</sup>
First four bytes are number of entries in the archive's file table<sup>(LE)</sup>


Line 21: Line 21:
| File List || 32-bytes per file, relative path to file, directories supported
| File List || 32-bytes per file, relative path to file, directories supported
|-
|-
| File Headers Table || Size seems to scale with number of files in archive
| File Metadata Table || Size seems to scale with number of files in archive
0x00-0x03 unknown ('''0x01''' for LOGO_GULTI.DDS in sc_logo.bin)<sup>(LE)</sup>
0x00-0x03 File ID<sup>(LE)</sup>


0x04-0x07 unknown ('''0x01''' for LOGO_GULTI.DDS in sc_logo.bin)<sup>(LE)</sup>
0x04-0x07 static 1<sup>(LE)</sup>


0x08-0x0B uncompressed file size?<sup>(LE)</sup> (131,200 bytes in sc_logo.bin, exactly 128K + 124 byte DDS_HEADER + 4 byte dwMagic)
0x08-0x0B uncompressed file size<sup>(LE)</sup>


0x0C-0x0F Size of compressed file data in archive, probably includes compression commands (ex: sc_title.bin - 311,009 bytes = '''0x00 0x8A 0xBE 0x04''' = 310,921 bytes + 88 bytes before, only 1 file in archive)<sup>(LE)</sup>
0x0C-0x0F Size of compressed file data in archive<sup>(LE)</sup>


0x10-0x13 Unknown
0x10-0x13 Boolean? (could also represent compression type), is data compressed<sup>(LE)</sup>.  Does not seem to support uncompressed data?


0x14-0x17 Offset from end of file headers table to start of file
0x14-0x17 Offset from end of file metadata table to start of file<sup>(LE)</sup>


Repeated for each file in archive
Repeated for each file in archive
|-
|-
| Compression commands? || See below
| File Data || The actual data for each of the archived files
|-
| File Data || The actual data for each of the archived files.  Appears to be have some endian weirdness, reversed every 32-bits?  (Xenon is 32-bit BE, makes sense?)
|}
|}


===Compression Commands===
====Compression Information====
 
Appears to be compressed with BPE ([https://en.wikipedia.org/wiki/Byte_pair_encoding Byte-Pair Encoding]) compression (quickbms compression method 25/55).  quickbms sometimes fails decompression, may require new utility or manual decompression.
 
quickbms's reinsert2 is bugged, and will not adjust archives appropriately if new zsize < old zsize.  Archives with files that are smaller than originally will cause the game to softlock when unpacking them.  The relatively simple fix is to ensure all repacked files are larger than their originals.  reinsert2 will then overwrite the entire original file with zeroes and adjust offsets appropriately.  This will create a small increase in loading times unless the files are manually (TBD on utility to automate this) stripped of padding.


Commands may be encoded both before and within the data.  Generally seem to be upper values of bytes (0xF0-0xFF), but occasionally lower.
====Strategies to avoid unpack softlock====


{| class="wikitable"
* '''Enabling mipmaps'''
|+ Commands
** Pros
|-
*** Simple
! Command !! Behavior !! Notes
*** Significant file size increase will almost certainly outweigh improved packing
|-
** Cons
| 0xF0 || Add 3 zeroes? || Changing the first instance of it after file data starts in sc_logo.bin\LOGO_GULTI.DDS to 0x00 or 0xF1 hardlocks in Xenia.  Untested on HW.  DDS textures need header length to be 32-bit, compressed archive stores it as 8-bit, so somehow command adds another 3 zeroes?
*** Only works on textures, and only those with mipmaps disabled originally
|-
*** Creates unnecessary bloat
| 0xF1 ||  ||
*** Significant VRAM usage increase
|-
 
| 0xF2 || Text related? ||
 
|-
* '''Packer-unfriendly data'''
| 0xF3 ||  ||
** Pros
|-
*** Works on all file types
| 0xF4 ||  ||
*** Can make files exactly the desired size to avoid softlock and reduce bloat/load times/VRAM usage
|-
** Cons
| 0xF5 ||  ||
*** Time consuming to do manually, unreliable via utility
|-
 
| 0xF6 ||  ||
 
|-
* '''Custom utility'''
| 0xF7 ||  ||
** Pros
|-
*** Solves all issues elegantly
| 0xF8 ||  ||
** Cons
|-
*** High up-front work to write and test utility
| 0xF9 ||  ||
|-
| 0xFA ||  ||
|-
| 0xFB ||  ||
|-
| 0xFC ||  ||
|-
| 0xFD ||  ||
|-
| 0xFE ||  ||
|-
| 0xFF ||  ||
|-
| Reserved || For || Future Expansion
|}




{{Internal Data}}
{{Internal Data}}

Revision as of 22:07, 20 March 2024

File Formats

GGXArchive Packed Files

Most of the game's files appear to be archives created with "GGXArchiver v1.00". These can contain various types of files, including but not limited to models, textures and text. Some entries are stored as little endian

GGXArchiver Format
Entry Format
Magic "GGXArchiver1.00" + 0x00
Archive Header 16-bytes, ex: 0x02 0x00 0x00 0x00 0x02 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 (/media/data/sc_logo.bin)

First four bytes are number of entries in the archive's file table(LE)

Second four bytes are number of "valid" entries in the file table (blank entries of 32 0x00 are "invalid")(LE)

Remaining 8 bytes are probably padding

File List 32-bytes per file, relative path to file, directories supported
File Metadata Table Size seems to scale with number of files in archive

0x00-0x03 File ID(LE)

0x04-0x07 static 1(LE)

0x08-0x0B uncompressed file size(LE)

0x0C-0x0F Size of compressed file data in archive(LE)

0x10-0x13 Boolean? (could also represent compression type), is data compressed(LE). Does not seem to support uncompressed data?

0x14-0x17 Offset from end of file metadata table to start of file(LE)

Repeated for each file in archive

File Data The actual data for each of the archived files

Compression Information

Appears to be compressed with BPE (Byte-Pair Encoding) compression (quickbms compression method 25/55). quickbms sometimes fails decompression, may require new utility or manual decompression.

quickbms's reinsert2 is bugged, and will not adjust archives appropriately if new zsize < old zsize. Archives with files that are smaller than originally will cause the game to softlock when unpacking them. The relatively simple fix is to ensure all repacked files are larger than their originals. reinsert2 will then overwrite the entire original file with zeroes and adjust offsets appropriately. This will create a small increase in loading times unless the files are manually (TBD on utility to automate this) stripped of padding.

Strategies to avoid unpack softlock

  • Enabling mipmaps
    • Pros
      • Simple
      • Significant file size increase will almost certainly outweigh improved packing
    • Cons
      • Only works on textures, and only those with mipmaps disabled originally
      • Creates unnecessary bloat
      • Significant VRAM usage increase


  • Packer-unfriendly data
    • Pros
      • Works on all file types
      • Can make files exactly the desired size to avoid softlock and reduce bloat/load times/VRAM usage
    • Cons
      • Time consuming to do manually, unreliable via utility


  • Custom utility
    • Pros
      • Solves all issues elegantly
    • Cons
      • High up-front work to write and test utility


Internal Data for Strike Witches: Silver Wing
Retrieved from "https://datacrystal.tcrf.net/w/index.php?title=Strike_Witches:_Silver_Wing/Notes&oldid=62180"