Difference between revisions of "ERF"

From Dragon Age Toolset Wiki
Jump to: navigation, search
(→‎List of Filenames)
Line 3: Line 3:
 
'''ERF''' ("Encapsulated Resource File") files are container files for game resources.  
 
'''ERF''' ("Encapsulated Resource File") files are container files for game resources.  
  
−
== List of common ERF Files ==
+
= List of common ERF Files =
  
 
ERF Files for Dragon Age Game Resources are stored in the (Dragon Age Install Dir)\packages\core\ Folder and Subdirectories. ERF's for Addons or Mods are usually in their respective Addon or Mod Folder.
 
ERF Files for Dragon Age Game Resources are stored in the (Dragon Age Install Dir)\packages\core\ Folder and Subdirectories. ERF's for Addons or Mods are usually in their respective Addon or Mod Folder.
Line 49: Line 49:
 
**packages\core\data_tools\... ---Contains Resources for the Toolset
 
**packages\core\data_tools\... ---Contains Resources for the Toolset
  
−
== Extracting Resources ==
+
= Extracting Resources =
  
 
<!-- As long as no technical explanation is present or needed, I may prefer seeing explanations of usage like this: -->
 
<!-- As long as no technical explanation is present or needed, I may prefer seeing explanations of usage like this: -->
Line 58: Line 58:
 
One may create a Folder on the HD named for example "gameresources". Then added with subfolders that suit the Purpose, for example "Materials" in which one extracts only Materials. Preferably the Folderscheme is similar to that of the ERF's, to make browsing simple and easy to remember.  
 
One may create a Folder on the HD named for example "gameresources". Then added with subfolders that suit the Purpose, for example "Materials" in which one extracts only Materials. Preferably the Folderscheme is similar to that of the ERF's, to make browsing simple and easy to remember.  
  
−
== Packing Resources ==
+
= Packing Resources =
  
 
The [[builder to player]] process packages up resources into ERF files for you, storing the generated ERFs inside [[DAZIP]]s.
 
The [[builder to player]] process packages up resources into ERF files for you, storing the generated ERFs inside [[DAZIP]]s.
Line 64: Line 64:
 
Additionally, you can edit any ERF by opening it directly in the Toolset, and add or remove files by drag& dropping them into the Toolset with the ERF open manually. In cases the above process fails or whenever.
 
Additionally, you can edit any ERF by opening it directly in the Toolset, and add or remove files by drag& dropping them into the Toolset with the ERF open manually. In cases the above process fails or whenever.
  
−
== Overriding Resources ==
+
= Overriding Resources =
  
 
ERF's are named by their general purpose. But not all material files need to necessarily be put into Materials.ERF as an example. Patches do add every Patched Resource into one single ERF for example. The Original Game Resource ERF Files however are best left alone, as it has a high risk of breaking the game.
 
ERF's are named by their general purpose. But not all material files need to necessarily be put into Materials.ERF as an example. Patches do add every Patched Resource into one single ERF for example. The Original Game Resource ERF Files however are best left alone, as it has a high risk of breaking the game.
Line 70: Line 70:
 
Files do not need to be present in ERF Files at all. When Overriding Gameresources as when modding, this is a pretty easy method. Overriding Files are of equal name to their original Gameresource counterpart, and are to be placed into packages\core\override\. The Game will then choose this File over the original.
 
Files do not need to be present in ERF Files at all. When Overriding Gameresources as when modding, this is a pretty easy method. Overriding Files are of equal name to their original Gameresource counterpart, and are to be placed into packages\core\override\. The Game will then choose this File over the original.
  
−
== File Format (version 2.0) ==
+
= File Format =
 +
 
 +
== Version 2.0 ==
 
The format of Dragon Age erfs is not the same as previous versions. There are no string arrays, only a header, a list of files and the files themselves.
 
The format of Dragon Age erfs is not the same as previous versions. There are no string arrays, only a header, a list of files and the files themselves.
  
Line 86: Line 88:
 
<TR><TD>16 bit integer representing the number of files encapsulated in this ERF</TD></TR>
 
<TR><TD>16 bit integer representing the number of files encapsulated in this ERF</TD></TR>
 
</TABLE>
 
</TABLE>
 +
 
=== Table of Contents ===
 
=== Table of Contents ===
 
The table of contents is a list whose length is the same as the number of files and whose entries are formatted like so:
 
The table of contents is a list whose length is the same as the number of files and whose entries are formatted like so:
Line 96: Line 99:
 
After the table of contents comes the encapsulated files which are in their own format, specified by the extension of the file name in the table of contents.
 
After the table of contents comes the encapsulated files which are in their own format, specified by the extension of the file name in the table of contents.
 
   
 
   
−
== File Format (version 2.2) ==
+
== Version 2.2 ==
 
In ERF version 2.2 all stored files are encrypted using blowfish. You may not extract such third party ERF files unless you have permission to do so. This is provided only for purposes when you want to encrypt your own ERF files.
 
In ERF version 2.2 all stored files are encrypted using blowfish. You may not extract such third party ERF files unless you have permission to do so. This is provided only for purposes when you want to encrypt your own ERF files.
  
Line 118: Line 121:
 
<TR><TD>16B</TD><TD>MD5 digest of the password used for encryption/decryption</TD></TR>
 
<TR><TD>16B</TD><TD>MD5 digest of the password used for encryption/decryption</TD></TR>
 
</TABLE>
 
</TABLE>
 +
 
=== Table of Contents ===
 
=== Table of Contents ===
 
The table of contents is a list whose length is the same as the number of files and whose entries are formatted like so:
 
The table of contents is a list whose length is the same as the number of files and whose entries are formatted like so:
Line 145: Line 149:
 
* Decrypt your content
 
* Decrypt your content
  
−
== File Format (version 3.0) ==
+
== Version 3.0 ==
−
 
+
−
<TABLE cellSpacing=2 cellPadding=2 width="10%" border=1>
+
−
<TR><TD>Header</TD></TR>
+
−
<TR><TD>List of Filenames</TD></TR>
+
−
<TR><TD>Table of Contents</TD></TR>
+
−
<TR><TD>Files</TD></TR>
+
−
</TABLE>
+
  
 
=== Header ===
 
=== Header ===
−
The header consists of:
 
−
<TABLE cellSpacing=2 cellPadding=2 width="35%" border=1>
 
−
<TR><TD>4 characters (8 bytes) for the type of ERF ("ERF ")</TD></TR>
 
−
<TR><TD>4 characters (8 bytes) for the version ("V3.0 ")</TD></TR>
 
−
<TR><TD>32 bit integer representing the length of the filenames table</TD></TR>
 
−
<TR><TD>32 bit integer representing the number of files encapsulated in this ERF</TD></TR>
 
−
<TR><TD>32 bit integer representing ERF flags</TD></TR>
 
−
<TR><TD>32 bit integer unknown</TD></TR>
 
−
<TR><TD>32 bit integer unknown</TD></TR>
 
−
<TR><TD>32 bit integer unknown</TD></TR>
 
−
<TR><TD>32 bit integer unknown</TD></TR>
 
−
<TR><TD>32 bit integer unknown</TD></TR>
 
−
</TABLE>
 
  
−
=== List of Filenames ===
+
{| border="1" cellspacing="2" cellpadding="2" width="100%"
−
This is a table of filenames encapsulated in this ERF. It is possible for some file TOC entries to not have a filename offset even others do. Beginning after the Header:
+
 
−
<TABLE cellSpacing=2 cellPadding=2 width="53%" border=1>
+
! width="20%" | Name
−
<TR><TD>Zero padded Strings with varying length, representing the Filenames contained in this ERF.</TD></TR>
+
! width="20%" | Type / Size
−
</TABLE>
+
! width="60%" | Info
 +
 
 +
|-
 +
| magic
 +
| utf16le[8]
 +
| "ERF V3.0", a total of 16 bytes.
 +
 
 +
|-
 +
| stringTableSize
 +
| uint32
 +
| Size of the file name string table.
 +
 
 +
|-
 +
| fileCount
 +
| uint32
 +
| Number of files encapsulated in this ERF.
 +
 
 +
|-
 +
| flags
 +
| uint32
 +
| Observed values: 0 and 0x20000000
 +
 
 +
|-
 +
| unknown
 +
| uint32
 +
|
 +
 
 +
|-
 +
| unknown
 +
| uint32
 +
|
 +
 
 +
|-
 +
| unknown
 +
| uint32
 +
|
 +
 
 +
|-
 +
| unknown
 +
| uint32
 +
|
 +
 
 +
|-
 +
| unknown
 +
| uint32
 +
|
 +
 
 +
|-
 +
| stringTable
 +
| char[stringTableSize]
 +
| Zero-padded strings referenced by offset in TOC entries. Strings are stored as ASCII. Note that not all TOC entries have to reference an entry in the string table.
 +
 
 +
|}
  
 
=== Table of Contents ===
 
=== Table of Contents ===
−
This is a list of 28 Bytes with the following:
 
  
−
<TABLE cellSpacing=2 cellPadding=2 width="53%" border=1>
+
{| border="1" cellspacing="2" cellpadding="2" width="100%"  
−
<TR><TD>32 bit integer, offset to filename in string table, is -1 if no name is available</TD></TR>
+
 
−
<TR><TD>64 bit integer, [http://en.wikipedia.org/wiki/Fowler–Noll–Vo_hash_function FNV64] hash of lowercase filename</TD></TR>
+
! width="20%" | Name
−
<TR><TD>32 bit integer, type ID ([http://en.wikipedia.org/wiki/Fowler–Noll–Vo_hash_function FNV32] hash of lowercase extension)</TD></TR>
+
! width="20%" | Type / Size
−
<TR><TD>32 bit integer, offset to the beginning of the File</TD></TR>
+
! width="60%" | Info
−
<TR><TD>32 bit integer, compressed length of the File encapsulated in this ERF</TD></TR>
+
 
−
<TR><TD>32 bit integer, uncompressed length of the File encapsulated in this ERF</TD></TR>
+
|-
−
<TR><TD></TD></TR>
+
| toc
−
</TABLE>
+
| [[#Entry|entry]][fileCount]
 +
|
 +
 
 +
|}
 +
 
 +
==== Entry ====
 +
 
 +
{| border="1" cellspacing="2" cellpadding="2" width="100%"
 +
 
 +
! width="20%" | Name
 +
! width="20%" | Type / Size
 +
! width="60%" | Info
 +
 
 +
|-
 +
| nameOffset
 +
| int32
 +
| Offset in stringTable of the file name for this entry. -1 if none.
 +
 
 +
|-
 +
| nameHash
 +
| uint64
 +
| [http://en.wikipedia.org/wiki/Fowler–Noll–Vo_hash_function FNV64 hash] of lowercased file name, including path.
 +
 
 +
|-
 +
| typeHash
 +
| uint32
 +
| [http://en.wikipedia.org/wiki/Fowler–Noll–Vo_hash_function FNV32 hash] of lowercased file extension.
 +
 
 +
|-
 +
| offset
 +
| uint32
 +
| Offset to entry file data, from start of ERF file.
 +
 
 +
|-
 +
| compressedSize
 +
| uint32
 +
| Compressed length of file data.
 +
 
 +
|-
 +
| uncompressedSize
 +
| uint32
 +
| Uncompressed length of file data.
 +
 
 +
|}
 +
 
 +
=== File Data ===
  
−
After the TOC comes the plain files data.
+
If compressedSize does not equal uncompressedSize, file data is compressed with [http://en.wikipedia.org/wiki/Zlib zlib], with a custom 1-byte (commonly observed as 0xF9) header (vs the normal 2-byte zlib header).
  
 
[[Category:File types]]
 
[[Category:File types]]

Revision as of 13:39, 28 February 2011

ERF editor

ERF ("Encapsulated Resource File") files are container files for game resources.

List of common ERF Files

ERF Files for Dragon Age Game Resources are stored in the (Dragon Age Install Dir)\packages\core\ Folder and Subdirectories. ERF's for Addons or Mods are usually in their respective Addon or Mod Folder.

Common ERF's of certain interests are,

  • Game Resources
    • packages\core\data\2da.erf
    • packages\core\data\anims.erf
    • packages\core\data\consolescripts.erf
    • packages\core\data\designerareas.erf
    • packages\core\data\designercreatures.erf
    • packages\core\data\designercutscenes.erf
    • packages\core\data\designerdialogs.erf
    • packages\core\data\designeritems.erf
    • packages\core\data\designerplaceables.erf
    • packages\core\data\designerplots.erf
    • packages\core\data\designertriggers.erf
    • packages\core\data\face.erf
    • packages\core\data\gui.erf
    • packages\core\data\guiexport.erf
    • packages\core\data\iterationtests.erf
    • packages\core\data\lightprobedata.erf
    • packages\core\data\materialobjects.erf
    • packages\core\data\materials.erf
    • packages\core\data\misc.erf
    • packages\core\data\modelhierarchies.erf
    • packages\core\data\modelmeshdata.erf
    • packages\core\data\pathfindingpatches.erf
    • packages\core\data\postprocesseffects.erf
    • packages\core\data\resmetrics.erf
    • packages\core\data\scripts.erf
    • packages\core\data\shaders.erf
    • packages\core\data\states.erf
    • packages\core\data\subqueuefiles.erf
    • packages\core\data\textures.erf
    • packages\core\data\tints.erf
    • packages\core\textures\high\texturepack.erf
    • packages\core\textures\medium\texturepack.erf
  • Patch Resources
    • packages\core\patch\... --- Contains Resources added by Game Patches
  • Toolset specific Resources
    • packages\core\data_tools\... ---Contains Resources for the Toolset

Extracting Resources

The Toolset is capable of extracting ERF Files Contents.

If you are modding Resources, the first step would be extracting Resources. You may start of by extracting all Resources of certain interest to you, or just a single File. To Maintain a simple and clean resource place, it is best to use a similar Folder Names to extract in.

One may create a Folder on the HD named for example "gameresources". Then added with subfolders that suit the Purpose, for example "Materials" in which one extracts only Materials. Preferably the Folderscheme is similar to that of the ERF's, to make browsing simple and easy to remember.

Packing Resources

The builder to player process packages up resources into ERF files for you, storing the generated ERFs inside DAZIPs.

Additionally, you can edit any ERF by opening it directly in the Toolset, and add or remove files by drag& dropping them into the Toolset with the ERF open manually. In cases the above process fails or whenever.

Overriding Resources

ERF's are named by their general purpose. But not all material files need to necessarily be put into Materials.ERF as an example. Patches do add every Patched Resource into one single ERF for example. The Original Game Resource ERF Files however are best left alone, as it has a high risk of breaking the game.

Files do not need to be present in ERF Files at all. When Overriding Gameresources as when modding, this is a pretty easy method. Overriding Files are of equal name to their original Gameresource counterpart, and are to be placed into packages\core\override\. The Game will then choose this File over the original.

File Format

Version 2.0

The format of Dragon Age erfs is not the same as previous versions. There are no string arrays, only a header, a list of files and the files themselves.

Header
Table of Contents
Files

Header

The header consists of:

4 characters (8 bytes) for the type of ERF ("ERF ")
4 characters (8 bytes) for the version ("V2.0 ")
16 bit integer representing the number of files encapsulated in this ERF

Table of Contents

The table of contents is a list whose length is the same as the number of files and whose entries are formatted like so:

32 characters (64 bytes) for the file name with trailing null characters if file name is not 32 characters long
32bit integer representing the offset from the beginning of the ERF to the beginning of the encapsulated file
32bit integer representing the length of the encapsulated file

After the table of contents comes the encapsulated files which are in their own format, specified by the extension of the file name in the table of contents.

Version 2.2

In ERF version 2.2 all stored files are encrypted using blowfish. You may not extract such third party ERF files unless you have permission to do so. This is provided only for purposes when you want to encrypt your own ERF files.

Header
Table of Contents
Files

Header

The header consists of:

SizeDescription
8B4 UTF-16 characters in LE encoding ("ERF ")
8B4 UTF-16 characters in LE encoding for the version ("V2.2 ")
4BNumber of entries in ERF file stored as 32-bit LE integer
4BYear since 1900 as 32-bit LE integer
4BDay since January 1 as 32-bit LE integer
4B0xFFFFFFFF as 32-bit LE integer
8BSome kind of data
16BMD5 digest of the password used for encryption/decryption

Table of Contents

The table of contents is a list whose length is the same as the number of files and whose entries are formatted like so:

64BFile name in UTF-16 LE encoding. Is null-terminated only when its (double-byte) length is less than 32.
4B32bit LE integer representing the offset from the beginning of the ERF to the beginning of the blowfish encrypted encapsulated file
4B32bit LE integer representing size of encrypted data (always multiple of 8)
4B32bit LE integer representing size of decrypted data

Files are encrypted as follows:

  • Generate random number (represented as string, base 10), such as "12345678"
  • Generate MD5 digest of the string (without null-terminating character) and store it in the header
  • Convert those digits to 64-bit integer (0x0000000000BC614E)
  • Store it in byte array in little-endian encoding (0x4e, 0x61, 0xbc, 0, 0, 0, 0, 0)
  • Use that array as key to initialize Blowfish ECB algorithm.
  • Pad your content to multiple of 8 (e.g. by zeroes)
  • Encrypt the data and write to ERF file

To decrypt:

  • You already know your password ("12345678")
  • Generate its MD5 digest and compare it with that in the header
  • If they don't match file is either damaged or you forgot your password. If you forgot, brute-force attack should take no more than 1 minute. You may not perform brute-force attack on ERF files that are not yours unless you have permission from its owner to do so.
  • Convert those digits to 64-bit integer (0x0000000000BC614E)
  • Store it in byte array in little-endian encoding (0x4e, 0x61, 0xbc, 0, 0, 0, 0, 0)
  • Use that array as key to initialize Blowfish ECB algorithm.
  • Decrypt your content

Version 3.0

Header

Name Type / Size Info
magic utf16le[8] "ERF V3.0", a total of 16 bytes.
stringTableSize uint32 Size of the file name string table.
fileCount uint32 Number of files encapsulated in this ERF.
flags uint32 Observed values: 0 and 0x20000000
unknown uint32
unknown uint32
unknown uint32
unknown uint32
unknown uint32
stringTable char[stringTableSize] Zero-padded strings referenced by offset in TOC entries. Strings are stored as ASCII. Note that not all TOC entries have to reference an entry in the string table.

Table of Contents

Name Type / Size Info
toc entry[fileCount]

Entry

Name Type / Size Info
nameOffset int32 Offset in stringTable of the file name for this entry. -1 if none.
nameHash uint64 FNV64 hash of lowercased file name, including path.
typeHash uint32 FNV32 hash of lowercased file extension.
offset uint32 Offset to entry file data, from start of ERF file.
compressedSize uint32 Compressed length of file data.
uncompressedSize uint32 Uncompressed length of file data.

File Data

If compressedSize does not equal uncompressedSize, file data is compressed with zlib, with a custom 1-byte (commonly observed as 0xF9) header (vs the normal 2-byte zlib header).

Retrieved from "http://www.datoolset.net/mw/index.php?title=ERF&oldid=15894"