Dragon Age Toolset Wiki - User contributions [en]

Installing the toolset

2009-11-07T03:03:00Z

ViresDroid: correcting the name of the default database server installed with the toolset

The Dragon Age Toolset installer comes with several pieces of third-party software that are needed for the toolset to function. They are:

* DirectX 9.0c
* Visual C Runtime Library
* Microsoft SQL Server Express 2005 Edition
* Nvidia PhysX drivers

These components will be installed along with the toolset by the toolset's installer.

== Location ==

The first screen you'll get to after starting the installer and passing the title page is the install location selector. The toolset can only install the program into a specific subdirectory of the Dragon Age game directory, so you won't be able to alter the installation path; it's for informational purposes.

'''Note:''' MSSQL Express 2005 is bundled with this toolset and is required for it to function. Unfortunately, MSSQL Express 2005's installer has a limitation that prevents it from installing to a directory path that's longer than 58 characters.<sup>[http://support.microsoft.com/kb/935371]</sup> If you've installed Dragon Age to a directory with a longer path than this you will need to uninstall Dragon Age: Origins and reinstall it in a directory with a shorter path length.

== Components ==

The next screen after location has been selected is where you can choose which components to install. There are only two selectable components here; Core Resources (the toolset itself) and Microsoft SQL Server Express. To run the toolkit you'll obviously need to install the Core Resources option. SQL Server is presented as a separate option so that if you already have a database server installed on your system you can use that instead of installing a whole new one.

If you don't have an SQL server installed, or if you don't know whether you have one installed, it's safe to go with the default and install SQL Server Express. SQLServer's installer will check the existing components and, finding that all required components are already in place, won't do anything after you click "finish."

== Manual database selection ==

If you choose not to install SQL Server Express, the installer will ask for a database server and database name to install the toolset's resource database to instead. The default server is <code>.\BWDATOOLSET</code> and the default database name is <code>bw_dragonage_content</code>.

== Installation ==

Once you have selected these options the Dragon Age toolset will begin the installation process. This can take a while. When the install process is finished you'll be given the option to read a readme.txt file.

The toolset will create a start menu folder named "Dragon Age". The start menu folder will include a link to start the toolset, a link to the toolset's uninstaller, and a link to the toolset's external configuration utility.

The installer will write a log file in your My Documents folder. If installation fails this log file will be helpful for debugging purposes.

If you are installing over a previous version of the toolset, you may need to update the format of your old database to make it compatible with the new version of the toolset. See [[database migration]] for details.

=== Lightmapper ===

The [[lightmap]]per is the tool used to calculate lighting in level layouts. You only need to use it if you're going to be creating or modifying level layouts. The lightmapper included with the toolset requires Python to run but Python isn't currently bundled with the toolset, so if you don't already have it installed you'll need to do the following:

#Install Python 2.5
#Install the win32all extensions for Python 2.5

Note you can do both of these steps at once if you install [http://www.activestate.com/activepython/ Activestate Python] 2.5, which comes with the needed extensions.

== Configuration ==

The external configuration utility is used to configure the toolset's database connection. You shouldn't need to run the configuration utility during routine usage of the toolset, and if the defaults set up by your installer work you may not need to run it at all.

The default database string is "<code>Provider=SQLOLEDB.1;Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=bw_dragonage_content;Data Source=.\SQLEXPRESS</code>". For advanced configuration, click on the ellipsis button ([[Image:ellipsis.png]]) to open the data link properties window.

Note that the configuration utility will first ask where the game build is. The game executable is not currently available for distribution but it is not necessary for the toolset to run so there's no need to be concerned about this.

== Uninstallation ==

An uninstaller shortcut is available from the same start menu folder as the toolset itself. It will autodetect the location you installed the toolset in, display it to you to confirm that this is correct, and then after you click "next" it will ask which components to uninstall.

* Core Resources - uninstalled by default, this is the toolset itself.
* Database - Since the uninstaller can't automatically determine if you're using SQLServer Express purposes other than just the Dragon Age toolset, SQLServer is ''not'' uninstalled automatically by default. To have the uninstaller remove the database server as well, unselect this checkbox.

The uninstaller is generated on installation and therefore isn't signed. This may cause Vista to complain under some circumstances but shouldn't affect the uninstaller's functioning.

== Troubleshooting ==

See [[installation troubleshooting]] for a list of common problems that can occur during installation and how to correct or work around them.

Note that there is currently an issue with installing the toolset for games that have been installed via Steam. See [[Installation with Steam]] for specific instructions on how to resolve this issue.

If you experience a crash logs can be found in <code>My Documents\BioWare\Dragon Age\Toolset\logs</code>. The log filenames will include the date on which the log file was created. Sending in a log will help greatly with bug fixing.

== Firewall issues ==

In its current pre-release state, the toolset requires constant internet access in order to deliver telemetry. If you use a router or firewall you will need to ensure that the toolset can connect properly with the outside world. If the internet connection is lost the toolset will pop up the authentication window again.

== Documentation ==

Documentation is currently provided via the wiki at:

http://social.bioware.com/wiki/datoolset/

[[Category:Toolset installation]]

Talk:Demo module

2009-11-07T03:00:02Z

ViresDroid:

I was able to get the demo to work, however I can't seem to find the "Generate FaceFX" menu option. I am using the initial toolset release from today.--[[User:ViresDroid|ViresDroid]] 23:52, 6 November 2009 (UTC)

:I've added links to screenshots of the menus to make them easier to find. The screenshots are of an older version, ignore the "to network" options (they were used internally and were removed for the end user version of the toolset). [[User:BryanDerksen|BryanDerksen]] 00:47, 7 November 2009 (UTC)

::Just wanted to stop by here to say I'm running into the same type of issue. It's as if your screenshot is from the "teacher's edition" of the toolset. Just to confirm, the version I downloaded and installed from Bioware today is v1.0.982.0. [[User:Falgoryn|Falgoryn]] 02:43, 7 November 2009 (UTC)

:: I included a screenshot of what the toolset looks like for me (missing the Generate FaceFX option): [[Media:no_generate_facefx.png|see here]] --[[User:ViresDroid|ViresDroid]] 03:00, 7 November 2009 (UTC)

Talk:Demo module

2009-11-07T02:59:54Z

ViresDroid:

File:No generate facefx.png

2009-11-07T02:57:31Z

ViresDroid: Just a screenshot of the editor displaying the missing Generate FaceFX menu option, which should be below Generate VO.

Just a screenshot of the editor displaying the missing Generate FaceFX menu option, which should be below Generate VO.

Talk:Demo module

2009-11-06T23:52:36Z

ViresDroid: Created page with 'I was able to get the demo to work, however I can't seem to find the "Generate FaceFX" menu option. I am using the initial toolset release from today.--~~~~'

GFF

2009-10-26T17:38:53Z

ViresDroid: changed the space inside of the "PC " platform quotes to nbsp; so it displays as two spaces instead of one (wikimedia collapes multiple spaces to one)

The Binary Generic File Format (GFF) is intended to be used for almost every file type in a game so it must support as many data types as are needed.

== General overview ==

In a GFF, data is grouped into structs, with the file level being the top level struct. Each struct can have any number of fields of any type including structs and variable size lists of any type.

== Differences between GFF V4.0 and GFF V3.2 ==

For those familiar with GFF V3.2, here is a short list of differences between V3.2 and the new version

* 4-byte numerical labels instead of 16 byte strings for faster access
* Supports lists of any type for convenience (instead of only structs)
* All struct data is stored in the data block (rather than fields) to allow for faster loads. Fields provide a mapping of where the data is stored.
* Supports references to mimic pointers to allow complex data structures
* A few fields were added to the Header
** File Version records the version of the file type using the GFF
** Target Platform records the intended platform for the resource
* In general, the format has been simplified and unnecessary sections were removed (label array, field indices array and list indices array)

Overall, the new format should result in smaller files with faster access times.

== File Format Conceptual Overview ==

=== Field Data Types ===
There will be a starting list of supported types and new types can be added as needed. The file format can support as many as 65,535 different basic types.

{| border="1"
|-
| Type || Type ID
|-
| UINT8 || 0
|-
| INT8 || 1
|-
| UINT16 || 2
|-
| INT16 || 3
|-
| UINT32 || 4
|-
| INT32 || 5
|-
| UINT64 || 6
|-
| INT64 || 7
|-
| FLOAT32 || 8
|-
| FLOAT64 || 9
|-
| Vector3f || 10
|-
| Vector4f || 12
|-
| Quaternionf || 13
|-
| ECString || 14
|-
| Color4f || 15
|-
| Matrix4x4f || 16
|-
| TlkString || 17
|}



Strings are stored as list of WCHARs.

There's also a "Generic" type that's only usable in references, with type ID 0x0000FFFF

=== Field Labels ===

The Binary GFF uses 4-byte IDs to label each field. Within each struct, each field must have a unique ID. These IDs could be string hashes or numerical IDs. The only requirement is that the reader and writer of these files agree on the IDs.

== File Format Physical Layout ==
=== Platform dependence ===

The endianness of the data is that of the target platform. For example, data files for intel processors should be little endian whereas data for power pcs should be big endian.

There may be other differences in the files generated for different platforms in order to achieve proper alignment or the desired in memory layout of the data.

=== Overall File Layout ===

<TABLE cellSpacing=2 cellPadding=2 width="25%" border=1>
<TR>
<TD>Header</TD></TR>
<TR>
<TD>Struct Array</TD></TR>
<TR>
<TD>Field Array</TD></TR>
<TR>
<TD>Raw Data Block</TD></TR>
</TABLE>

==== Header ====
The header is located at the start of the file and contains the following values.

<TABLE cellSpacing=2 cellPadding=2 width="90%" border=1>
<TR>
<TD> <STRONG>Value</STRONG></TD>
<TD> <STRONG>Description</STRONG></TD></TR>
<TR>
<TD>GFFMagicNumber</TD>
<TD>All GFF files will start with the hexadecimal value 0x47464620, which is the ASCII value for “GFF “.</TD></TR>
<TR>
<TD>GFFVersion</TD>
<TD>4 bytes representing the version of the underlying GFF format. This should be “V4.0” or 0x56342E30 for all files using this format.</TD></TR>
<TR>
<TD>TargetPlatform</TD>
<TD>4-byte field indicating the intended target platform for this file.
*“PS3 “ or 0x50533320 for the Playstation 3
*“X360” or 0x58333630 for the XBOX 360
*“PC  “ or 0x50432020 for the PC
There will most likely be more specialized targets for the PC in the future.</TD></TR>
<TR>
<TD>FileType</TD>
<TD>4-byte field used to identify this file type. By convention it should be the three letter file extension followed by a space.</TD></TR>
<TR>
<TD>FileVersion</TD>
<TD>4-byte version of the FileType. By convention it should be “Vx.x” or “xx.x” where X is a digit.</TD></TR>
<TR>
<TD>StructCount</TD>
<TD>4-byte unsigned number of elements in the Struct Array.</TD></TR>
<TR>
<TD>Data Offset</TD>
<TD>4-byte unsigned offset from the beginning of the file to the Raw Data Block.</TD></TR>
</TABLE>

The first five fields are always in big endian and never byteswapped. This keeps those fields human readable on any machine.

==== Struct Array ====

A struct is a grouping of data. A struct definition describes which data is in a struct. Many instances of a struct type may occur in a single file but there will only be one definition for each struct type.

The Struct Array starts immediately after the header. The first element in the Struct Array is the Top-Level Struct for the GFF file and it describes what the file looks like at the top level. Since the Top-Level Struct is always present, every GFF file contains at least one element in the struct array.

The Struct Array looks like this:

<TABLE cellSpacing=2 cellPadding=2 width="35%" border=1>
<TR>
<TD>Struct 0 (Top-Level Struct)*</TD></TR>
<TR>
<TD>Struct 1</TD></TR>
<TR>
<TD>Struct 2</TD></TR>
<TR>
<TD>...</TD></TR>
<TR>
<TD>Struct N-1**</TD></TR>
</TABLE>

<nowiki>*</nowiki>Struct 0 is always present<br/>
<nowiki>**</nowiki>N = Header.StructCount

The GFF Struct contains the values listed in the table below.

<TABLE cellSpacing=2 cellPadding=2 width="35%" border=1>
<TR>
<TD> <STRONG>Value</STRONG></TD>
<TD> <STRONG>Description</STRONG></TD></TR>
<TR>
<TD>StructType</TD>
<TD>4-byte programmer defined ID</TD></TR>
<TR>
<TD>FieldCount</TD>
<TD>4-byte number of fields in the struct</TD></TR>
<TR>
<TD>FieldOffset</TD>
<TD>4-byte unsigned offset from the beginning of the file to the first field in the struct</TD></TR>
<TR>
<TD>StructSize</TD>
<TD>4-byte unsigned size of the chunk of data representing the struct</TD></TR>
</TABLE>

All the fields for a struct are contiguous so knowing the address of the first one and the number of fields is enough information to access any element in the struct.

==== Field Array ====

The Field Array starts immediately after the Struct Array. Each field entry describes a piece of data contained in a struct. Each struct’s fields must be contiguous in the array and appear in increasing order of their labels. The fields for the Top Level Struct appear first in the array.

The Field Array looks like this:

<TABLE cellSpacing=2 cellPadding=2 width="35%" border=1>
<TR>
<TD>Struct 0 field 0</TD></TR>
<TR>
<TD>Struct 0 field 1</TD></TR>
<TR>
<TD>Struct 0 …</TD></TR>
<TR>
<TD>Struct 0 field Struct.FieldCount — 1</TD></TR>
<TR>
<TD>Struct x field 0</TD></TR>
<TR>
<TD>Struct x field 1</TD></TR>
<TR>
<TD>Struct x …</TD></TR>
<TR>
<TD>Struct x field Struct.FieldCount — 1</TD></TR>
<TR>
<TD>…</TD></TR>
</TABLE>

Each field looks like this:

<TABLE cellSpacing=2 cellPadding=2 width="75%" border=1>
<TR>
<TD> <STRONG>Value</STRONG></TD>
<TD> <STRONG>Description</STRONG></TD></TR>
<TR>
<TD>Label</TD>
<TD>4-byte label used to look up the field</TD></TR>
<TR>
<TD>FieldType</TD>
<TD>4-byte describing the type of the field (see below for explanation)</TD></TR>
<TR>
<TD>Index</TD>
<TD>4-byte unsigned offset to the location of the data</TD></TR>
</TABLE>

The label is just a 4-byte value used to find the correct field. They could be string hashes or some other numerical ID.

The index field stores the location of the data as an offset from the beginning of the struct in the data block. This can result in padding within the structs which can be garbage data for all we care (although by convention we usually start with 0xFF). In particular this happens when trying to maintain alignment for data types such as 16 byte alignment for vectors, etc.

The type is broken up into two 2-byte values that describe the type of the field.

The type looks like this:
<TABLE cellSpacing=2 cellPadding=2 width="50%" border=1>
<TR>
<TD> <STRONG>Value</STRONG></TD>
<TD> <STRONG>Description</STRONG></TD></TR>
<TR>
<TD>Flags</TD>
<TD>2-bytes of bit flags</TD></TR>
<TR>
<TD>TypeID</TD>
<TD>A 2-byte unsigned number indicating the type </TD></TR>
</TABLE>

The following flags are currently defined starting from the most significant bits.

<TABLE cellSpacing=2 cellPadding=2 width="90%" border=1>
<TR>
<TD> <STRONG>Bit</STRONG></TD>
<TD> <STRONG>Flag Description</STRONG></TD></TR>
<TR>
<TD>1</TD>
<TD>List Flag. If this flag is set then this type is a list of the described type.</TD></TR>
<TR>
<TD>2</TD>
<TD>Struct flag. If this flag is set then this type is a struct.
If the struct flag is not set then the BaseType indicates the type of the field by the integer id of that type. If the struct flag is set then the BaseType indicates the index of this struct’s description in the Struct Array.</TD></TR>
<TR>
<TD>3</TD>
<TD>Reference flag
If the reference flag is set then the data stored in the data block is actually an offset from the beginning of the data block to the location of the actual data. References can be used to mimic pointers in a GFF.</TD></TR>
</TABLE>

==== Raw Data Block ====

The data block is where the actual data is stored. The data for the top level struct is stored at the beginning of the block. All other data will be fields in the struct or accessed by reference.

=== Lists ===

The address pointed to by a list is actually a reference to another location in the file which stores the list. The first thing in the list is a 4-byte unsigned length of the list followed by the elements.

This is what the list looks like:

<TABLE cellSpacing=2 cellPadding=2 width="20%" border=1>
<TR>
<TD>Length</TD></TR>
<TR>
<TD>Element 0</TD></TR>
<TR>
<TD>Element 1</TD></TR>
<TR>
<TD>...</TD></TR>
<TR>
<TD>Element Length - 1</TD></TR>
</TABLE>

Empty lists can just store a null reference to prevent creating another block.

=== References ===

A reference is a 4-byte unsigned offset from the beginning of the data block to the location of the data.

Null references are stored as 0xFFFFFFFF.

== Improvements and optimizations ==

=== Field look-up ===

Field look-up is faster in this version of the GFF because it requires an integer comparison instead of a string comparison. Additionally, storing fields in order allows efficient search for the specific field. Finally, knowing what the data structure should look like allows the program to make a good guess as to where the data should be. If the guess is wrong (because the file is an older version) then the program only has to start searching at the initial guess, which is still a faster search than without a good guess.

=== Binding tables ===

It is possible to build a binding table for a type of file that will be read in often. The binding table will allow data to be loaded from the GFF without parsing through the file each time.

=== Direct loads to memory ===

Using this format it is possible to optimize specific file types by writing the GFF in exactly the way it will be written in memory. After verifying that the file type is up to date, the game can read the data block into memory and cast the pointer it directly to the in C data type. This optimization would not result in any loss of generality since the GFF will still be accessible in the usual way.

[[Category:File types]]