Add support for BTD (Bethesda Terrain Data) files

[Wully616]

@niftools/pyffi-reviewers

Issue Overview

BTD files store landscape mesh and texture information for Fallout 76 (\Fallout76\Data\Terrain\Appalachia.btd), instead of being stored in BSA or BA2 files

The file format itself may be unknown as of right now, as I've not found references anywhere about its structure. It may be similar to other bethesda archive formats.

I tried to reverse engineer the file and could find this information about the header at least:

Header

char[4] - magic (BTDB)
uint32 - version/content flag (6)
32 bytes - unknown
uint32 - number of materials stored in the file
uint32[number of materials] - an array of uint32s which contain either pointers or sizes, not sure

The rest of the data appears to be compressed in some format, i've not been able to reverse engineer which format from the fallout 76 exe yet.

I believe the file itself contains data relating to these files:

Data\Meshes\Terrain\Scratch%s.TMP.%u.%i,%i.HEM
Data\Meshes\Terrain%s%s.%u.%i.%i.BTR
Data\Textures\Terrain%s%s.%u.%i.%i.DDS
Data\Textures\Terrain%s%s.%u.%i.%i_msn.DDS

[moraxy]

Here's what I got so far after a quick look:

After your uint32[number of materials] are 80802 floats, possibly values for the height map.
The 32 bytes - unknown are

(float) -700 the lower limit in the chunk of floats
(float) 38209.99609375 the upper limit
(u32) 25728
(u32) 25728
(i32) -100
(i32) -100
(i32) 100
(i32) 100

The -100/-100 and 100/100 could be the extent of the X/Y grid. The 25728 then might also be grid related as it divides cleanly by 201, just like most other numbers.
E.g. 80802 floats / ( 201 * 201 ) = 2 floats per grid point?

After the chunk of floats, there are at least 3 more chunks:

0x0004ef60 ?
0x002c641c first guess: u16 counters, maybe indices?
  this chunk is 15513984 bytes long, and again: 15513984 / 2 / 80802 = 96?
0x01191d9c <u32, u32> tuples, 97557 of them apparently
  u32 relative offset in the next chunk
  u32 bytes to read
0x01250644 data for the tuples

The data snippets in the last chunk are compressed with zlib, haven't yet looked at them.

[Wully616]

Oh wow! Well done! That is a good step forward.
I think you're correct about the grid definitions. But where are you getting the value 201 from? - nevermind I got it, the -100 to 100 grid

[fo76utils]

This code can unpack the BTD file: https://github.com/fo76utils/fo76utils/blob/main/src/btdfile.cpp

There is a brief description of the file format at the beginning of the source code. Also, for the land textures, here is a table of the corresponding LTEX form IDs, material (.bgsm) files, and texture files: https://github.com/fo76utils/fo76utils/blob/main/ltex/fo76.txt