Archive Library

The Archive Library supports reading and writing files from the PC releases of the game.

File List contains one or more path strings. Each starting with C:\ff8\Data\ ending with \r\n. These are in the same order as the File Index. You scan the File List to find the items you want, while doing this you would increment a counter. The counter is used to know which entry to get from File Index. Lastly, you use the File Index to read from the File Source.

ZZZ contains multiple files including FLFIFS archives of different languages. The ZZZ header starts with the number of entries followed by FileData entries. File Source functions can accept FileData entries to extract from ZZZ files.

References:

Archives

struct open_viii::archive::Archives

Reads a path looking for ZZZ or FIFLFS archives. It remembers the locations of the archives, and serves serves as a front end to access them.

Public Functions

template<ArchiveTypeT archiveTypeT>
inline const auto &get() const noexcept

Get archive via ArchiveTypeT

Return

Template Parameters

  • archiveTypeT:

template<ArchiveTypeT archiveTypeT>
inline auto &get() noexcept

Get archive via ArchiveTypeT

Return

Template Parameters

  • archiveTypeT:

inline explicit operator bool() const noexcept

convert to bool

Return

inline auto get_nested(const std::initializer_list<std::string_view> &nested_archive) const

Gets the nested FIFLFS files from Field archive.

Return

all results.

Parameters

  • nestedArchive: string to filter results. {} will get all nested archives.

Archives() = default

This will be in an invalid state but it’s so I can default construct and move in valid values later.

inline Archives(const std::filesystem::path &path, std::string_view lang)

Preloads all archives in the path.

Parameters

  • path: that contains FIFLFS files or ZZZ files.

template<std::intmax_t minT = static_cast<std::intmax_t>(ArchiveTypeT::begin), std::intmax_t maxT = static_cast<std::intmax_t>(ArchiveTypeT::end), typename lambdaT>
inline bool loop(const lambdaT &lambda) const

Loop through each of the archives.

Return

true if ran to completion

Template Parameters

  • minT: min archive

  • maxT: max archive

  • lambdaT: type of lambda function that returns bool and takes archive types. The bool returned short circuits the loop.

Parameters

  • lambda: lambda of type lambdaT

template<ArchiveTypeT... aT, typename lambdaT>
inline bool specify(const lambdaT &lambda)

run lambda on listed types.

Return

Template Parameters

  • aT:

  • lambdaT:

Parameters

  • lambda:

inline bool test_set() const

if true all archives are valid.

Return

template<ArchiveTypeT... aT>
inline bool test_set()

if true listed archives are valid

Return

Template Parameters

  • aT:

template<bool nested = true, typename lambdaT, typename filterT = decltype(default_filter_lambda)>
inline bool execute_on(const std::initializer_list<std::string_view> &filename, lambdaT &&lambda, filterT &&filter_lambda = {}) const

execute on all archives the following lambda.

Return

Template Parameters

  • nested:

  • lambdaT:

Parameters

  • filename:

  • lambda:

template<bool nested = true, ArchiveTypeT... aT, typename lambdaT, typename filterT = decltype(default_filter_lambda)>
inline bool execute_on(const std::initializer_list<std::string_view> &filename, lambdaT &&lambda, filterT &&filter_lambda = {}) const

execute on all listed archives the following lambda.

Return

Template Parameters

  • nested:

  • aT:

  • lambdaT:

Parameters

  • filename:

  • lambda:

ArchiveTypeT

enum open_viii::archive::ArchiveTypeT

There are 6 main FIFLFS archives for ff8 and 2 2 main zzz archives for ff8 remaster. These enums are for get<> in archives to ask for the one you want.

Values:

enumerator battle
enumerator field
enumerator magic
enumerator main
enumerator menu
enumerator world
enumerator zzz_main
enumerator zzz_other
enumerator count
enumerator first
enumerator last
enumerator begin
enumerator end

FIFLFS

template<bool HasNested>
struct FIFLFS

Grouping

template<std::ranges::contiguous_range T>
struct open_viii::archive::Grouping

Grouping contains the location of the file and/or the raw data.

Public Functions

inline const std::filesystem::path &path() const noexcept

get path to file containing archive

Return

inline std::size_t offset() const noexcept

get offset in bytes to start

Return

inline std::size_t size() const noexcept

get Size of file / also defaults size if value is 0.

Return

size_t

inline const T &data() const noexcept

get loaded data buffer

Return

inline const std::string &base() const noexcept

stem of filename upper cased

Return

inline const std::filesystem::path &nested_path() const noexcept

get path inside file

Return

inline explicit operator bool() const

convert to bool

Return

true means value is set and loaded.

File List
namespace open_viii::archive::fl

Functions

inline constexpr void clean_path_string(std::string &input) noexcept

FL files contain internal file structure paths. As a flat text file. This class is used to search the strings for a filename or grab all the entries. The entry will be a string paired with an int that is the line number. This is used to ID the FI entries.

Todo:

refactor this to reduce parameters and reduce functions.

Remove the C:\ from the start, remove the \r from the end, and change \ to the correct slash. added skipFixed if data is set then i probably fixed slashes already.
Return

void

Parameters

  • input: updates this string

  • skipFixed: if false skip removing the \r from end and skip replacing slashes.

inline std::string clean_path_string(std::string &&input) noexcept

Remove the C:\ from the start, remove the \r from the end, and change \ to the correct slash. added skipFixed if data is set then i probably fixed slashes already.

Return

modified input

Parameters

  • input: updates this string

  • skipFixed: if false skip removing the \r from end and skip replacing slashes.

inline std::string clean_buffer(std::string &&in_buffer)

Take out carriage returns and replace slashes.

Return

cleaned.

Parameters

  • in_buffer: multi line string of paths.

inline constexpr std::size_t get_max(const std::size_t &count, const std::size_t &limit)

Decide how much to reserve based on known count or a set limit.

Return

0 or count or limit;

Parameters

  • count: computed max count

  • limit: manual limit placed

inline void sort_entries(std::span<std::pair<std::uint32_t, std::string>> vector)

Sort the strings. to make it easier to choose the correct string first. shorter length and then what ever str < str2 does.

Parameters

  • vector: pairs of ints and paths

inline auto sort_entries(std::vector<std::pair<std::uint32_t, std::string>> &&vector)

Sort the strings. to make it easier to choose the correct string first. shorter length and then what ever str < str2 does.

Return

sorted vector

Parameters

  • vector: pairs of ints and paths

inline std::vector<std::pair<std::uint32_t, std::string>> get_all_entries(const tl::read::input &cont, const size_t &offset, const size_t &size = 0U, const size_t &count = 0U, const std::initializer_list<std::string_view> &needle = {}, const size_t &limit = 0U)

Eagerly populate a vector with pairs of (id,path), then sort it.

Todo:

make needle a predicate lambda.

Note

If size, limit, count are all 0 it’ll read till end of file. Or till it reads an empty line.

Parameters

  • cont: wrapper on istream or span. For the source data.

  • offset: to start of data.

  • size: + offset is the end of the data.

  • count: max count detected from FI filesize / 12U.

  • needle: set of strings to search for.

  • limit: manually set max count.

inline auto get_all_entries(const std::filesystem::path &path, const size_t &offset, const size_t &size = 0U, const size_t &count = 0U, const std::initializer_list<std::string_view> &needle = {}, const size_t &limit = 0U)
inline std::vector<std::pair<std::uint32_t, std::string>> get_all_entries(const std::string &data, const size_t &offset, const size_t &size, const size_t &count, const std::initializer_list<std::string_view> &needle, const size_t &limit)
inline std::vector<std::pair<std::uint32_t, std::string>> get_all_entries(const std::filesystem::path &path, const std::string &data, const size_t &offset, const size_t &size = 0U, const size_t &count = 0U, const std::initializer_list<std::string_view> &needle = {}, const size_t &limit = 0U)

Get All entries sorted from file or data buffer.

Return

matches

Parameters

  • path: filename path.

  • data: buffer of bytes.

  • offset: bytes from start of data to start looking.

  • size: of FL file if known; 0 == unlimited

  • count: expected number of matches; calculated from FI file size / 12; 0 == unlimited

  • needle: possible string matches; {} == all

  • limit: max matches; 0 == unlimited

template<typename T>
inline auto get_entry(const T &data, const std::initializer_list<std::string_view> &needle, const size_t &offset = 0U, const size_t &size = 0U, const size_t &count = 0U)

Get a single entry that is the first match for needle.

Parameters

  • data: contains buffer of chars

  • needle: is a group of strings to filter the output with.

  • offset: is the number of bytes to skip.

  • size: is max number of bytes. 0 is unlimited.

  • count: is max results returned. 0 is unlimited.

inline auto get_entry(const std::filesystem::path &path, const std::string &data, const std::initializer_list<std::string_view> &needle, const size_t &offset = 0U, const size_t &size = 0U, const size_t &count = 0U)

Get a single entry that is the first match for needle.

Parameters

  • path: contains path to file

  • data: contains buffer of chars

  • needle: is a group of strings to filter the output with.

  • offset: is the number of bytes to skip.

  • size: is max number of bytes. 0 is unlimited.

  • count: is max results returned. 0 is unlimited.

Variables

static constexpr auto EXT = std::string_view(".fl")

File extension

File Index
struct open_viii::archive::FI

FI is the file index for the FL and FS files.

Public Functions

inline constexpr auto uncompressed_size() const noexcept -> std::uint32_t

Uncompressed size in bytes.

inline constexpr auto offset() const noexcept -> std::uint32_t

Offset is distance from beginning in bytes.

inline constexpr auto compression_type() const noexcept -> CompressionTypeT

Compression type (uncompressed, lzss, or lz4).

Public Static Attributes

static constexpr std::size_t SIZE = 12U

Expected size of the FI struct, used for a static assert after.

static constexpr auto EXT = std::string_view(".fi", 3U)

File extension for FI

File Source
namespace open_viii::archive::FS

Functions

template<is_default_constructible_has_data_size_resize outputT = std::vector<char>>
static outputT get_entry_lzss(tl::read::input &input, const std::uint32_t uncompressed_size)

Get entry and uncompress via lzss

Note

not meant to be used directly

Return

output() filled with uncompressed data.

Note

lzss doesn’t need the uncompressed size to extract the data but it’s used to reserve the memory before uncompressing.

Template Parameters

  • outputT: type to be returned: example std::string, std::vector<char>

  • fiT: FI or FI compatible type.

Parameters

  • input: buffer adapter that holds a std::span<char> or a std::istream

  • uncompressed_size: number of bytes expected to expand to.

template<is_default_constructible_has_data_size_resize outputT = std::vector<char>>
static outputT get_entry_lz4(tl::read::input &input)

Get entry and uncompress via lz4

Note

not meant to be used directly

Return

output() filled with uncompressed data.

Note

L4Z header contains size of total section as uint32, 4 byte string, and a uint32 of the uncompressed size.

Template Parameters

  • outputT: type to be returned: example std::string, std::vector<char>

  • fiT: FI or FI compatible type.

Parameters

  • input: buffer adapter that holds a std::span<char> or a std::istream

  • uncompressed_size: number of bytes expected to expand to.

template<is_default_constructible_has_data_size_resize outputT = std::vector<char>, FI_Like fiT = FI>
static outputT get_entry(tl::read::input input, const fiT fi, const std::size_t offset = 0U)

Get entry and uncompress via lz4

Return

output() filled with uncompressed data.

Template Parameters

  • outputT: type to be returned: example std::string, std::vector<char>

  • fiT: FI or FI compatible type.

Parameters

  • input: buffer adapter that holds a std::span<char> or a std::istream

  • fi: contains compression type, uncompressed size, and offset

  • offset: additional offset value added to fi.offset()

template<is_default_constructible_has_data_size_resize outputT = std::vector<char>, FI_Like fiT = FI>
static outputT get_entry(const std::filesystem::path &path, const fiT &fi, const size_t &offset = 0U)

get file entry and decompress it

Return

uncompressed file

Template Parameters

  • outputT: type being returned

  • fiT: type of FI or FileData that contains offset, size, compression.

Parameters

  • path: to file

  • fi: FI or FileData

  • offset: to file data in bytes

template<is_default_constructible_has_data_size_resize outputT = std::vector<char>, FI_Like fiT = FI>
static outputT get_entry(std::span<const char> data, const fiT &fi, const size_t offset = 0U)

get file entry and decompress it

Return

uncompressed file

Template Parameters

  • outputT: type being returned

  • fiT: type of FI or FileData that contains offset, size, compression.

Parameters

  • data: buffer with embedded file

  • fi: FI or FileData

  • offset: to file data in bytes

Variables

static constexpr auto EXT = std::string_view(".fs")

File Source

See

http://wiki.ffrtt.ru/index.php?title=FF8/PC_Media#.fs_.28File_Source.29 Extension

ZZZ

struct ZZZ

ZZZ file archive from FF8 remaster.

See

https://github.com/myst6re/qt-zzz

FileData

struct open_viii::archive::FileData

FileData is an FI Like that contains the filepath, offset and size.

Public Functions

inline  FileData (decltype(m_filename) filename, const std::unsigned_integral auto offset, const std::unsigned_integral auto size)

Piecemeal constructor

Parameters

  • filename: path to file

  • offset: path to

  • size:

inline constexpr auto total_size() const noexcept

size of this file entry in the zzz file. This is for appending a new entry. I don’t know if it works.

Todo:

used in tests no where else. might want to make this a free function of the tests.

Return

inline auto get_path() const

gets path as a std::filesystem::path

inline constexpr auto uncompressed_size() const noexcept

alias for Size that should mirror FI

inline constexpr auto offset() const noexcept

get offset of file

inline auto get_path_string_view() const

gets path as a std::string_view

inline auto get_path_string() const

gets path as a std::string

template<std::size_t I>
inline auto get() const noexcept

gets member variables

Note

required to structured binding support

Public Static Functions

static inline constexpr auto compression_type() noexcept

Compression type required to match FI like concept.

Return

no compression