{{
`${userDetail.user.displayName}${userDetail.user.cachedBungieGlobalDisplayNameCode ? `#${standardizeBungieGlobalCode(userDetail.user.cachedBungieGlobalDisplayNameCode)}` : ""}`}}
{{getUserClans()}}
originally posted in:BungieNetPlatform
View Entire Topic
3/3/2015 6:09:45 AM
7
How to Read the Manifest (Reference)
So you might have just discovered the Destiny APIs and/or have already spent some time banging your head against the table trying to make sense of the Manifest file. Back when I initially started playing around with it, I was trying to figure this out using comments pulled off of various threads in this Group, and even then, I think I still had to figure out some things on my own using PHP.
The manifest can be located at:
http://www.bungie.net/platform/Destiny/Manifest/
It will give you something like this:
[spoiler]array (
'version' => '43982.15.02.02.1746-3',
'mobileAssetContentPath' => '/common/destiny_content/sqlite/asset/asset_sql_content_7c23eefeaa0003dbc141793135dae141.content',
'mobileGearAssetDataBases' =>
array (
0 =>
array (
'version' => 0,
'path' => '/common/destiny_content/sqlite/asset/asset_sql_content_7c23eefeaa0003dbc141793135dae141.content',
),
1 =>
array (
'version' => 1,
'path' => '/common/destiny_content/sqlite/asset/asset_sql_content_5eed6e9a4cb8e84eeb081b93acf267c4.content',
),
),
'mobileWorldContentPaths' =>
array (
'en' => '/common/destiny_content/sqlite/en/world_sql_content_41764675ffa70c7fca0a6873b77aabeb.content',
'fr' => '/common/destiny_content/sqlite/fr/world_sql_content_5c56a7c5c166383195afdcdaa64f4575.content',
'es' => '/common/destiny_content/sqlite/es/world_sql_content_a784ce4972713d7d2f665749a4727d7b.content',
'de' => '/common/destiny_content/sqlite/de/world_sql_content_f0263aae5495546025628408dbdf2fc0.content',
'it' => '/common/destiny_content/sqlite/it/world_sql_content_1726e5fda614c91686b38b6a5debefb0.content',
'ja' => '/common/destiny_content/sqlite/ja/world_sql_content_554c8560f931fedb73e9454481fb4bd4.content',
'pt-br' => '/common/destiny_content/sqlite/pt-br/world_sql_content_12e4e5a8fd13e8efa699396f80a93740.content',
),
)[/spoiler]
The main files you will probably want to look at are the "mobileWorldContentPaths" (the other 2 I believe are for getting assets needed to render 3D models). Each of these files are an SQLite database file that has been put into a Zip file.
Extracting them in PHP is pretty straightforward (get_url is a function I wrote that basically does a cURL request which includes stuff like cookies and my API key etc, file_get_contents can be used here, but it can be much slower than using cURL):
[spoiler]$cacheFilePath = 'cache/'.pathinfo($path, PATHINFO_BASENAME);
file_put_contents($cacheFilePath.'.zip', get_url($path));
$zip = new ZipArchive();
if ($zip->open($cacheFilePath.'.zip') === TRUE) {
$zip->extractTo($_SERVER['DOCUMENT_ROOT'].'/cache');
$zip->close();
}[/spoiler]
Now if you are like me, and don't have an SQLite viewer on hand, you have an SQLite file that you don't know what tables it contains or what columns are in said tables. Basically each table will generally have two columns, "id" or "key" and "json". Most of the time, "id" is not the entry's hash so you can't just query for a given entry like you can the /Manifest/{type}/{hash} API. So instead, we need to extract these SQLite files further into something we can actually look at and be able to access.
Note: $dbtype is referring to the section in the manifest the file is from (gear, asset, world).
[spoiler]if ($db = new SQLite3($dbfile)) {
$result = $db->query("SELECT name FROM sqlite_master WHERE type='table'");
while($row = $result->fetchArray()) {
$result2 = $db->query('SELECT * FROM '.$row['name']);
$data = array();
while ($col = $result2->fetchArray(true)) {
$json = json_decode($col['json'], true);
if (isset($col['id'])) {
$data[$col['id']] = $json;
} else if (isset($col['key'])) {
$data[$col['key']] = $json;
} else {
$data[] = $json;
}
}
if (!file_exists($dbtype)) mkdir($dbtype);
file_put_contents($dbtype.'/'.$row['name'].'.json', json_encode($data));
}
}[/spoiler]
You should now have some json files you can open up and view. When loading these json files, you should load them in so that the {dataType}Hash value becomes the key so you can reference them directly, though not all tables use a hash to identify objects.
If anyone has solutions for doing this using other languages/frameworks, feel free to post it below so we can have a consolidated guide in one place.
English
-
5 RepliesEdited by purin: 5/2/2015 1:24:17 AMNote - You don't actually need to extract the JSON to get the hash for a given row - you CAN use the id column. OP is correct that they don't always match - but this is for a very simple reason. itemHash appears to be an unsigned 32bit integer, but SQLite is interpreting the value as a signed int. See this query: SELECT * FROM DestinyInventoryItemDefinition WHERE CAST(SUBSTR(json, 13,10) AS INT) != id AND CAST(SUBSTR(json, 13,10) AS INT) != 4294967296 + id No rows are returned, because either the itemHash matched the id (first case of WHERE clause) or converting the unsigned itemHash to a signed int causes it to match the id (second case of the WHERE clause). So, if you're using a C-like language with casts, just make your itemHash an unsigned 32 bit int, cast it to a signed int, and then use that value as the id. If you're using a weakly-typed language like PHP, or if your language doesn't allow you to specify 32-bit ints, you can just convert it manually: SELECT * FROM DestinyInventoryItemDefinition WHERE id + 4294967296 = <ItemHash> OR id = <itemHash> Just replace <ItemHash> above with the item hash you're looking for. And obviously, replace "DestinyInventoryItemDefinition" with the table you're interested in. Should be significantly more efficient than parsing the entire database, and should also be much faster for lookup.
