How to Give Cartridge-Based Video Game Data an Extra Life

The following is a guest post by Christopher Fox, a student employee at the National Institute of Standards and Technology (NIST). For over four years, Christopher has provided NIST’s National Software Reference Library project with data entry services and the application development demands that support those services. He is currently completing his senior year at Hood College in Frederick, Maryland, where he is earning a Bachelor’s in Computer Science. He started working on the video game software part of the Cabrinety project when the group hit a dead end trying to collect data from gaming cartridges. Since Chris enjoys gaming and found a way to collect save data from older games for his own purposes, he offered to explore and share methods of extracting data from these obsolete formats.

Disclaimer: Trade names and company products are mentioned in the text or identified. In no case does such identification imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the products are necessarily the best available for the purpose.

After experimenting with a USB adapter called Retrode, I discovered a method for retrieving forensically viable data from cartridge-based media in the Cabrinety collection.  The Retrode device collects ROM from Sega Genesis and Super Nintendo (SNES) games, and with special plug-in devices, collects ROM from Game Boy, Atari 2600, and Nintendo 64 games. Users can retrieve any saved files from cartridges plugged into the device and view the files from the directory. The Retrode displays as an external storage device along with its accompanying configuration file.

 

 

 

 

 

 

 

 

The idea is to use an emulator to play the ROM, back up the ROM, and save files onto the user’s computer in case of cartridge damage. Older gaming cartridges use batteries as a way of saving data, but when the batteries eventually die, the saves are lost and the cartridges are unable to create new saves. Using the Retrode to back up data to a user’s computer allows the game to be played and saved while preventing data loss from dying batteries.

The Retrode includes two SNES and two Sega Genesis controller plug-ins for playing games with the original controllers.

I encountered at least one problem with the Retrode while attempting to collect the ROMs. To check for consistency, I took the SHA1 of each ROM generated by the Retrode. Each time they were mounted, every cartridge had a different level of consistency in its SHA1's. Getting different SHA1 values made collecting data from the cartridges more challenging because it was impossible to determine what a "good" ROM’s SHA1 value was since I couldn’t replicate the value.

I had to find another way to identify whether or not the ROM generated was forensically viable.

To better understand how gaming cartridges work, here is a brief synopsis of the ROM makeup of these cartridges.

Every gaming cartridge has metadata (data that provides information about the cartridge) stored at specific hexadecimal addresses in the ROM. This is referred to as the "ROM Header." Every type of cartridge – SNES, Sega Genesis, Game Boy, etc. – has its respective metadata stored at different addresses. The place where the metadata begins depends on a cartridge’s "offset." Most cartridges have one specific offset where the metadata addresses begin.

For example, the Sega Genesis' and Game Boy's metadata start at the hexadecimal address 0x100. However, some cartridges have multiple addresses where the metadata begins. The SNES cartridge has two "Bank sizes" called, "LoROM" and "HiROM." The Bank size determines where the offset is and consequently, where the metadata starts. LoROM starts at 0x7FC0 and HiROM starts at 0xFFC0. Starting at the offset, cartridges have different types of data that they have stored.

Types of data include game name, manufacturer name, copyright information, and ROM size. The most important data stored in the metadata, at least when determining if the ROM is good, is the "checksum." A checksum is used by companies like Nintendo and Sega to fight game piracy. The checksum is calculated through each company’s unique algorithm. At first, the ROMs were tested in emulators to see if they would run. If they passed the company’s checksum algorithm, they were considered good ROMs of the game. If a single byte in the ROM was different, the checksum wouldn’t calculate properly and thus, not run.

 

 

 

                      

1 Good ROM                                                  2 Bad ROM

I did some research to find the algorithm for each type of cartridge. So far, I have only confirmed how to calculate the SNES and Sega Genesis checksums. I did find the Game Boy checksum algorithm, but I haven’t been able to confirm that it is correct since I cannot collect the ROM from these cartridges. I wrote a script in PERL to test them, and I have listed the code for each algorithm below each cartridge type.*

*A note regarding the PERL scripts:  "ord" is used to get the numerical value of the bytes for addition. "$bytes" is an array of each byte in the entire ROM. You need to convert it back to hex format and get the right bytes in the correct order from the sum according to each respective algorithm for the final comparison against the checksum.

The algorithms are:

SNES:

The checksum is 16bits long with the lower 8bits stored first, followed by the upper 8bits starting at address 0x7FDE for LoROM and 0xFFDE for HiROM. To calculate the checksum, divide the entire contents of the ROM into 4Mbit groups and then add all of the bytes in these chunks. Once you have the sum of each of these chunks, add all of the chunks together for a final sum and take the lower 16bits of that result, which should equal the checksum. Some cartridges have ROM sizes of 2Mbits or 10Mbits and are not equally divisible by 4Mbits. In this case, with the last 2Mbits that are left over, multiply the sum of the last 2Mbits by 2. (Fortunately, these sizes are not that common.)

2Mbit: sum of whole ROM multiplied by 2
10Mbit: 4Mbit + 4Mbit + (2 x 2Mbit)

PERL:

if ($length ==  "262144"){#File is 2Mbits  

for (my $i=0;$i<=$length;$i++){  

$sum += ord $bytes[$i];  

$sum = $sum*2;  

}

}

elsif($length == "1310720"){#File is 10Mbits

for (my $i=0;$i<="1048576";$i++){#Go through first 8Mbits

$sum += ord $bytes[$i];

}

for (my $i="1048577";$i<=$length;$i++){#Go through last 2Mbits

$sum2 += ord $bytes[$i];

}

$sum = $sum + ($sum2*2);#Excess 2Mbits needs to be multiplied twice to make even 4Mbits

}

else{#File is a multiple of 4Mbits

for (my $i=0;$i<=$length;$i++){

$sum += ord $bytes[$i];

}



Sega Genesis:

The checksum is also 16bits long starting at 0x018E. To calculate the checksum:

1.             The checksum equals 0.

2.             Start at address 0x200. 

3.             Read the first byte at that address and shift the bits to the left 8 (this can also be done by multiplying          the ASCII value of the byte by 256) and add this to the total checksum.

4.             Take the next byte in the ROM and sum it to the checksum.

5.             Repeat steps 3 and 4 until you reach the end of the file.

6.             Take the first 16bits of the result and discard the higher bits.

PERL:

my $start = 512 #200h

for (my $i=$start;$i<$length;$i+=2){#Starts from 200h to end of file

$checksum += (ord ($bytes[$i]) << 8) + (ord $bytes[$i+1]);

}



Game Boy (Not confirmed. Based on several resources with this algorithm):

The Game Boy is slightly different in that it has two checksum values: a "header checksum" that is 8bits stored at address 0x014D and a "global checksum" that is 16bits stored at address 0x014E. Only the header checksum is verified before running.  To calculate the header checksum, start at the address 0x0134. For each byte until address 0x014C, the calculated checksum (starting at 0) equals the calculated checksum - the byte you are currently at - 1.

PERL:

my x=0; #x is the calculate checksum

for(my $i=0134h;$i<=014Ch;$i+=0001h){

x=x-$bytes[$i]-1;

}

Once the algorithm is calculated, compare the result to the specific address in the ROM Header where the checksum is stored.

                                         3 Script Output

 I created a script that will calculate the checksum of SNES and Sega Genesis ROMs and verify that it has a correct checksum. Once the checksum is confirmed, it will print out some of the     metadata to also help confirm that the ROM is good. If the checksum is valid, then we know it will run on  an emulator and use it for collecting data. 

 

 

 

 

                  5 SNES Metadata                        4 SNES Checksum (Lower Bits first)

Works in progress:  

Game Boy ROMs

As mentioned before, I haven’t yet been able to collect ROMs from Game Boy games. The Retrode does include a plug-in adapter for Game Boy games, but currently it only works with a very small selection of games. The Retrode has had several firmware updates that have helped with reading a wider variety of Game Boy cartridges; unfortunately, it is not enough for the vast number of cartridges within the Cabrinety collection. Most of the cartridges that can be read are the European versions (also known as PAL) of the game, which are not included in the Cabrinety collection. Currently, when the cartridges are plugged into the Retrode with the adapter, either it gives a raw binary empty dump of the cartridge or the data is completely scrambled on the ROM.

There is a way to force the format of the dump to a Game Boy ROM, but these will not run in emulators and the ROM is bad. For example, I was able to get the ROM to start up for the American version of the Pokémon Yellow (NTSC) game, but then it immediately crashed. The data in a hex editor showed that the metadata was scrambled. For now I will not be using the Retrode for collecting Game Boy ROMs for the Cabrinety collection until a new firmware update is released that offers help with reading more Game Boy cartridges.

After some extensive online research, I have found an alternative way that claims to be almost 100% successful in reading and collecting the ROMs from all Game Boy games tried. However, the method used to collect the data is a bit obscure.  Here is the procedure:

First, you need a Nintendo 64(N64) with the Expansion Pak (to help ensure compatibility) and a N64 controller with a “Transfer Pak.” The Transfer Pak is a device made by Nintendo that allows the user to plug in certain Game Boy games to be used in combination with a N64 cartridge. At first, I was a little worried, because I thought the Transfer Pak only supported certain games. But a game called PD Ultraman Battle Collection 64 states that users can plug in any Game Boy and Game Boy Color game in order to use some special features in the game. Because of this, I assume that the Transfer Pak will recognize all games for this method of dumping the ROM.

Next, you need a 64Drive device. You plug an SD Card (not in CF format), with the agbd program loaded on it. Plug this into the N64 to boot it up. Take this, along with the Game Boy game in the Transfer Pak and controller, and run the program to put the ROM onto the SD card.
 
Basically, plug in the Game Boy game, run agbd to transfer the ROM from the Game Boy cartridge onto the SD card, and then plug the SD card into a computer to run the script that I write (if this works and we get a correct ROM) to verify that the ROM is good.  It is a tedious process, but it should get the job done.
 
Smart Boy is another device I have come across for Game Boy ROM dumping.  The user simply connects the device with a USB and then plugs in the Game Boy cartridge. Using the program that comes with the device on a CD, simply press “Dump ROM.”  This may be the simpler option.
 
 
6 Game Boy Plug-in
 
Nintendo ROMs
 
The Retrode does not currently support Nintendo (NES) games for data collection. If ROMs are going to be collected it will have to be through another method.  I haven’t yet performed enough research with these cartridges in terms of ROM headers and checksum algorithms, but I managed to find a potential way to collect ROMs from the cartridges since the Cabrinety collection has so many of them.
 
CopyNES is a hardware NES that can be soldered into the NES Motherboard. CopyNES has a USB port on it and software that will let you do a lot with a NES cartridge, including dumping the entire contents to the connected computer. I haven't yet been able to confirm how well this works, or if it works at all, but I believe it should work with all NES cartridges that can be read in the authentic NES console. The challenge has been finding one because the device is sold out everywhere I have looked and is no longer in production.
 
Nintendo 64 
 
I have completed a lot of research on N64 cartridges, and while I couldn’t find these cartridges in the Cabrinety collection, it is beneficial to have this information in case we discover some. I do not have access to a N64 plugin for the Retrode, but I learned that N64 cartridges may be as troublesome as the Game Boy cartridges. The list of games that have had the ROM successfully taken off with the Retrode is short.

I found one other way to make it work, but the process is time-consuming.  A user would need an operational N64 with an Expansion Pak, a Gameshark v.3.3 device with a Parallel Cable, and a computer that has a parallel port. The process dumps all 512Mbits from the N64 cartridge to the computer. Since parallel ports are painfully slow, this entire process would take at least an hour, especially if the parallel port times out and the process has to be started all over again.

I still plan to test the Retrode with the N64 cartridges once I gain access to the Retrode plug-in. I have noticed that calculating the checksum of N64 cartridges is much more difficult than calculating the checksums of any other cartridge types I have come across. It seems everywhere I look there is a different way to calculate the checksum. Since I haven’t been able to dump a ROM from a N64 cartridge successfully, I can’t verify that any of the checksum algorithms are correct.
 
Resources:
 
Retrode Home Page: http://www.retrode.org
SNES ROM Makeup/Checksum: http://www.emulatronia.com/doctec/consolas/snes/sneskart.html
Genesis ROM Makeup/Checksum: http://www.emulatronia.com/doctec/consolas/megadrive/genesis_rom.txt
Game Boy ROM Makeup/Checksum: http://gameboy.mongenel.com/dmg/asmmemmap.html and http://gbdev.gg8.se/wiki/articles/The_Cartridge_Header
Transfer Pak Wiki and link of compatible games: http://nintendo.wikia.com/wiki/Transfer_Pak
Agbd Game Boy transfer program: http://lacklustre.net/n64/agbd/
Smart Boy cartridge: http://atariage.com/forums/topic/142114-smartboy-cart-in-stock-now-smartboycart-is-best-gameboy-develop-system/
CopyNES description: http://www.retrousb.com/product_info.php?cPath=24&products_id=36
List of N64 games that work on plug-in: http://forum.retrode.org/index.php/topic,132.0.html
N64 Dumping ROMs: http://www.nesworld.com/n64-howtodumproms.php
N64 Checksum: http://www.goldeneyevault.com/wiki/index.php?title=ROM