The added bonus of figuring out what goes into frontend web development is nice too.

_{Realizing as I'm typing my first blog post that I forgot to support images...}

_{More importantly, because everything is in a flat structure, it's definitely going to be a pain to properly isolate images per post... but maybe I can just call it a feature and have a common image directory.}

Preface

A recent study I saw popping up on some social media commented on how information density over time across spoken languages was fairly similar despite vastly different approaches (different syllable counts, general rate of speech, etc...). There are some obvious criticisms, of course. For example, language tends to bias towards specific cultural requirements and so there may be specific contexts where the language excels and conversely, specific ones where it falls short... but that aside, it did get me thinking a bit about written information density, especially when it comes to displaying things on fairly low-spec systems (_{yes I mean the Gameboy}).

I've done a fair amount of reverse engineering on old GB stuff to enable translations from Japanese. I think I'm generally rather slow overall, but I do have a process:

1. Identify what's writing the text out
   • Text is usually represented as a set of indices into a larger tileset
2. Use the knowledge from (1) to figure out where the text itself is stored
   • Intuitively, text is probably being written out as part of a larger scripting system, but we generally don't need all of it to just translate text
   • Sometimes it's embedded in the script itself (like most old GBDK titles), which means we have to figure out the scripts to an extent anyway...
3. (Optionally) Dump the tileset to png and allow fo rebuilding with it cleanly
   • Sometimes, like with Sakura Wars GB2, the text itself already supports English font so I just reuse it
4. Extract the text into some common format (usually CSV) and rebuild with it cleanly (text extraction + reinsertion)

Generally after text extraction/insertion, the next big task to overcome is... how do we actually fit English text in the space originally meant for Japanese text? ...The answer is usually "it depends" from game to game and I won't bore you with the details (for now).

Instead, with the above context, I'd rather dive in a bit into Japanese text itself and how exactly it compares to the translated English. Japanese text in old games tends to be fairly space efficient and storage efficient. Since it's New Years' day and I'm tired from all the festivities, I'll just briefly talk space efficiency here.

Space Efficiency in Dragon Warrior 3

Randomly, on Cohost this year, a user @Bek0ha (bsky) went and found and recreated a cool font that caught my eye:

Bekoha went so far as to even provide me an 8x8 version of it:

Anyway, I figured I'd see how it would look in Dragon Warrior 3, which I had disassembled and gotten text reinsertion working for a while back for a re-localization project.

_{From left to right: The original Japanese, the original English, Kinema 8-bit, and Kinema 8-bit narrow}

Let's look at these lines (note that things surrounded by brackets are a single tile):

_{(Did you notice how the Japanese actually has enough leeway to indent lines around the asterisk? The English definitely didn't...)}

I suspect if you have some common variable-width font for English enabled, the English text actually looks like it takes significantly less space than the fixed-width Japanese:

ねえ!　オルテガさんの　子供が 生まれたんですって!?

Is it true that Ortega had a baby?

So let's see some stats as measured on the Gameboy:

_{(The GIFs are direct 2x scale, so I just split the frames and used those to measure the pixels :D)}

Conclusions

It's not quite as apples-to-apples, but English taking 200 characters to convey what Japanese does in 148 is indicative of a larger trend we see that causes a lot of trouble when it comes to the actual storage of English text when memory is limited... This is a storage problem though, and not one of actual visual space. If we're willing to spend the computation to render English text dynamically, then we can fit as much, if not more information within the same visual space by using a variable-width font!

Now if we could just figure out a way to do it in a way that didn't involve needing to draw text dynamically while maintaining the same text storage efficiency... I suppose that's a deep dive for another time, or research for someone smarter than me to get into. For now, at least, I'll focus on shoving VWF into as many projects as I can.

Some final personal notes

One of my New Years' resolutions was to write a bit more and to get generally better at articulating my thoughts. It has a few other nice benefits too...

• I finally get to live out the HTML/CSS dream I missed because I scoffed at MySpace and personal blogs in my teenage years
• It's quite therapeutic to put things to paper and/or keyboard
• Writing things down in this format really lets me dig into topics I wouldn't really spend as much time on, especially with things like gathering stats!

Also, since I'm starting to forget more things recently, I figured it might be a good time to start really investing in archiving whatever knowledge I've accumulated... if for no one else's benefit but my future own.

So that being said, thanks for reading!

The original verison of this post from yesterday actually relied on some ancient binaries from old versions of XCode, but thanks to the dedicated efforts of several people in the last decade, we have a fairly recent ld we can build on anything with Clang 10 or higher and some useful information.

For reference, you can see the original markdown for that post here.

It's pretty cool how many people still seem to want to keep these old things alive.

Anyway, here's what I've accumulated to build the rgbds toolchain for powerpc-apple-darwin8.

My updated Dockerfile for different versions of GCC and SDKs can be found here.

References

• Bluzt3's Blog
• OSDev Wiki
• OSXCross/cctools-port

Pre-requisites

• clang > 10
• CMake > 3.4.3
• Make > 4
• Whatever GCC might need (I needed to do brew install texinfo)
• llvm-devel if you want LTO support (recommend it)

The following instructions all operate based on the environment variables below and from a root directory.

Here's the set of installed binaries on Ubuntu 24.04

apt-get update -q && 
    apt-get install -y 
    cmake 
    ninja-build 
    build-essential 
    clang 
    llvm-dev 
    git 
    python3 
    wget 
    flex 
    texinfo 
    file 
    autoconf 
    libssl-dev 
    libz-dev 
    libtool-bin 
    && apt-get autoremove -y --purge 
    && apt-get clean -y 
    && rm -rf /var/lib/apt/lists/*

Environment

export ROOT_DIR="$(pwd)"
export TARGET_PREFIX="$(pwd)/install/target"
export TARGET=powerpc-apple-darwin8
export MAC_SDK_VERSION=MacOSX10.4u
export MACOSX_PPC_DEPLOYMENT_TARGET=10.4
mkdir -p "$TARGET_PREFIX"

NOTE: We are using the 10.4 SDK but will need to rebuild crt1.o. This is due to the issue with 10.4's crt1.o. 10.5 should theoretically work too.

cctools

Reference this long discussion here where a lot of people chime in on how to get this all working. Or don't, and just note the final results below, that's cool too.

dsymutil

Not included as part of the rest of cctools, so we'll quickly build llvm 7.1.1, which is the last version with powerpc-apple-darwin8 target support with dsymutil.

This is easily the longest part of the whole process because apparently we need to build a ton of LLVM before we can build dsymutil...

mkdir -p $ROOT_DIR/llvm-project && 
    cd $ROOT_DIR/llvm-project && 
    git init && 
    git remote add origin https://github.com/llvm/llvm-project.git && 
    git fetch --depth 1 origin 4856a9330ee01d30e9e11b6c2f991662b4c04b07 && 
    git checkout FETCH_HEAD

mkdir -p $ROOT_DIR/build/target/llvm-project && 
cd $ROOT_DIR/build/target/llvm-project && 
CC=clang CXX=clang++ cmake -G Ninja $ROOT_DIR/llvm-project/llvm 
  -DCMAKE_BUILD_TYPE=Release 
  -DLLVM_TARGETS_TO_BUILD="PowerPC" 
  -DLLVM_ENABLE_ASSERTIONS=OFF && 
ninja dsymutil && 
mkdir -p $TARGET_PREFIX/bin && 
cp bin/dsymutil $TARGET_PREFIX/bin/$TARGET-dsymutil && 
cd $ROOT_DIR && 
rm -rf $ROOT_DIR/build/target/llvm-project && 
rm -rf $ROOT_DIR/llvm-project

ld, as, ar, lipo, nm, ranlib

mkdir -p $ROOT_DIR/cctools-port && 
cd $ROOT_DIR/cctools-port && 
git init && 
git remote add origin https://github.com/tpoechtrager/cctools-port.git && 
git fetch --depth 1 origin 6694f27d56923e64e6190c8d3eb149413768e9b7 && 
git checkout FETCH_HEAD && 
cd cctools && 
./autogen.sh

mkdir -p $ROOT_DIR/build/target/cctools-port && 
cd $ROOT_DIR/build/target/cctools-port && 
$ROOT_DIR/cctools-port/cctools/configure CC=clang CXX=clang++ --prefix="$TARGET_PREFIX" --target=$TARGET && 
make -j2 && 
make install -j && 
mkdir -p $TARGET_PREFIX/$TARGET/bin && 
for i in ld ar as lipo nm ranlib strip dsymutil; do 
  ln -s "$TARGET_PREFIX/bin/$TARGET-$i" "$TARGET_PREFIX/$TARGET/bin/$i"; 
done && 
cd $ROOT_DIR && 
rm -rf $ROOT_DIR/build/target/cctools-port && 
rm -rf $ROOT_DIR/cctools-port

(Note we also symlink for convenience here)

MacOSX SDK

wget https://github.com/phracker/MacOSX-SDKs/releases/download/11.3/$MAC_SDK_VERSION.sdk.tar.xz
tar xf ./$MAC_SDK_VERSION.sdk.tar.xz -C $TARGET_PREFIX

We explicitly copy it into the prefix so it should benefit from the --with-sysroot option, at least according to the GCC docs: If the specified directory is a subdirectory of ${exec_prefix}, then it will be found relative to the GCC binaries if the installation tree is moved.

We then also need to extract the ppc binaries out of the universal binaries, allowing us to actually link against them without worrying about universal binary support. Note that we don't want to hit the ppc7400 binaries, since those are for 10.5 and the goal here is just for a Tiger focused SDK.

cd $TARGET_PREFIX/$MAC_SDK_VERSION.sdk/usr/lib
for i in **/*.dylib **/*.a **/*.o; do
  if file "$i" | grep -q "for architecture ppc)"; then
    mv $i $i.universal
    $TARGET_PREFIX/bin/$TARGET-lipo -thin ppc $i.universal -output $i
  elif file "$i" | grep -q "for architecture ppc7400)"; then
    mv $i $i.universal
    $TARGET_PREFIX/bin/$TARGET-lipo -thin ppc7400 $i.universal -output $i
  fi
done
cd -

Building gcc

Some minor modifications to libgcc's dynamic library generation. Since it calls lipo to generate universal binaries, which our cctools-built linker doesn't support, I've created a simple patch to skip calling it when there's only one architecture.

mkdir -p gcc
cd gcc
git init
git remote add origin git@github.com:VariantXYZ/gcc-13-branch.git
git fetch --depth 1 origin 908dbc96f1271f995759c87fc9d32879d6f49756
git checkout FETCH_HEAD
./contrib/download_prerequisites
cd -

Just the compiler

Note we make some hard-coded configuration settings to avoid gcc making assumptions about as and ld features we don't have. This is also in that GitHub issue discussion linked above.

mkdir -p $ROOT_DIR/build/target/gcc && 
cd $ROOT_DIR/build/target/gcc && 
echo "" > config.site-$TARGET && 
echo "gcc_cv_as_mmacosx_version_min=no" >> config.site-$TARGET && 
echo "gcc_cv_as_darwin_build_version=no" >> config.site-$TARGET && 
echo "gcc_cv_ld64_demangle=0" >> config.site-$TARGET && 
echo "gcc_cv_ld64_platform_version=0" >> config.site-$TARGET && 
CONFIG_SITE=$ROOT_DIR/build/target/gcc/config.site-$TARGET $ROOT_DIR/gcc/configure CC=clang CXX=clang++ --disable-nls --disable-multilib --enable-languages=c,c++,lto --with-dwarf2 --prefix="$TARGET_PREFIX" --target=$TARGET --with-sysroot="$TARGET_PREFIX/$MAC_SDK_VERSION.sdk" CXXFLAGS_FOR_TARGET="-O2 -g -mmacosx-version-min=$MACOSX_PPC_DEPLOYMENT_TARGET" CFLAGS_FOR_TARGET="-O2 -g -mmacosx-version-min=$MACOSX_PPC_DEPLOYMENT_TARGET" LDFLAGS_FOR_TARGET="-O2 -g -mmacosx-version-min=$MACOSX_PPC_DEPLOYMENT_TARGET" --disable-bootstrap && 
make all-gcc -j2 && 
make install-gcc -j

Before we get back to gcc, we need to go rebuild crt1.o as mentioned above.

Rebuilding Apple crt1.o

This one is a bit hacky compared to the rest, but we only need to rebuild one part of Apple's runtime libraries (crt1.o). This is only necessary if we're using the 10.4u SDK, at any rate.

mkdir -p csu
cd csu
git init
git remote add origin git@github.com:VariantXYZ/Csu.git
git fetch --depth 1 origin 06967ab403a7a04e30e2f32479285d68152cedd2
git checkout FETCH_HEAD
cd -

(For the record, I just renamed files to let GCC handle assembly preprocessing, otherwise the cctools assembler has trouble)

cd csu
make CC=$TARGET_PREFIX/bin/powerpc-apple-darwin8-gcc ARCH_CFLAGS="-arch ppc -D__private_extern__= -isysroot $TARGET_PREFIX/$MAC_SDK_VERSION.sdk" ./crt1.v1.o
mv ./crt1.v1.o $TARGET_PREFIX/$MAC_SDK_VERSION.sdk/usr/lib/crt1.o
cd -

The rest of the GCC runtime/libraries

cd build/target/gcc
make -j && 
make install -j
cd -

...and that's it, that's the toolchain. Well, the target toolchain anyway, we still need the GBDev one :^).

Building rgbds for powerpc-apple-darwin8

CFLAGS and CXXFLAGS must include -static-libstdc++ and -static-libgcc since we won't have dynamic libraries on the target, but other than that, it should be feasible to just reference CC and CXX to the powerpc toolchains. Note we're still operating in the same directory.

libpng

We need libpng for rgbgfx, so we'll build it first and install it into our prefix.

mkdir -p libpng
cd libpng
git init
git remote add origin git@github.com:pnggroup/libpng.git
git fetch --depth 1 origin f753baae52e1f0fd5451c25de8f8361ec5aea95f
git checkout FETCH_HEAD
cd -

mkdir -p build/target/libpng
cd build/target/libpng
PATH=$TARGET_PREFIX/bin:$PATH LDFLAGS="-static-libstdc++ -static-libgcc" CXX=$TARGET_PREFIX/bin/$TARGET-g++ CC=$TARGET_PREFIX/bin/$TARGET-gcc $ROOT_DIR/libpng/configure --prefix=$TARGET_PREFIX/$TARGET --host=$TARGET
PATH=$TARGET_PREFIX/bin:$PATH make -j
PATH=$TARGET_PREFIX/bin:$PATH make install -j
cd -

rgbds

Working on tag v0.9.0 (a good way to test that C++20 works). PNG should be in our search path automatically, but we need to make sure it doesn't use the host-installed libpng.

mkdir -p rgbds
cd rgbds
git init
git remote add origin git@github.com:gbdev/rgbds.git
git fetch --depth 1 origin d63955eccd7aa69794960a626144ea247d638957
git checkout FETCH_HEAD

mkdir -p $TARGET_PREFIX/$TARGET/deploy
PATH=$TARGET_PREFIX/$TARGET/bin:$PATH make install -j LDFLAGS="-static-libstdc++ -static-libgcc" PNGCFLAGS="" PNGLDFLAGS="-lpng" PNGLDLIBS="" CXX=$TARGET_PREFIX/bin/$TARGET-g++ CC=$TARGET_PREFIX/bin/$TARGET-gcc CXXFLAGS="-D__STDC_FORMAT_MACROS" PREFIX=$TARGET_PREFIX/deploy

Note: * We need to add -D__STDC_FORMAT_MACROS when using g++ in our case since inttypes.h will not enable certain macros otherwise (C99 locks it behind a macro, C++ doesn't seem to care) * We specify the path to make sure strip is called such that the install utility can use it, instead of the system strip

Anyway, after all this, you can verify it with file rgbasm:

% file rgbasm
rgbasm: Mach-O executable ppc

Horray! We actually have to more thoroughly test it... but maybe that's a problem for another day.

Making 2025 the year of productive PowerPC usage

If you recall my earlier post, I had gotten GCC 14.2 building things for Mac OS X 10.4 PowerPC (darwin8). Of course, after going through all that trouble, I did want to actually use the thing.

I figured an old PPC G4 could still find life as a GameBoy development machine, so I got to work trying to build everything necessary to checkout a repository, build a GB ROM, and actually debug and test it.

This meant building:

• rgbds
• Python 3
• git with openssl support
• An emulator

Simple enough, right? Well, sorta. Turns out, all software is just built on top of other software so there's quite a bit of work to do...

Here's a glimpse at what it takes to build Git, for example:

_{(I wish GitHub would let me properly re-use build flows in a sane way)}

I won't bore you guys with all the details, but there was a lot of random little changes in various projects and a handful of forks and research.

All of that work can be found in the GitHub Actions here and the forks I reference in them.

Everything culminated in being able to build and run the Medarot 3 patch at a glacial framerate:

Well, patched up enough of SDL 2.0.3 and added some stuff from 2.0.4 to build SameBoy.

If you listen closely, you can hear Ikki's voice played at 5 Hz.

[image or embed]

— Variant (@vxyz.me) February 16, 2025 at 9:49 PM

...Alright, so we still have some work to do here, but it's a good(?) transition into what's been eating up my time more recently.

Wonder what it'd take to write an emulator?

SameBoy is well known as being a highly accurate GB/GBC emulator that's fairly feature rich, but... it's not really designed nor optimized for a single-thread machine from 2003 that only supports OpenGL 1.1. The video above showcases a pretty common issue I've had.

Other emulators exist, but they either opt to drop accuracy for speed (so, not great for development and they usually don't have debugging features), or are even worse performing for a richer feature set. I even had Calindro, the Emulicious dev, reach out and had me test some builds but to no avail. Nothing really achieves a practical debugging experience on a Mac OS X 10.4 PPC that I could find...

So, of course, here I am with a pretty unique problem statement and set of resourcees, and I figure... if I've got a C++ 20 compiler and an understanding of the exact workflow I'm trying to build a tool for, maybe I can start writing an emulator with the features I need myself? It would also finally give me an opportunity to play around with C++20 and I could probably spin that as some form of professional development... at least, that's what I told myself.

The project, aptly named PowerGB, is still a work-in-progress. I've been finally getting to the stage of adding instruction parsing after working on it on-and-off after work. If you're wondering why it's taking me so long, it's because I keep finding new things in C++20 that make me interested and I waste days doing things like writing a comprehensive result wrapper instead of doing actual work.

Though, to be honest, I'll probably put this on hold because there are other things I'd like to get done...

Medabots GBC titles before 2030!?

Yeah, the Medabots GBC titles are still a slow work-in-progress... but we're still making progress.

Medabots 3 "US Localized" Version

andwhyisit handled some of the graphics work for the Medabots 3 US-English localization patch and I finally got around to writing that text replacement script, so the first (unfinished) nightly release of tr_EN-US is here.

_{Poor Shrimplips is almost entirely covered by the text :(}

The other GBC titles

The development aside, the 4th title is about 40% more text than the 3rd, so I have a feeling I might not be the major bottleneck... eventually anyway.

I think the 5th title should be a bit smaller, at least.

At the rate we're going it might be cutting it close to get it all done by 2030, but uh, I'm sure we'll be fine if I don't get distracted by other projects.

Work and other things

My job tend to be quite technically challenging so my brain wants to shut-off at times. I suspect things might get easier in the coming months, but phew, it's a real motivator to try and save up as much as I can to retire early so I can devote more time into projects and other things... alas, if only it were that simple.

Games! Actually playing them!

In the last several years, investing any amount of time into a title has been pretty difficult. I beat Radiant Historia over the course of 2 years, Dragon Quest 11 took me a year, and even now I'm finally approaching the end of the DQ3 remake at about 40 hours. I'm thankful that I'm finding some time to play, but phew, 40 hours really feels like the limit of a reasonable sink... maybe I'm just getting old.

That being said, DQ3's Recall mechanic being streamlined is such a nice feature. I wish every JRPG had a button I could press to let me remember NPC dialog at any moment. It's such a nice touch, and probably my favorite feature of DQ3 in general.

Finishing up

Anywho, I guess we'll all see how things shape up in the coming year. At a later date, maybe I'll do some more technical deep dives... but for that, I might need to dive in a bit deeper myself first.

I've been feeling pretty mentally exhausted from work and life things, and so the only projects I've really put time into have been ones that are more creative than investigative. That is to say, I haven't felt like reverse engineering things for a little bit but I'm sure I'll get back into it soon.

On the creative side though, my little pet GB emulator project PowerGB has been an exercise in seeing how far I can get pure portable C++20 to go.

A GameBoy Emulator should have a Philosophy

I imagine for a lot of people, a GameBoy emulator is usually a really fun sounding project. It's a well-documented system with a large support community, a ton of people who've gone through the same struggles you'll run into, and a lot of open-source working examples to reference. It's an absolutely phenomenal platform for development experiments and learning.

You'll find implementations that are highly accurate down to the CPU tick, ones that provide Python interfaces for automation, and even things that do JIT and static-based recompilation.

So given what's already out there, I figured I wanted to approach this with my own philosophy as I study the newer features of C++, originally propagated by my earlier post where I documented my adventures getting a C++20-compatible cross-compiler working on OSX Tiger.

As close as we can to a consteval world

As I went through thinking and defining various parts of the process, which I'll go into more detail below, I attempted to push as much of the work as possible to compile-time evaluation and correctness, even if it meant fighting the language a little. This meant extensive use of static_assert to make sure things were being evaluated at compile-time.

The easier we can prove to a compiler that something can be optimized out, the better.

Life's no fun without a few complications

I think C++ introduces opportunities for overengineering for meager advantages a lot, and really emphasizes the fun in the journey rather than the destination. I've really come to embrace this in personal projects because no one'll fire me for taking too long thinking about minor things. I wonder how people get away with reimplementing boost in production code though...

Debugging sucks, so let's make sure error scenarios are clear

I spent a lot of time putting together a pretty flexible results system within a single header, a little like Rust's own Result<...> type.

It works like this: * A Result is defined via a string literal description and whether or not it should be considered a success * A ResultSet is a wrapper around several results and a particular object (a return value)

It's possible to cast a ResultSet of one function into another as long as the target result set can handle all the possible results of the source set.

So every single result is expected to be explicitly handled, or caught at compile-time when it's not. The results themselves have string literals that can be propagated up to callers.

So you can go from one result type to another while preserving the underlying Result value, as long as the states can be determined at compile-time:

{
    using ResultSetTestInt   = ResultSet<int, ResultSuccess>;
    using ResultSetTestFloat = ResultSet<float, ResultSuccess, ResultFailure>;

    int  a                   = 255;
    auto resultInt           = ResultSetTestInt::DefaultResultSuccess(a);
    TEST_CHECK(static_cast<int>(resultInt) == 0xFF);

    auto resultFloat = static_cast<ResultSetTestFloat>(resultInt);
    TEST_CHECK(static_cast<float>(resultFloat) == 255.0f);
}

The caller will (must) always know what it's calling can actually return. No offline integer-to-string lookups or dynamic exception handling.

Basic Building Blocks

So in another fun over-complication, I decided to think about how to represent the underlying register types.

The GameBoy has a register file that contains a handful of registers that can also be referenced in pairs, including: * An 8-bit accumulator register A and "4-bit" (8 with the lower 4 bits always 0) flag register F * Several registers accessible as 8-bit pairs or a single 16-bit register (BC or B and C, DE or D and E, HL or H and L) * A program counter and stack pointer, each accessible as 16-bit registers (PC and SP, respectively)

Each register has different access expectations and would benefit from slightly different access patterns, with the flexibility to be used in multiple configurations. While it's pretty trivial to just create some uint16_t values and some neat little bitmasking functions, that's not what I'm about. Enter the Block class: a representation of data that's meant to emphasize specific access patterns while allowing for others.

For example, given a template Block<Size, AccessGranularity>, we can have each register above defined like so:

//// Accumulator
Block<8, 8> _A; // 8-bit register that's expected to be accessed as 8-bit

//// Flag
Block<8, 4> _F; // 8-bit register expected to be accessed by Nibbles

//// General purpose
Block<16, 8> _BC; // 16-bit register expected to be accessed in 8-bit chunks
Block<16, 8> _DE;
Block<16, 8> _HL;

// Program counter
Block<16, 16> _PC; // 16-bit register expected to be accessed as 16-bit

// Stack pointer
Block<16, 16> _SP;

Allowing for access to each of these registers with varying access patterns (e.g., a 4-bit Nibble, an 8-bit Byte, or a 16-bit Word) but prioritizing the provided access granularity. With some good compile-time definitions in release builds, I suspect we can make this into the zero-cost abstraction meme.

A Registry in Many Parts

The last bit for this particular update is my dream of making a compile-time registry of instruction handlers, in several files for organization.

To explain what that means, first let's get some points about instructions: * Every instruction can just be broken down into a set of operands and operations on the memory and CPU state (for simplicity, I just map things through a single memory map for everything) * Instruction operations can generally be categorized (loads, arithmetic, logic, etc...) * Each instruction can map to an opcode (1 byte or 2 bytes) and some variable number of operands (0 to 2 bytes)

So we have instructions like NOP, LD [HL], A (load the value from A into the memory address represented by HL), and INC BC (increment BC).

Referencing gekkio's technical reference, it's possible to get the general timing and behavior of all the instructions as well. We even have some tests based on the Ares emulation core we can use to validate them!

The intuitive approach to this would be to have some functions for each instruction and maybe use templates to reduce boilerplate, and then finally just have a single large array to 'decode' opcodes.

Seems pretty straightforward... but also feels kinda boring.

Firstly, I don't want to keep track of a big array and I know with C++ I can define an array at compile-time pretty easily with something like:

static constexpr auto CreateCallbackMap() noexcept
{
    // An array of callbacks mapped to the bytes
    std::array<std::size_t (*)(MemoryMap&) noexcept, 0xFF + 1> callbacks{nullptr};

    // Fill the callback array somehow

    return callbacks;
}

Second, I want to organize each instruction by category in separate files so I can define category-specific operations neatly.

Finally, I want all of this to happen at compile-time so no shenanigans like a static constructor to handle registration.

Given the above conditions, the obvious problem is that "creating a registry at compile-time" means somehow maintaining state at compile-time... luckily, there is a solution! One that's technically within the bounds of the language and requires careful use to not violate the one-definition rule. Enter: friend-njection and Argument-Dependent Lookup (explained well in this StackOverflow post).

To not bore you with many details, the actual registry header is here.

Effectively, we have an InstructionDecoder class template used that ties an opcode with functions and metadata (e.g., CPU ticks) and also appends to a registry.

The registry itself maintains a callback map that simply gets filled by:

((callbacks[Decoders::Opcode] = &Decoders::Execute), ...);

Then each instruction category can be defined in separate files and as long as they end up in the same translation unit (e.g., with cpu.cpp), we'll have a compile-time registry defined with everything in a separate file!

Marching onwards

There were definitely a few things I skipped above, but those were the most interesting I think. Using known tests, I've been able to implement instructions without much trouble, albeit slowly and whenever I find time. It doesn't help that all of this work is happening on my old XPS 13 from 2016 with 8 GB of RAM. I might do another post later about my adventures in trying to speed up build times (those compile-time shenanigans definitely have a cost somewhere...).

In personal projects, especially ones related to reverse engineering, I've generally learned that there's a lot of value in taking the time to understand problems and build a solid foundation. Being able to take each step and validate it is a fantastic privilege that not many fields get to experience, so it feels a bit wasteful to not do it when I can.

My old man philosophy aside, this has been a really fun project so far and I'm looking forward to researching and implementing the clock and interrupt behavior next, and then graphics!