I've been putting more work into the boot loader this week and I thought I should post a quick update regarding some changes. I'll put a summary at the end, so feel free to skip over all the technical stuff.
The original plan for self-patching games was to put the patch code/data in a HEX file on the SD card and use the existing game-loading code to flash it to the chip. After looking into it more closely I don't believe this is the best way to achieve this. Under normal circumstances a boot loader transfers control to the application and doesn't get control back until the next reset. In the case of Gamebuino the application can transfer control back to the boot loader to load a new game, but again it's a one-way process. Game patching involves a new scenario in which the game needs to transfer control to the boot loader but then the boot loader needs to return control back when it's done, and in the absence of an underlying OS we immediately run into the problem of memory management. Both applications have sizable memory requirements and each doesn't know how the other is managing RAM.
One way around this is for the calling application to pass the boot loader a "scratch pad" buffer to use during patching, but the size of this buffer needs to be 539 bytes exactly, or about a quarter of total available RAM. It also means that any future boot loaders cannot exceed this 539 requirement (or any other we set) for any reason whatsoever without breaking compatibility with existing games that use the game-patch feature.
To get around this I've implemented this feature as a function that simply allows the user to pass in individual 128-byte pages and the boot loader will flash it into memory for you. If you want to load code and/or data off SD card then that's fine, you just use the full SD library in your app to load and flash one page at a time. It also means the files can be in sub-folders, which the boot loader can't handle. If you only need to flash a portion of a page then that's fine too, just buffer it in RAM, make your changes and re-flash the whole thing. This ability to flash partial pages from RAM opens the door to one very nice feature indeed: self-modifying code!
It also comes with another unexpected benefit. A few days ago I posted about the possibility of storing user settings in Flash memory i.e. default user name, sound settings etc. With software flash functionality we get this feature automatically without the need for any additional boot loader code. We simply set aside a single 128-byte page in flash memory somewhere and the game library can use it however it wants.
Which leads me nicely to the final issue: if this feature were to be added then it would have to go at the end of the application space rather than inside the boot loader area. First of all we're not going to have much space in the boot loader. More seriously, Gamebuino would have to be shipped with the lock flags set to allow the boot loader to be overwritten. If the user then tries to load an application via USB that spills into boot loader memory then they will brick their Gamebuino, at least until they can get a hardware programmer on it to fix it.
Summary:
- Game patching can currently be done by calling a function in the boot loader to flash pages
- Self-modifying code is now possible
- A page for user settings is possible e.g. for allowing customization of the game loader screen, but it would have to reside at the top of app space. If the user uploads a large sketch they'll overwrite it, although this could be easily detected by the library code.
Current status: all of this is implemented and working well but the boot code is currently 32 assembly instructions too long. I still haven't done any size optimization though so I'm very confident of hitting the 2KB target over the next few days.
Feedback from everyone welcome as always. Rodot, I'm going to need feedback from you regarding the user settings page because if it's something you want to officially sanction then I'll need to put code in the game loader to prevent users overwriting that page...it won't stop them trashing it themselves when programming via USB of course but at least it's something. Either way, as soon as I get the green light from you I'll start wrapping things up ready for release.