Porting SimBlob to the web

From Red Blob Games's Blog

I first saw Emscripten a few years ago. It compiled C++ to Javascript. My mind boggled! How cool! This year at Game Developers Conference there was a talk showing that Emscripten plus asm.js could run C++ games in the browser with good performance. I was amazed.

After publishing my article on hex grids, I decided I should do something different for a little while. I remembered Emscripten looked intriguing, and I should try it out. Could I make BlobCity, an OS/2-only game from 15 years ago, run in the browser?

TL;DR: Yes, try it out here!

Here's what I started with, a game that only runs in OS/2:

Porting to SDL

The first problem is that SimBlob (the simulation/game engine for BlobCity) is written for multithreaded OS/2, using the Presentation Manager graphical library. The game isn't cross-platform, and nobody uses OS/2 anymore. Even I couldn't run my own game, even as a reference for this port. To use Emscripten, I needed to make SimBlob work with either SDL or OpenGL. I chose SDL as a better match.

I'm developing on a Mac with Homebrew, so I used brew install sdl to get SDL. I went through a few examples from this tutorial, then used the SDL docs as a reference.

I took inventory of the SimBlob modules. Only one third of the code was independent of OS/2. Yuck. I was hoping that it'd be better than that. That means I have 10,000 lines of code that I need to port. Ugh. Do I really want to do this?

I decided it'd be ok to port just part of the game. After all, my main goal was to play with Emscripten, not to make BlobCity run. I first got the simulation code running, and output the map in ASCII. Then I made some simple rendering code in SDL. After a bit of fighting with SDL, I got things to work. The game simulation was running successfully, and showed up in the minimap!

I started wondering how much work it would be to make the main map display. I started digging into the code, cursing at my 15-year-ago self for making it so convoluted and undocumented. I commented out OS/2 specific code, or created extra typedefs and no-op functions to make it compile. After I had it compiling, over the next week I either ported OS/2-specific code to SDL, or I emulated parts of the OS/2 API in SDL. Some notes:

• OS/2 has a vector graphics object called "PS" in my code. It handles text, line drawing, rectangles, etc. If I were porting directly to Canvas or SVG or Flash, I would have equivalent vector graphics available. SDL has only a subset of this functionality in SDL_gfx and SDL_ttf, and Emscripten has only a small subset of that, so I decided not to port that code.
• OS/2 has multiple overlapping windows, each with an event handler. SDL has none of this. At first I decided to comment out the code, and hard-code the main map and minimap. As the week went on, and I wanted the toolbar, status bar, and tabbed information pane, I ported the display code to SDL and emulated multiple windows with their own event loops.
• The multiple layers in my code made porting easier in some ways. The Window hierarchy had a subclass that only dealt with bitmaps, so I could port that to SDL, and the Glyph hierarchy didn't deal with OS/2 at all. Most of the game graphics are rendered to an platform-independent bitmap.
• Without the vector graphics and text rendering, I couldn't generate the procedural graphics at run time. The code generated these and saved them to files, so I was able to reuse those files. I can't regenerate them but at least I can use them.
• The OS/2 blitting code used multiple approaches, WinDrawBitmap, GpiDrawBits, DIVE. I also used heuristics to choose blitting on the fly (each frame) based on the size of the dirty rectangles. This extra layer made it easier to plug in SDL as a rendering target.
• I had wrappers around OS/2 low-level data types: colors, sizes, points, rectangles, damage regions, mutexes, event semaphores. I reimplemented these to not use OS/2. Since I wasn't going to use threads this time, mutexes and related constructs became no-ops.
• I also found a bug that's been there for over 15 years. If more than one builder tries to do a job, the first one does it and the second one undoes it. In a typical game you have only one builder working at a time, so this bug escaped my detection until now.

Over the period of a week, I continued to either port OS/2 specific code or emulate OS/2 APIs, and I ended up getting almost all of the code running. The only thing I didn't tackle was the OS/2 menubar.

Getting a little bit drawing on the screen encouraged me to work on more.

Here's a screenshot of the SDL version – looks the same as the OS/2 version except for the menus!

Using Emscripten

Setting up emscripten: I installed llvm and clang through homebrew (brew install llvm --using-clang). Note that clang on the Mac isn't enough. It doesn't have the right version number, and it doesn't include the rest of LLVM. And using the Homebrew regular install (brew install llvm) isn't enough. I needed to install both through Homebrew.

I used the Emscripten FAQ and the #emscripten IRC channel on irc.mozilla.org as references. I also read through some of the Emscripten code when I needed to understand what was going on.

The first thing the Emscripten limitations page says is that it "CANNOT compile Code that is multithreaded and uses shared state. JS has threads - web workers - but they cannot share state, instead they pass messages." SimBlob is very much multithreaded. Yikes!

Other notes:

• OS/2 is multithreaded. This was one of the reasons I was using OS/2 instead of Windows 3. For SimBlob, I had multiple main loops running simultaneously, communicating with event semaphores, shared state, and lock-free data structures. To make this work with Emscripten, I had to switch to a single threaded model. I had to merge all the main loops together into a single SDL event loop. It wasn't as bad as I had feared. Although I could've used threads with SDL, Javascript is event-based, and Emscripten wouldn't work if I left the game multithreaded. Emscripten needs the standard SDL event loop changed to this.
• SimBlob uses an 8-bit palette. I got this ported to SDL, but I had some trouble making it work in Emscripten. I decided it'd be easier to switch to 32-bit color. Since the game graphics draw to my own bitmap structures, I used the palette there, and expanded the palette when copying it over to the SDL surface.
• There's an Emscripten open issue that recommends disabling "copy on lock" to make palettes work. I had tried this out, but didn't need this anymore once I switched to 32-bit color. But I saw no harm in keeping it disabled. As far as I can tell, if you disable it, it will not copy the browser canvas back to your internal SDL surface. I don't need that copy.
• The binary includes SDL, so it's relatively large. Use -O2 to shrink it. I also needed -O2 to make the simulation code run acceptably fast.
• SDL's event.type is Uint8 in the SDL docs, but Emscripten needs more than 8 bits. I ended up using an int instead, but keep this in mind when porting your code.
• I had considered implementing the OS/2 menus on the HTML side, and then using Javascript to send those events back to the C++ code. To do this, you need to export some of the C++ code using "C" linkage, and then listing those functions in the command line flags, -s LINKABLE=1 -s EXPORTED_FUNCTIONS "['main', '_invoke_command']" You can then import the function into the Javascript side with invoke_command = Module.cwrap('invoke_command', 'void', ['number']) although for reasons I don't yet understand I was able to call _invoke_command directly without the wrapper. I added buttons for each of the OS/2 commands, and had those buttons call into the C++ code.
• I also used these flags: --jcache -s ASM_JS=1 -s WARN_ON_UNDEFINED_SYMBOLS=1
• I wanted the game to start with the window size I used back in 1997, but a large window makes the game more fun to play. I added resize support to the SDL version, then added a "Zoom" button to resize in the browser. You can call Browser.setCanvasSize(width, height) to do this. I experimented with full screen but haven't gotten that working.
• It was awesome to see the game running in the browser, but even more awesome to see it running on my phone! Emscripten needs Float64Array which is supported in iOS 6 but not iOS 5. The game also runs on Android 4.1; I haven't tried it on older versions of Android. It runs very slowly but it runs.
  
• Once I saw it running on the phone, I decided to add touch event support. I trapped touchstart, touchmove, and touchend and redirected them to SDL mouse down/move/up. This makes it feel much nicer on iOS. It didn't seem to help as much on Android.
• I had to switch drawing byte order when using the putpixel() code from SDL's docs. I filed a bug, but the workaround is easy. It's one of the few places where I have #ifdef EMSCRIPTEN in my code.
• SDL_GetKeyState() wasn't supported; I worked around it by tracking down/up events myself.
• Shift, control, alt modifiers didn't show up in Emscripten. SDL event.key.keysym.mod always contained 0. I filed a bug, and inolen on irc gave me a branch with a fix.

The process of getting Emscripten to compile the game to HTML5 was surprisingly easy. The OS/2 to SDL port took most of my time; after that, emscriptening (is that a word?) took only a few tweaks.

Thoughts

Try it out here!

I'm still amazed Emscripten is possible. Javascript doesn't support pointer arithmetic, unsafe casts, unsafe unions, etc., and yet it all works! I dug into how it works and I was amazed even more. I'm also quite impressed by how fast asm.js is. In some microbenchmarks, I found that C++ code compiled to Javascript ran faster than equivalent Java code.

I've not been able to run BlobCity for 15 years. Although I could have ported SimBlob to SDL, it was Emscripten that motivated me to do it. This was a fun project. I love being able to run the game in the browser. I don't plan to do anything more with SimBlob/BlobCity but I will probably use Emscripten for a future game project.

Labels: emscripten , programming , project