Saturday, December 31, 2016

"Space, the final frontier. These are the voyages of the starship Enterprise. Its five-year mission: to explore strange new worlds, to seek out new life and new civilizations, to boldly go where no man has gone before." –Star Trek

Five years ago I started Red Blob Games to help game developers with interactive tutorials and also to make my own games. I had already been interested in interactive tutorials since 2004 and likely even earlier than that, but it wasn't something I had spent much time on. In August 2011 I made a explanation of "blind spots" when driving a car and discovered that by playing with the diagram, I learned things I hadn't ever learned when I had merely read about the topic. I decided to make many more interactive explanations of math and computer science topics, especially for game development. It was something I wanted to see more of in the world.

Around the same time, I discovered the work of Bret Victor. He wasn't writing interactive tutorials for game development, but he was thinking much bigger. Much, much, much bigger. His articles on “explorable explanations” and “ladder of abstraction” made me realize that there's a lot more to this than I realized.

The first interactive tutorial I published under Red Blob Games was at the beginning of 2012, about damage roll calculations and probability. Also early in 2012 I became pessimistic about making my own games so I decided to focus on helping others with my tutorials.

With the probability page, I followed a "textbook style" of starting with the basics and working my way up. I think that style didn't work so well. The top of the page was too bland; it didn't make people want to read more. For my second interactive tutorial, 2d visibility, I decided I needed more of a "journalist style" of starting with the juiciest thing at the top of the page and then the details later. I think this worked better.

My third interactive tutorial was for curved roads and tracks. This project made me rethink my strategy. In 2012 I had been exploring "small" topics I didn't know much about while learning visualization, javascript, d3, svg, etc. This project took five months, and it was a topic not many people cared about. The combination of I'm learning something new (takes a lot of time, as I'm a slow learner) and it's not very useful stung badly and I had to do something different.

In 2013 I was more focused. I had some topics to explore, and I was better about prioritizing and "shipping". I wanted something that was both a topic I knew about and a topic that would be useful. I started with my guide to hexagonal grids. Not only did I want to have the best reference to hexagonal grid math on the web, I wanted to make better visualizations and a cleaner underlying implementation than I had for my previous articles. I'm quite happy with the way it turned out.

Also in 2013 I used Emscripten to port my old OS/2 game to run in the browser, I spent a month learning signal processing, and finished by writing a page about signal processing concepts and procedural map generation.

In 2014, I wanted to follow the success of 2013 with another topic I knew about and that people would find useful: pathfinding. I started with a tutorial for Tower Defense pathfinding (flow fields), then wrote a longer page about Breadth First Search, Dijkstra's Algorithm, and A*.

One thing that had gone wrong in earlier tutorials was that I explained the theory but not how to implement the algorithms. I had stayed away from implementations because I didn't want to pick a programming language, only to have a different langauge become popular a few years later. I want my pages to last for decades. But at the same time, it was definitely a weakness of my tutorials. I decided to write an implementation guide for the A* page, with both Python and C++ code, as well as a little bit of C#. To make sure that the code I presented on the web page actually ran correctly, I used Emacs Org Mode to write code that would get run during the html export process, and both the source code and its output would be shown on the page. This solved a problem I had with previous pages in which the code on the page would differ slightly from the code I had tested.

My style evolved from "journalist style" to "action movie style". In a James Bond movie, our hero might be falling out of an airplane, fighting the henchman for a parachute. Then the movie switches to Bond talking to his boss about the geopolitical situation in some country. The first scene is exciting but gives no explanation; the second scene is quiet and explains things. The movie alternates between visual-excitement scenes and story-informative scenes until we have a final battle in a secret volcano lair, as exciting as the opening scene, but the viewer now understands the plot and motivations. I now aim to start my tutorials with something cool looking, but without much explanation of how it works. Then I give explanations. Then I give something cool looking again. Then more explanations. Maybe something doesn't work out. Maybe we have to try something different. At the end, there's some demo that's cool, but better because the reader understands the algorithm.

Towards the end of 2014 I went to a workshop that Bret Victor organized to bring people together to talk about interactive documents. It was at this workshop we ended up with the Explorable Explanations page maintained by Nicky Case.

The workshop was absolutely amazing. It inspired me to write up notes on how I write tutorials (draggable markers, A* page). Unfortunately it also paralyzed me when I tried to write new tutorials. I started overthinking everything for a while. When I get stuck on complicated things I find it useful to go back to something simpler. I wrote a quick page about straight lines on a grid.

In 2015, I wanted to try something more ambitious, so I tried procedurally generating downloadable code libraries. I wanted my readers to pick a programming language, formatting style, and hexagonal grid characteristics (offset vs axial, y-up vs y-down, size, skew, etc.), and then my procedural generator would create a downloadable library that implements the hex grid algorithms for their needs. Neat, eh? I wrote a series of blog posts about this (1 - 2 - 3 - 4 - 5 - 6 - 7). I wrote an implementation guide and also generated code for C++, Python, Javascript, C#, Haxe, Java, Typescript, and Lua. I also procedurally generated unit tests for each of those languages and ran all of them as part of the publish process.

I also got a bit distracted working on pathfinding optimizations. I wrote a partial page about differential heuristics (which I think is a fantastic optimization with large speedups from little code) and a partial page about waypoint graphs instead of grids (which I'm less excited about now). 0fps wrote a nice optimized grid pathfinding library; I tried it and it ran A* in a mere 1-3 milliseconds on an 800x800 grid map. Pretty cool. But I didn't understand the algorithm and didn't write a tutorial about it.

I realized at some point that my most successful pages were about problems that I had in a real project, and the least successful pages were about problems that I expected to have in a future project. Curved roads and tracks weren't something I needed for a real project, as I've always used grids in building games. Regular A* was fast enough for all the projects I had worked on, so optimizing it further had never been something I needed to do for a real project. Every time I ventured into a topic where it didn't matter for my own projects, that tutorial failed.

This hit me hard. Over the past three years, half that time was wasted writing things that didn't really work out.

I decided that I needed to work on more real projects. I dug through existing projects to see if there was anything I hadn't written much about, and I found one: simple procedural map generation in 50 lines of code. The rest of 2015 I spent updating my skill set (C++, OpenGL, physics, networking, shaders, distance fields, etc.) instead of tutorials, so that I could work on real projects.

In 2016, I updated some of my existing pages (1 - 2), but lost motivation to work on new tutorials. I tried several times to work on a tutorial about coordinate systems but failed each time. I continued learning things by working on small projects (React.js, SVG, Lua, WebAudio, Pixi.js, neural networks, hexagonal tiling of spheres, stylized map rendering) but I don't have much to show for it. I've worked with game developers on small projects but don't have anything I can share yet.

So that's the past five years.

  1. The first year I worked on small tutorials and learned how to use Javascript, text, and diagrams to build the kinds of explanations I wanted to build.
  2. The second year I made the guide to hexagonal grids and a few other things.
  3. The third year I made the A* pathfinding tutorial, started adding implementation guides to my pages, and came up with a style I liked for my tutorials.
  4. The fourth year I wrote some fun projects but nothing had the impact of my earlier work, and I started to figure out why.
  5. The fifth year I learned a lot but lost motivation to write big tutorials.

I'm very happy with what I accomplished during this five year mission. The hexagonal grid guide and the pathfinding tutorial are the best things I've written. I loved exploring the capabilities of the medium. Web articles can be interactive explorations of topics and don't have to be static like magazine and newspapers articles. When I started, very few people wrote interactive essays, and now I see many people writing them. I think we'll see a lot more of them.

I love writing tutorials and I love working with game developers, but 2016 convinced me that I need to change my path. I don't know what's next for me. Will I extend my mission? Will I write my own game? Will I move on to something else? I don't have a plan for 2017.

20 comments:

Myzreal wrote at January 01, 2017 9:16 AM

I love your tutorials, they're very helpful, I hope you'll continue. Maybe something on AI and state machines? Or entity systems, such as Ashley? Just throwing some ideas around

Amit wrote at January 01, 2017 12:01 PM

Thanks Myzreal! Those topics are all interesting, but one of the lessons I learned from this journey was that the tutorials work best when I've needed that technology for a real project, and so far I've not done much with those topics. I'm thinking that if I work on a real project or two, I will have more topics I can turn into tutorials…

Moofy wrote at January 01, 2017 12:40 PM

Forgive me if this sounds strange, but how did you afford to eat whilst spending five years doing all this?

Rasmus Praestholm wrote at January 01, 2017 1:51 PM

Have followed your work for years and it has been a great journey :-)

Not to seem overly selfish or anything, but have you considered contributing to existing open source projects? Like maybe one that already built a thing or two based on your tutorials? Would be something to build on further ... :D

(Terasology's PolyWorld implementation, for instance)

No matter what I hope you will continue to chronicle. Have a great 2017! :-)

Scott Turner wrote at January 01, 2017 4:49 PM

My plan is to spend the entire 2017 fussing around trying to generate good mountain symbols for fantasy maps :-). (More seriously, it's certainly an entirely different skill to write about something that you aren't intimately using and excited about.)

Dan Butler wrote at January 01, 2017 6:34 PM

What about tutorials on things that high school and college students absolutely need to learn for their classes? Trig, calculus, algebra, geometry. Maybe some basic physics and electromagnetism. There are plenty of video lectures on these topics, but not as many interactive explanations.

Maxwell Joslyn wrote at January 01, 2017 6:45 PM

Amit, your tutorials on hex grids and pathfinding have been invaluable as I work on computer programs for D&D DMing. Thank you for running Red Blob and good on you for not giving up hope after a year of losing motivation. I have had such years before and I know they can be dreadful for one's mental state. I wish you a fruitful 2017.

Anonymous wrote at January 01, 2017 7:03 PM

@Moofy:

Bio on his homepage says he was releasing popular BBS software in 1990 and finished his PhD from Stanford in 2001. He has served his time :)

gaurav gautam wrote at January 01, 2017 7:42 PM

Hi,
I really like your page. Especially the one in which there was a demonstration of pathfinding using graph based algorithms. And do you know who showed me that page? Peter Norvig. The Peter Norvig. I think he really likes your work too. I have been meaning to start a similar effort myself since past year. I even made a page with a simple ai agent that I wrote in javascript. But the code got really bulky and it wasn't as beautiful as your website. I have learnt a lot since then. And I am planning to start again using typescript this time so the code is scalable and everything. Anyways, happy new year and good luck for the next five years. :D

A.F. wrote at January 01, 2017 7:45 PM

I'll join the chorus here: Amit, your tutorials have been invaluable. They made the internet useful and interesting to me and I'm sure to many others. I think I've been reading your tutorials since forever, way more than 5 years, probably 20. Is that possible?

Whatever you choose to do next will be welcome!

Nathan Pointer wrote at January 01, 2017 8:35 PM

Also just wanted to say how greatful I am for all the work you've put into your tutorials - they are easily some of the best on the internet. They always proove insightful and worth working through. Whatever path you choose I wish you the best of luck & happy a new year!

DDR wrote at January 02, 2017 5:22 AM

Thank you for your generous work. I'm always happy to see the Red Blob Games header at the top of a page, it has been a useful source of understandable, interesting tutorials over the years. Many, indeed, were not problems I've had myself though.

Shane wrote at January 02, 2017 6:02 AM

Just piling on to say how much I appreciate the effort you put in; and more than just appreciating it, have seen how much love and artistry and attention to detail has gone into it all. Not to be too dramatic, but sites like yours give me hope for the world -- the fact that one person's efforts can produce so much multiplicative good in the world lets me believe that maybe I could do some good in the world too. Thank you.

Christophe wrote at January 02, 2017 10:17 AM

If you're interested in interactive tutorials, this one on neural networks is a great one : http://neuralnetworksanddeeplearning.com/chap4.html

Amit wrote at January 02, 2017 10:36 AM

@Moofy: my income has come from non-game development work. I'm not spending all my time on Red Blob Games. You may also find https://www.whywork.org/about/features/stories/semiretirement.html and http://sachachua.com/blog/experiment/ to be interesting.

Amit wrote at January 02, 2017 10:58 AM

@Rasmus: I've considered it. There are lots of options available :)

@Scott: thank you for making your blog; I've been enjoying every post!

@Dan: I agree those would be valuable. You may enjoy Steven Witten's http://acko.net/blog/how-to-fold-a-julia-fractal/ . There's also http://immersivemath.com/ila/ , http://betterexplained.com/articles/intuitive-trigonometry/ (this site also has some pages about calculus and other math topics), http://world.mathigon.org/

@gaurav: thanks! Peter and I have talked about an interactive version of his AI textbook but it's a huge amount of work. Feel free to email me at redblobgames@gmail.com ; I have some code you can look at and use. For most of my tutorials I've had better luck making it *not* scalable and reusable.

@A.F. thanks! Yes, I started my first site in 1994 so that's over 20 years :) However it's my school account which only gave me 100MB so I needed to put my new stuff on a different site (redblobgames.com).

@Shane: thank you — there's never been a better time for sharing and helping others and I hope to see much more of it!

@Christophe: I agree! (He mentions Bret Victor and me on that page too)

@Maxwell: thanks!

@Nathan: thanks!

@DDR: thanks! I like to have a mix of things that are useful right away and ideas that shape the way you think about problems you see in the future :)

xaminmo wrote at January 02, 2017 9:31 PM

I know for me, part of the goodness of your blog is sentimental value. I mean, I still have an SRE license key in my filing cabinet. I like seeing the things you think about. I feel your logic processes to be very familiar, even though I drifted out of programming decades ago (too slow, no projects).

I use LiveJournal as an RSS to email gateway to track your game and personal blog.

If you keep writing things about coding, I will be happy to keep reading.

If you write about other things, I'll read that.

If you give up on babbling on the internet, I'll be a little disappointed. (However, it's not all about ME.)

Anonymous wrote at January 03, 2017 12:24 PM

To add to the other comments here. I too am very grateful for the work you have put into this site over the years. I have been dabbling with my own game, and your information on hex grids and path-finding have been extremely valuable.

I hope you continue to experiment and share the results and your observations with us. I find myself reading back over your posts fairly often to reacquaint myself with the material. What a great resource you have created!

Don't feel every post has to hit the ball out of the park with an amazing demo, even thoughts on what you are working on, resources you are referencing, etc. would be great to keep up with.

Amit wrote at January 03, 2017 7:45 PM

@xaminmo: wow, that's more of SRE than I still have :-) I plan to keep writing but for now it will be smaller, less polished things like the hexagons-on-spheres article and the stylized map rendering article. Those took a week or two. It's the bigger many-month projects that I've had trouble with so I'm putting those on hold until I figure out a way to make that work.

@Anonymous: thanks! Yes, I think part of the problem is that as I got better at making demos, I felt like I *had* to make it polished from the start, and that made it harder and harder to make new things. So I'm going to try writing more quick things and put the polished longer pages on hold for now.

Bobomite wrote at January 07, 2017 3:23 PM

I have come back to your articles and tutorials many, many times over the years. I wanted to say thanks very much for what you have created and shared.

If it counts for anything, I'd say start building your own games. And of course keep your training and reference articles up.

I reckon you'll have more new tutorials to make in time.

Anyway time to get back to reading the ones you already made! I got distracted from my research when I read your blog post. :)