Violin, Theoretical Physics, and Indie Games

For this blog post, I wanted to interview Nathan Reed. He's currently a VR rendering engineer at NVIDIA, and someone who inspires me greatly. 

What do you do for work?

I primarily do real-time graphics programming. Most recently, I’ve been working at NVIDIA on virtual reality rendering technology—researching ways to render VR scenes as fast as possible on their GPUs, and helping to design APIs to give developers access to those features. VR could be a really interesting sea change for real-time graphics: a lot of the established wisdom we have in our industry about how to do things is being rethought in light of the new challenges that VR brings (such as needing low latency, high framerate, and stereo rendering).

Before I was at NVIDIA, I spent several years at Sucker Punch Productions, where I did graphics programming for the Infamous series of games for PS3 and PS4. Over the course of those projects, I got to touch practically every aspect of the games’ rendering—such as low-level optimization for the console hardware, gamma-correct HDR lighting, dynamic wet materials, and postprocessing effects, to name but a few.

How did you get into orchestra?

I was fortunate enough to go to public schools with music programs—something it seems is getting ever rarer in the USA. :( When I was in fifth grade, our class tried out three different instruments: clarinet, trumpet, and violin. I couldn’t figure out how to blow into the clarinet correctly to sound even a single note, despite weeks of trying. On the trumpet, I managed one note, once. The violin was the only instrument that didn’t make me feel like a complete idiot. You can actually see what you’re doing; you just put your finger down and draw the bow on the string, and sound comes out! Easy!

Well, maybe not so easy—I’ve been playing for almost 20 years now, and I still feel like I can only kinda sorta do it. :)

But from that point on I was a violinist. My parents got me the instrument and lessons; I played in school orchestras and youth orchestras every year until I graduated college, then in all-volunteer community orchestras after that. Currently, I play with the Nova Vista Symphony. If you’re in the San Jose area, you should catch one of our concerts!

Playing music is very therapeutic for me. It engages a different part of the brain from the part that does engineering for my day job, so it’s a nice way to relax and unwind. It’s a great way to get into a meditative, flowy state of mind, especially when you can lose yourself in an ensemble. And it even gives you a bit of exercise—you can definitely work up a sweat playing an energetic piece, and you get that endorphin high after a rehearsal, a little like going for a run.

What got you interested in physics?

I’ve been studying theoretical physics as a hobby for many years. That’s a sentence that often seems to induce a little consternation in people I talk to! But I don’t think it’s really such a singular pursuit.

It’s an interest that’s deepened gradually over time. As a kid I was always interested in learning how things work, and was constantly asking questions that my parents were hard-pressed to answer. Someone gave me a copy of Stephen Hawking’s A Brief History of Time, which first exposed me to some of the more bizarre phenomena like relativity, quantum mechanics, black holes, and the Big Bang. It clicked with me, and over the next few years, I read a lot of books of that sort: physics books for the non-technical audience. Eventually I wanted to explore the subject more deeply and seriously, so I took some classes and started reading more technical books; I also watched a lot of Leonard Susskind’s The Theoretical Minimum lectures on Youtube, which are a great resource. Today I’m making very slow progress in learning quantum field theory. I’m also reading an interesting physics/philosophy crossover book, about many-worlds theory and decoherence.

Everyone tries to make sense somehow of how the world works and what our place in it is; sometimes people do that through religion and politics, sometimes through history and literature, and sometimes through science and math. For me, physics is sort of existentially reassuring, in a way, in that beneath all the bustle and strife of the everyday world there is this very elegant and mathematically beautiful hidden world. And it feels “cosmically worthwhile” to understand it, even if knowing stuff like how to calculate the wavefunction of an electron or simulate the expansion of the universe isn’t ever going to be directly useful in my career or life.

What makes you passionate about indie games?

Probably very similar reasons to why other people are interested in them! Games are a uniquely immersive medium (and VR even more so): you can literally enter a fictional world. The medium can do so many things, so it’s frustrating to see the AAA industry largely turning out the same games with slight variations year after year. A massive amount of work and craft goes into them, and those developers are just as hard-working and passionate about games as anyone…but once you’ve been around for a while, there’s a sameness to AAA games that’s dispiriting.

Indie games seem to be where people are doing more interesting things—exploring art and gameplay styles and topics that AAA games don’t, breaking away from old patterns and stretching the horizon of the medium. And while a team of a few indies can’t match the massive technology and production scale of a hundreds-strong AAA team, games like The Witness and Firewatch prove that being indie doesn’t need to mean compromising on production values, either. Plus, it gives you a level of ownership and control over your game that you don’t get to experience as a proverbial cog in the AAA machine.

How do each of these topics influence each other?

I think my physics studies have definitely informed my programming in some ways. When you get down to it, a large amount of computer graphics is about simulating the interaction of light with different materials, and that’s a branch of physics! It’s not so much that any specific results from physics help me do graphics—I can’t use electron wavefunctions to solve for BRDFs or anything like that (if I could, I’d be off writing SIGGRAPH papers about it). It’s more of a general physical intuition, plus a few skills like dimensional analysis and making simple models (or good approximations of complicated models), which are broadly useful.

Music and programming naturally go together too. I haven’t done too much music/audio programming so far, but it’s something I’d like to experiment with more (in my copious free time, of course). One cool thing I did for a college class was build an electronic music box. It had a microcontroller hooked up to some audio synthesis components, and I programmed it to play songs extracted from MIDI files. That was a pretty fun and interesting project.

What are you interested in creating in the future?

So many things! I have a file with a “future projects” todo-list about five pages long. :D

One concept I’m excited about at the moment is creating a VR game that puts you in the role of an orchestra conductor. It’s perfect for VR, because you just stand in one place and everything is controlled through gaze and gestures. Imagine being in control of the orchestra, playing a piece and being able to direct the music how you want! (Okay, so that one is only going to appeal to the 10 or so people in the world who are interested in both VR and classical music. It’s still a great idea, says I.)

Another project I’m interested in pursuing sometime is a tool for automatically finding approximations for mathematical functions or data sets. This is something that comes up a lot in graphics programming, where we have some complicated shading equation or something, and we’d like to find an approximation that’s cheaper to evaluate. This is hand-wavy, but I imagine using search algorithms to build up candidate formulas from primitives like polynomials, rational functions, power functions, and so on, then using curve-fitting to optimize the error over a domain you’re interested in. The tool could give you a spectrum of options ranging from cheaper and more approximate to more expensive and accurate. (You can do a lot of the curve-fitting and error analysis parts of this using computer algebra systems today, but the exploration of possible formulas is still pretty manual.)

What other topics would you like to explore?

Programming language design is another side interest of mine. I did my college bachelor’s thesis on a module system for a toy language one of my professors invented, and I’ve experimented with hobby languages of my own, though I’m far from an expert in this area.

Like a lot of programmers out there, though I use mostly C++ on a daily basis, I’m not very satisfied with the direction it’s is going these days: off the deep end of unnecessary complexity, making it ever harder for programmers to know what their code is doing and harder for compilers to do a good job, while simultaneously doubling down on bad decisions from years past, in the name of backward compatibility. It seems like C++ is ripe to be replaced with something simpler and easier for both programmers and compilers to reason about, that will serve our needs better for production-scale high-performance code. Of course, C++ has a massive incumbency advantage and will be used for decades yet—but the way it’s going, I can’t see it still being the best trade-off forever. Unfortunately, the main C++ competitors at the moment, such as C#, D, Go, and Rust, all have issues of their own that make them unattractive for various reasons. So perhaps the language that will take down C++ hasn’t been created yet.

On a more down-to-earth note, shading languages are another case where the incumbents, HLSL and GLSL, don’t seem to be serving our needs as well as they could, for a variety of reasons. I’m definitely interested in exploring how new shading languages could make graphics programmers’ lives easier, and this seems like an area where I could realistically make a difference. The tricky thing about a shading language is that you can’t compile it all the way to GPU machine code without IHV support (or insane reverse-engineering hackery), so I’d have to compile to HLSL and GLSL (or now SPIR-V), and that limits some of the prospects for improving on the basic execution model. Still, I think there’s a lot of exciting possibilities in this space!

Happiness List

Programming with People