A visual effect breakdown of my NotITG playable arrow music video made for six impala's "eyes in the water"
A few months ago, I released a NotITG modchart set to six impala's "eyes in the water". If you haven't seen it yet, I highly recommend you view it; in its entirety it took me about a week (plus extra time for polish).
This article will be a write-up of the many, many effects it features! I'll try not to use mods terminology to make this more intuitive to read to those outside of the ITG modding space (or even VFX in general!), so take this as just a plain visual effects breakdown.
The palette
A very big aspect that went into the making of the file is the palette that's applied on top of every single effect. It's a simple, 16-color palette I designed in an afternoon and refined over the making of the file.
But why, you might ask? Well, simply put, it exaggerates the colors I'd like to see the most that give it the spooky atmosphere it has, while also making my life picking colors under the palette pass much easier.
Here's what the file looks without the palette effect applied - you'll notice there's a lot of very plain colors. Plain red, plain gray, plain black - with no sense for any color theory or matching colors. However, put under the palette, it all blends together quite nicely.
I was highly inspired by Inscryption, specifically an interview with Dan Mullins where he described why exactly a similar post-processing effect was added:
I was just experimenting with that and it really came together when I got Tomasz Stobierski—he’s in the credits—who did this shader. It was an idea that I had but I couldn’t make it myself, where the darker colors posterize, which means they lock to the nearest color, but the lighter colors don’t. Which is how it ended up with these very hard shadows, but if you do that to all the colors, it’s hard to see the lighter stuff, and you can’t read text, and it looks bad.
This is why there are so little dark colors, yet so many vibrant, brighter colors. The one key difference between Inscryption and NotITG mod reading however is that arrows need to pop out as well to make them easier to read, which is why reds, greens and blues, their primary colors, were prioritized in the palette.
However, that's not all that goes into this effect - if you just apply this on its own, it'll look really sharp, and more like a badly compressed JPEG than anything.
Which is why I've done some makeshift dithering behind the scenes!
Before the shader gets to posterize (crunch the image into the palette) the image and return a compressed version, a slight noise texture is applied. Two of them, actually. A simplex noise texture, which looks something like this:
And a basic white noise texture:
These two, applied only subtly, offset the colors of the image ever-so-slightly, giving every border that slight staticy, wobbly feel, which plays into the atmosphere of the chart a lot.
Generally you would do this with a Bayer dithering filter, but this fits better because it's less rational and methodical and more noisy, messy, which is exactly the kind of feel the chart goes for.
And that's it! This effect applies throughout the entire chart with no exceptions, which was a rule I set out for myself, and it works really well.
The vein-like backgrounds
During the drops, there's a particular background in play, which at first sight looks really complicated:
But I assure you, reader, that it's actually quite shrimple.
First, we start off with what makes the shapes of the background. It's a simple bit of Voronoi noise, which on its own looks like this:
Now, this might look slightly useless at first, and that's because it kind of is. Typical Voronoi noise has the color correspond to the distance from the middle of the cell. This matters to us because this means the color of the boundaries, so to speak, are inconsistent and weird.
I picked up a version of this type of noise from Shadertoy, which would handle distances from the borders instead.
Now, if we keep just those borders, we get something a lot more similar to what we're looking for.
Perfect! Now let's ease the animation of the cells to make it more punchy:
Okay, so now what do we do about the colors pulsating? Well, what I wanted is to have them pulse from where the player was, but that was going to be hard to code in the shader. So I resorted to making the engine render out little ripples for me:
Let's use that as a texture for our Voronoi shader!
And that's it! No, seriously, that's it! Now we just let our post-processing effects handle the rest:
The palette shader really does wonders for this, and was what I was designing the Voronoi effect with. It let me cut a lot of corners, and yet it still looks really impressive.
The pulsating arrows
During the drop, arrows are presented in a particular way:
They seem to have a really complicated way of appearing, and a really complicated way of pulsating and then disappearing. What's going on here?
What you're looking at is an effect made with an SDF. An SDF, in short, is an image where every pixel's color is determined by its distance to an object. For instance, here's what the arrow used here looks like:
When we run it through an SDF generator, we get an output like this:
What this means for us is that you can filter by specific colors to turn this blurry mess into a shape. For instance, if we pick every color the alpha of which is below 0.5, we get this:
However, tweak our threshold of 0.5 and the arrow seems to be expanding and shrinking in size, without having to compute this all on the GPU constantly:
Tweak our threshold to be everything above 0.5 and below 0.6, and we get an outline:
And so on, and so forth. You can tweak the algorithm that takes the SDF alpha channel as the input a lot. And that's what I did! After lots, lots and lots of tweaking, playing around with random numbers, and various cursed math, I ended up with this:
This may not look all too fancy, but put through our palette shader I keep praising so much (and for good reason!), we get something quite nice-looking.
This renders extremely quickly because, well, you're just doing simple math to the alpha channel of an image, so we can now render it multiple times per frame to form the patterns in the file. Each is independently animated with a uniform value, and that's all there is to it.
A Secret?!
It's worth mentioning that if you peek inside the files, you won't see an ordinary SDF arrow like I showed above, you'll see something more like this:
This is because I'm only encoding useful information in the alpha channel. Every other channel can be safely discarded, so I've put it to use to hide a little secret as part of.. many other secrets. It's a neat trick, and it's actually done quite a lot in the assets.
The ending
This one's a little funny - the holds at the very end are peculiar in that everything around them is warping, while they are holding still.
This is conceptually a really simple trick. The holds are placed on a sine wave with bumpyx, and then the sinewave is accounted for in post-processing with a shader:
float beat = (uv.y - (0.5 - px.y * RECEPTOR_ARROWS_Y_STANDARD)) / (ARROW_SIZE * px.y) / (speedmod * 0.873);
float off = sin(beat * PI*2) * px.x * ARROW_SIZE * (bumpy / 160.0);
This is a really hacky trick; you'll notice there's magic numbers and constants everywhere. This is mainly because the game is stupid, but it's also partly because shaders just don't have that much control over the game. You will not be able to get this to work on a setup that isn't exactly like mine, and even then you can very clearly tell something's up with the holds.
Regardless; I think it came out well and it adds quite nicely to the atmosphere. The hold heads being black was completely accidental; and I'll be honest, I have no clue how to undo that. Genuinely. I did something wrong somewhere, but oh well, I decided it should stay.
Summary
This is far from covering every trick in the chart, of course! There's loads of little visual treats in the background and in tiny little bits of the song. There's accents and details and little design considerations for every noise the song ever makes; and it's quite fun to just look through it bit-by-bit.
The shaders for the file are all available in the file and are (mostly!) documented; so if there's anything you'd want to take or otherwise reference, you are completely free to do so.
I hope you enjoyed reading through this all!
👁️