What I realized was, the shader design of running code for every single pixel of an image isn't only applicable when your GPU allows you to do it.
So back in September this year, I decided to stream music to one of my friend groups. The plan of it was: I create a funny-looking overlay in OBS, then go in a Discord VC and stream the layout and music through a fake audio input. It worked well, but the part that this post is about is the layout.
It looked something like this:
Ignoring all the mess of dancing GIFs, you'll notice it's literally just tmux with a music visualizer, nano and.. a black spot?
Yeah, so you'll notice all of the software used here isn't original. I slapped some Linux software together in a tmux shell and called it a day. But I wasn't satisfied with it, so I decided to code up a tiny script for the top-right segment.
My mental image of it was this: it'd show the currently playing song, and have some extra fancy visuals if possible. I knew this was possible with MPRIS - a standard D-Bus interface which allows you to control music playing on the system, read its metadata (which is the part we're interested in!) and do this all through any music player on the system.
And since I was already going to broadcast the audio through a music player, this was the prefect solution:
So of course, logically, I started with the most optional part: the background.
The background might look like it's really hard to make, but surprisingly, not really?
So, in short, to render anything on a GPU you need a shader, specifically a pixel shader. What a pixel shader does is go one-by-one on each pixel, runs some code, and puts the resulting color (in form of a 3D vector) on the spot it ran the function for.
For example, in a pseudo-GLSL, you could do something like this:
void main() {
// uv is a variable that converts the screen into
// a 2D vector from 0.0 to 1.0
vec2 uv = gl_FragCoord.xy / resolution.xy;
return vec3(uv.x, uv.y, 1.0);
}
And this pseudo-GLSL code will render something like this:
This may seem weird and unintuitive, but in reality shaders are a lot more powerful than you think. For example, my homepage's background is made entirely through a single shader! It makes sense if you think about it: you start with something simple like this:
Then, you add a bit of UV distortion, making the image use "wrong" positions for the image:
Add some more lines on top, do some color correction, and you're done!
I won't dwell on shaders too much, but my point is - they're sick. People have made really impressive stuff with them, and there's an entire website dedicated to archiving, creating and sharing shaders.
You may think, "well, this applies to GPUs, cool, but how does this relate to the music visualizer script"? Well, what I realized was, this design of running code for every single pixel of an image isn't only applicable when your GPU allows you to do it.
A terminal only has a few "pixels" in it - in reality, if you have, say, a 60x30 terminal, that means it'd be able to display 1,800 characters, which means it's a very small canvas for rendering actual images. You could try, but all you get is vague shapes..
Turns out, vague shapes are exactly what we need for a weird-looking funky background! And since a terminal is so tiny, you can do this pretty efficiently! So I got to work, and made a script that does just that in Node.JS (I still wish I could learn a better language overnight).
The shader I chose was a classic plasma shader - I won't go too in-depth into how they're done, but you can check out this really good article for a better explanation of how one is made.
After you render out a plasma shader (like seen above) to a tiny terminal, you can now convert each color to a character. If you have an array of characters like so:
const plas = [' ', '.', '*', '/', '0'];
You can now assign every single "brightness" value its own character. If an image is fully dark, it'll display blank, letting the background color of the terminal show up. If an image is fully bright, it'll displays as zeroes, which take up more space than the other characters, it'll seem as if the color is brighter, as there's more of the terminal foreground color showing up.
So we end up with a very demoscene-looking terminal display! Exactly what we wanted. And once we slap on the rest of the display, we end up with this!
And that's pretty much it. The MPRIS, metadata handling part of the code isn't very fun to discuss, as it's just D-Bus jank, so I've left it out.
As with all of my other projects, you can check out its source code! I definitely don't think it's very readable or nice-looking, but it gets the job done. The text rendering specifically was a very very jank system to implement, despite being the thing terminals were made to display...
I hope you learnt something fun about computer graphics today. I like them computer graphics. They make me happy. :)