My face is just an array of numbers
An experiment that never happened
At one point while I was working on my exercise tracking companion super mega app I thought of an experiment. Can I make it count reps automatically by looking at me through the laptop's camera?
The app is not mobile. It's designed to be used on a proper personal computer. That way I can clearly see the time left, the information I need for the current rep, etc. And it just looks better. (The small screens are a bit sad.)
So I wrote a small "probe" code file - a self-contained runnable that I can play with separately from my app. No "beautiful code" there. Just raw experimentation.
The biggest chunk of the probe was to set up the whole pipeline of MacOS. The integrated camera means that Apple gives you a nice API with your system.
I got to a place in my experiment where I could get raw data from the camera.
You see, for my experiment I didn't need the colors of each pixel that the camera "sees". I just needed the brightness. The idea of the experiment was that the brightness was going to change significantly while I did my reps in front of the camera - colors didn't provide me with any useful signal.
And then I stopped. The experiment demanded more and more work - we were getting into the math zone, normalization, etc. (What if there's a cloud - the brightness changes significantly, but not because I did anything...).
This required more energy from me. At the same time the work started to ramp up and most of my energy went there - multiple burnouts followed. Another story.
Many-many months later I think about a donut
I guess this is how creativity works sometimes: you have a thing in your brain and it randomly meets another thing, unrelated.
For some reason I've remembered the donut.c experiment.
Some years ago the dev influencers space was full of videos about rendering a rotating donut. In a terminal. Using ASCII characters.
Back then I was still watching random youtube videos thrown at me by the algorithm, so I saw the donut analysis, etc.
The idea stuck in my brain: a pixel on a black background can be represented by its level of brightness; the brightness can be a number (0-255, for example, the bigger, the "whiter") or a character. Some characters take less space in a terminal's cell, for example a full stop (.), and some take more space, like an eight (8) or a zero (0). The more space the character takes from the cell the brighter the cell (the pixel) becomes.
A very simple idea.
But wait... I have this unfinished experiment that stopped at me getting an array of pixels and their brightness levels. What if I just... convert the raw brightness number into an ASCII character that gives an illusion of brightness in a terminal...
The failure
Today was the day to do this. I felt fine. I had my hot caffeinated drink. I hadn't done any hobby coding for some time.
So I opened the old probes/camera.odin file and got to it.
The idea was yet again simple. Get the size of the terminal using Darwin's syscall (every OS probably has one for that). Loop through all the cells of the terminal. Figure out which pixel from the raw array that the camera gives us goes where in the terminal.
The loop:
for row in 0..The so-called ramp - characters from least "bright" to most "bright" - was taken from the donut experiment. It's this:
ramp := [?]u8{'.', ',', '-', '~', ':', ';', '=', '!', '*', '#', '$', '@'}You can clearly see how the character fills more and more space in its allotted cell.
I fixed all the integer casting issues and worked on performance and got something like this:
Something is not right. I get some noise on a mostly dark screen.
Let there be light
Before debugging I decided to do one thing first - move my butt to the couch where I have a lamp.
And it worked!
This was very exciting, not gonna lie. I felt... like a kid who had made his first Basic program run a word (his name, of course) across the screen. The joy of programming was still here.
And because it was a camera I was getting frames, so I didn't just render a static image. The oldest trick in the book: if you clear your screen and then render and then repeat these two steps, you get a movie (an animation).
You get... fun.
The gif is converted and cut to be small. I do have a 10s video on my laptop. And you can experiment with my code. It's a part of my Couch repo. It's in the probes folder.
The end?
Will I do anything with this experiment?
I don't know. That's the whole point of experimenting and probing. I can work on them and then stop and then get back to them when I get my dev senses tingling again.
It would be cool to plug this into video call software somehow (imagine a Google Meet where you're rendered in ASCII... so... elite hacker style :D).
But for now... I will go and enjoy some fresh air. It's Saturday.
GLHF