@haverholm during the first lockdown, Sean from Autechre was streaming some experiments on Twitch with a Max patch that was controlled in part by a video https://www.youtube.com/watch?v=EP37LslDAvo - clues on how it might work on blank screens and statics screens. I assume it's using the value (brightness?) of selected pixels to control parameters of the sound generation
@haverholm on Linux, Puredata + Gem could probably do something similar, Gem has an object that allows you to get pixels values https://puredata.info/downloads/gem/documentation/manual/manual/list-of-gem-objects
The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!