You would really think the world would have figured out how to share one set of peripherals between multiple computers, but apparently that's just fucking impossible.

Since DisplayPort is such a hot mess, I'm just gonna buy new monitors, build my own KM switch with audio, and write a little daemon that switches the monitor inputs over software when the right USB devices appear or disappear.