Welp, I've contacted a favourite podcast to correct their statement that neural networks are impossible to understand. I'll do more of this in the future, I think, but I'll probably wait until I'm done the paper I'm currently writing since it has some new approaches not in my previous publications.
@Vanessa Oh gosh. Everyone has to do that at some point u.u
For serious - ANNs have been around far too long for them to be a mystery.
@Nafrondel it's so weird to me. Why is this story of ANNs as black boxes so persistent? People even sometimes still tell that old tale about the military tanks that were accidentally trained to only fire on sunny days. Despite there being no evidence that any such thing ever happened and it now being recognized as apocryphal. It's not magical gnomes inside; it's numbers. We can inspect numbers.
@Vanessa Ikr. It's like the "Holy fluff, Google translate unexpectedly invented a language! Machines are evil o_o" thing last year.
Idk if it helps, but the tank thing as I learned was that they were using a very basic in memory machine vision learning (not ANNs), and taught it friendlies on runways, and enemies on sand. Because the tanks were minor artefacts, it couldn't tell friend from foe in the field. Just airbases from sand. So it was never put near a tank, and wasn't a mystery.
@Nafrondel But, yeah, the fear of them becoming evil also comes up a lot. I have a looot to say about that, but the short version is that an artificial consciousness of advanced capabilities is unlikely to care we exist. Humans just aren't that important or interesting or useful really. My friend Damien writes about ethics and machine learning a fair bit if you're into that. You can find some of his work here:
http://afutureworththinkingabout.com
I think he's decidedly excellent.
@Vanessa Doesn't help that Hawking keeps saying "Moore's law means if it can happen, it will happen".