“be prepared to accept that what seems obviously true now may turn out to be false later.” —Philip E. Tetlock, *Superforecasting: The Art and Science of Prediction*
Words to live by. This book is 🔥.
“#Beliefs are hypotheses to be tested, not treasures to be guarded.”
*Superforecasting: The Art and Science of Prediction*, by Philip E. Tetlock
1/2 “#police officers spend a lot of time figuring out who is telling the truth and who is lying, but research has found they aren’t nearly as good at it as they think they are and they tend not to get better with experience. That’s because experience isn’t enough. It must be accompanied by clear feedback. …” —Tetlock & Gardner.
2/2 “… Psychologists who test police officers’ ability to spot lies in a controlled setting find a big gap between their confidence and their skill. And that gap grows as officers become more experienced and they assume, not unreasonably, that their experience has made them better lie detectors.” —#Tetlock and Gardner.
(Yes I’m still thinking about Steve Yegge’s https://sites.google.com/site/steveyegge2/being-the-averagest and Anders Ericsson and the thread from last month: https://octodon.social/@22/99637730196296990)
@22 Kahneman in /Thinking Fast And Slow/ also looks into this WRT intuition. He had a disagreement with his collaborator Tversky - he thought intuition was largely useless, Tversky thought it highly useful.
It turns out they had been studying different groups - Tversky studied firefighters, and fires give extremely obvious and immediate feedback, and experienced firefighters intuitively made extremely good predictions about fire behaviour.
Kahneman had been studying investment managers… 😁
@RAOF Tetlock talks about this too! His spectrum goes from meteorologists who get fast and good feedback (and thus improve with experience, assuming they want to) to police who get no feedback (and thus without justification use experience as a proxy for skill). Tetlock cites an off-hand remark Kahneman made to him during the 80s as the start of the Expert Judgement Project and Good Judgement Project 😄!
@22 I have the opposite problem. Experience has taught me to defensive, slow, write a lot of code, to prefer a copy rather than to add another layer, and so on and now I think I’m good but I fear my younger colleagues don’t see it and I find it increasingly harder to communicate these values. “Imagine you will think about this code in ten years!?” … “Ten years ago I was in middle school!?” … “…”
@kensanata @22 ...and in 2028, almost nothing written in 2018 will be comprehensible, supported, or maintainable enough in whatever the new context is to tell whether it was a good idea in 2018. ecosystem churn and dependency proliferation have eaten the feedback loop.
@kensanata @22 (or possibly i'm just disgruntled at a few years of learning that the vast majority of my accumulated knowledge is the functional equivalent of a specialization in cobol a few years back.)
@brennen @22 @kensanata Given the current trend towards build systems randomly pulling stuff in from Github and other places, I'd be surprised you could even build most software in ten years.
I mean software stops building after 2 months. @nolan
@loke @nolan @22 @brennen Yeah but you guys are talking about a different world than I am. I’m in corporate land, with legacy apps running on Java 6 or older, where Oracle 8 is definitely out but not so sure about later editions, where I’ve been involved in software projects I helped start ten years later, and that stuff isn’t built on quicksand like npm and Android but it’s old and weird and a pain to set up, that part is very similar.
@loke @kensanata @brennen
I don’t think it’s all so bad. NPM stores hashes of each version, and that will be mirrored on site by JS devs after npmjs.org dies.
Also, while I appreciate that ecosystem churn blocks feedback for JS devs, I expect that devs in banks and defense contractors who have no problems with that are still suffering just as much from lack of feedback driving skill growth as JS devs.
And poor @nolan was complaining about SemVer fails upstream, also a universal problem.
@kensanata thanks for weighing in! I don’t see that as the opposite problem but another one. Experience does teach us many things, and it is difficult to transmit that to the next generation of coders (unless you have a strong apprenticeship mentorship program for your new hires). I’m channeling the Bob Paradox from Steve Yegge’s awesome post “Being the averagest” https://sites.google.com/site/steveyegge2/being-the-averagest where we substitute experience for any real measure of skill—more police than meteorologists.
@22 @kensanata also it is difficult to get a new generation to design something to work in 10 years time, when they are often no longer given secure jobs and instead have to rely on the "gig economy" or short term contracts where there is little training and what is given is often rushed and incomplete (this is also the case for system integrators, project engineers and customer support staff in hardware)
@vfrmedia @22 @kensanata Thankfully, new generations of meteorologists still program in Fortran ^_^
@vfrmedia @22 @kensanata And I'm not being ironic here.
I'd been reading about that recently, possibly from links I followed about research conducted by Met Office, DWD and KMNI (as a cyclist in Northern Europe, I do have some interests in meterology 😉 )
I' also read an article somwhere mentioning how Fortran was once widely used for maths in electronic design and RF circuitry (maybe it still is in some places..)
@vfrmedia @22 @kensanata I work with meteorologists and climatologists. I've looked at a lot of simulation and forecasting code, and it is always in Fortran, and I hope it stays that way for the next 20 years.
For the scientists, improvement is not about writing better code, it's about creating better models.
@wim_v12e @vfrmedia @kensanata
I’m seriously going to learn enough fortran to write a bunch of numerical codes in it. Enough to compare it to Numba, Julia, C (ugh), Futhark?…
@22 @vfrmedia @kensanata I think Fortran-95 is quite a nice language, esp. for writing numerical code. What I like in particular are the powerful array manipulation features and the attribute-based type system, and the module system with selective imports and aliases, and interfaces. What I don't like so much is the IO system.
Fortran 2008 is a much bigger language with support for OO etc, I prefer a much smaller subset. But it does offer native constructs for parallel programming which is nice.
@wim_v12e Thanks, that's super-helpful. I am very interested in SIMD/multithreading baked into the language (I was reading the new OpenMP spec with delight), so I'll write some sequential code in F95, get my bearings, and then study enough of 2008 to see what I need for parallel.
I don't mind the poor IO, I expect to embed the resulting Fortran library into a Python/Julia/Node.js/etc. module.
Is it worth getting ifort (via their free program for open source devs) or just go gfortran?
@22 I compared ifort, gfortran and pgi fortran 2 years ago, there was no appreciable difference in performance for my test code. So I think gfortran is just fine.
@22 But do coders really have to be good? I mean, maybe it is not such a big problem that programmers think they're great and reality disagrees? Maybe they're still good enough?
@wim_v12e well, replacing “coders” with “meteorologists” or “firefighters” or “police”, yeah I think it’s important to improve, especially if you’re bad. You’ll be outcompeted by those who do. You’re holding back progress. Plus nobody wants to embody Dunning-Kruger 😝
@22 I was looking at it from a different perspective. There is clear evidence that humanity progresses, both technologically and socially. This is despite the fact that people think that they are better than they objectively are, so this delusion does not stop progress.
I guess it means we improve regardless of it.
@wim_v12e Well, sure, on a cosmic scale, but on a personal scale, being a better coder should on average make me more money trading options, so that's my reason to want to improve :).
And about "clear evidence that humanity progresses", is this Steven Pinker's argument? You know I'm ambivalent about that. Yes, we have eradicated smallpox. And yes, Kennedy's thinking the first day of the Cuban Missile Crisis was "at a minimum… to authorize preemptive air attacks on the Soviet missile launchers".
@22 Well, I'm personally all for improving, but on the other hand, how can I be sure that I am improving? Like you say, lack of feedback. Also of course for me it should be being a better academic, and what does that mean anyway?
As for the progress: I don't know Steven Pinker so no, it's not his argument. There are many indicators (child mortality, literacy, poverty) that are improving globally. But if you disagree, fine with me.
@wim_v12e yes, that’s the question, “how can we improve?” We could seek a way to get better feedback sooner, or! we could do the equivalent of psychologists testing police’s lie detection skill, but both seem really hard, for coding and academicing! It’s a miserable situation! I’d like to keep thinking about it and see if a creative solution along either of these lines suggests itself. Otherwise we might have to content ourselves with not caring about how good we really are :)
@22 I care about how good *I* am, but although I also care a lot about trying to help other people to get better, I don't care so much how good they really are.
I wonder if, just for coding, it would be possible to come up with some measurable criteria that people could use to assess if they have improved and/or are doing the right things to improve. That would be a great help.
@22 Oh, I remember now, we that this exchange a while ago about technological progress, so I guess that's where you are referring to.
@wim_v12e yeah the progress principle. I really resonated with the counterfactual-driven pessimism here https://www.scottaaronson.com/blog/?p=3654 the roll of the 20th century dice came up very favorable for us, but I think a lot about the other rolls that were equally possible. But yeah this is a more cosmic outlook, not so relevant to day to day work :)
This is exactly how I feel about coders. We get paid to write code for years. We ship products. We mentor and interview. We *have* to be good, right? Maybe, maybe not. We lack a way to measure how good we are, we have very slow feedback (only later do bugs and inextensible design become apparent). I bet I’m pretty bad at programming relative to how great I think I am.