I was gonna spend an hour on this and then I spent my whole day on this long thing nobody's going to read.
But here's some more musing on "systems lisp" things, whatever the hell that means: "A Guile Steel smelting pot" https://dustycloud.org/blog/guile-steel-smelting-pot/
@cwebber "Compilation By Program Transformation" is such a weird title... it's like "Walking by Moving Legs"
@cwebber but ohhhh Christine... this post is giving me the feels.
I really want to believe. I want to live in this dream world we've all been envisioning where we can finally live free from the curse of the Original Sin of using C.
but that dream feels like a black hole that just sucks in all the work you can possibly give it without a credible promise on the other side of something that can work for everyone.
for every person who hacks on an ideal application, you have someone who's going even further to develop their own ideal language, and someone who goes further than that to develop an ideal (done right this time!) VM, and beyond that someone with their ideal OS that finally learns from the mistakes of the past, who of course is surpassed by someone with their amazing, pure, ideal instruction set and CPU architecture which will finally set us free (once we solve the pesky problem of fabrication).
@cwebber and even with all this that everyone has done, can you really credibly tell me that this "ideal" world is for everyone? have we found a way to render Chinese, Arabic, Hebrew, and Russian without harfbuzz? have we found a way to implement TLS 1.3 in sectorlisp? are we prepared to reimplement the entire universe of browser-based applications in this from-the-ground-up done-right-this-time world?
meanwhile we're having this conversation on a network which runs on web browsers and unix and mountains of unspeakably awful, insecure, hacked-to-bits C code but it's this incredible place where people can actually own their conversations and live their lives without corporate pressures breathing down their neck every second.
it feels so indulgent and elitist to spend time on the former when the latter is here today and is actually demonstrably changing lives, I guess?
but I'm as guilty as any, some wanker who builds compilers recreationally. I dunno. wish I could do more.
@technomancy @cwebber I really *really* sympathise with this fear, and feel that it's the hard work that needs to be done. is it impossible? maybe, and these foundations are certainly colossi of code and absorbed knowledge and practice, but they are not intractable or irrepeatable -- I believe we do not rebuild them largely because of the sunk costs, not because it is impossible.
The question is: how to make this challenge tractable? A few ideas:
* in the before times the majorit of the established world was proprietary, so you couldn't use automated systems to analyse and incrementally incorporate older systems. Now harfbuzz, chrome etc default to open
* there is evidence that re-implementation is getting easier: we've gone from Linux being a multi-year bootstrap to hobby OSes emerging a regular basis, including web platform support e.g https://github.com/SerenityOS/serenity
* there's increasing commonality between well-funded commercial research goals and the needs of alternative platform makers: e.g. the porting of chrome to Fuchsia, Google's cap-based OS, Microsoft and ARM's funding of OS ports and toolkit support for CHERI https://msrc-blog.microsoft.com/2022/01/20/an_armful_of_cheris/
* there are also some hardware trends that are promising -- on one hand we now have a handful of promising open hardware platforms (and no more than a handful!) which gives some hope to having a limited support matrix. on the other, the chip building stack is opening up, promising some ability to bend the hardware to the needs of our goals instead of struggling the other way
I think the biggest social challenge, as you note, is how hard it is to get everyone to focus on one "perfect" platform rather than everyone pursuing their own utopian dream setup. This certainly divides resources and attention. It's the classic problem of decentralized development! But that just means we must pursue the hard work of connecting and co-operating, just as the builders of open systems before us did
@mala @technomancy I agree with *both* of you on *everything* there. And I think we're at a stage where there's enough happening with people not helping to each other that writing a "smelting pot" like post is good. Eventually you need to make choices, have focus, try to drive core work in a particular area, but right now I think some cross-dialogue is good.
Also, the choice to focus *largely* on PreScheme is not accidental here. You can see a lot of parallels between PreScheme and Fennel: both are targeting an existing ecosystem rather than trying to build an entirely new world, and in that way, provide a clever way to survive it.
but then i harnessed some of that existential angst towards something more practical and installed gotosocial on my home server, and it looks like A) it's really easy to install and B) the federation bug that bit me last time appears to be fixed.
so it's possible to strike a balance between building towards this ideal world and building technologies that help people today where they are.
(but I gotta say, after setting up a home fedi server, if you somehow found a way to defeat the evils of NAT and port forwarding, I think that would be a greater force for user empowerment than all the lisps ever created put together)
re: so I guess this whole thread is just a weird elaborate way to say ...
@technomancy @mala At any rate, I think it's worth persuing multiple exciting directions. It's hard to anticipate which things will take off, giving boosts of encouragement to things that look interesting I think is good.
RISC-V and Blender are both things that I now regularly see hitting my timeline that I was really excited about for a long time but there was a lot of "yeah, this is never going to happen though, it would have by now". Hell, ActivityPub itself is in that category. It could be pretty demoralizing working on it when the main thing I had ringing in my ear was "this is never gonna happen". I'm glad I didn't listen to that, and put in the work.
Lots of other things I've been excited about *didn't* make it. But you really don't know what will. It's worth encouraging, trying to promote and inspire things that might, from the extremely ambitious stuff (a new architecture?!) to the more down to earth (the thing we want that compiles to the thing we don't as much!)
(as much I can say it's a great and necessary thing that gotosocial is taking over the niche that pleroma originally occupied, trading elixir for golang means it's nearly impossible for me to summon up any enthusiasm to hack on its codebase directly.)
@technomancy @mala btw I have many, many times thought "what the fuck am I wasting my time on this stuff for when the world is literally BURNING" and considered dropping all the programming stuff to try to focus on climate change because if society doesn't survive what's the point of all this?
but let's be honest, I would have near zero impact on that. so I keep hacking on the stuff I can.
A lot of your work on compilers, your enthusiasm for it, your gamedev stuff, have been *infectious* for me, and at various times where I felt completely depressed with everything and giving up, I turn to the people I admire for inspiration. Your stuff is straight up in that category.
Keep going, you're on the right track. And if you want to figure out how to make a more direct impact on the social network stuff while doing lisp, well heck, there are ways we can collaborate :)
@cwebber @technomancy Me too -- and *my* day job is (literally!) working out how to to thread all of the energy and ideas being worked on into something that has immediate impact for end users. (Oh, and also trying to solve hole-punching through NATs, though that's mainly #libp2p so I don't think that's a solution for you right now. Bizarrely, ipv6 consistently works in that space -- talking about quixotic quests)
@csepp @cwebber @technomancy right there with you for the last decade or so :) https://cpj.org/author/danny-obrien-cpj-internet-advocacy-coordinator/page/2/
This needs creative mediation for any outcome.
Tech can play a role in social movements #activertypub shows that, but this is clearly not anufe.
Your chart is ready, and can be found here:
Things may have changed since I started compiling that, and some things may have been inaccessible.
In particular, the very nature of the fediverse means some toots may never have made it to my instance, in which case I can't see them, and can't include them.
The chart will eventually be deleted, so if you'd like to keep it, make sure you download a copy.
You know I'd never connected the two :D but now that I think about it..
My Gran took me to see Spirited Away in the cinema when I was pretty young and the image of the river spirit throwing up all the garbage really stuck with me (especially the bicycle, in hindsight I'd already lost a few!) I didn't understand it at first but the movie had a big impact on my life and Ghibli films in general introduced me to Animism.
Having spent a good chunk of my life trying to bootstrap a new language stack and OS from scratch and gotten a visceral education in all the ways that's difficult and, worse, ill-posed, I now have a new answer to the most impactful thing I can work on:
A more decentralized approach to software governance.
This connects up with capitalism, bear with me.
Projects evolve over time, especially growing monotonically more complex. Again and again I see projects slow with age. Is a clean new project really an advance? Maybe it's just younger? Conversely, I don't use Emacs, but I think it's the healthiest software project in the world today. Extremely old, continuously in development for decades, still sporting incredibly vibrant releases.
If good platforms might be mirages over time, and bad platforms can participate in good projects, what's the pattern here? My best guess: incentives. Emacs may have survived all these years partly because no business makes money from it and so has an incentive to capture its governance.
In this worldview the problem with C is not its safety issues. It's that compiler projects are captured by companies to interpret UB more and more aggressively.
If I'm right, what will move the needle is not a better point-in-time design but somehow making projects capture resistant. And I think the best way to do this lies in the direction of 10x/100x more forks. It will put each fork closer to the sub-communities that use it. Communities of users will be closer in size to the projects they depend on, no more fan out/blast radius of 1M-100M users.
When producers and consumers are more similar in size, you get greater accountability. Accountability to _people_ rather than companies with revenue.
Emacs creates incredible value in the world, but it contributes zero to GDP. This is an indictment of GDP. We need better measures.
Yes, the world is burning. Yes, we need to do things about it. But doing requires organization. Healthier modes of organization are hugely valuable.
Asimov reference: fixing incentives is the Second Foundation.
@akkartik I've been thinking about "career-oriented open source software."
I embarked on writing a cluster-oriented language + bytecode/virtual machine that would be the core by which an ecosystem of "low profit" data technologies could connect into and be marketed in a "co-op" manner.
The motivation for the endeavor is discussed in these slides:
The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!