I was gonna spend an hour on this and then I spent my whole day on this long thing nobody's going to read.
But here's some more musing on "systems lisp" things, whatever the hell that means: "A Guile Steel smelting pot"

@cwebber "Compilation By Program Transformation" is such a weird title... it's like "Walking by Moving Legs"

@cwebber but ohhhh Christine... this post is giving me the feels.

I really want to believe. I want to live in this dream world we've all been envisioning where we can finally live free from the curse of the Original Sin of using C.

but that dream feels like a black hole that just sucks in all the work you can possibly give it without a credible promise on the other side of something that can work for everyone.

for every person who hacks on an ideal application, you have someone who's going even further to develop their own ideal language, and someone who goes further than that to develop an ideal (done right this time!) VM, and beyond that someone with their ideal OS that finally learns from the mistakes of the past, who of course is surpassed by someone with their amazing, pure, ideal instruction set and CPU architecture which will finally set us free (once we solve the pesky problem of fabrication).

@cwebber and even with all this that everyone has done, can you really credibly tell me that this "ideal" world is for everyone? have we found a way to render Chinese, Arabic, Hebrew, and Russian without harfbuzz? have we found a way to implement TLS 1.3 in sectorlisp? are we prepared to reimplement the entire universe of browser-based applications in this from-the-ground-up done-right-this-time world?

meanwhile we're having this conversation on a network which runs on web browsers and unix and mountains of unspeakably awful, insecure, hacked-to-bits C code but it's this incredible place where people can actually own their conversations and live their lives without corporate pressures breathing down their neck every second.

it feels so indulgent and elitist to spend time on the former when the latter is here today and is actually demonstrably changing lives, I guess?

but I'm as guilty as any, some wanker who builds compilers recreationally. I dunno. wish I could do more.

@technomancy @cwebber a c compiler can generate lisp, so it's not impossible to leverage these libraries though. That's what the lisp machines did back then.

@technomancy @cwebber Is your compiler building medicinal or recreational?


Practitioners usually are driven both by utility and pleasure.

So, I might misunderstand your question but I think it's a false dichotomy?

We create toys and play with them because the learning inherent to the process is fun. There is no substitute for that process really--formalizations and applications of it around standards, curricula, pedagogies are essentially extractive, reaping the fruits of play in some compatible, scalable way.

@technomancy @cwebber

@idlestate You took the question way too seriously.

@technomancy @cwebber I really *really* sympathise with this fear, and feel that it's the hard work that needs to be done. is it impossible? maybe, and these foundations are certainly colossi of code and absorbed knowledge and practice, but they are not intractable or irrepeatable -- I believe we do not rebuild them largely because of the sunk costs, not because it is impossible.

The question is: how to make this challenge tractable? A few ideas:

@technomancy @cwebber

* in the before times the majorit of the established world was proprietary, so you couldn't use automated systems to analyse and incrementally incorporate older systems. Now harfbuzz, chrome etc default to open
* there is evidence that re-implementation is getting easier: we've gone from Linux being a multi-year bootstrap to hobby OSes emerging a regular basis, including web platform support e.g

@technomancy @cwebber

* similiarly, the toolkit around emulation or building harnesses has improved: VMs/sandboxes are not only commonly used but code has adapted to living in these spaces -- a program that doesn't run in docker or virtualized hardware isn't much use thse days

@technomancy @cwebber

* there's increasing commonality between well-funded commercial research goals and the needs of alternative platform makers: e.g. the porting of chrome to Fuchsia, Google's cap-based OS, Microsoft and ARM's funding of OS ports and toolkit support for CHERI

@technomancy @cwebber

* there are also some hardware trends that are promising -- on one hand we now have a handful of promising open hardware platforms (and no more than a handful!) which gives some hope to having a limited support matrix. on the other, the chip building stack is opening up, promising some ability to bend the hardware to the needs of our goals instead of struggling the other way

@technomancy @cwebber

I think the biggest social challenge, as you note, is how hard it is to get everyone to focus on one "perfect" platform rather than everyone pursuing their own utopian dream setup. This certainly divides resources and attention. It's the classic problem of decentralized development! But that just means we must pursue the hard work of connecting and co-operating, just as the builders of open systems before us did

@mala @technomancy I agree with *both* of you on *everything* there. And I think we're at a stage where there's enough happening with people not helping to each other that writing a "smelting pot" like post is good. Eventually you need to make choices, have focus, try to drive core work in a particular area, but right now I think some cross-dialogue is good.

Also, the choice to focus *largely* on PreScheme is not accidental here. You can see a lot of parallels between PreScheme and Fennel: both are targeting an existing ecosystem rather than trying to build an entirely new world, and in that way, provide a clever way to survive it.

@mala @technomancy This is also arguably the smartest choice that Rust made, to target C ABI compatibility (even though there is no real C ABI, heh)

@cwebber @mala but yeah I'm not criticizing the Prescheme work; more wringing my hands in existential angst over taking the Prescheme idea to what seems to be to be its natural conclusion.

but then i harnessed some of that existential angst towards something more practical and installed gotosocial on my home server, and it looks like A) it's really easy to install and B) the federation bug that bit me last time appears to be fixed.

so it's possible to strike a balance between building towards this ideal world and building technologies that help people today where they are.

(but I gotta say, after setting up a home fedi server, if you somehow found a way to defeat the evils of NAT and port forwarding, I think that would be a greater force for user empowerment than all the lisps ever created put together)

so I guess this whole thread is just a weird elaborate way to say ... 

@cwebber @mala

"don't quit your day job"?

re: so I guess this whole thread is just a weird elaborate way to say ... 

@technomancy @mala I especially won't if it can continue to be written in lisp, even if it's not *about* lisp ;)

re: so I guess this whole thread is just a weird elaborate way to say ... 

@technomancy @mala At any rate, I think it's worth persuing multiple exciting directions. It's hard to anticipate which things will take off, giving boosts of encouragement to things that look interesting I think is good.

RISC-V and Blender are both things that I now regularly see hitting my timeline that I was really excited about for a long time but there was a lot of "yeah, this is never going to happen though, it would have by now". Hell, ActivityPub itself is in that category. It could be pretty demoralizing working on it when the main thing I had ringing in my ear was "this is never gonna happen". I'm glad I didn't listen to that, and put in the work.

Lots of other things I've been excited about *didn't* make it. But you really don't know what will. It's worth encouraging, trying to promote and inspire things that might, from the extremely ambitious stuff (a new architecture?!) to the more down to earth (the thing we want that compiles to the thing we don't as much!)

@cwebber @mala yeah I guess I should be totally extra clear here that this whole time I'm describing my own emotional journey and emphatically NOT telling other people what they should be doing with their time!

@cwebber @mala but it's also about me personally wondering if there's some way to spend my own efforts more on things that could have end-user impact rather than toys for nerds.

(as much I can say it's a great and necessary thing that gotosocial is taking over the niche that pleroma originally occupied, trading elixir for golang means it's nearly impossible for me to summon up any enthusiasm to hack on its codebase directly.)

@technomancy @mala I actually think your toys *and* tools for nerds are very important and more , and you should keep working on the things that excite you.

games and fun drive everything interesting.

Show newer

@technomancy @cwebber @mala Since we're baring our souls and talking personal emotional journeys here..

Having spent a good chunk of my life trying to bootstrap a new language stack and OS from scratch[1] and gotten a visceral education in all the ways that's difficult and, worse, ill-posed[2], I now have a new answer to the most impactful thing I can work on:




A more decentralized approach to software governance.

This connects up with capitalism, bear with me.

Projects evolve over time, especially growing monotonically more complex. Again and again I see projects slow with age. Is a clean new project really an advance? Maybe it's just younger? Conversely, I don't use Emacs, but I think it's the healthiest software project in the world today. Extremely old, continuously in development for decades, still sporting incredibly vibrant releases.

Show newer
Show newer

@mala @cwebber yes and no. because when I see things like SerenityOS it actually makes me want to do a lispy OS even less! here I'm looking at the one alternate OS that has a (slim) chance of succeeding. I can either switch to it (and give up on ever escaping from C) or compete with it divide efforts even worse than before.

(the argument that docker makes it easier doesn't count for much to me when docker is symptomatic of all the problems I want to avoid.)

but in the end as you've noted, it's the ancient fated Lisp Curse which is the greatest threat.

@mala @cwebber it's particularly telling that generations of lispers have tried and failed at uprooting Emacs from its C. Emacs, the most beloved-by-lispers program ever written, has a dozen half-finished attempts to "do it right this time" and liberate it from its humble and yucky origin point. if we can't even get something like that off the ground, how can we talk about entire operating systems or chipsets?

@technomancy @mala @cwebber I thought Zmacs qualifies as an uprooted emacs? It pretty much had to be, I think, since Genera is written entirely in Lisp.

@technomancy @mala @cwebber Reimplementing the C core of Emacs is easy, and has been done many times. Zmacs, Climacs, Second Climacs, Lem, Hemlock, etc.

The problem is that just because you have a foundation that is just as powerful as Emacs doesn't mean you now have a replacement. The Emacs ecosystem is so much more though, and just the editor itself isn't interesting at all. It becomes useful when you add all the thousands of libraries that have been written for it.

These libraries are written in Elisp, using very low-level functionality of Emacs, and both Elisp the language and the low level API's are terrible. No one is interested in reimplementing that in Common LIsp, or any other language for that matter.

@loke @mala @cwebber

"No one is interested in reimplementing that in Common LIsp"

maybe not in CL, but deuce, guile-emacs, and remacs all have a goal of reimplementing full elisp compatibility outside C.

@technomancy @loke @mala guile-emacs! By Robin Templeton! I actually RAN IT! It managed to load my entire emacs config and theme and worked and I was able to run org-mode and tetris!

It was slow, but it worked! It still needs to be optimized tho

@cwebber @technomancy @mala Yes, but what's the benefit? Scheme is too different from Elisp to be a reasonable replacement. You really don't want a language with two distinct and incompatible strings. And that's just one of the issues.

@loke @technomancy @mala guile-emacs ran elisp. And you could mix scheme and elisp code together.

@loke @cwebber @mala well, I didn't say it was a good idea necessarily; just that there are people who are trying. =)

"that emacs clone died" as an argument is kind of ... well, as far as I know *every* emacs clone is dead, or in the process of dying. so it doesn't tell you much about the differences between them.

@technomancy @loke @mala
I don't remember where I got the phrase from but one of the things I like to say out loud is: "I'm dying! Slowly! Over the years!!!"

@technomancy @cwebber @mala remacs wasn't an emacs clone though. It was the poster child for when "rewrite it in rust" is ill advised.

Personally, I don't really have a horse in this game. I'll use GNU Emacs until a better alternative exists, and I frankly don't care what language the core is written in. I wouldn't spend time on rewriting it, and rather work on something better. Climacs was a good first and second attempt. Perhaps the third will be worth working on. 🙂

@loke @technomancy @mala I guess a big difference is that Remacs was very much a fork, and kind of a slightly antagonistic one.

guile-emacs could have just become the future of GNU Emacs. That would have been a cool future.

@cwebber @technomancy @mala I think guile was a bad choice though, for multiple reasons. CL would have made more sense, but even there you'd be bitten by a lot of the same problems, just slightly fewer, thanks to CL being closer to Elisp.

But, as I said earlier, I really would prefer a new foundation. Emacs is the best we have at the moment. It's just sad that the best we have is so terrible.

@loke The discussion whether some Scheme or CL is a better foundation for a new Emacs is a horse that's been beaten to death already. Actually so long ago that you'll hardly find a trace of the bones anymore. I guess these days people just do what they want to do and hope to attract a community.

Outside of the Emacs bubble people shrug and keep on using VSCode or whatever. @cwebber @technomancy @mala

@technomancy @mala @cwebber Well, remacs died (to be honest, it was never really alive, as admitted by the project's creator in the justification for its creation).

Guile-emacs is unlikely to go anywhere, for plenty of reasons. One of which being that they have only one developer, who isn't really working much on it anymore (at least last I looked).

@loke @technomancy @mala I think it sucks that when I tried to advocate it to the core GNU Emacs mailing list though one of the emacs maintainers at the time was dismissive :\

I was looking through the "Guile emacs thread (again)" from 2014; it didn't seem to me that the Emacs maintainers were so negative about the proposal at the time.

However, if I understand correctly, the elisp support was never fully merged into Guile, and since then there've been major changes, such as Guile 3.0, and Emacs with native compilation and concurrency. Might be hard to resurrect.

@loke @technomancy @mala

@mpjgregoire @loke @technomancy @mala It's been a while, my memory might not be so good. but I remember being an interesting article.

@mpjgregoire @loke @technomancy @mala Stefan Monnier, who was either maintainer then or maintainer sometime near to that time period, was fairly against the idea. I think the project needed a swell of support from the core maintainers, and it didn't happen...

@cwebber @mpjgregoire @loke @mala I wonder how much of that was just CL vs Scheme factionalism. I feel like that factor was worse when CL and Scheme were the only two viable lisps and everyone felt like they had to be for one and against the other.

I read that LWN article. You remember better than me, I think: the Emacs maintainers didn't shut the door completely, but they saw lots of challenges of various levels of technical difficulty. Given that the implementation was slow and compiled with difficulty at the time, they didn't see much gain either.

Am I correct that no released version of Guile has added Robin Templeton's patches?

@loke @technomancy @mala

@cwebber @mpjgregoire @technomancy @mala I think the root problem is that 8 years after that article was written, the exact same problems are still there.

I believe that the only way this would move forward is either by having a fork which then happens to be so popular that the main Emacs branch loses momentum, or that the core maintainers of Emacs all resign are are replaced with people who really want it to happen.

The former is unlikely because even if it worked perfectly, there would be no reason for regular users to switch. So the new fork would have to bring something new in addition to the guile implementation. Perhaps I lack imagination but I don't see that happening.

The latter is even less likely, as there isn't a huge number of people out there capable of maintaining Emacs to begin with. If most of the current team resigned then Emacs development would probably stop, or at least slow down significantly, at which point it doesn't matter if the guile port becomes official or not, as nothing would really happen.

@technomancy @mala @cwebber Is the success worth it in other aspects, though? I can understand SerenityOS being more comfortable or technically interesting for people (even though my techncial knowledge is nowhere near as deep as that of the rest of you). But in the spirit of doing things anew and better this time, I would like to see the next successful system embrace and protect user freedom (and possibly do things in other fields more ‘right’), in addition to being technically excellent.

With regard to this, when I look at Serenity OS, I see that all of it is under a permissive licence, and that their main (and so much worse: only) communication platform appears to be Discord, and I find it very difficult to like it or root for it, since the projects shows little care for values I deem absolutely crucial.

@tirifto @technomancy @cwebber so specifically, i don't cite #serenityos as the OS you should get behind, but the possibility of OSes being a less significant decider, in the face of us directly tackling the complexity of interoperability. An enormous amount of work went into this in the early *nix days in an attempt to knit together a movement from incompatible unixen

@tirifto @technomancy @cwebber ultimately, that cross-compatibility burden led to consolidation; you shoild absolutely be able to choose your OS in the face of your ethics, but others will disagree with your interpretation of even similiar goals, and fracture effort. We need to work on how to let us work and connect together anyway, and provide your choice in a wider universe of possibilities

Sign in to participate in the conversation

The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!