Ahmed FASIH is a user on octodon.social. You can follow them or interact with them if you have an account anywhere in the fediverse.
If you don't, you can sign up here.
Them: "How have you dealt with data that's too big to fit comfortably in memory?"
Me: "I ask the CEO to buy me a 1 TB RAM workstation."
This exchange is a bit embarrassing but I don't think I can be condemned as practicing bad engineering without a lot of careful thinking.
Also, 1 TB RAM makes many of our most important datasets "small data" again and it is So SO GOOD to be able to spend my time on real engineering instead of big data linux bullshittery.
