I'm looking at GHCN and getting back to the fractional Laplacian texture-shaded terrain elevation map (https://fasiha.github.io/post/texshade/) because my family needs a proper app for teaching and exploring and writing about geography!
My first project that I'm seriously thinking about monetization from the get-go. How scary!
I see ~4000 weather monitoring stations in #OpenStreetMap:
I can even see a static map of them.
How do I download all of them?
The Global Historical Climate Network (https://www.ncdc.noaa.gov/ghcn-daily-description) currently has 113'951 weather stations, and I might add them but.
Working with GHCN data gave me an idea: let's write a course, Data Wrangling Katas, containing a variety of increasingly-complicated exercises that look at various aspects of data parsing, cleaning, analysis, etc.
"MD5 checksums are available for each file in x-amz-meta-md5sum header field. You can check it with curl --head url."
Lol, it's fun and interesting to see a made-for-developers app! You certainly can't say anything so cryptic and incomplete to a general audience :P
Also TIL AWS gives an easy way to do md5 checksum of downloads (as long as your DNS isn't attacked).
“We never store a copy of this common key without encryption and don't use any escrow mechanism to recover your encrypted data. This means that if you forget your login password, we cannot recover your data and we can't even reset your password.”
I'll try it, but right now I hate this. I'm much less anxious on Mastodon than Secure Scuttlebutt because of "edit and redraft". I don't think this "can't reset" will catch on, but maybe password manager is the crucial part.
Object.entries, Object.assign, Array.map… I'm creating and destroying arrays left and right, which throw a lot of performance on the floor. The perf-conscious alternative is not very expressive: to write nested for-loops.
‘Wherever Protestant English men put down family roots, agricultural production for global markets on lands held as private property became the key to prosperity and the measure of masculine self-rule. Anyone who stood in the way of these men—Native Americans, landless European newcomers, obstreperous servants and slaves, people of differing religious views, would-be English royal governors—did so at their peril.’ —Richter, "Before the Revolution"
The history that we're familiar with, even if a bit more complete than usual, frequently has these deeply-relevant but unknown antecedents.
I remember first seeing that WW1's prefiguring of WW2 was mirrored in the way the Franco-Prussian War prefigured WW1, and again how the Napoleonic wars prefigured the Franco-Prussian War, and …
Much of the American founding fathers' understandings came from Charles and Cromwell.
Columbus' background in Genoese slavery easily led to Caribbean slavery.
(cont.) ‘Honed in late medieval campaigns in places such as Moorish Granada and Celtic Ireland, exported to such islands as the Canaries and Azores, perfected in the Spanish conquests of Hispaniola, Mexico, and Peru, and brought to fruition in disastrous English efforts to colonize at Roanoke and Jamestown, patterns of ruthless violence, enslavement, and oppressive rule of indigenous peoples laid an ugly base for all that would follow.’ —Richter, Before the Revolution
‘In crusades sponsored by monarchs assembling primitive states, driven by freelance warriors seeking power and wealth, and justified by zeal for spreading what was called the True Faith, Western Europeans spilled out into the Atlantic basin. The exploits of these conquistadores—Protestant & Catholic—owed more to the loyalties and beliefs of the Middle Ages than to anything we would recognize as “modern” or “capitalistic,” although they nonetheless contained the germs of those later traits’
“a period of global warming … Beginning in the decades after the year we now call 900 C.E., strikingly parallel agricultural revolutions produced strikingly different civilizational forms in North America and Western Europe. Distinctive systems of power and authority, of family and kinship, of religion and spirituality, of production and exchange took shape on the two continents … as the climate cooled in the 14th century, each system began unraveling…”
—D. Richter, “Before the Revolution”
After talking about the anxiety following small initial successes, worrying about that success evaporating, an anxiety that never went away:
“I decided to think of this anxiety about the future as a proof that I’m really doing what I want to do.”
Very useful insight from Takuya Matsuyama. https://blog.inkdrop.info/grit-46c57af01d6d
Kids ask the most interesting thing since. And that means we can ask Siri the darndest questions.
How many people are born every second? 4.3 Hz.
How many people die every second? 1.8 Hz.
Until an edict from the Prime Minister’s office in 2016 requiring all companies with more than 300 employees release numbers on female participation and pay, nobody had any idea how bad it was. The government requirements for transparency shone a light there, and the concomitant shame has been part of why some improvements have been happening.
Kathy Matsui of Goldman Sachs coined the phrase ”Womenomics”. Surprising findings:
Japan female labor participation is higher than US.
Accounting for all the sources of foreign labor that constitute Japan’s stealth immigration policy, Japan has higher foreign labor participation than Germany, and almost as high as the US.
In OECD and among Japanese prefectures, female labor participation is POSITIVELY correlated with fertility.
Funny story, I began reading the Iliad in college out of an effort to honor the American Founding Fathers, most of who apparently studied Greek and Roman classics, and during discussion with my roommate who studied Greek in high school he lets drop “Achilles kills Hector” and I’m outraged, “Dude, spoiler much???”
He argued that you can’t spoiler a three thousand year old epic.
But I disagreed because, well, when exactly does the statute of limitations for spoilers expire? 100 years? 1000?! 😂
Achilles’ face when he’s trying to outrun the enraged river god Scamander trying to drown him, when he cries out that he was promised a death at the walls of Troy by Apollo’s shining darts and not like a clumsy boy slipping into a mountain stream, when the ocean god Poseidon shows up and assures him, “you need not fear death by drowning, I promise you”! 😼
I’m loving this graphic novelization of the #Iliad by Hinds. My fave English translation remains Stanley Lombardo but this is delightful.
I've had a slow-burn project to publish 95% daily VaRs on # of visitors to (or editors on) various Wikipedias daily.
I want to use Gaussian processes (GPs) https://scikit-learn.org/stable/modules/gaussian_process.html because they can provide probability estimates, allowing me to find the 95% VaR.
But they didn't have GPs in the 1990s right??
https://www.value-at-risk.net/value-at-risk-inference-procedures/ says just run a regression and assume Normal standard errors to get percentiles but, like, ew, gross.
Brown says they used sports betting tools…?