Abu Godot is a user on octodon.social. You can follow them or interact with them if you have an account anywhere in the fediverse.
If you don't, you can sign up here.
So do search engines just take a long time to update based on robots.txt? I set up a "block everything" robots.txt like 9 hours ago but just checked Google, Bing, and DuckDuckGo and a search for "site:toot.cafe" is still showing content on all three. Should I just be patient? 😕 Pretty sure my robots.txt is good: https://toot.cafe/robots.txt
@nolan They'll only see your robots.txt when they recrawl the site. So it can definitely take some time. In urgent cases you can use Google Webmaster Tools (now called Search Console) to ask them to remove specific URLs temporarily, until the robots.txt kicks in.
