@Canageek Meh, I understand the reasoning but yeah, it can be a bit of a pain to find anything again.

@craigmaloney I don't? What is it?

@Canageek UNfortunately full-text searching can be used to find certain posts to harass folks. It's a protective measure.

@craigmaloney .... That is really dumb. You can't enforce it, as evidenced by the fact Twidere has it built in. So harassers can still do it, but it inconveniences everyone, and the is no way to tell if people are doing it. The are commend line clients- log the federated timeline, then run grep on that.

@Canageek @craigmaloney You wouldn't even need to write or use an actual Mastodon client. Masto publishes a user's activity as an Atom feed; other instances subscribe to this feed to get their toots. But anyone who wants to can also subscribe to it, and then all you need is a routine for parsing Atom (not hard, it's just namespaced XML) and dumping the content into a flat file or database or whatever. Put that on a cron job and boom, you're done.

@jalefkowit @Canageek @craigmaloney i don't think anyone has the impression it's impossible

but it doesn't have to be impossible for there to be benefit

if it is more of a pain in the ass it will almost certainly happen less often, and that's a good thing

@bea @jalefkowit @craigmaloney Nah. It will happen less often until someone writes a tool for it, then it will be common, and we still won't have full text search.

@Canageek @jalefkowit @craigmaloney that will be an unfortunate day!

when it comes though i hope i'm well enough to take pleasure from writing a patch that will fuck that tool up~

@bea @jalefkowit @craigmaloney As it has been explained to me, can't be done. Just download the timeline atom feed to your PC then run a search on it there.

@Canageek @jalefkowit @craigmaloney for now? sure

not hard to change the way that works to make it hostile to an indexer though

probably can detect such an indexer as well

like i said, i hope when the time comes i feel well enough to enjoy the challenge

other instances can do what they will including have fulltext search

but i won't have my instance being indexed for the benefit of anyone but my users i think

@bea @jalefkowit @craigmaloney So you'd have to break federation then, right?

@Canageek @jalefkowit @craigmaloney mm no, i'm pretty sure it can be avoided

@bea @jalefkowit @craigmaloney Might want to talk to @craigmaloney about they? Individual user feeds are public atom based on the one he just sent me

@Canageek @jalefkowit @craigmaloney craig has been tagged in this whole conversation...

@bea @jalefkowit @craigmaloney Install one if the commend line clients. Log all toots in the federated timeline to disk. Run grep on it?

I'm told it is even easier with atom feeds?

@Canageek @jalefkowit @craigmaloney if you have a well behaved client or act like a well behaved client you miss all the data from the past and have to process everything at realtime

i think for anyone who wants to be an asshole this is already a pretty serious setback

@bea @jalefkowit @craigmaloney .... Yeah, this the logging to disk. Then once you have a log run grep? What an I missing?

@Canageek @bea @craigmaloney If you can't get the data for the entire instance out of it easily, it's harder to identify individual targets within the herd for harassment.

(If you already have a target identified, you're already past the point where anything related to search could stop you.)

@jalefkowit @bea @craigmaloney Doesn't Mastodon already provide that? Isn't that what the federated timeline is? I mean, I'm looking at it. If text arrives on my computer the must be a way to log and then search it.

@Canageek @jalefkowit @craigmaloney sure but then you have only the data for the present and the future

when to identify individuals for some nefarious purpose you want data from the past

@craigmaloney @jalefkowit @Canageek also this is not quite the whole story because the local and federated timelines contain only public posts

@bea @jalefkowit @craigmaloney Oh sure, that would be harder. But a) If you set it up now and have a big enough hard drive in ten years you'll have close enough all the past data, and b) I would be fine with real time text search. I want to find people talking about role-playing games now, not last year....

Plus, wouldn't you just set your toots to not appear on local timeline of you wanted to be private?

@Canageek @bea @craigmaloney It arrives on your computer because you've manually created an account and used it to manually follow a bunch of people.

A harasser _could_ create an account on an instance, then write a bot to log in periodically and scrape the local timeline. That would take some work, but not much.

Admins could theoretically watch for accounts that behave like bots and close them. But then you have to define what "bot like" means, etc.

@jalefkowit @bea @craigmaloney That is more of a barrier. I can string cli tools into bash files but not do anything web based.

Still, the fact there is an emacs client means it wouldn't be hard to capture the public timeline, right?

@jalefkowit @bea @craigmaloney (Also, if anyone has suggesting on how to archive public toots that match a hashtag let me know, want to run one before a fiction event so I can make a log of it)

@jalefkowit @craigmaloney So do I just need to know the URL of that feed? What is the format for that? I'd like to play around with this. Would be useful for a fiction event I have planned.

@Canageek @craigmaloney Yeah, here's mine:

https://octodon.social/users/jalefkowit.atom

@craigmaloney @Canageek Note also that Mastodon's user profile pages have feed autodiscovery meta tags, so if you hit them using an autodiscovery-capable browser you can pipe the user's feed to your feed reader with a single click.

@jalefkowit @craigmaloney Is the one for the local timeline in general?

@Canageek @craigmaloney I'm not sure there is a separate feed for the local timeline, as that doesn't get federated across instances. You'd need to check the Mastodon source to confirm though.

@jalefkowit @craigmaloney Ah, so no way to get all the messages without manually downloading every atom feed?

@jalefkowit @craigmaloney So how are the local and federated feeds created? They just take all those feeds and mush then together?

@Canageek @craigmaloney This question goes deeper into Mastodon internals than I can speak to from experience, so I can't say for sure.

My _guess_ is that the local and federated feeds are generated dynamically by Masto for each user, based on their follow list. i.e. they don't exist in Atom format themselves; Masto takes the Atom feeds for all the users that user follows and them mashes them together.

@jalefkowit @craigmaloney Doesn't every Mastodon user see the same local feed? I thought that was just every non-private toot on the instance?