Furbland@lemmy.world to

196@lemmy.blahaj.zone · 3 months ago

rulebots.txt

309

rulebots.txt

Furbland@lemmy.world to

196@lemmy.blahaj.zone · 3 months ago

Chat

TriflingToad@lemmy.world
link
fedilink
arrow-up
3·
2 months ago
really? ddg will show me reddit links, did they have to make a webscraper or something
- morgunkorn@discuss.tchncs.de
  link
  fedilink
  arrow-up
  4·
  2 months ago
  There’s a cutoff date, anything indexed before the robots.txt was changed stays in the index

196@lemmy.blahaj.zone

196@lemmy.blahaj.zone

You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !196@lemmy.blahaj.zone

Be sure to follow the rule before you head out.

Rule: You must post before you leave.

Visibility: Public

This community can be federated to other instances and be posted/commented in by their users.

2.06K users / day
4.7K users / week
9.46K users / month
20.9K users / 6 months
1 local subscriber
16.4K subscribers
16.8K Posts
155K Comments
Modlog