I understand that lemmy doesn’t have this feature due to it being written in rust
The language has nothing to do with this. The way search engines work is they crawl the web and index everything they find. Then when you do a query for certain terms they use their magic sauce algorithm to find results they believe are most relevant and present them to you.
Reddit is a convenient search term for three reasons:
it has predictable page locations, making it easy to crawl,
it has a lot of cross-links meaning it bubbles up to the top of search results easier, and
it gets crawled often because its content changes very fast.
So for Lemmy to have equivalent indexability it needs lots of links, frequent changes, and it needs to have a predictable method for discovering content via crawling. The first two are entirely based on users - the more users contribute the more content gets posted, more comments, are made, and more links to the content show up organically on other sites. The last one is trickier - being federated Lemmy content cannot be crawled in the same way as standard websites. However the fediverse does have a deterministic way of getting content from federated servers, so it could well be that we’ll see the search engines create their own federation server to slurp up content instead of crawling the sites.
The language has nothing to do with this. The way search engines work is they crawl the web and index everything they find. Then when you do a query for certain terms they use their magic sauce algorithm to find results they believe are most relevant and present them to you.
Reddit is a convenient search term for three reasons:
So for Lemmy to have equivalent indexability it needs lots of links, frequent changes, and it needs to have a predictable method for discovering content via crawling. The first two are entirely based on users - the more users contribute the more content gets posted, more comments, are made, and more links to the content show up organically on other sites. The last one is trickier - being federated Lemmy content cannot be crawled in the same way as standard websites. However the fediverse does have a deterministic way of getting content from federated servers, so it could well be that we’ll see the search engines create their own federation server to slurp up content instead of crawling the sites.