Bots and Google

[Rich]

While reading a response to a post I made, I decided to do
a search to see if the Google bots had catalog it yet.
These were the first two entries that popped up:


None of the following entries related to Tinycore. But when
I reached the bottom of the page, I was a little surprised
to find this:


Seven out of eight suggestions referred to Tinycore.

This was the post I made:
https://forum.tinycorelinux.net/index.php/topic,27848.msg180072.html#msg180072

[Paul_123]

I guess that's something for all that scraping that gets done.   Its a never ending battle to keep them tamed.  Some of it is good, it would just be nice if there were not hundreds of bots doing the same damn thing.  Just scanning the logs right now,  ClaudeBot, PetalBot, GoogleBot, AmazonBot, bingbot, ThinkBot, AwarioBot, DuckAssistentBot, GPTBot are all active.

Most do respect the robots.txt directive I have to limit rates.  I did tell Bytedance and TikTok Bots to knock it off......and so far they have respected the robots.txt