• VonReposti@feddit.dk
    link
    fedilink
    English
    arrow-up
    17
    ·
    10 days ago

    They’re spamming all web logs too with an advertisement for their services in the user agent. I decided to ban them from all my websites because the logs took up too much space.

      • VonReposti@feddit.dk
        link
        fedilink
        English
        arrow-up
        5
        arrow-down
        2
        ·
        10 days ago

        They’re saying who they are, what they do, and are linking to their website and sometimes sends hundreds of requests in a minute. It might not say "For only €49.99 you can get your very own thing!”, but that does not mean they aren’t throwing their name up in every website owner’s arse whether they like it or not.

          • VonReposti@feddit.dk
            link
            fedilink
            English
            arrow-up
            6
            arrow-down
            1
            ·
            10 days ago

            That could have been done by just having a single entry called GoogleBot or BingBot, not an entire sentence explaining their product offering let alone hundreds of times a minute.