pathetic piss kitten (@kitten)

pathetic piss kitten @kitten@elizabeth.cat

10mo

TIL robots.txt was standardized only 3 years ago

1❤️1

4

1

8

2

Summer Jo

☀️

@ShadowJonathan@tech.lgbt

10mo

cynicism

@kitten and just in time for it to get ignored en-masse by AI scrapers

❤️1

0

1

0

1

Alba 🌸

@mildsunrise@tech.lgbt

10mo

@kitten and the X-Robots-Tag, which is the thing people should actually be using instead of robots.txt, isn't standardized yet :(

0

1

0

Ry @Ry@meow.social

10mo

@kitten TIL robots.txt was standardized

0

1

0

~: Liera

@FamiliarDraco@evy.pet

10mo

@kitten oh yeah indeed RFC is only 3 years old. Would have expected it to be standardized earlier given how long robotstxt.org has been a thing. Guess people have just been rocking with their draft up until then

0

1

0