you want to block crwlers from accessing your entire website what robots txt entry would you use 48784

You want to block crwlers from accessing your entire website. What robots.txt entry would you use?

by

Advertisement

  • User-agent: Allbots Disallow: * “` ““
  • User-agent: / Disallow: * “` ““
  • User-agent: * Disallow: “` ““
  • User-agent: * Disallow: / “` ““
Join our list

Subscribe to our mailing list and get interesting stuff and updates to your email inbox.

Thank you for subscribing.

Something went wrong.

We respect your privacy and take protecting it seriously

Leave a Comment

Share via
Join our list

Subscribe to our mailing list and get interesting stuff and updates to your email inbox.

Thank you for subscribing.

Something went wrong.

We respect your privacy and take protecting it seriously

Send this to a friend