After much rumour about an Apple Web Crawler, Apple has finally posted a help document approving the presence of AppleBot.
Apple said, Applebot is the web crawler for Apple. AppleBot is “used by products including Siri and Spotlight Suggestions,” the company said.
The user-agent will usually follow the following string but will contain “Applebot” in it always:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1)
Apple declares it will respect the usual robots.txt guidelines and robots meta tags. AppleBot currently originates in the 17.0.0.0 net block. If you don’t mention AppleBot in your robots.txt directive, Apple will follow what you mention for Googlebot. So if you want to block AppleBot and GoogleBot, you can just block GoogleBot, but it is recommended to block each independently.
If you figure our some unfamiliar AppleBot activity, you can always reach Apple about it at Apple-NOC “at” apple.com.
It is still uncertain if Apple plans on constructing and challenging with Google on search, but this is one step closer to that.
Follow @socialentellus
No comments:
Post a Comment