Twitterbot
Twitterbot/0.1
Twitter’s crawler respects Google’s robots.txt specification when scanning URLs. If a page with card markup is blocked, no card will be shown. If an image URL is blocked, no thumbnail or photo will be shown. Twitter uses the User-Agent of Twitterbot (with version, such as Twitterbot/1.0), which can be used to create an exception in the robots.txt file.
What the bot does:
Metrics & Site Analytics
Content Scraping
Who runs the bot:
Social Media, Chat, or Shorteners
Content Aggregator & RSS
Is this bot visiting your website?
Sign up to find out!