Yandex
YandexBot is the primary web crawler for Yandex, Russia's dominant search engine serving over 60% of Russian-speaking internet users.
Current Rank
#8
Traffic Share
3.01%
Operator
Yandex
Category
Search Engine Crawler
YandexBot is the web crawler for Yandex, the most popular search engine in Russia and the CIS region. It indexes web pages to power Yandex Search, which serves hundreds of millions of queries daily.
Yandex also uses YandexBot data to power its AI assistant Alice and various other Yandex services, including Yandex Browser and Yandex News.
YandexBot has been active for over two decades and is known for being one of the more aggressive crawlers, which has led some webmasters outside of Russia to block it.
To block YandexBot from crawling your site, add the following to your robots.txt:
robots.txt
User-agent: YandexBot Disallow: /
Blocking YandexBot will remove your pages from Yandex Search results. If you have an audience in Russia or CIS countries, this may significantly impact your traffic.
User-Agent string: Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)
Respects robots.txt and supports Crawl-delay directive
IPs can be verified via reverse DNS (*.yandex.com, *.yandex.net, *.yandex.ru)
Supports JavaScript rendering
Crawl rate configurable in Yandex Webmaster Tools
YandexBot is primarily important for visibility in the Russian-speaking market. If your target audience includes Russia, Belarus, Kazakhstan, or other CIS countries, Yandex Search is a critical traffic source.
For sites that don't target the Russian market, YandexBot's crawling primarily contributes to server load without significant SEO benefit. Many webmasters outside of Russia choose to block or rate-limit it.
LLM Pulse monitors how AI models mention your brand. See which crawlers visit your site and how that translates into AI visibility.