User-agent: * crawl-delay: 30 User-agent: Yandex Disallow: / User-agent: spbot Disallow: /