User-agent: * Disallow: /cgi-bin/ #do not spider GET parameters Disallow: /*? Crawl-delay: 120