sunaweb is a web crawler (web spider) that has been designed and developed to work within the Site Update Notification Agent framework. It identifies changes made to external websites that provide data to be displayed on a customer's site, for example in a mashup system.
Why are you crawling my site?
The sunaweb web crawler is currently being tested and used in the research of web search engine technologies. There are a number of reasons why sunaweb may be visiting you.
-
The most likely reason for sunaweb visiting your site is that a user of the change notification service has registered your site for us to monitor and provide a notification when your site is updated.
-
Your site is used as a data supplier to a mashup system which is being used in one of the above scenarios.
-
One of the alpha testers of the Site Update Notification System has configured their test environment to use your site. This is may be because your site uses some unusual techniques in its source.
How do I stop sunaweb visiting my site?
sunaweb adheres to the Robot Exclusion Standard and should not be invasive to your web site, or cause any problems for you. If it does, then please e-mail us and we'll investigate the problem.
We would like to continue to crawl your site, but respect your decision if you would prevent not to let sunaweb access it. To prevent sunaweb crawling your site, add the following to your robots.txt:
User-agent: sunaweb
Disallow: /