NewsHuntBot is the news crawler of NewsHunt, a news search and reading service. It reads the latest articles from news websites so that NewsHunt can show the headline, a short summary and a link back to the publisher.
NewsHuntBot/0.1 (+https://newshunt.io/bot)Signature-Agent: "https://newshunt.io". Our public key is published at /.well-known/http-message-signatures-directory. Crawl-delay, that delay is used instead, however long it is.NewsHuntBot (case does not matter), only that group is used. Otherwise the * group is used.Allow and Disallow: the longest matching rule wins. When an Allow and a Disallow rule are the same length, Allow wins.* matches any characters and $ marks the end of the URL.Crawl-delay (in seconds) is read from the group that applies, as described above.Retry-After time if it is longer.Retry-After (seconds or an HTTP date) for every request to that host. Without it we wait at least 60 seconds after a 429.Add this to your robots.txt:
User-agent: NewsHuntBot Disallow: /
Questions or problems: please contact us.
Newshunt ले तपाईंलाई साइन इन राख्न र तपाईंको भाषा र देश सम्झन आवश्यक कुकीहरू प्रयोग गर्दछ, ताकि साइट तपाईंले अपेक्षा गरेअनुसार काम गरोस्। तपाईंको अनुमतिमा, हामी Newshunt लाई मानिसहरूले कसरी प्रयोग गर्छन् भन्ने बुझ्न र समयसँगै सुधार्न एनालिटिक्स कुकीहरू पनि प्रयोग गर्न चाहन्छौं।
स्वीकार गर्नाले एनालिटिक्समा मात्र असर पार्छ। थप जान्नका लागि, हाम्रो हेर्नुहोस् गोपनीयता नीति or नियम र शर्तहरू.