Sophie

Sophie

distrib > Fedora > 16 > i386 > by-pkgid > e344e5e676109c917d477821b5e1ee26

perl-WWW-RobotRules-6.01-3.fc16.src.rpm

Description:

This module parses /robots.txt files as specified in "A Standard for Robot
Exclusion", at <http://www.robotstxt.org/wc/norobots.html>. Webmasters can
use the /robots.txt file to forbid conforming robots from accessing parts
of their web site.

Generated packages:

Other version of this rpm: