在之前的一篇\\”解决WordPress占用资源过大 导致主机商暂停账户的方法\\”一文中,蜗牛基本上比较全的通过5个解决方法解决了在我们建立站群的时候导致占用CPU资源过大,主机商暂停我们主机账户的问题。尤其是在使用类似ixwebhosting主机这类的站群主机的时候尤为的出现这样的问题较多。在我解决之后,发现又来TK给我说占用资源还很大。
于是官方给我的HTACCESS文件解决各种搜索引擎没有必要的蜘蛛限制,占用资源的问题。还好这次的暂停没有把所有的网站都暂停,而是暂停了其中的一个占用较大的站点,他告诉我FTP开着,让我通过修改HTACCESS文件。
RewriteEngine OnRewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]RewriteCond %{HTTP_USER_AGENT} ^Bot\\\\ mailto:[email protected] [OR]RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]RewriteCond %{HTTP_USER_AGENT} ^Download\\\\ Demon [OR]RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR]RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR]RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]RewriteCond %{HTTP_USER_AGENT} ^Express\\\\ WebPictures [OR]RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR]RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR]RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR]RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR]RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR]RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR]RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR]RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR]RewriteCond %{HTTP_USER_AGENT} ^HMView [OR]RewriteCond %{HTTP_USER_AGENT} HTTrack [NC,OR]RewriteCond %{HTTP_USER_AGENT} ^Image\\\\ Stripper [OR]RewriteCond %{HTTP_USER_AGENT} ^Image\\\\ Sucker [OR]RewriteCond %{HTTP_USER_AGENT} Indy\\\\ Library [NC,OR]RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR]RewriteCond %{HTTP_USER_AGENT} ^Internet\\\\ Ninja [OR]RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR]RewriteCond %{HTTP_USER_AGENT} ^JOC\\\\ Web\\\\ Spider [OR]RewriteCond %{HTTP_USER_AGENT} ^larbin [OR]RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR]RewriteCond %{HTTP_USER_AGENT} ^Mass\\\\ Downloader [OR]RewriteCond %{HTTP_USER_AGENT} ^MIDown\\\\ tool [OR]RewriteCond %{HTTP_USER_AGENT} ^Mister\\\\ PiX [OR]RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR]RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR]RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR]RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR]RewriteCond %{HTTP_USER_AGENT} ^Net\\\\ Vampire [OR]RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR]RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR]RewriteCond %{HTTP_USER_AGENT} ^Offline\\\\ Explorer [OR]RewriteCond %{HTTP_USER_AGENT} ^Offline\\\\ Navigator [OR]RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR]RewriteCond %{HTTP_USER_AGENT} ^Papa\\\\ Foto [OR]RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR]RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR]RewriteCond %{HTTP_USER_AGENT} ^RealDownload [OR]RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR]RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR]RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR]RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR]RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR]RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR]RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR]RewriteCond %{HTTP_USER_AGENT} ^Teleport\\\\ Pro [OR]RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR]RewriteCond %{HTTP_USER_AGENT} ^Web\\\\ Image\\\\ Collector [OR]RewriteCond %{HTTP_USER_AGENT} ^Web\\\\ Sucker [OR]RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR]RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR]RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR]RewriteCond %{HTTP_USER_AGENT} ^WebGo\\\\ IS [OR]RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR]RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR]RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR]RewriteCond %{HTTP_USER_AGENT} ^Website\\\\ eXtractor [OR]RewriteCond %{HTTP_USER_AGENT} ^Website\\\\ Quester [OR]RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR]RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR]RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR]RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]RewriteCond %{HTTP_USER_AGENT} ^Widow [OR]RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR]RewriteCond %{HTTP_USER_AGENT} ^Xaldon\\\\ WebSpider [OR]RewriteCond %{HTTP_USER_AGENT} ^ZeusRewriteRule ^.* - [F,L]
建议我们把上面的文件添加到.htaccess文件最下面。具体的内容我看的不是太懂,大概就是限制爬行一些目录和限制一些蜘蛛的爬行。如果有需要的朋友可以把这第六个方法作为处理解决方法。
评论前必须登录!
立即登录 注册