Littafin Adireshi
Baidu Spider yana ɗaukar bayanan keɓanta bincike: Menene zan yi idan soket ya karanta kuma ya rubuta ba daidai ba?
Tsammanin cewa Baidu bai haɗa gidan yanar gizon ku ba, dole ne ku fara aiwatar da bincike na rarrafe gizo-gizo akan dandalin neman albarkatun Baidu.
Me zan yi idan mai rarrafe na Baidu ya kasa jan rarrafe hanyoyin bincike?
Idan binciken rarrafe na Baidu ya gaza sau da yawa, mai yiwuwa bangon wuta ya toshe shirin crawler.
Baidu Neman Dandali Hanyar Hanya > Ganewar Garkuwa > Bayanin Banbancin Rarrafe: Kurakurai karanta da rubuta soket ▼
- Musamman lokacin amfani da Cloudflare CDN, ana toshe shi ta tsohuwa.
- A Intanet, an ce don ƙara adireshin IP
xxx.xxx.xxx.xxx/24 - Duk da haka, gwada hakan bai yi nasara ba.
Ban toshe gizo-gizon Baidu akan sabar ba, don haka matsalar yakamata ta zama WAF na Cloudflare!
Shiga zuwa Cloudflare → Tsaro → WAF → Dokokin Wuta → Ƙirƙiri Dokar Wuta
- Nemo ka'idodin WAF masu alaƙa da masu rarrafe akan Cloudflare, kuma sami zaɓi na "madaidaicin ɗan rago na robot" ▼
- Bayan ƙirƙirar ka'idodin Tacewar zaɓi, jira na minti 10, sannan a kama ganewar asali, kuma an yi nasarar kama su duka!
Menene ke damun Baidu crawler gazawar taswirar rukunin yanar gizon da ƙarewar haɗin gwiwa?
Idan ka ƙaddamar da adireshin fayil ɗin taswirar gidan yanar gizon akan dandalin neman albarkatu na Baidu, za a sami matsaloli kamar gazawar rarrafe da ƙarewar haɗin gwiwa ▼
Magani ga gazawar Baidu crawler don ɗaukar taswirar rukunin yanar gizon
Shiga zuwa Cloudflare → Tsaro → WAF → Dokokin Wuta → Ƙirƙirar Dokokin Wuta ▼
- filin, zaɓi "Agent User"
- afareta, zaɓi Ya ƙunshi
- Ƙara sabon wakilin mai amfani, danna "Ko" na ƙarshe
- Darajar, bi da bi shigar da wakilin mai amfani na Baidu Spider UA mai zuwa:
-
Baiduspider/2.0 -
Baiduspider-image -
Baiduspider-render/2.0 -
http://www.baidu.com/search/spider.html -
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) -
Mozilla/5.0 (Linux;u;Android 4.2.2;zh-cn;) AppleWebKit/534.46 (KHTML,like Gecko) Version/5.1 Mobile Safari/10600.6.3 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html）
Bayan an gama, sake gwada ɗauko, kuma sakamakon ya dawo HTTP header 200, yana nuna cewa an yi nasara.
-
抓取诊断 > 抓取详情以下是百度Spider抓取结果及页面信息： -
提交网址： https://www.etufo.org/sitemap_baidu.xml -
抓取网址： https://www.etufo.org/sitemap_baidu.xml -
抓取UA： Mozilla/5.0 (compatible; Baiduspider/2.0; -
+http://www.baidu.com/search/spider.html) -
抓取时间： 2022-11-11 19:03:44 -
网站IP： 172.***.***.149 -
下载时长： 0.868秒 -
返回HTTP头：HTTP/2 200
Wakilan masu amfani da wasu gizo-gizo da masu rarrafe suma suna iya nemo kansu ta hanya guda.
Hope Chen Weiliang Blog ( https://www.chenweiliang.com/ ) shared "Baidu Spider Crawl Failure Diagnosis Diagnosis Inal Information Abin da za a yi idan an karanta Socket da Rubuta Kuskuren Haɗin Kan Kuskure", wanda ke taimaka muku.
Barka da zuwa raba hanyar haɗin wannan labarin:https://www.chenweiliang.com/cwl-29315.html
Barka da zuwa tashar Telegram na Chen Weiliang's blog don samun sabbin abubuwa!
📚 Wannan jagorar ya ƙunshi ƙima mai yawa, 🌟Wannan dama ce da ba kasafai ba, kar a rasa ta! ⏰⌛💨
Share da like idan kuna so!
Rarraba ku da abubuwan so sune ci gaba da ƙarfafa mu!