Akwụkwọ ndekọ ederede
Baidu Spider weghara ozi ewepu nchoputa: Kedu ihe m ga-eme ma ọ bụrụ na oghere na-agụ ma dee na-ezighi ezi?
Na-eche na Baidu edepụtaghị aha webụsaịtị gị, ị ga-ebu ụzọ mee nchọpụta nrụrụ udide n'elu ikpo okwu ọchụchọ Baidu.
Kedu ihe m ga-eme ma ọ bụrụ na Baidu crawler emeghị njikọ nyocha?
Ọ bụrụ na nchoputa crawler crawler daa ọtụtụ oge, firewall nwere ike igbochi mmemme crawler.
Ọchụchọ ihe enyemaka Baidu > Nchọpụta Nchọpụta > Ozi Nwepu Crawl: Socket gụọ na dee mperi ▼
- Karịsịa mgbe ị na-eji Cloudflare CDN, a na-egbochi ya na ndabara.
- Na ịntanetị, a na-ekwu na ọ ga-agbakwunye adreesị IP
xxx.xxx.xxx.xxx/24 - Agbanyeghị, nwara nke ahụ enweghị isi.
Anaghị m egbochi spiders Baidu na sava ahụ, yabụ nsogbu kwesịrị ịbụ WAF Cloudflare!
Banye na Cloudflare → Nchekwa → WAF → Iwu Firewall → Mepụta Iwu Firewall
- Chọta iwu WAF metụtara crawlers na Cloudflare, wee chọta nhọrọ nke "robot crawler ziri ezi" ▼
- Mgbe ịmepụtara iwu firewall, chere maka nkeji 10, wee jide nchoputa ahụ, wee jide ha niile nke ọma!
Kedu ihe na-eme na Baidu crawler Sitemap ọdịda na njedebe oge njikọ?
Ọ bụrụ na ị nyefere adreesị faịlụ saịtị saịtị n'elu ikpo okwu ọchụchọ Baidu, a ga-enwe nsogbu dị ka ọdịda ịrara arị na njedebe oge njikọ ▼
Ngwọta maka ọdịda nke crawler Baidu ijide maapụ saịtị ahụ
Banye na Cloudflare → Nche → WAF → Iwu Firewall → Mepụta Iwu Firewall ▼
- ubi, họrọ "Onye ọrụ Agent"
- onye ọrụ, họrọ Nwere
- Tinye onye ọrụ ọhụrụ, pịa "Ma ọ bụ" ikpeazụ.
- Uru, n'otu n'otu tinye onye ọrụ Baidu Spider UA ndị a:
-
Baiduspider/2.0 -
Baiduspider-image -
Baiduspider-render/2.0 -
http://www.baidu.com/search/spider.html -
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) -
Mozilla/5.0 (Linux;u;Android 4.2.2;zh-cn;) AppleWebKit/534.46 (KHTML,like Gecko) Version/5.1 Mobile Safari/10600.6.3 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html）
Mgbe emechachara, nwalee mbubata ahụ ọzọ, nsonaazụ ga-eweghachite nkụnye eji isi mee HTTP 200, na-egosi na mbute ahụ na-aga nke ọma▼
-
抓取诊断 > 抓取详情以下是百度Spider抓取结果及页面信息： -
提交网址： https://www.etufo.org/sitemap_baidu.xml -
抓取网址： https://www.etufo.org/sitemap_baidu.xml -
抓取UA： Mozilla/5.0 (compatible; Baiduspider/2.0; -
+http://www.baidu.com/search/spider.html) -
抓取时间： 2022-11-11 19:03:44 -
网站IP： 172.***.***.149 -
下载时长： 0.868秒 -
返回HTTP头：HTTP/2 200
Ndị ọrụ nke spiders na crawlers ndị ọzọ nwekwara ike ịchọ onwe ha n'otu ụzọ ahụ.
Hope Chen Weiliang Blog ( https://www.chenweiliang.com/ ) kekọrịtara "Baidu Spider Crawl Failure Diagnosis Diagnosis Adịghị Eke Ozi Ihe ị ga-eme ma ọ bụrụ na agụ Socket wee dee njehie agwụla", nke na-enyere gị aka.
Nnọọ ka ị kesaa njikọ nke akụkọ a:https://www.chenweiliang.com/cwl-29315.html
Nabata na ọwa Telegram nke blọgụ Chen Weiliang ka ị nweta mmelite kachasị ọhụrụ!
📚 Ntuziaka a nwere nnukwu uru, 🌟Nke a bụ ohere dị ụkọ, echefula ya! ⏰⌛💨
Kekọrịta na-amasị ma ọ bụrụ na-amasị gị!
Ịkekọrịta na mmasị gị bụ mkpali anyị na-aga n'ihu!