Robots.txt

N'isiokwu a, anyị ga-enyocha ọrụ bụ isi nke faịlụ robots.txt na ijikwa okporo ụzọ na weebụsaịtị, tụlee mkpa ọ dị na ọnụnọ ya, ma nye ndụmọdụ maka ịtọlite ya maka njikwa ntinye akwụkwọ dị irè. Na mgbakwunye, anyị ga-enyocha ihe atụ nke ojiji ntuziaka ziri ezi na faịlụ robots.txt wee nye ntuziaka maka otu esi elele izi ezi nke ntọala ya.

Ihe kpatara eji achọrọ Robots.txt

Robots.txt bụ faịlụ dị na sava saịtị ahụ na ndekọ mgbọrọgwụ ya. Ọ na-agwa robots search engine ka ha ga-esi nyochaa ọdịnaya nke akụrụngwa. Iji faịlụ a eme ihe nke ọma na-enyere aka igbochi ntinye akwụkwọ nke ibe ndị na-achọghị, na-echebe data nzuzo, ma nwee ike melite arụmọrụ SEO kachasị mma na visibiliti nke saịtị na nsonaazụ ọchụchọ. A na-eme nhazi nke robots.txt site na ntụziaka, nke anyị ga-eleba anya n'ihu.

Ịtọ ntọala na Robots.txt

Onye ọrụ-Agent

A maara ntuziaka bụ isi dị ka Onye ọrụ-Agent, ebe anyị debere isiokwu pụrụ iche maka robots. Mgbe achọpụtara okwu a, rọbọt ahụ ghọtara na e bu n'obi iwu ahụ kpọmkwem maka ya.

Tụlee ihe atụ nke iji User-Agent na faịlụ robots.txt:

User-Agent: *
Disallow: /private/

Ihe atụ a na-egosi na robots niile na-achọ (nke akara ngosi nọchiri anya ya).*") kwesịrị ileghara ibe ndị dị na /nkeonwe/ ndekọ.

Nke a bụ ka ntuziaka ahụ si dị maka rọbọtị ọchụchọ kpọmkwem:

User-Agent: Googlebot
Disallow: /admin/

User-Agent: Bingbot
Disallow: /private/

N'okwu a, na googlebot search robot kwesịrị ileghara ibe na /admin/ ndekọ, mgbe binbot kwesịrị ileghara ibe dị na ya anya /nkeonwe/ ndekọ.

Kwụsị

Kwụsị na-agwa robots ọchụchọ nke URL ga-awụpụ ma ọ bụ ghara ịdepụta na webụsaịtị. Ntuziaka a bara uru mgbe ịchọrọ izobe data nwere mmetụta ma ọ bụ ibe ọdịnaya dị ala ka nke injin ọchụchọ na-edepụta ya. Ọ bụrụ na faịlụ robots.txt nwere ntinye Kwụsị: /directory/, mgbe ahụ, a ga-agọnarị robots ịnweta ọdịnaya nke akwụkwọ ndekọ aha akọwapụtara. Ọmụmaatụ,

User-agent: *
Disallow: /admin/

Uru a na-egosi na robots niile kwesịrị ileghara URL ndị na-amalite na /admin/. Iji gbochie saịtị ahụ niile ka robot ọ bụla depụta aha ya, tọọ ndekọ ndekọ mgbọrọgwụ ka iwu:

User-agent: *
Disallow: /

ekwe ka

Uru "Kwe ka" na-arụ ọrụ megidere "Kwụsị": ọ na-enye ohere ka robots ọchụchọ nweta otu ibe ma ọ bụ akwụkwọ ndekọ aha, ọ bụrụgodị na ntuziaka ndị ọzọ dị na faịlụ robots.txt machibidoro ịnweta ya.

Tụlee ihe atụ:

User-agent: *
Disallow: /admin/
Allow: /admin/login.html

N'ọmụmaatụ a, akọwara na anaghị ekwe ka rọbọt banye na ya /admin/ ndekọ, ma e wezụga maka /admin/login.html ibe, nke dị maka indexing na nyocha.

Robots.txt na saịtị saịtị

Sitemap bụ faịlụ XML nwere ndepụta URL nke ibe na faịlụ niile dị na saịtị nke ngwa ọchụchọ nwere ike ịdepụta ya. Mgbe igwe nchọta na-abanye na faịlụ robots.txt wee hụ njikọ na faịlụ XML sitemap, ọ nwere ike iji faịlụ a chọta URL na akụrụngwa niile dị na saịtị ahụ. A kọwapụtara ntuziaka a n'ụdị:

Sitemap: https://yoursite.com/filesitemap.xml

A na-edobe iwu a na njedebe nke akwụkwọ ahụ na-ejikọtaghị ya na onye ọrụ-onye ọrụ ma na-edozi ya site na robots niile na-enweghị isi. Ọ bụrụ na onye nwe saịtị anaghị eji sitemap.xml, ọ dịghị mkpa ịgbakwunye iwu.

Ọmụmatụ nke Robots ahaziri ahazi.txt

Ịtọlite Robots.txt maka WordPress

N'akụkụ a, anyị ga-atụle nhazi nhazi maka WordPress. Anyị ga-enyocha igbochi ịnweta data nzuzo yana ikwe ka ịnweta ibe ndị bụ isi.

Dịka ngwọta dị njikere, ịnwere ike iji koodu na-esonụ:

User-agent: *
# Block access to files containing confidential data
Disallow: /cgi-bin
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/themes/
Disallow: /wp-login.php
Disallow: /wp-register.php
Disallow: /xmlrpc.php

# Allow access to the main site pages
Allow: /wp-content/uploads/
Allow: /sitemap.xml
Allow: /feed/
Allow: /trackback/
Allow: /comments/feed/
Allow: /category/*/*
Allow: /tag/*

# Prohibit the indexing of old versions of posts and parameterized queries to avoid content duplication or suboptimal indexing.
Disallow: /*?*
Disallow: /?s=*
Disallow: /?p=*
Disallow: /?page_id=*
Disallow: /?cat=*
Disallow: /?tag=*

# Include the sitemap (location needs to be replaced with your own)
Sitemap: http://yourdomain.com/sitemap.xml

Ọ bụ ezie na ntụziaka niile na-esonye na nkwupụta, ka anyị nyochaa n'ime nkwubi okwu.

Robots agaghị atụpụta faịlụ na akwụkwọ ndekọ aha nwere mmetụta.
N'otu oge ahụ, a na-ahapụ robots ịnweta isi peeji na akụrụngwa nke saịtị ahụ.
akwadoro mmachibido iwu na ịdepụta ụdị posts ochie yana ajụjụ ndị a na-akpachapụ anya iji gbochie mbịgharị ọdịnaya.
Egopụtara ebe saịtị saịtị ahụ maka ntinye aha ka mma.

Ya mere, anyị atụlewo ihe atụ n'ozuzu nke nhazi dị njikere, nke ụfọdụ faịlụ na-emetụ n'ahụ na ụzọ na-ezobe indexing, ma ndị isi akwụkwọ ndekọ aha na-enweta.

N'adịghị ka ọtụtụ CMS na-ewu ewu ma ọ bụ saịtị ederede ahaziri ahazi, WordPress nwere ọtụtụ plugins na-akwado ịmepụta na njikwa nke faịlụ robots.txt. Otu n'ime ihe ngwọta na-ewu ewu maka nzube a bụ Yoast SEO.

Iji wụnye ya, ịkwesịrị:

Gaa na panel nchịkwa WordPress.
Na ngalaba "Plugins", họrọ "Tinye ọhụrụ".
Chọta ngwa mgbakwunye "Yoast SEO" wee wụnye ya.
Tinye ngwa mgbakwunye.

Iji dezie faịlụ robots.txt, ị ga-emerịrị:

Gaa na ngalaba "SEO" na ngalaba nchịkwa nchịkwa wee họrọ "General".
Gaa na taabụ "Ngwaọrụ".
Pịa na "Faịlụ". N'ebe a, ị ga-ahụ faịlụ dị iche iche, gụnyere robots.txt.
Tinye iwu indexing dị mkpa dịka ihe ị chọrọ.
Mgbe ịmechara mgbanwe na faịlụ ahụ, pịa bọtịnụ "Chekwa mgbanwe na robots.txt".

Rịba ama na nhazi faịlụ robots.txt ọ bụla maka WordPress bụ ihe pụrụ iche ma dabere na mkpa na njirimara nke saịtị ahụ. Enweghị ndebiri zuru ụwa ọnụ nke ga-adabara akụrụngwa niile na-enweghị isi. Agbanyeghị, ihe atụ a na iji plugins nwere ike ime ka ọrụ ahụ dị mfe.

Ntọala akwụkwọ ntuziaka nke Robots.txt

N'otu aka ahụ, ịnwere ike ịtọ nhazi faịlụ gị ọbụlagodi na enweghị CMS dị njikere maka saịtị ahụ. Onye ọrụ ahụ kwesịkwara bulite faịlụ robots.txt na ndekọ mgbọrọgwụ nke saịtị ahụ wee kọwaa iwu ndị dị mkpa. Nke a bụ otu n'ime ọmụmaatụ, nke egosiri ntuziaka niile dị:

User-agent: *
Disallow: /admin/             # Prohibit access to the administrative panel
Disallow: /secret.html	      # Prohibit access to a specific file
Disallow: /*.pdf$	      # Prohibit indexing of certain file types
Disallow: /*?sort=	      # Prohibit indexing of certain URL parameters
Allow: /public/		      # Allow access to public pages
Sitemap: http://yourdomain.com/sitemap.xml # Include the sitemap

Otu esi elele faịlụ Robots.txt

Dị ka ihe enyemaka mgbe ị na-elele faịlụ robots.txt maka njehie, a na-atụ aro ka iji ọrụ ịntanetị.

Tụlee ihe atụ nke Yandex Webmaster ọrụ. Iji lelee, ịkwesịrị itinye njikọ na saịtị gị na mpaghara kwekọrọ ma ọ bụrụ na ebugolarị faịlụ na sava ahụ. Mgbe nke ahụ gasịrị, ngwá ọrụ n'onwe ya ga-ebu nhazi faịlụ ahụ. Enwekwara nhọrọ iji aka tinye nhazi ahụ:

Na-esote, ị ga-arịọ maka nlele wee chere nsonaazụ ya:

N'ihe atụ enyere, ọ dịghị njehie. Ọ bụrụ na ọ dị, ọrụ ahụ ga-egosi mpaghara nsogbu na ụzọ isi dozie ha.

mmechi

Na nchịkọta, anyị kwusiri ike ka ọ dị mkpa faịlụ robots.txt dị maka ịchịkwa okporo ụzọ na saịtị ahụ. Anyị nyere ndụmọdụ maka otu esi edozi ya nke ọma iji jikwaa ka ibe nchọta ihe nchọta. Na mgbakwunye na nke a, anyị lebakwara anya n'ihe atụ nke otu esi eji faịlụ a eme ihe n'ụzọ ziri ezi wee nye ntụziaka maka otu esi elele na ntọala niile na-arụ ọrụ nke ọma.

❮ Edemede gara aga Otu esi ejikọ na sava Linux site na SSH

Akụkọ na-esote ❯ Otu esi ahazi sava weebụ (Apache-PHP-MySQL/MariaDB) na Linux