Crawl & indexation

Robots.txt & Sitemap Tester

Timapeza robots.txt yake ndi sitemap.xml kuchokera pa seva yathu, tikuwonetsani zoletsa zosakira ndi sitemaps zovomerezeka, ndikuwunikira ma URL a sitemap a query-string omwe amayambitsa kulephera kwa zinthu zosagwirizana.

Momwe mungayesere robots.txt ndi sitemap.xml

Gawo lachiwiri, palibe kulembetsa. Chowonadi chokha chimayenda pa server-side.

1

Pezani URL ya malo

Muyenera kulemba URL ya tsamba la homepage la tsamba lililonse. Simuyenera kulemba njira yonse ya robots.txt, chifukwa timachokera ku woyang'anira.

2

Tikupeza kuchokera ku server yathu

M'malo mwake, server yathu imafuna robots.txt ndi sitemap.xml iliyonse yovomerezeka, kotero kuti kuyesa sikunalepheretsedwe ndi browser yanu kapena cache yanu.

3

Onani malangizo ndi sitemaps

Gawo 2: Sinthani sitemaps Yambani ndi sitemaps yanu. Pezani ma sitemaps omwe amagwiritsidwa ntchito kwambiri, ndi ma sitemaps omwe amadziwika bwino.

4

Kusintha zomwe zafotokozedwa

Kuchotsa chilichonse cha URL cha sitemap cha query-string kapena kulephera-koma-kuphatikizidwa kotsutsana ndi injini yosakira yomwe imachepetsa zodutsa.

Chiyani ichi tester amayesa

Ma guardrails omwe timayendetsa m'maganizo athu asanatulutse tsamba limodzi.

📜

Crawl directives

Anapanga masamba onse a ogwiritsa ntchito ndi malamulo ake Ovomereza / Oletsa kuti mudziwe zomwe zili zoletsa.

🗺️

Kuyankha sitemaps

Amasonyeza ma sitemap onse omwe amapezeka mu robots.txt, amapeza aliyense, ndipo amadziwitsa mtundu wake ndi kuchuluka kwa URL.

⚠️

Kufunsa-mzere bloat

Amalemba ma URL a sitemap a query-string omwe amapanga masamba opitilira muyeso ndi kuyambitsa zoyipa zazolemba zoyipa.

🔗

Kupeza chitetezo

Amatenga sitemaps omwe adalengezedwa mu robots.txt kuti alephere, kubwezeretsa kapena kubweza XML. Izi ndi zowopsa zowopsa.

Mafunso ofunsidwa nthawi zambiri

URL imodzi yofunsira-mzere mu sitemap ingagwiritse ntchito mamiliyoni a masamba opitilira muyeso. Ma injini osaka amawasaka, amapeza zomwe zilipo pansi pa zinthu zosiyanasiyana, ndipo amachepetsa magulu onse azinthu zofanana. Izi zimawapatsa ma flags asanachitike.
Sichoncho. A robots.txt wopanda kapena wopanda ntchito amalola kutsitsa zonse. Mphamvu ndi yosiyana: kukana kopitilira muyeso komwe kumaletsa masamba omwe mukufuna kusinthidwa, kapena sitemap yomwe idapereka mu robots.txt yomwe imabweretsa vuto.
Sichoncho. Timapeza fayilo ya robots.txt ndi sitemaps yomwe imalengeza, kenako timawerenga chitsanzo cha URL kuchokera pamodzi. Sitimafufuza patsamba lililonse, chifukwa chake kuyesa kumatha mwachangu ndipo sikungatulutse seva yanu.
Ndikofunika kuzindikira kuti kulumikizana kwa machitidwe ndi REST API kumapezeka pa machitidwe olipira omwe amayendetsa.
Kwa opanga mapulogalamu

Kuchita izo mu pipeline yanu

Mumatumiza URL, backend yathu imapeza ndi kugawa robots.txt ndi sitemaps zomwe zafunsidwa, kenako imabwezeretsa tsambali ndi zotsatira. Programmatic REST API ya CI / CD ndi kuyankhulana kwa masamba osiyanasiyana amapezeka pa maphunziro olipira. Onani zolinga →

Wopanga API

Kugwiritsa ntchito izo programmatically

Gawo lililonse limapezeka pogwiritsa ntchito REST API yotsimikizika pa Pro ndi API.Ikani mu CI, CMS kapena dashboards yanu.

curl -X POST https://seo.bike/api/v1/robots-check \ -H "Authorization: Bearer sk_live_yourkey" \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com/"}'
OK
{ "ok": true, "result": { ... } }
Pezani chida cha API → Mtengo wowerengedwa, wolipira pafoni. Mtengo wopanda malire wa chida cholumikizana pamwambapa.
Pezani patsamba lililonse la tsamba lanu, si chimodzi panthawi imodzi
Pro imagwirizana ndi Google Search Console, imapeza tsamba lililonse lomwe Google lidasiya, tsamba lililonse lowonongeka ndi tsamba lililonse lomwe limalowetsa pansi, ndipo limalemba zomwe zimayenera kusinthidwa pamodzi.
Onani momwe Pro imagwira ntchito