Robots.txt (overwrite)
Posted: Wed Oct 21, 2026 1:47 pm
I set up robots.txt and WWB exports the lines below:
User-agent: *
Allow: /
Allow: /images/
I have set "allow all" but for some individual pages I set "Disallow" ,"noindex" etc
Why I do not see the "Disallow" lines in robots.txt but only in every page code?
Is it working like that ?
User-agent: *
Allow: /
Allow: /images/
I have set "allow all" but for some individual pages I set "Disallow" ,"noindex" etc
Why I do not see the "Disallow" lines in robots.txt but only in every page code?
Is it working like that ?