Robots.txt (overwrite)

[alextsampas]

I set up robots.txt and WWB exports the lines below:

User-agent: *
Allow: /
Allow: /images/

I have set "allow all" but for some individual pages I set "Disallow" ,"noindex" etc
Why I do not see the "Disallow" lines in robots.txt but only in every page code?

Is it working like that ?

[BaconFries]

A robots.txt file only contains site-level rules that apply to crawlers before they fetch pages.
This means that when you set an individual page to "noindex", the do not add a Disallow rule to robots.txt won't be added to it instead they place a meta robots tag in the page HTML, such as
<meta name="robots" content="noindex,follow">
or
<meta name="robots" content="noindex,nofollow">
If it was added like in the following:
User-agent: *
Disallow: /private-page/
then Google may not process a page-level noindex tag this is why it is not added to the robots.txt and add in the meta tag as shown.

[alextsampas]

Thank you for your reply !