Lost Robots.txt after migration

Hello @johan-stam

Thanks for reaching out about your robots.txt file. It should be at the root of your domain (on the server) and not stored in the database. To create a robots.txt file, go to WordPress > Yoast SEO > Tools > File Editor > Robots.txt > Create robots.txt file

However, if you don’t see the ‘File Editor’ option in WordPress > Yoast SEO > Tools, file editing may be disabled on your WordPress installation. Please enable file editing or edit the file through FTP. If you need help with that, please get in touch with your web hosting provider.

Hi,

I know how to create a robots.txt. That’s not the point. I think a Yoast update removed the data.

Hey Johan,

Maybe the migration method you’ve used did not migrate non-WordPress files which were present in de directory where WordPress was originally installed? If you migrate a site and the file is not present on the target server, then it hasn’t been copied over.

That’s why asked where the values are stored if you create a robots.txt in the Yoast plugin.

We don’t store it in settings. We just output the defaults to the file. If the file is no longer present, you should be able to rebuild it on Yoast SEO > Tools > File Editor > Robots.txt (this only works if WordPress has write permissions. If it does not have it, you can create the file manually again).

The content we write to it is what you can read here too: https://yoast.com/help/how-to-edit-robots-txt-through-yoast-seo/

Hope this helps.