Disable robots.txt override

  • Resolved Dima Stefantsov

    (@dima-stefantsov)


    4 years ago

    I am using other robots.txt plugin ‘wp-robots-txt’. It just does default add_filter('robots_txt', 'cd_rdte_filter_robots', 10, 2);

    In your code I see init_front_end_filters(): \add_filter( 'robots_txt', [ $this, 'robots_txt' ], 10, 2 );

    Could you please add a filter (would be common sense to make an admin interface checkbox, but just filter is ok for me) to disable your robots.txt override?

    I am looking into using your TSF instead of yoast seo since they have too many ads I don’t like it, but you are currently missing this kind of common sense filter. Let’s see if TSF could be a good choice ??

    Or you could just straight swallow wp-robots-txt, all it has is a textbox with robots.txt content I’d like to have: https://4.stefantsov.com/s/2021.02.17_10.08.12_qa1gs10s.png

    Thank you.

Viewing 3 replies - 1 through 3 (of 3 total)

  • Hey Dima, if you do have a file named robots.txt you create yourself, TSF will not create our virtual robots.txt file. Will that work for your care?

    Or you could just straight swallow wp-robots-txt,

    As per GitHub issue #59. We thought about making an extension many moons ago, but only a very few users asked for it, so it never became a priority. I hope you understand we need to budget our developer time. We might do in the future… but honestly it is really rare request and 99% of the users do not need it, so it is not in our current pipeline.

    Thread Starter Dima Stefantsov

    (@dima-stefantsov)

    @lebaux I know how routing works, of course if I have a static file “robots.txt” wp will not get this request. As I said, I’m using another wp robots.txt plugin, that just creates a text field where I can edit my robots.txt from wp admin.

    “hope you understand we need to budget”
    in autodescription\inc\classes\init.class.php:283 replace your

    // Overwrite the robots.txt file
\add_filter( 'robots_txt', [ $this, 'robots_txt' ], 10, 2 );

    with

    if ( \apply_filters('the_seo_framework_overwrite_robots_txt', true ) ) {
	// Overwrite the robots.txt file
	\add_filter( 'robots_txt', [ $this, 'robots_txt' ], 10, 2 );
}

    Your budget saved!
    I am just asking you to create a filter for me to be able to disable your robots. Here is the code, styled like you style it. Ctrl+C, Ctrl+V please.

    Plugin Author Sybre Waaijer

    (@cybr)

    Hello!

    That’s possible. In TSF I added hooks after TSF is done hooking into other hooks, via which you can unhook its hooks, like so:

    add_action( 'the_seo_framework_after_init', function() {
	remove_filter( 'robots_txt', [ the_seo_framework(), 'robots_txt' ], 10 );
} );

    I saw you took notice of the_seo_framework_overwrite_titles and the_seo_framework_manipulate_title — I added those there because we brute-force remove filters one might want to gain more control over. Such a boolean-wrapper would be redundant in the case of the robots-filter.

    It takes a bit of time perusing TSF’s code to understand what’s going on. But, I hope it’ll become common sense to you soon.

    Please mind that blocking /wp-admin and /wp-includes can cause search engines to fail rendering your pages, for they might require resources from those locations. To learn more, see https://yoa.st/robots-txt (yes, they created a great write-up!).

    I hope this helps! Cheers ??

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘Disable robots.txt override’ is closed to new replies.