[Plugin: Search Engine] Indexing times out

Can you supply additional information to help me figure out what might be going on here?

What version of PHP are you running? How large is your site? Who is your web host?

Here’s my phpinfo page – which should help in that regard.

The site is hosted on my own server, on a Mac, running Apache.

The site itself has a shopping cart with a couple of thousand items in it.

In the Search Engine’s back office, using the Wizard’s advanced settings, I tried setting the depth to 1, and also tried setting the URL masking to ignore anything with a URL containing a question mark, like this..

https://www.swing-thing.com/wp-admin/admin.php?

This should have prevented it from traversing the shopping cart itself.
But neither approach seemed to help.

The website is https://www.swing-thing.com

The following is the log produced when I try a search without masking or setting any advanced options.. NOTE: this is all I get before the timeout happens.

[Log file moderated as per the Forum Rules. Please use the pastebin]

It’s indexing for me right now, so I’m wondering what your log says. Can you send it to scott (at) skcdev .com

Sure – you want the Apache Log?
I can use Console’s matching filter method to try and isolate your indexing process; what search string should I use?

Well – rather than wait for your reply to my last question, I’ve email you a download link to my Apache2 Access_Log – I hope this is what you wanted?

Sorry, thought the log you were talking about was the one that Search Engine produces on the Indexing Wizard page (there’s a button you can click that shows that output).

Could you e-mail me that?

It’s my experience that there is a timeout in a modest-sized blog with just a couple hundred blog entries, no external links, etc. Telling it to Continue “should” allow it to run to completion. After doing this a number of times it looks like the blog is indexed, but honestly I’ve never seen it actually complete.

I think it depends on your server configuration at some point. This plugin seems to work best on certain hosts, while others have difficulties. I can’t really afford to buy hosting packages with every provider so it’s bug reports like these which help determine the solutions either for my recommendation in the plugin’s FAQ, or things that can be tweaked within the plugin to improve it for the specific case.

In this case, it looks like it’s server configuration related, at some point it’s stopping the indexing much earlier than certain configurations allow for.

I’m having what sounds like the same issue. My site, OpenSourceCMS.pro, behaves in similar fashion when attempting to get the site indexed.

Happy to share logs / info with you, Scott. Just let me know.

best,
rico!

ricoflan – are you seeing actual output in the advanced details section while running the spider? Feel free to send logs to lol (at) scottkclark dot com

SKC,

Yes, I am seeing output in the Advanced Details window.

Happy to send a log file — which log file is useful to you?

TIA,
rico!