Hi olidell2013,
You’re absolutely right–the import process can be painful at times. I’ll explain some of the complexity here.
First, we don’t have a way to use FTP to import database updates. The reason for this is that FTP doesn’t give you the database access you need anyhow. You have to have a direct connection to the DB to do updates.
But suppose for a second we did have a direct connection to the DB–how would that work? We have to update a number of tables (wp_posts, wp_categories, wp_meta, various BD tables) and it’s actually pretty complex. Way more than the average user can probably do on their own. That’s why we have the CSV import function–to handle all that complexity behind the scenes for you and get the data where it needs to go.
The 50-record batch is limited by a number of things. The biggest limit we’ve seen is simply the time of execution that PHP allows for a single request. Most of the time, that’s around 30 seconds but I’ve seen some web hosting providers put this as low as 8 seconds, which is absurd. That means we have to process as much data as we can in that time before the connection drops. That’s most likely what you’re seeing here. When we have 30+ seconds, we can easily pull in a few thousand records at a time.
Another thing that limits data import is size of the images. If you pull in large images, that tends to slow things down because the image must be imported at the same time as the listing. Big images take longer to process, which reduces the amount of data you can get in.
I hear what you’re saying here. This is on our list of things to improve, but doing so will be a pretty massive change over what we have now so we need to plan it carefully to get it right instead of making a bad situation worse.
