CSV Import Failure

  • My database has three different member types. Each are specified in the “Category” column. We regularly import three csv files per month, one for each category. This has worked fine for a couple of years.

    Recently, two of the three files import fine, the third fails with the error referring to the column header “Category”. This is strange as the column header is exactly the same, as we use with the other two files. No spaces or funny characters. We’ve even copy/pasted the header row from the files that worked, but no help.

    We’ve also split the file in two, eliminating half of the data. When doing this, half of the file still fails, the other half shows the import as being successful. However, when we look at the data in the “List Participants” screen, none of them are there. If we try looking up a record, none are found.

    Ideas?

Viewing 15 replies - 1 through 15 (of 16 total)
  • Plugin Author xnau webdesign

    (@xnau)

    I’ve seen this problem before: the header throwing what seems to be a spurious error on the CSV header when the server has an object cache on it.

    Really very strange that 2 of the 3 import OK, and their headers are all the same. Have you looked for any other differences? Possibly the encoding? (must be UTF-8).

    If records are not getting imported (especially when the message says they were) then really the only way to see what’s happening is to get your php error log set up. and see what comes in. The plugin’s debugging log can be helpful, but it will miss any server issues that happen before the plugin can catch them.

    I’d suggest first, clearing the cache, especially any kind of object cache (such as LiteSpeed) that is set up on the server. Then try the import again.

    Thread Starter mikepetersonwi

    (@mikepetersonwi)

    I’ve flushed the caches on LiteSpeed and Asset Clean Up Pro: Page Speed Booster. I reran the import and received a “success” message. However, when looking at the “List Participants” screen, the records are not there.

    When I look in the debugging log I’m seeing the following for each of the records I imported.

    [04/13/22 1:50pm UTC]
xnau_CSV_Import::insert_from_csv
    
columns:id, business_name, category, description, contact_person, street, city, state, postcode, county, telephone, toll, fax, email, website, maps, city_st_zip
  
csv line= Array
(
  [id] => 103436
  [business_name] => West Bend Mutual/NSI
  [category] => Company
  [description] => West Bend provides property/casualty insurance products throughout the Midwest. The company offers a broad personal lines package, a full range of commercial lines products and services including specialty lines.
  [contact_person] => Ms. Anna Maas
  [street] => 8401 Greenway Blvd Ste. 1100
  [city] => Middleton
  [state] => WI
  [postcode] => 53562
  [county] => Dane
  [telephone] => (800) 760-9250
  [toll] => 
  [fax] => (608) 410-3412
  [email] => [email protected]
  [website] => https://www.thesilverlining.com
  [maps] => 8401 Greenway Blvd Ste. 1100 Middleton WI 53562
  [city_st_zip] => Middleton, WI 53562
)
    Plugin Author xnau webdesign

    (@xnau)

    This is the initial import of the data for a single record. The write to the database should also be there. If you’re not seeing the database write in the log, something is preventing it.

    This functionality relies on the WP Cron, do you have any plugins that affect the cron?

    Also, if you’re not seeing the database write in the log, this is where the php error log will help because it is something that happens before the plugin starts its logging.

    Thread Starter mikepetersonwi

    (@mikepetersonwi)

    The entire debug log is here

    Plugin Author xnau webdesign

    (@xnau)

    OK, thanks, you need to get your php error log set up, the database write is not happening, and we’re not seeing the specific reason for that.

    Also, check into the Cron issue I mentioned, probably you’re not doing anything with it, but if it has been modified (some plugins do this) that could prevent the write from happening.

    Plugin Author xnau webdesign

    (@xnau)

    Oh, one more thing: it could still be the cache. If the options table is cached, then it’s possible the import queue won’t be processed.

    Thread Starter mikepetersonwi

    (@mikepetersonwi)

    I did flush both caches that I’m aware of and turned off the plugins associated with them. That didn’t seem to help.

    I’ll work on setting up the php error log.

    Thread Starter mikepetersonwi

    (@mikepetersonwi)

    piaw-wp-admin-error_log.txt

    piaw-error_log.txt

    Do these help?

    Plugin Author xnau webdesign

    (@xnau)

    I cannot access those files, you need to post them somewhere publicly accessible.

    Thread Starter mikepetersonwi

    (@mikepetersonwi)

    Sorry about that

    Error Log

    Admin Error Log

    Plugin Author xnau webdesign

    (@xnau)

    Great, the second one is the correct log, best way to catch the problem is to clear the log, then perform the action where you see the problem, then check the log.

    Thread Starter mikepetersonwi

    (@mikepetersonwi)

    All caches are off and I ran the import. It looked as if it worked, but there are no new records in the “list participants” screen. The error log is at Error Log

    Plugin Author xnau webdesign

    (@xnau)

    It looked as if it worked….can you be more specific about what you saw? Was there an import report?

    The link you provided wasn’t the error log, it needs to be the php error log that is set up on your server so it catches any errors that happen in the background.

    Thread Starter mikepetersonwi

    (@mikepetersonwi)

    I delete the error logs and re-imported the csv. The initial message after uploading:

    The file PIA-Company-List-4-12-22.csv has been uploaded.
    File upload complete: 57 lines received.
    The data is importing in the background, refresh the page to get the current status of the import.

    Looking at the List Participants table shows none of the new recorded uploaded.

    The two error logs are: Error Log 1

    Error Log 2

    Forgive me if these are not the correct logs. If they are not, let me know the path to the log you are looking for and I’ll delete the log and re-import.

    Plugin Author xnau webdesign

    (@xnau)

    What is probably happening is that the once the file is uploaded and the imported records are placed in the import queue, the queue is never fired, and so the record data is not written to the database. This can happen if there is an issue with the WordPress “cron” functionality. Sometimes, this means that the actual import is delayed, sometimes it never happens. If you have any plugins that change the way the WP cron works, or if your WordPress is configured to use al alternate cron, this can cause problems with the background import.

Viewing 15 replies - 1 through 15 (of 16 total)
  • The topic ‘CSV Import Failure’ is closed to new replies.