Batch Process Suggestion?

  • Resolved justinbevansck

    (@justinbevansck)


    3 years, 11 months ago

    Hello! I have custom plugin that lets admins set some filter for a single file multi-form export (Using the below for inspiration):
    https://www.ads-software.com/support/topic/download-all-results-of-multiple-forms/#post-11634401

    Everything is working great except… we have a lot of entries to export now! And running into a situation where even trying to do a single day of exports results in timeouts despite generous PHP settings.

    Do you have any suggestions for handling large exports? I’m guessing I’m just going to have to forego having multiple forms in one export but just curious if you had a better idea. The ideal situation is to be able to export a month’s worth of entries at a time (~500k entries over ~13 forms).

    Thank you!

Viewing 1 replies (of 1 total)
  • Plugin Author Doeke Norg

    (@doekenorg)

    @justinbevansck , sorry for the delay. Unfortunately I can’t come up with something for this.

    The problem is that where a CSV or TXT file can be appended too, and thus be filled in multiple processes, the excel file cannot. This is because this file is in fact a zip of multiple files. You cannot just append data, so the whole file must be written at once, and therefore be in memory completely.

    So the more results, the more time and memory it takes. This is a know issue with the software my plugin uses.

    I would suggest forgoing on the multisheet approach and just export multiple files. This way you have multiple process that take less time indicudually and less memory usage.

    Perhaps you can combine those afterwards somehow.

    Very sorry about this.

Viewing 1 replies (of 1 total)
  • The topic ‘Batch Process Suggestion?’ is closed to new replies.