How many products can import & sync Woocommerce by cron job?

  • Hello,

    I’m working on an online store. My intention is to have around 250-300.000 products inside. All these products (sizes, colors and parent are included in these 300K), are imported by a cron job. The bad part is that are took too much time for sync…days..and I need to make all in at least once/24h in order to have the stock and the prices updated with the supplier database.

    The question is: What characteristics of the server are important in order to speed up the import process? I mean: what to improve as hardware or as settings? Now I have this:

    Server VPS

    • Memory: 32 GB
    • Procesor: 16 vCPU
    • Space: 400GB SSD
    • Traffic Web: Unlimited, Uplink up to 1000Mbps

    Apache web server. MariaDB database 10.3
    + 100% server security with Imunify360
    Automatic Malware and Virus scanning with automatic cleaning
    Web Application Firewall with Machine Learning
    Intrusion Detection and Protection
    Proactive protection for PHP sites
    WebShield with CAPTCHA integration
    + Litespeed Enterprise web server
    Exceptional PHP execution speed and delivery
    Native cache plugin for WordPress, Magento Prestashop

    On the WordPress, these are the figures now:

    WooCommerce version:7.3.0

    WordPress memory limit:4 GB

    Server info:LiteSpeedPHP

    version:8.1.14

    PHP post max size:256

    MBPHP time limit:3600PHP

    max input vars:15000

    cURL version:7.87.0, OpenSSL/1.1.1

    sSUHOSIN installed:–

    MySQL version:10.3.37-MariaDBMax

    upload size:256 MB

    Thank you very much!

    Alin

    • This topic was modified 1 year, 10 months ago by Jan Dembowski.
Viewing 6 replies - 1 through 6 (of 6 total)

  • What is the data source of the products? XML or CSV? Or do you access something external?

    In my experience, the fastest way for any import is via WP CLI. There is no timeout in the way and it can go through in one go – even if it will surely take hours with the amount. How you could use WP CLI for this depends on the data source.

    Thread Starter alinh

    (@alinh)

    Hi Threadi. The file is csv.

    I import via a plugin (wisexml). It have “behind” some settings and there is no timeout too. The bad part is that he can sync aprox 1000 products/hour. And in this way, are 24K/day. I need 10 days for 1 sync. I need to do something (without reduce the products size…let sey 240.000 products).
    I don’t know if is manageable this ? What settings can modify?

    Thank you,

    Alin

    If you want to continue using wisexml, contact their support: https://www.wisexml.com/support – since this is a commercial plugin, they can’t help you here on the forum regarding this.

    If the source was XML, it would work via the already mentioned WP CLI. See: https://developer.www.ads-software.com/cli/commands/import/ – of course your hosting must provide this. If necessary, contact the support of your hoster.

    Thread Starter alinh

    (@alinh)

    Hello Threadi,

    Unfortunately, after 3 weeks spent with them and with the hosting provider, I didn’t succeed anything. Each of the confirm that the plugin and the server are well configured, and is a problem from WordPress, plugins or other.

    Which settings are mandatory for a fast upload?

    Basically are 140.000 products in my store, and this API sync the price and stock. In the current situation, they check around 10 products/minute. I don’t have time to run once/day the cron job with all products.

    What suggestion you have?

    Thank you,
    Alin

    As described in my answer above, I would recommend the WP CLI. It has no timeouts. If you can use it, your hoster can tell you. How you can use it depends on your data format, which I don’t know.

    If necessary, it would make more sense for you to look for someone who supports you personally. You can find someone like that here: https://jobs.wordpress.net

    Hi there

    I run a woocommerce website with 150.000 EAN’s, working fine and fast, the only thing is indeed, importing prices and stock, iùm also figuring out how to speed up the process. Will lokk at WP CLI, which i will setup up.

    I you want we can share our solutions

Viewing 6 replies - 1 through 6 (of 6 total)
  • The topic ‘How many products can import & sync Woocommerce by cron job?’ is closed to new replies.

Tags