Duplicate Events in Database

Hi Matt,

I’ve not seen duplicates like this before. Would you mind checking the API response directly in order to confirm that CCB isn’t actually sending duplicates?

The easiest way to test this is to connect to the endpoint in your browser and inspect the items that are being returned. The format of the request would be:

https://<username>:<password>@<subdomain>.ccbchurch.com/api.php?srv=public_calendar_listing&date_start=2018-04-11&date_end=2018-04-22

Where <username> <password> are the API credentials and <subdomain> is your unique CCB church subdomain.

You can of course change the date range if needed (this is 2 weeks starting today).

One last thing to clarify, but I’m sure you probably know this… If an event is a recurring event, you will get duplicates (in the sense that you’ll see multiple Event Names as posts in WordPress) but they would all have unique dates in their post meta.

Matt,

I have a couple other thoughts…

1) I noticed you’re doing a SQL query to check for the duplicate events. Be aware that if any WordPress posts get updated after a synchronization, you will get an additional duplicate record in wp_posts but the post_status would be a revision. Do any duplicates have a post_status other than publish perhaps?

2) Another way we might confirm whether the CCB API is sending duplicates versus the plugin creating duplicates is to manually delete all of the event posts in WordPress and do a single sync. Then you can see if the data is clean. Next, do another sync and see if you get duplicates again.

• This reply was modified 6 years, 6 months ago by Jared Cobb.

Hi Jared,

Thanks so much for the quick reply. Here’s the status on the tests you suggested …

Looking directly at the XML from the API, using the method you outlined above, there are NOT any duplicates coming through. I then confirmed that there are still duplicates (for the same date range) in the database after doing a sync with the plugin.

Next, yes, I understand that recurring events (of which we have many) come through with the same title, but in SOME cases, we are also seeing them with the same title and date in the database (a duplicate).

Next, yes, I had previously confirmed that these duplicates are all marked with a post_status of “publish”.

Lastly, I went through and killed all events in the database, confirmed they were all gone and did a sync. NO DUPLICATES.

Then I waited a little bit and did another sync, DUPLICATES ARE BACK. So back to square one. At this point there were 278 events in the DB with “publish” status.

Did another sync, now 291 events in the DB with “publish” status. More duplicates obviously.

Did another sync, now 305 events in the DB with “publish” status.
Did another sync, now 315 events in the DB with “publish” status.

… you get the idea. We are accumulating a larger and large number of dups with each sync. Interestingly, there are zero events with a status other than publish. Should there be some with a different status, like revision?

Man, any help you can offer is greatly appreciated. We’ve hit a wall.

thanks!
Matt-

Matt,

Thanks for confirming all of that! The plugin definitely seems to be causing the issue, and for some reason I haven’t been able to replicate it on my church’s account.

Would you be able to create another API user for me and grant the user rights to the public_calendar_listing service? This way I can try to replicate the issue using the same data. (And once I get this fixed you can delete my API user).

Also, do you know what PHP version you’re running? If not you can temporarily install this plugin https://www.ads-software.com/plugins/wordpress-php-info/ and then remove it.

• This reply was modified 6 years, 6 months ago by Jared Cobb.

Matt,

Disregard my last post, I’m actually able to replicate this. It seems there’s a bug if the data range starts in the past (i.e. in the settings if the “How Far Back” setting is greater than 0).

I’ll have time to debug this later today and figure out what’s wrong. I’ll be able to publish an update (hopefully today or tomorrow).

Thanks again for reporting this!

Awesome, we’ll hang on until we hear back from you.

In case it helps, here are versions we are using:

OS: linux
Apache: 2.4.33
PHP: 5.6.35
MySQL: 5.6.38

Thanks!
Matt-

• This reply was modified 6 years, 6 months ago by Modmacro.

Great. This actually had nothing to do with the date range starting in the past after all.

As part of the synchronization process, the plugin fetches all the existing event posts and compares them to the API response. It turns out there is a scenario where fetching the posts can result in not actually getting all of them (and then the plugin thinks those events are “missing” and needs to insert them again).

I’m also surprised this hasn’t come up before. It’s possible something has changed in a recent WordPress update with how WP_Query handles batched & looped queries (which is how I’m fetching the existing posts for performance reasons).

I have a fix in place, but I won’t be able to publish a new version until this evening.

• This reply was modified 6 years, 6 months ago by Jared Cobb.

@modmacro I published a new version that fixes this issue (1.0.5). When you have a chance can you confirm it’s working for you as well? Thanks again for reporting this.

Jared

Dude! I think you got it. With each successive sync there are (so far) no duplicates being created – data looks clean. Nice job!

One thing to note for anyone else who runs into this issue. After updating to plugin version 1.0.5, it DOES NOT clean up the previous duplicates in the database. So I removed all events, then did a sync. Seems to be good after that.

I’ll keep an eye on it for a week or so to ensure no issues, but all signs are positive.

Thanks for killer plugin support, Jared! I usually just write my own plugins as support tends to be poor, or none. But you’re doing a great job!

thanks,
Matt-

That’s how I do!