LJ import has thousands of duplicate comments

Edit to add: It seems to have imported each comment 159 times.

I did the math on a few posts, and this is the number I come up with.

John

Another addition…

So far (I have only looked at 30 pages of the oldest comments so far) if I start at the end… the older comments were not duplicated. It seems the oldest (maybe archives were not duplicated?) were not duplicated. Still going with manual delete, with 1322 pages of comments still to go. Damn. 15 duplicates of each would have been much easier. ?? Before the comment import it was about 9 pages…

John

Okay, it looks like anything commented on or after Jan 7th 2011 is the problem.

Before that, it all has looked good.

Hmmmmmmm…..

Reminds of another problem I have seen in other systems. And I do not mean Y2K…

One of these systems (WP or LJ) has a bug in date interpretation. It is all just clock clicks from an arbitrary date. Somewhere in 1970 for most systems. The problems come with the role-overs… sigh.

Here I go again, having to compensate for myopic programmers who think in terms of weeks or months, and not years.

Am I the only one that has seen this problem?

Did someone not realize 2011 would sooner or later happen?

Okay, I would like to thank you (WP employees, programmers, and shills) for all of your help.

Your silence as you respond to dozens of other “common” issues speaks volumes.

At least an “I don’t know, tell me more about the problem” would have been appreciated. That is how I treat people I work with, professionally, personally, and even just in forums I participate in.

We all know it is easier to ignore a problem than it is to admit there is one. That is the bane of all users of software products, yours and others. If issues come up, talk about the easy ones… ignore the rest. Of course it is easy to blast others (like MS) while doing it yourself.

I guess that is why so many 3.1 upgrades have gone bad. Just looking around at the forum, it is “assumed” in the replies that your final code is good. I doubt it from what I have read. If it was, there wouldn’t be 100’s of complaints a day.

Is there any QC in your programmers group? Or is it all 20 year old coders that feel they know everything and if it is not being done the way they envisioned, it is wrong?

I have recommended WP do dozens of people, and now I need to clean up all of the problems it has caused them. My bad. It will not happen again. In fact, the exact opposite…

Once again, thanks for the good work. {/sarcasm and former trust in WP support off}

Most of the time, I figure things out for myself when there is a problem. I am not a person that needs my hand held at every turn. It would help if there was some admission that there as a problem. At least I would know how to proceed.

Actually, I guess I do… don’t ask for help here. Sigh… I hoped this would be better, or at least for a simple response.

Once again… Thanks.

Back to work now. I have a deadline to fix this problem that has already passed, and I am not being paid. Sucks after a 10 hour day working on this… and knowing it is going to take at least 2 more days.

I think if you look back over my contributions they haven’t been restricted to trivial stuff. I have only come on after the weekend (I keep strict limits about time) and saw your post.

I nearly went past it as as there were several posts counted and thought someone else was dealing with you.

I don’t use LJ so can’t help there, but surely you can do a SQL query and cut out the comments? If you inported the data is the XML/SQL import file still around and does that have the error in it?

If not what about a selective table restore using Mysql tools not wordpress?

I want to start off by apologizing for my last post. I had gotten some very bad news yesterday evening when all of this was happening and it was wrong of me to take it out on you. I hope you will accept my apology. I was upset and frustrated. Normally I am a very calm thoughtful person.

I will look into cutting out the comments and doing another import. I will have to wait until the person I was doing it for is available to do a restore to the previous database. I will let you know how it goes.

And thanks, really thanks. I will try to behave better in the future.

No problems – I throw my toys out of the pram on a regular basis with WP….

let me know how the restore goes, If you want me to look at anything I can drop you my email.

Help. This is awful. I have >2800 comments, most duplicates. I don’t have time to click through 145 pages of duplicated comments. Please tell me there’s a way to fix this.

Okay, sorry, actually it’s not that bad, I think. At first I thought it was every post, but it’s not. It seems as if the problem extends backwards for me though, well into late 2010. I haven’t posted much this year, though, so maybe there’s something else going on. I’m in the process of clicking through and deleting about 200 duplicated posts.

For me it started slow at end of 2010, one per post… then two… then three. The farther back, the fewer duplicates. But by early 2011 it grew fast, with the most recent 2 months being the worst. I was told by the person I am doing this for (my girlfriend) the comments were not so important.

But after reading them I reminded her this was part of her business history. I guess I am my own worst enemy here. So I went to the comments section in the control panel and have been working my way back from the end. This is a low volume blog, few posts, few comments for the most part. But at 159 dupes per comment (at this point in the posts), it is a slow process. I am up to January 24th, and still have 1062 pages more to clean. I wish there was a delete duplicate command.

Also, it is not in a language I can read (I have to use a translator program if I want to actually read them)…. which makes it even more of a challenge. But after an hour or so, you get a rhythm to the deleting. So I do a few hours every day. Sooner or later it will be done.

Yes, that’s exactly it. I didn’t have that many posts, so it wasn’t a big deal, but there was a gradient. As it got closer to the first duplicated posts there were just 6 then 3 then 2.
Good luck!

Thanks. It is another one of those “Live and learn” experiences.

I am curious what our blogs might have in common that caused this. Or maybe we are the only ones importing from LJ. ??

Either way, I am sure it would be useful to us and others if we knew the “why” to this happening. So if you figure out any more patterns or details, post them. It may not help us, but maybe we can help others in the future.

Do either of you have the original import file still?

It might be possible to sort out the issue from that. As I said earlier in this thread an SQL statement might be able to go through the files in the DB and cut out any duplicates. If either of you have the original or can make a copy of your DB now with phpmyadmin I might be able to help by looking at the data.

I didn’t use a file. I directed WP to authorize import from my livejournal account at enigmakaty.livejournal.com.

I no longer need help deleting the comments, but I’m happy to help track it down if you need help. I deleted the comments manually, though, because there were only about 200.

Hi John,

If those duplicate comments had unique ID numbers, it would have been much simpler and quicker to use phpmyadmin and SQL statements to delete them, like what datasoftict suggested. Just something to keep in mind should it happen again, so you don’t have to waste hours on it.

Also, there are some misconceptions you have about WordPress I think you should be aware of. Firstly, there’s no official WP support. The people helping here are all volunteers who aren’t getting paid for it either.

Also, as an open source project, many of the programmers, testers, QC staff etc, are all volunteers who work on it in their free time.

But precisely because it is an open source project, everyone can contribute. If you would like to help, there’s a procedure for reporting bugs.