HowTo: preselected meta keys for export

This is not an ideal suggestion ( bit lazy, as you pointed out ) – all your preferences will be lost on the next upgrade and it’s only a personal fix – a plugin needs something more accessible.

I did start work on a simple saving mechanism, but other work has taken over – I hope to complete that this winter sometime.

Q

of course it is not ideal and you never should change core files but sometimes you really need an essential feature ??

About 9 weeks ago I did a version of a way to save which fields you’ve selected. It’s based on version 0.9.4, so it’s a release back. I’ve got it up at https://www.dropbox.com/s/7falfqdoojo7mse/export-user-data.php
for the moment. I’m not intending a fork of the code or anything, but it might help someone.

Thanks @cwjordan – when I get some extra time I’ll take a look at this, but if you get a chance, please could you update it for the latest plugin version?

Q

Okay, I’ve updated it to the latest plugin version, 0.9.5. I also included -Eli’s code for clearing the cache to work around the out of memory problem (I didn’t include that part very prettily, and I feel like the underlying cause could use more research, so I don’t know if you’d want to release it like that, but it does work for me. I really think there is something in the Buddypress code that is not playing nicely with us).

I just noticed when I exported something with serialized data in one of the fields it showed up as “Array”, instead of as the serialized data. I wonder if I broke something with my change, I’ll have to look through that more carefully tomorrow – be wary.

URL is the same as before: https://www.dropbox.com/s/7falfqdoojo7mse/export-user-data.php?dl=0

@cwjordan – thanks for your work on this, I’ll have some time at the end of this week to review all the new code additions and see how this all works out.

Q

Okay, I’ve fixed the problem where my Buddpress extended profile fields that had serialized data in them were showing up as “Array”. I’ve also fixed the problem where selecting an offset and total number of users wasn’t working. URL is the same as above.

I’m going to take a little look at what is up with memory usage & Buddypress but I fear that may be beyond me. Looks like it’s just caching everything very aggressively. I don’t see a way to disable that for a particular query.

@cwjordan – did you include these fixes in your hosted copy of the plugin on DropBox?

Q

Yes, those changes are in the copy on dropbox.

I looked a bit into the memory usage issue and I don’t see a better solution than the one Eli proposed. If I understand what Buddypress did correctly for 2.0, they rewrote BP_XProfile_ProfileData::get_all_for_user so that instead of making it’s own SQL queries it calls BP_XProfile_ProfileData::get which calls BP_XProfile_ProfileData::get_data_for_user. get_data_for_user in turn checks if the data is in the cache already and if not it makes the SQL query, populates the cache with it, queries the cache for it and returns the data it got from the cache.

I guess I can see why they did it that way, but that doesn’t work well for exporting most or all of the data from a site, it just fills up the cache. I suppose an alternative would be to do our own SQL query, but it’s better to use the Buddypress functions to access the Buddypress data. I’m not using a cache plugin, I don’t know if that would make a difference.

@cwjordan – thanks for looking into this – I often find the BP documentation a bit sparse, so we have to chase functions calls up the chain to find out how they relate to one another.

Seems a shame that we can’t by-pass the cache-creation programmatically – but I certainly agree that we should stick to using provided top-level functions and avoid custom SQL, it’s always inadvisable.

I’ll take a look at everything you’ve been working on at the weekend.

Thanks!

Q

I thought I’d drop this note in this topic also:

First push of 0.9.6 is up on github for testing:

https://github.com/qstudio/export-user-data

There are a few bits missing and a few changes, mostly to the saving of previous export data:

– I moved this from the wp_options table to the wp_usermeta – this will allow several users to store data on the same install
– I moved some of the saving and retrieving options around in the layout and reformatted the methods a little, to match the rest of the plugin
– I had to comment out the array data sanitizing lines for now, noted with an @todo – @cwjordan, please can you take a look at this, as it’s including the saved array as a key within itself?

I tested a 1500 member export and all the saving and loading functions, but most testing is required before this goes up on WP.

Any questions, just ask.

Q

Q,

Thanks! I’ll take a look.

Regarding using wp_usermeta instead of wp_options, that’s up to you. The use case I had in mind was that one person might want to create a report, and then hand it off to someone else to run. In that case having the same report visible to more than one person would be useful. I can see why some people might want the reports to be only visible to them though.

Q,

It gives me an error that it can’t load css/export-user-data.css. That looks like a new file, could you please add it to github also?

Regarding that way the sanitize works, I can see where that is unclear. I think I did it that way because I first wrote it to save just one report, then I changed it to do multiple reports and that was an easy way.

The get_option( 'q_eud_options',array() ); in the constructor is going to pull an array of report names, and each report name is itself an array. It puts that into q_eud_options. When I update options I just need to update the one key in q_eud_options for a given report name, and then save the whole thing. So when it came to updating and sanitizing that data it was easier to have an array, $options of one element, whose key was the report name to be updated, and use key($options) (which will return the only key in a one element array) to get that name instead of adding a separate argument for $report_name when I called the function.

Looks like you aren’t planning to support multiple report names anyway, so you don’t need that top level of array at all.

@cwjordan – thanks for your reply.

Please take deeper look at the code, multiple reports are still supported – and I feel it’s more likely that different users will want to run their own exports, at least that’s the common usage on a large site I use this on.

I keep the same logic you had in terms of saving, loading and deleting reports ( I called them exports ) – but I switched the code around a bit and added some more validation and error checking.

The exports are still saved in one large array, with named keys, the name the user gives them at the time of saving.

I’ll add that missing css to github in a minute.

Cheers

Q

@cwjordan – I’ve pushed that .css file to the gihhub repo.

Q