wp_set_object_terms returns wp_error

Hi,

We created something with which we can post a post on multiple blogs in a certain category. (WPMU) Now I want to use the function wp_set_object_terms to add the category to it, but it returns an error.
I do the following:

wp_set_object_terms( $post_id, array('categgory-to-add'), 'category'))

I have switched to the correct blog, I add the correct post_id and I add an array with the category names and I want to add a category.
It returns the following error:

wp_error Object
(
    [errors] => Array
        (
            [invalid_taxonomy] => Array
                (
                    [0] => Invalid Taxonomy
                )
        )
    [error_data] => Array
        (
        )
)

invalid taxonomy, I have tried it as well with ‘post_tag’ or any other random name and none of them work.

Anyone can help?