Grr, I think I found the issue, now I just have to figure out how to fix it.
I used what you wrote and it still didn’t work for me, so I just went about debugging line by line the whole call.
The query_vars do actually get added properly…until do_action(‘template_redirect’); gets called in template-loader.php. Then all the whole request gets rewritten and my added query_vars get lost.