Opening Images in Attachment.php On Per Category Basis

I have tried to do this on my own with little success. In fact, more like dismal failure.

Specifically, I am attempting to do the following:

1. Make a post to a specific category with a custom template for that category (done easily with Kaf’s PTbC plugin). This post will contain images uploaded via the WP built in attachment uploader. Images are auto thumbed and “linked to page”.

2. Make the full size images on attachment.php use the custom template I created for that category to which the parent post is assigned.

To put this in a different way for clarity, let’s assume I have a music site with the category “Rock”. In it I have a custom category for “The Who” and “Rolling Stones”. I make a post about The Who and upload some pics using WP’s built in uploader. I want the thumbs to use attachment.php, but I don;t want a generic attachment.php used… I want it to inherit the template I did for The Who category to avoid “competition” from any other category on the same page.

I am simply unsure how to go about creating a category specific attachment.php that is customized entirely to a specific category, and assigned to the post parent itself, of course.

Help?