Does anyone know how to take off the rear side panels? Should I remove the back seat entirely and then proceed from the bottom up removing the clips? Also the rear deck panel, I am guessing I have to remove the clips from the trunk, is there a special tool I need?

What are you trying to do? Change out the 6x9's in the rear? You have to remove a lot of things, such as the seatbelts from the front to the rear, the back seats completely...its a very big hassle. I would have a shop do it. Save yourself the trouble, just put it this way man...It took the shop I went to with 2 guys working on it 3 hours to install my 6x9's.
So its a pain to do, just my 2 cents.

Calvin