You got all the screws i believe, but now around the edge of the panel there are little pop-rivet things.. start at the bottom of the door. Get your fingers in between the plastic panel and the metal door and pull out. If you cant fit your fingers in there, you can use something like a wall puddy spreader thing (haha im not actually sure what its called), just be sure to wrap the tool in fabric or something so you don't scratch up your door.
after that, just pull the entire panel up. It helps if you have the window down for this, but when you put the panel back on, i felt it easier if the window was rolled up a little bit. This helps line up the weather stripping that goes in between the glass and the door. Its kind of hard to explain, but once you get there, you'll know what i mean.