There are two screws (each side) that you will notice when the doors are open. You will need to remove those as well.
And just to be clear, you need to remove the bolts from the frame, so the whole trim with the metal brackets will come off all together.