If it gives you piece of mind, look at it this way:
Any car manufacturer will take great caution in adding anything to a vehicle that would cause damage when removed (ie - stickers, badges, decals, trim tape, etc).
With that being said, it's been on there for a long time so take caution and go slow. Warm it up, have some Goo Gone on hand, and use dental floss or a plastic scraper and it should come off no problem.
**Also consider this, the sun has obviously affected that area over the years, enough to turn the vinyl yellow. When you remove that square, your paint underneath will be as clean as the day it rolled off the lot. You will have a square piece of clean paint there, that will not match the surrounding panels...... I don't know which is worse. I've seen some people use Rhino liner or black vinyl strips on the lower leading edge (about 15 inches?) of the doors/sills. That might be a neat look since your black plastics look like they are in good condition?