You need to remove the panel, there are three rivets, (or sometimes Torx head screws), under the door rubber, and either replace them, have them wrapped, or spray them.
That's quite a common problem and there are lots of solutions out there.
My one's are the same but worse, so they will be getting painted when I have time.