That is absolutely true. I am just coming from collecting experience of past residency applicants I have spoke to for the specialty I am going for, and all of them did not rank programs which did not interview because unless stated otherwise in the VIRMP, places would definitely interview you if they were actually thinking about ranking you. So everyone have their choice, I guess for me it made more sense to not bother about places who did not offer an interview, especially academia because I did my fair homework by asking 4 different institutes during my visits and they all concurred that they wouldn't offer interviews to applicants they were not interested or rank.
For internship sure, residency; I think I have seen enough, at least for my specialty to know how they roll.