Is second/third home ownership ethical in this economy?

Especially when those 2nd, 3rd, + properties are being used as passive short term rentals. Observing the state of the housing situation "Hmm there aren't enough homes for normal families to each have a chance, I should turn this extra property of mine into a vacation rental." does this make said person a POS?