Nonfiction readers. Do you feel guilty reading fiction?

Do you feel guilty when you read fiction some times? Do you feel like it's a frivolous pursuit? Sometimes, I do, because I'd think to myself "might as well watch a TV show", and I hardly ever watch TV shows because, to me, they're a waste of time. But damn it, some of these novels are so good and I can't stop once I started reading them.