I think we notice it more because there are some really ****** analysts on TV. With the good ones, especially if they have a good rapport with the play-by-play guy, you really don't notice it as much.
For better or worse though, that's what they're getting paid to do. Producers don't want dead air. And out of the 4 main sports, football has the most downtime between plays, so there is this "need" to fill that time with something. Baseball is somewhat similar, although much less so, especially now that we have the pitch clock.
Basketball (and I assume hockey, although I've never called a hockey game) is easy because there is constant action to describe/talk about.