Two thoughts. First, Star Trek was based on the theory that life was seeded throughout the universe and that other life would have some similarities.

Second, there is a logical fallacy called proof by the negative. I've always thought Fermi relied on that a bit. Just my amateur opinion but...