Attracting birds but not mice?

[Tripecac]

I love watching birds, especially the chickadees, jays, towhees, and juncos.

In the past, we've used birdfeeders on our deck to attract birds (very effectively). However, this spring we caught 3 field mice in our garage, and our pest control guy said that birdseed also attracts mice.

(NOTE: The mice had chewed open a bag of birdseed and build a large mound of it inside our heater! I've since moved the birdseed to a huge airtight tupperware container in the kitchen.)

Is there a way to attract birds w/o increasing the chance of a mouse infestation? Or are mice omnipresent in our area, with or without birdseed?