I have lots of the little black ants running pretty much everywhere in and around my house (up the outside wall and into the attic mostly). They've also taken up residence in passion fruit vines that are climbing some of the patio poles. Is there a real worry of them doing damage to the house, emphasis on real.