Are the Tea Baggers going to cause long term losses & damage to Republican Party even after the election gains?

Even though Tea Baggers won some House seats last election, now that many have seen their policies in action, will it lead to a long term loss for Republicans and causing the party to become a Party of Fringers and Far Right loons?