I'm trying to figure out the name of a children's book that I remember my mom reading to me when I was a child?

I grew up in the 80's, but I'm not sure what year the book was written. I can't for the life of me remember the title or the author! I remember from the story being about either a man or some little creature who finds an ant in his house. I think he started out by trying to hit the ant with a hammer and ended up making a hole in the floor. The ant got down in to the floor and he tried to hit it again and busted a pipe. The book ends with the ant walking away and the house in shambles around him. Anyone know either the title or the author?