Did I do the right thing in preventing the fight (long one)?

So OK I'm outside my house with my friends (two of them are, while the other one I don't even consider an acquaintance) and we were just talking about whatever it was we were talking about as we were all insulting that one guy that "thinks" he's my friend. Then the subject of drinking came up and one of my friends insisted on that we should. I told him that we could and the "thinker" thought that we could maybe pop open a few in the back of my house. It bothered me after awhile because he kept insisting, annoyingly might I add, that we should do it there in the back, which ultimately lead me to be extremely irritated. We've drank in my backyard before, but today was not one of the days where we could indulge in the act, so no was my answer every time anyone asked if we could. So after awhile the "thinker" started hassling me and my other two buds for beer money and then I snapped and asked him if he had any money to buy booze as well, to which he painstakingly replied with "no, I'm just the planner, I plan these things". I noticed it bothered him when I asked, so he started saying that he was gonna go get himself a tall-boy and said he was going to drink right outside of my house, in which I replied by telling him that "if he did, I was going to kick him off my property". He laughed and that's when I just started insulting him and then he decided to pick a fight with yours truly. I stood up for myself and told him that he shouldn't be coming around my place to insult me. We were about to square off when I just backed up against the wall. After that my friends tried to calm us down a little, but we were still pretty furious. So I decided not to engage him and made it all seem like it was just a F%@$&^G joke. And I guess he kind of got the idea and stopped insisting on doing any sort of underage drinking and just kept asking what we were going to do. So my question is, did I do the right thing? I understand that violence is not the answer, but sometimes, I am most certain, there are people that deserve a serious A@# kicking. And I ask this because I sort of feel less of a man for not really taking my fist to his face. By the way this guy is our minor; he's 18 exactly. I'm almost 20 and my friends ages are 20 and 24. So did I do the right thing or what? Am I considered a P%^$Y for not handing his A$# to him? Or did I overreact?