Male Chauvinism

The other day one of my boyfriend's friends told us all a story about trying to get a new phone after he had dropped his into the ocean on accident.  He had insurance on the phone and paid ten dollars a month for the insurance.  He called his phone's insurance company and the young lady who answered could not help him get a new phone, for whatever reason.  He said he tried to keep his patience, but finally, he said "Sweetheart, I know the last thing you want to do is deal with a ticked off cutomer, but you really need to understand that I need my phone replaced and I need it now."  This is the point I quit listening and became utterly outraged.  I asked him why he called this young lady over the phone, sweetheart.  He could not come up with an answer.  I asked him if he would have called a man sweetheart at this point?  He, of course, said no; but, we all know he would not have called a man sweetheart. 
Call me a feminist, but I think calling someone sweetheart that you do not know is crossing the line.  Men feel it is perfectly fine to use gendered, language they would not sue when speaking to men, but they feel is acceptable when talking to a woman.  I am not sure I will be able to be friends with this particular friend of my boyfriend.  He says things without thinking, which in my book is criminal, I am all for being spontaneous, but not at the expense of using offensive terminology which belittles other people.  I hope this person can soon realize that in speaking this way to women he treats them as a lesser class.  Unfortnately, I am now the friend in the group with the nickname "sweetheart" because I had the gall to speak my mind in front of a group of men who feel it is suitable to use familar terms with a woman, but not with a man.