Avoiding a confrontation may be worse than just having it out. Say what you have to say as matter-of-factly as possible.