I think the allegations of abuse were aimed towards Barb and a partner she had before she met Bob, when Cole was much younger. Cole would have been much older when Barb married Bob and I'm not even sure he lived with them, and if he did it wouldn't have been for very long. Cole hated Bob more for political and personality reasons. I'm sure they didn't get along well, but I doubt Bob was ever physically abusive towards Cole, and Cole would have been old enough and physically mature enough to defend himself by that time.