It depends on the circumstances. Remember that Billy's goal was to change the world; to cut off the head of the rotting society. Penny's goals were ultimately the same thing. The only variable is how they both want to achieve that. I think Penny could've forgiven certain things (use of force, etc.) but that there would be a line — such as killing someone — that would've alienated her from the resultant Billy.

Notice through the whole film that Billy does keep some of Horrible's more villainous tendencies in check. He doesn't battle in the park where there were kids and he did not want to kill anyone. The ultimate tragedy is that Penny made Billy stronger. He wanted to make a freeze ray; not a death ray that kills people. He wanted the freeze ray so that he could "find the time to find the words".