Divorce Drama: When do I no longer have to answer to my "husband"?

We have been married for 10 years. We've struggled with our relationship for many of those years and lived together off and on for at least the past 6. After years of infidelity on his part, I moved out and filed for divorce. He's now claiming he's a changed man and that we(his family) mean everything to him, etc. He refuses to see the children unless I come over with them. But those visits are always about him trying to get with me and aren't really for the kids. I've arranged times for the kids and they'll come over, bags packed, be in his car, ready to go and if I refuse to come he tells them to get out and they can't come cos Mom doesn't want to, so to be mad at me. I hate being made out to be the bad guy. I am SO intimidated by him that I feel the need to be submissive and do what he wants. I am only tough on the phone and when face-to-face I cave. I really son't WANT to see him anymore and I wish we could just act civilly. But when I don't conform to what he wants, he follows me everywhere I go, blows up my phone/text/email, threatens to call my work, my boss' wife and claim we've been having an affair(false) or to reveal to everyone that I had two abortions during the course of our marriage. (For the record: ALL pregnancies were from him, but he had girlfriends at the time, unsupportive and I already have 4 kids, again all his, to care for on my own.) I don't live with him and he doesn't compensate me financially at ALL. I work and make decent money on my own but my job requires that I travel about once a month. Because of previous experiences, I despise telling him anything about my agenda because he goes crazy and shows up either before hand ruining my plans or where I'm at doing the same. Now that the background has been explained, I guess my question is: When do I not have to answer to him anymore. He is still using the excuse of the fact that I am still legally his wife so I have to inform him of everything I do in my life. It's exhausting! I'm so paranoid EVERYWHERE I go, always looking over my shoulder because I never know when he's just going to show up. I could go on and on, but what can I do??? I just want to move on!