Here's what you are missing out on by leaving now: NOTHING.
My uNDh pulled the same shit on me, just stranded me with no sex, no explanation except, "it's not your fault, I just don't want to". As we now know, there was an unspoken addition to that, "...with you, anyway. But it's nice to keep you hanging around wondering and hoping and loving me".
Your plan is perfect. You're seeing things clearly. You'll be way happier. His loss, for sure.
Seriously, let the dude Hoover away. Don't believe him for a second. Have as little contact as possible.
I got a vision of him as the Wicked Witch, watching you in the crystal ball, screeching, "I'll get you, my pretty! And your little dog, too."
You found out way earlier than I did. I had already moved away, for FIVE YEARS. He took his time, Hoovered me back, and then made sure I knew that he was laughing in my face for being so stupid as to believe him.
Get out quick, get out clean, don't look back.