Still feel like a dirty little secret years after affair started

Dear Coleen I’M A woman in my early 40s and have never been married or had children.

- 

I've been with my partner for 10 years and we got together by having an affair - he’s 10 years older than me and has four children with his ex, ranging in age from 14 to 21.

When his wife found out about our affair, she kicked him out and he went to live in a flat but we now live together.

The issue is, he still hasn't divorced his ex and he also sees her way too much in my opinion, often just popping round to hers, and he's constantly on the phone to her about the kids or stuff to do with their family home.

He doesn't involve me in any of it. I've met his children only a couple of times and they've never come to our place. I'm not a part of any of it.