Then I don’t know if it’s too big or not, but another possibility is that it is very soft as well. So if it’s snug inside the ear canal and you pull it out fast enough, the snugness may hold the outer part in place and the softness inverts the dome before the outer part gets dislodged.
What happens if you pull it out slowly by wiggling it a little bit? How easy is it to return it to the inside in shape? Maybe over time, it’ll harden up more with use and stop turning inside out.
Of course you can try a smaller side double dome. But if you get to a smaller size double dome that is not snug enough and starts slipping out, then I’d stick with the next size up that wouldn’t cause slippage anymore and not worry about it turning inside out upon retrieval as long as it feels comfortable inside your ear canal.