While it's definately preferred to keep the original part in place it's not a complete disater if the parts pop out. You do want to avoid dislodging the parts if at all possible.
I've had it happen to me a couple times, you can still get a useable mold you just have to take a lot of extra time for wiggling bending the silicone to make sure the part reseats correctly.
In a worst case scenario when you can't get it back into place you can always start over. Chop up the mold half and recycle it as filler on the new molds that way you aren't wasting material.