I would do a large (80%) waterchange anytime you decide to remove all the fish and replace them with new fish, especially after fish as heavy in the waste production area as goldfish.
(Water doesn't hold your good bacteria, so doing the 80% waterchange, as long as you refill it immediately (within 10-30 minutes) of draining it, you won't risk any problems with your cycle. Make sure not to vaccuum more than half of the substrate at a time and not to scrub down decorations beyond a gentle cleaning.

)
Good job on asking.
