You will be totally fine without feeding them for 3 days,
As long as water quality is and you do a water change before you go it will be fine
I suppose if it is a 1/2 inch fry a 5 gallon tank is fine but when it gets larger it will require a larger tank. Other than that it should be ok for 3 days without food (though generally speaking frys should be well fed for good growth). Having said that do not put a bunch of extra food in the tank just before you leave. It will foul the water and that is more likely to kill the fish than the lack of food.
Btw gold gourami will eat a little plant/algae matter so if your tank has plants (and some algae) it will munch on that if hungry.
As said I have returned 1.Now I have 1 left in my 5gallon tank