Yep, what species? And what do you mean by gravel? What size? All my fish "eat" sand and spit it out. They are foraging for food hidden on the gravel which is normal. I have heard of goldfish choking on big gravel. I guess if your fish can put decent sized gravel in your mouth they are kind of big.
Fish like African cichlids build nests by moving around sand/gravel while goldfish (carp descendants) are natural bottom feeders which means they vacuum the bottom of river/lakes in search of food. It seems like instinct to me. I personally wouldnt remove the gravel cause the possibility is not enough for me but its there.