Options could be soaking it in a water solution for a day, then add bleach. let the bleach sit for a few hours to a day, and then wash very well. I have done this when I ran tanks with artificial decor.
I would also recommend leaving the algae on the decoration, since I tried this with fake driftwood, and when the algae reconstitutes it will flake off and be good food for several fish species.
Or just soak in water and then brush off the algae.
Toothbrush and water is what I'd do. Maybe vinegar water. I don't like using any chemicals like bleach with my fish stuff cuz it scares me, so I've never done that. Hydrogen peroxide can work too. But I think just water and toothbrush would be fine for most.
I don't clean off anything but live plant leaves if necessary. I have a skull that came with my second-hand tank and it's turning brown, but I'll pry just leave it. There's good bacteria on there and the fish doesn't care what color it is.