I've read (regarding amphibians, at least), that tannins MIGHT provide a mild antibacterial agent in the water. I don't know if I believe that, but some of the things I've read claI'm that this can be beneficial to some species, like neon tetras. A lot of the info on tannins is lore, as far as I can tell. I've read a lot of different things, and I haven't actually found solid support (beyond rumor, or one person's experience) to support any of it. The general consensus is that they're safe, as long as you're ok with the tinting to the tankwater. Actually, if anyone out there knows of any scientific papers/sites that actually study tanins, I would love to read more (beyond the word-of-mouth that I keep stumbling into.)
Adding Activated Carbon to your filter will remove the brown tea color from the water. Driftwood can lower your pH levels but probably not a great deal as mentioned above but I do suggest that you keep an eye on your levels.
I have a large piece in my 38g and a small one in my 10 and agree with what everyone here has posted. I don't feel personally there are any drawbacks health wise for your fish although like stated above you will get a tea coloring to the water. I have tried to boil(on the large piece) for hours and still ended up getting the brown. The benefits are it really does lower pH but only a small amount and I believe the driftwood is actually needed or suggested if you plan on having a Pleco. In my opinion nothing to worry about though.