a bag of polished stones from walmart for $4, but I may take them back...it says good for aquariums, but it was sold in the silk plant/vase/styrofoam section of the store - not the pet part of the store - so...I don't know...
Polished stones from WalMart work for me - depending on the fish. I use larger stones for my bettas, but I've had at least one smaller fish get stuck underneath them.
As far as decor goes, my plants cost $3 each, and I have three of them. I also have 99cent glass vases, two dollar glass cubes, and several 3" diameter PVC connectors. Cheap, easy... and the fish like them.
A $20 bag of polished smooth baby blue gravel for my 10 gallon Betta tank (needed polished gravel to protect his delicate tail) and about $15 for a Greek Temple Ruins ornament with airstone for my other 10 gallon Betta tank.
My ten gallon glass tanks were only $11.99 each, cheaper than the gravel or ornaments :-[
I will probably spend a bundle once I start landscaping with live plants!