Yes my local fish store is outrageous one small plant that you would put in a Betta tank is upwards of 10$ it's crazyI normally go on eBay to mine local fish stores are a total rip off!
I will have too look into that store and just see of they have anything on saleI actually have some river rock that I clean very well for one of my tanks (I have a few stones left over too), that I cleaned very well and scrubbed before letting dry out and then sticking into my tank (I did not bake them as I didn't trust them not to explode, my angels and kuhlies absolutely love them!). I have a few terracotta pots I got from walmart for under 2$ a piece. The rest has been bought driftwood (petmountain, the shipping is the expensive part, but not too bad specially if you catch them on sale).