For standard basing supplies, one trip to the pet shop can most likely set you up for life - and very cheaply too.
In most cases you will not be spending very much, but you will get vast amounts of material, so if you know someone who has pets then try nicking a little of their stuff instead! Even cheaper!
So, what's on our shopping list:
Sand - usually in the reptile, occasionally in the bird section. You can get this at a garden centre as well, but at the pet store you will most likely get a wider choice of colours and consistency.
Bird Grit - Again in the bird aisle (surprisingly). Mix a little of this in with your sand to give the apperance of small rocks and stones. Gives a very natural look, and you'll get a large bag of this for about £1.
Aqua grit - The stuff used for the bottom of fish tanks. Again you'll get a large bag very cheap, but this is only really useful if you intend to do water or beach scenes and want pebbles. You will get a few jagged ones in there, and you get a fair mixture of rock types.
Kitty Litter - You'll want the non-clumping lightweight clay stuff, usually the cheapest one they have. You can get this at supermarkets in giant sacks, again very cheap, though somoene you know probably has some to hand.
Kitty Litter gives you a fantastic selection of natural sized rocks or rubble, anything under boulder size. You can also mix this up with glue to form a paste which makes awesome debris.
That's all folks, see you in the pet aisle.