Actually you could argue that diamonds are worth less than other gem stones (to anyone other than the initial seller) because De Beers have all but shut down any second-hand market, so, by the old adage that something is only worth what you can sell it for, they're worthless. Most jewelers won't buy a diamond off you, or sell second-hand diamonds as De Beers have told them they'll cut off their supply of new ones if they do.
Obviously there are shops that deal in second-hand but not those that need a shiny new supply.
I love buying my wife jewellary (no special occasion required) but she (luckily for my pocket) dislikes diamonds (and gold) and loves semi-precious stones in silver (as do I). For us, it's not the cost of the piece, it's the beauty of it, the style of the setting etc. She's more delighted with the beauty of a (human enhanced) piece of mystic topaz, or a particularly lovely bit of jasper, or iron tiger's eye, or quartz with some rutile inclusions, or moonstone or whatever than some mass produced, all looks the same, over-priced, colourless, boring diamond.