Burn tests, as yyhhsj0521 points out, are done. If the object is made of keratin, it will smell like burning hair. I've never burned ivory, but I imagine it would also give off a unique smell (e.g., think drilling in a dentist office). Of course, as biotech advances, DNA tests and the like will be cheap and prevelant, and those will be used to establish authenticity.
There have been some substitutes developed for ivory, but they're readily identifiable. These substitutes include the tagua nut [1] and nitrocellulose plastic [2].
