The thing to do is to look at the composition of the alloy -- what different elements are in the mix. For the most part, metals in everyday use are not a single, pure metal. They have other metals and elements added to give specific properties, which is why you have such things as silicon bronze, free-machining brass (with a small amount of lead added to give it the free-machining quality), and so on. Straight aluminum tends to be very soft, so (if I remember correctly) most alloys have some copper and some zinc thrown in.
Now, once you know what's in the metal, you take a look to see if it matches any known Earthly-use alloys exactly. If there was a close match, I'd be suspicious. However, if the proportions were off considerably, and there were other elements thrown in, then at least you could reasonably entertain the suggestion that the item in question wasn't cut from an old Wear-Ever aluminum cooking pot.
That by itself doesn't prove an other-worldly origin. I've got a small foundry in the back yard, and could whip up some witches' brew allow out of the scrap in the shop waste can, cast a part, machine it to some interesting shape, and present it as a lug nut from a space cruiser. I might have a problem if I used a standard 4-40 thread in a screw hole, however.
Machine parts wouldn't be conclusive, because they are too easy for anyone with a minimal machine shop to counterfeit. Doesn't it strike anyone as peculiar that the only "parts" found from a spaceship are like gears, sheet metal scraps, and relatively primitive simple shapes? There are no examples of fasteners, screws, rivets, bonding agents such as adhesives, sealants, or glues. These are the items which would show something of the technology which produced the supposed craft. The supposed alien parts are always so damned simple, like flat or round shapes. No one has produced something that, while simple, requires a better-than-high-school-shop level of skill to produce, like a length of metal tubing, or some kind of pressure fitting or coupling.
A dedicated forger would have to develop their hoax piece entirely from the ground up, starting with the materials, to the geometries of the part (I'd immediately reject any gear with a standard pitch and tooth form), the dimensions (funny how the aliens use parts machined to the English measurement system, working in quarter or eighth inch increments, or are surprisingly Metric), down to what tools were used to make the part (they ALL leave distinctive, identifiable marks), cutting fluid residues, and lastly, just what the part is supposed to do.
Regards;
Beanbag