6 traits of an investment scam:
1. They're Unsolicited.
Honest investment houses, and the brokers that work for them don't call out of the blue to sell you the next big thing.
2. It looks too good to be true.
If they salesman promises extraordinary rates of return, run! If someone is selling an investment that's up 20%, while its peers are down 30%, something is wrong. For example, a guy calls you up and tells you about a great bank stock that was up in the fall of 2008 (while every bank stock is imploding), it's a scam.
Bernie Madoff's hedge fund/ponzi scheme is another terrific example. By some accounts, that fund was up 15% year after year. The stock market as a whole averages 10% for a much longer time (and was actually down over much of Madoff's stellar returns).
3. The investment is a "penny stock", or microcap..
Both microcap and penny stocks have relatively low numbers of outstanding shares, and are easily manipulated. The classic penny stock scam is by far the "Pump and Dump." In a pump and dump, the scam artist buys hundreds of shares of a penny stock, for example: ACME Widgets Inc, then calls or emails hundreds of people (unsolicited) telling them that ACME Widgets Inc is going to make it big real soon, and they ought to get in on the ground floor before it's too late.
This becomes a self fulfilling prophecy as more and more people buy the stock and drive the price up. Once the price hits what the scam artist feels is the high point, he dumps the stock leaving everyone else holding the bag.
4. The person contacting you is overly pessimistic.
This is the opposite of the "pump and dump". Where the scam artist in the pump and dump is overly optimistic and tries to talk you into buying, this one is trying to get you to sell. This scam is commonly called a "short and abort" because it preys on investors fears to drive the price down so the scam artist can "short" the stock for a profit. This type of scam also tends to focus on penny stocks, though not always.
5. There really isn't an investment.
As silly as it sounds, scam artists do make a living tricking people into investing in stocks or real estate that simply does not exist. Do your research and verify that any potential investment actually exists - take nothing for granted.
6. The broker is not a broker.
A companion scam to the non-existent investment is the illegitimate broker. I think this one works because of all the others listed above. Most people pride themselves on being savvy investors, and know to look out for penny stock scams and the like. But this scam is all about the artist and his charisma. He's selling a real investment, but what people are buying is the salesman. By the time they realize they don't actually own the investment, the scam artist is long gone.
What you can do.
Always check a potential broker out at www.NASDR.com or www.SEC.gov. If they're legit, they'll have a record of it.
If you suspect a scam, check out www.fraud.org or www.fraudbureau.com. If you know you were scammed, you can file a fraud claim.
Get to know the sec and learn how investments work.
Finally, take a tip from Warren Buffet - only invest in what you understand.