The folks at Mozilla who put out the Firefox browser (which is great) understand the Trial and Error economy. They recently wrote up a description of an experiment they did in marketing their download web site. (Hat tip to Steve Levitt at Freakonomics.)
Here’s the test: if you go to a search engine to download the Firefox browser, you get the official website as the number one result from most of the popular search engines. So should they pay Google or Yahoo for a pay-per-click ad on those search results when they are already number one? It turns out that they get a little more traffic when they do.
One lesson from this, though, is that the majority of the traffic that comes from pay-per-click advertising would have come anyway; a majority but not all of the traffic. So your cost per customer is not really what you are paying per click; it’s your total costs of the clicks divided by the customers who would not have come to you without the pay per click.
In one experiment, it appears that about seven out of every ten people who clicked the paid advertisement would have clicked the regular search result (called "organic search") if the paid ad had not been present. So all of your click payments only brought you those three out of ten who otherwise would not have come to you.
It takes a lot of work to get a good experiment, and you often have to keep running more experiments to get the results fine tuned. But you can learn far more, and boost your profit tremendously, by regularly experimenting in your business.