When evaluating market research, there are several important concepts to keep in mind, as different research firms conduct their research in different ways, and your needs may differ according to your goals and internal capabilities. Some research is geared to those who know relatively little about their market and need lots of guidance. Other research is intended for directors of strategic product planning or product managers who have their own expertise but need reliable data to operate on. The following discussion examines some of the things to keep in mind when deciding what type of research you need.

EDC specializes in demand side primary research in the developer community. This is a unique service and has been filling a market need amongst Product and Marketing Managers, Directors, and VPs at major tool and platform companies for the last several years. No other firm specializes in this market focus and this technique. Most other well-known research firms in the high-tech arena (IDC, Gartner, etc) provide either secondary research or supply side research or sometimes both.

PRIMARY VS SECONDARY RESEARCH - Primary research and secondary are two entirely different services. EDC provides primary research.

Primary research always involves actually going out to the field and conducting some type of survey or fact finding investigation among the actual people in a given universe. This is what we do. We conduct thousands of interviews with actual real developers each month to create the reports in our subscription series. And we deliver the results quickly. EDC provides lots of very fresh data direct from the developers themselves.

Secondary research involves examining all the various types of research that may be available to the public, collecting the data, and usually forming some type of opinion about it. Many research firms specialize in this type of research and the main value they provide is in organizing publicly known trends and providing "experts" to give their opinions as to what it all means. Many companies who provide this type of research also do product reviews and paid white papers, etc for clients.

DEMAND SIDE VS. SUPPLY SIDE RESEARCH - These are two different types of research, each with its own particular strength.

Demand side research attempts to find out what the consumer (in our case the developer) wants, needs, thinks, uses, etc by directly contacting the developer and asking him through a survey format. This is the only type of research that can measure any of the patterns of actual perceptions, day to day usage, attitudes or intentions amongst the universe of developers themselves. This is the type of information EDC delivers. What this type of research cannot do is predict the total size of the universe of developers or the revenue of products shipped.

Supply side research is conducted by polling product vendors and asking them how many units of their products they have shipped within a given time frame and for how much money. IDC is the most well-known provider of this type of research though other firms do this too. This type of research can predict the market size and revenue of the total universe and can be used to estimate market share, etc. What it cannot do is give any insight into what the developers themselves are actually doing, thinking, what their needs, attitudes and perceptions are, or their intentions for adopting a particular technology.

When conducting demand side primary research it becomes important to recruit the participants (or samples) from sources that are as unbiased as possible. During the five years that EDC has been recruiting developers to participate in surveys this ideal has continuously been foremost in our efforts. Consequently, though we have used over 100 different individual sources for recruiting, the following principles have always been and will always be applied:

• No vendor lists have ever been used in EDC subscription surveys and none have ever been added to the panel
• No platform specific lists have ever been used in any EDC general subscription surveys and none have ever been added to the general panel*
• No language specific lists have ever been used in any EDC subscriptions surveys and none have ever been added to the panel

In this way we provide the most eclectic and unbiased sample available anywhere. With thousands of developers chosen in a deliberately unbiased way from a wide variety of neutral lists, our data truly provides in-depth looks at representative samples of the developer population.

Note: our Linux Development survey does use lists targeted for the Linux platform, however all developers recruited for that survey are kept in a separate database and are not used in any surveys other than Linux specific ones.

The EDC panel of developers includes about 20,000 professional developers in more than 70 countries. The EDC panel, having been recruited and assembled over the last five years strictly from neutral developer lists represents the finest example today of an unbiased and representative sample of developers. As the panel continues to grow, the same principle of neutral recruitment will continue to be applied, thus assuring clients of the most representative sample possible.

EDC uses an unweighted sampling and data processing technique. All of the respondents in the survey are treated as equals or without bias as to their circumstances.

All surveys have a confidence level and confidence interval or margin of error built in which can be determined by mathematical formulas. The confidence level tells you the degree of accuracy about the results of a survey. Confidence intervals are determined by sample size (or number of people who actually completed the survey).

Due to a rule of calculus, the confidence intervals of all populations having more than 300,000 have the same confidence interval depending on the sample size. The most useful values are as follows:

• For a confidence interval of (+/-) 3%, the sample size must be 1060
• For a confidence interval of (+/-) 4%, the sample size must be 600
• For a confidence interval of (+/-) 5%, the sample size must be 384

Thus if a survey has a sample size of 600 , then the confidence interval and the margin of error will by plus or minus 4%. Other values can be worked out for sample sizes in between the values listed above.

All EDC subscription surveys have sample sizes above 400, with some of the surveys ranging up to 800 respondents. Confidence levels and intervals should be taken into account when evaluating results of quantitative surveys. For example, if one answer has 1.5% more responses than another one in a survey of 500 developers then the two answers will fall within the margin of error and definitive relative rankings between the two should not be made.