When a census, survey or poll is conducted, several factors can affect the results. These factors are called biases and can occur at any stage of the information gathering process. In order to avoid them, here is a list of the most common sources of bias.
|
|
|
|
This list is not complete. There are several other factors that can bias the results of a survey. However, this list presents the main sources of bias.
Here are some sources of bias that can arise when constructing a sample.
To ensure that the sample is representative of the population, it is important to have a reasonable sample size compared to the size of the target population.
Adequate sample size
To get the opinion of Quebec City residents, approximately |800\ 000| in total, on the construction of a new public transit system, a survey was sent to |5000| people by mail.
Inadequate sample size
To get the opinion of Quebec City residents, approximately |800\ 000| in total, on the construction of a new public transit system, a survey was sent to |100| people by mail.
Contrary to what one might think, the sample size does not necessarily depend on the size of the population.
In fact, the sample size is generally determined according to the degree of precision needed and the desired significance level. Also, with a random sample survey, the sample size rarely exceeds |1000| respondents, since the responses do not vary much and the costs involved are high.
Like sample size, it is important that the characteristics of the sample reflect those of the population. Before starting a survey, it is important to have a good understanding of the target population to construct a sample that resembles it.
Sample with good characteristics
To determine parents' perspective on road safety in school zones, residents who live within |2| kilometres of an elementary school are interviewed.
Sample not having the right characteristics
To determine parents' perspective on road safety in school zones, company owners in an industrial district are interviewed.
In this case, it is essential to interview people who are directly affected by the topic to add credibility to the investigation.
Here is one source of bias that can arise when making a questionnaire.
When writing survey questions, it is important to keep a neutral stance on the topic and be as concise as possible.
Proper formulation of a question
“How would you characterize the work of your MP?"
“Do you agree that Quebec should become a sovereign nation?"
Inadequate formulation of a question
"Do you sometimes think that your MP isn’t doing a good job?"
"Do you agree that Quebec should become a sovereign nation after having formally offered Canada a new economic and political partnership? This partnership is within the framework of the bill on the future of Quebec, from the agreement signed on June 12, 1995. Yes or no?"
In the first question, it is obvious that the amount of negation as well as the use of the word "sometimes" makes the question more difficult to understand. The second example seems far too long and refers to events that some people might not know about. In both cases, it would be difficult to answer the question spontaneously and honestly.
Here are some sources of bias that may arise when collecting information.
To ensure the most honest answers possible, it is important that the interviewer not overtly support their question(s) with arguments or an attitude that could influence the respondents.
Investigator with proper attitude
“Hello! I work for an independent survey firm and I would like to speak with you to get your opinion on the quality of the police work in your municipality. Do you have a few minutes for me?"
Investigator with inadequate attitude
"Hello! I have just been the victim of a theft at my home, and I would like to speak with you to get your opinion on the poor quality of police work in our municipality. Do you have a few minutes for me?"
In the second example, it seems obvious that the investigator wants to gather information in order to criticize the work of police officers. It follows that the respondent might feel uncomfortable giving answers that are contrary to the interviewer's goal. Therefore, the respondent may change his or her answers to agree with those of the interviewer and the results would be biased.
For some respondents, in order to give a truthful opinion, they need to know that their answers and their identity will be kept confidential.
Respecting confidentiality
A study was conducted in the city of Montreal and the results showed that |9\%| of people love baseball.
Failure to respect confidentiality
A study was conducted in the city of Montreal and the results showed that not everyone on Monardes Street likes baseball.
Knowing that their answers could be openly revealed, many respondents may be worried about potential consequences and may refuse to answer or simply give incorrect answers. If the responses do not represent reality, then the entire survey loses its value and credibility.
It is important to choose a suitable time and place during the day to ensure a good response rate from respondents.
Opportune time and place
A survey firm sets up its kiosk in the main hall of a shopping centre and interviews people who are not carrying any purchases.
Inopportune time and place
An investigator asks people exiting a grocery store questions as they struggle to carry bags full of food back to their cars.
By approaching those not carrying any purchases, the investigator has a better chance of getting their cooperation than interviewing a customer who has their hands full and is doing their best not to drop anything.
Here are some sources of bias that can arise from the analysis of responses.
For a proper analysis of the results, the interviewer must collect enough data to paint a fair picture of people's opinions. To do so, the interviewer must do everything possible to ensure that respondents properly complete the survey.
Good response rate
By completing the survey, respondents had the chance to earn travel credits with the airline of their choice. As a result, |90\%| of those approached agreed to answer the questions.
Low response rate
Having no motivation to complete the survey, only |30\%| of those approached took the time to complete it.
A low response rate reduces the credibility and value of the interpretations of the results. Therefore, caution should be exercised in making decisions related to the survey. In other words, there is a risk that many people will be dissatisfied.
Despite all the goodwill of the interviewer, some respondents may provide answers that make no sense in order to complete the survey as quickly as possible.
Reliable answers
|63\%| of respondents chose "dog" in response to the question: "What is your favorite pet?"
Unreliable answers
|2\%| of respondents chose "elephant" in response to the question: "What is your favorite pet?”
As a data analyst, it is essential to keep all the results obtained. However, if some data appears abnormal, it is preferable not to draw any conclusions. If too many responses seem unreliable, it may be wiser to modify the survey and restart the process.
At this stage of the survey, it is important to present the overall picture of the results. In other words, avoid isolating a result to create a stir or to spread a false public opinion.
Good presentation of the results
In the late-night newscast, the results of each of the five questions on a death penalty survey are presented to viewers.
Poor presentation of results
In the late-night newscast, only the result of the question "Are you for or against the death penalty?" was presented to viewers.
In this case, the death penalty is a delicate subject. Some people may be in favour only under very specific conditions. Therefore, It is important to mention these conditions to paint a fair portrait of public opinion.