Opinion and analysis of Reza Behrouz and Mohammad R. Jahan-Parvar on IranPoll and the pitfalls of opinion polls in an authoritarian environment.
————————————————– ————————————————– ——–
Every 6 to 12 months, a survey is conducted by a private Toronto-based company called the Iran Poll in collaboration with the University of Maryland Center for international and security studies. This is a telephone opinion poll of people residing in Iran. The results are then published online and almost always presented by Washington think tanks such as the Atlantic Council. Despite a partnership with a major American research university, the results of Iran Poll are never subject to academic review or published in a peer-reviewed journal.
Iran Poll prides itself on using solid scientific knowledge methodology, which includes sequential randomizations, a catch-all geographic scope, and the ability to interview in multiple ethnic languages and dialects, in addition to Persian. Yet the intent of the survey and its results often arouse controversy and outrage, rather than offer practical information conducive to policy making.
In reaction to Iran Poll, there is always a deluge of indignation on the part of Iranians inside the country and in the diaspora who consider the results far-fetched and detached from reality. The main argument raised by the Iranians against Iran Poll is the unrealistic expectation that politically oriented investigations can ever be conducted in a totalitarian dictatorship such as the Islamic Republic. Opinion polls administered in authoritarian states naturally question the authenticity and honesty of participants when answering politically sensitive questions. There is no doubt that the fear of retaliation influences the way respondents respond to certain politically sensitive questions.
There are also features associated with Iran Poll which are scientifically wrong. The survey methodology was briefly described on IranPoll’s website, leaving ample room for doubt as to the validity of the data collection process. For example, participants are selected from Iranians who have access to landlines, excluding a large segment of the population who exclusively use mobile devices. This reduces the test sample and introduces bias from the start. The process for obtaining participants’ telephone numbers is of course not detailed. It should be noted that each landline number is attached to a physical address, which could theoretically be accessible by the security apparatus of the Islamic Republic. This factor can influence the authenticity and integrity of responses.
Demonstration in Tehran. July 16, 2021
The investigation goes through a succession of awkward randomizations to register the participants. Excessive stratification using multiple randomizations eliminates balanced participant enrollment, counteracting the effect and value of random sampling.
Iran Poll States, “[w]When a residence is reached, a qualified respondent is randomly selected from that household, often using the random array technique. Iran Poll the documentation does not describe the inclusion criteria for the “qualified” respondent. Assuming that being an adult is the primary criteria, randomization would fail if a household consists of only one adult. This riddle also applies to a situation where the randomly selected respondent is disabled and unable to participate. It is not known how these situations are handled by investigators.
The cooperation rate for the latest Iran Poll poll was 79%. The typical cooperation rate in public opinion polls is around 30%, with anything above 50% being considered “excellent”. An unincentive cooperation rate of almost 80% is astronomical, given Iran’s stifling political atmosphere. We are forced to consider this result as “too good to be true”.
The same argument can be made about the overall response rate of 60%. A realistic response rate is between 5% and 30%. A response rate of 60% and a cooperation rate of 79% does not mean that every respondent has completed the survey and answered all the questions. If the survey had been offered online, these numbers would be more convincing, but it is inconceivable that such a large proportion of people would hold the phone line long enough to complete the survey in its entirety. This is why nowadays multi-questions surveys are usually conducted online (or on paper forms), while polls are mainly carried out by telephone request.
Iran Poll of course does not offer any information on completion rates. Assuming the cooperation rate means answering one or more survey questions, Iran Poll intentionally conceals the speed at which individual questions were answered. The more participants refrain from answering a specific question, the smaller the sample size per question and the less accurate the results. For example, if only 50 people answered the question about the favor of President Ebrahim Raisi, it would take 39 favorable opinions to reach the rate of 78% demonstrated in the results. Fifty respondents constitute only 5% of the entire sample. This pitfall applies to all survey questions; from President Joe Biden’s popularity among the Iranian people to his opinion on US sanctions and the nuclear deal.
Besides, Iran PollThe results of s contrast sharply with the results of multiple-question online surveys such as the Netherlands-based polls GAMAAN (Group for analysis and measurement of attitudes in IRAN). For example, the proportion of respondents in Iran Poll who have indicated their intention to vote in the presidential elections of June 2021 exceeds that of GAMAAN by 24 percentage points. Another example is the level of trust in the broadcasting service of the Islamic Republic of Iran (IRIB). Iran Poll indicated that around 50% of their respondents trusted IRIB reports “most of the time”, while the number of GAMAAN was only 14%.
GAMAAN’s methodology certainly has its own shortcomings, but its online format alleviates the challenges associated with multi-question telephone surveys. Research has consistently shown that respondents tend to give more extreme and positive responses to attitude items than when asked the same questions over the web. There is also indications that the social interaction inherent in a telephone interview can put pressure on respondents, affecting the way they answer questions. Although the glaring discrepancies between the results reported by the two surveys can be attributed to ” trend “, the differences in responses by survey mode are generally not significant. Thus, the extent of the differences between GAMAAN and Iran PollThe results are too important to ignore.
Overall, there are notable flaws in IranPoll ‘s methodology, as well as the way the results are presented. To our knowledge, the method by which this survey is conducted has never been validated. In other words, its replicability and generalizability are highly questionable. Iran Poll correlates the demographics of its sample with that of the Iranian national census and the Central Intelligence Agency’s Factbook, and attempts to validate by matching the statistics it generates with the official data disclosed by the regime itself. For example, Iran Poll produced a turnout of 52.9% in the presidential elections of 2021. This figure was then juxtaposed with the turnout of 48.9% reported by the Islamic Republic in an attempt to ostensibly demonstrate accuracy and reproducibility. It is impossible to ensure the credibility of the electoral participation declared by the regime when the elections are far from free and fair.
Finally, it is also worth understanding who are the key players behind the University of Maryland program. Iran Poll. To this end, we encourage the reader to consult a item by Ahmad Rafat in Kayhan’s life.
Given Iran PollBecause of its socially reprehensible results, its methodological flaws, the way the results are presented, and the fact that it escapes peer review and scientific scrutiny, an impartial observer is compelled to consider the ‘survey with suspicion. Such an observer can reasonably conclude that this investigation is little more than a propaganda ploy by the Isalmic Republic to influence US policy towards the regime. The data Iran Poll provides that each cycle is outrageously predictable and often downright absurd. Many Iranians and members of the diaspora see this investigation as a source of humor and an object of ridicule. Think tanks that extol Iran Poll as a stunning and genuine revelation from inside Iran should thoroughly examine the merits and validity of this investigation. Likewise, the University of Maryland should consider whether associating its name with such a scientifically dubious and politically controversial project is in the institution’s best interest.
Dr Reza Behrouz is an Iranian-American physician and medical researcher based in San Antonio, Texas, USA.
Mohammad R. Jahan-Parvar PhD is an Iranian-American economist based in Washington, DC, USA.
The opinions expressed by the authors do not necessarily reflect the views of Iran International