Threats to Internal & External Validity in Quantitative Research Essay

Research Theory, Design, and Methods
Walden University
Threats to Internal Validity
(Shadish, Cook, & Campbell, 2002)
1. Ambiguous temporal precedence. Based on the design, unable to determine
with certainty which variable occurred first or which variable caused the other.
Thus, unable to conclude with certainty cause-effect relationship. Correlation
of two variables does not prove causation.
2. Selection. The procedures for selecting participants (e.g., self-selection or
researcher sampling and assignment procedures) result in systematic
differences across conditions (e.g., experimental-control). Thus, unable to
conclude with certainty that the “intervention” caused the effect; could be due
to way in which participants are selected.
3. History. Other events occur during the course of treatment that can interfere
with treatment effects and could account for outcomes. Thus, unable to
conclude with certainty that the “intervention” caused the effect; could be due
to some other event to which the participants were exposed.
4. Maturation. Natural changes that participants experience (e.g., grow older,
get tired) during the course of the intervention could account for the
outcomes. Thus, unable to conclude with certainty that the “intervention”
caused the effect; could be due to the natural change/maturation of the
participants.
5. Regression artifacts. Participants who are at extreme ends of the measure
(score higher or lower than average) are likely to “regress” toward the mean
(scores get lower or higher, respectively) on other measures or retest on
same measure. Thus, regression can be confused with treatment effect.
6. Attrition (mortality). Refers to dropout or failure to complete the
treatment/study activities. If differential dropout across groups (e.g.,
experimental-control) occurs, could confound the results. Thus, effects may
be due to dropout rather than treatment.
7. Testing. Experience with test/measure influences scores on retest. For
example, familiarity with testing procedures, practice effects, or reactivity can
influence subsequent performance on the same test.
8. Instrumentation. The measure changes over time (e.g., from pretest to
posttest), thus making it difficult to determine if effects or outcomes are due to
instrument vs. treatment. For example, observers change definitions of
behaviors they are tracking, or the researcher alters administration of test
items from pretest to posttest.
9. Additive and interactive effects of threats to validity. Single threats interact,
such that the occurrence of multiple threats has an additive effect. For
example, selection can interact with history, maturation, or instrumentation.
© 2016 Laureate Education, Inc.
Page 1 of 2
Research Theory, Design, and Methods
Walden University
Reference
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasiexperimental designs for generalized causal inference. Boston, MA:
Houghton-Mifflin.
© 2016 Laureate Education, Inc.
Page 2 of 2
You can find quantitative articles by searching in the Library databases using methodology terms
as keywords. To find a quantitative study, possible keywords include the type of study, data
analysis type, or terminology used to describe the results.
Example quantitative keywords
•
ANOVA
•
Distribution
•
Quantitative
•
Time series
•
Chi square
•
Experiment
•
Reliability
•
Validity
•
Correlation
•
Pretest
•
Statistical
•
Variable
•
Data
•
Posttest
•
T-test
•
Variance
The following search uses Thoreau, our multi-database search tool, to find examples of
quantitative research studies. However, you can search in any article or dissertation database for
quantitative studies.
1. On the Library homepage, type your general term in the main search box:
quantitative.
2. Sign in with your myWalden username and password when prompted.
3. Type more methodology terms in the first search box. Use as many alternative terms as
are relevant to your search. Use the remaining search box(es) to narrow your search to a
specific topic of interest.
4. Click the Search button.
Here is an example search set up:
First search box:
Quantitative OR Statistic* OR Correlation*
Second search box:
Post-Traumatic Stress Disorder
Here is an image that shows the search setup:
Search Tips
•
Connecting the alternative terms with OR tells the database to search for any of these
terms.
•
Using the asterisk (*) truncates the search. The database will search for the part of the
word you typed before the asterisk, along with any possible endings of the word. Using
statistic* tells the database to search for statistics, statistical, etc.
•
Some methodologies are rarely used for certain research topics. You may need to broaden
your search topic to find a study that uses your methodology.
•
Many articles and dissertations will include methodology terms in the abstract or title. To
make sure that you have an example of your methodology, be sure to look at
the methodology section in the full text. This will provide detailed information about the
methodology used.
To find more results or if you are searching for a very specific type of study design you can try a
different search setup.
1. Type your terms into the first search box.
2. To the right of that, change the Select a Field (optional) drop-down menu to TX All
Text.
3. Type your other keyword term into the second search box
For example:
First search box:
Pretest AND Posttest
Second search box:
post-traumatic stress disorder
Here is an image of the search set up:
Search Tip: Connecting these terms with AND tells the database to search for both of these
words.
Watch a video showing how to search for a quantitative article in the Library databases:
Video: Find Quantitative Articles
(2 min 2 sec) Transcript
More Information:
•
How do I find an article that uses a specific methodology?
•
How do I find a qualitative article?
•
How do I find a mixed-method article?
•
How do I find original research studies that include empirical data?
•
Learn more about methodologies by searching encyclopedias and SAGE Research Methods
Online.
Do you have other methodology search questions? Ask a Librarian!
Litmus Test for a Doctoral-Level Research Problem
Background on these “litmus test” questions
• The distinguishing characteristic of doctoral-level research (as opposed to masters level) is that it must
make an original contribution to the field. However, students may struggle to identify what will
authentically contribute to their field or discipline.
• The most critical step in making such a contribution is to first identify a research problem with the 4
doctoral hallmarks below. Identifying a doctoral-level research problem is “necessary, but not
sufficient,” to produce doctoral-level capstone.
REQUIRED DOCTORAL HALLMARKS OF THE RESEARCH PROBLEM
In Walden’s scholar-practitioner model, a research problem shows
promise of contributing meaningfully to the field ONLY if the answer
to ALL of the following questions is “yes.”
1. JUSTIFIED?
Is there evidence that this problem is significant to the professional field?
There must be relevant statistics (expressing an unjust inequality,
financial impact, lost efficiency, etc.), documentable discrepancies (e.g.,
two models that are difficult to reconcile), and/or other scholarly facts
that point to the significance and urgency of the problem. The problem
must be an authentic “puzzle” that needs solving, not merely a topic that
the researcher finds interesting.
2. GROUNDED
IN THE
LITERATURE?
Can the problem be framed in a way that will enable the researcher to
either build upon or counter the previously published findings on the topic?
For most fields, this involves articulating the problem within the context
of a theoretical or conceptual framework. Although there are multiple
ways to ground a study in the scientific literature, the essential
requirement is that the problem is framed in such a way that the new
findings will have implications for the previous findings.
3. ORIGINAL?
For research doctorates (Ph.D.):
Does the problem reflect a meaningful gap in the research literature?
For the professional doctorates (Ed.D. and D.B.A.):
Does the problem describe a meaningful gap in practice?
4. AMENABLE
TO
SCIENTIFIC
STUDY?
Can a scholarly, systematic method of inquiry be applied to address the
problem?
The framing of the problem should not reveal bias or present a foregone
conclusion. Even if the researcher has a strong opinion on the expected
findings, scholarly objectivity must be maximized by framing the problem
in the context of a systematic inquiry that permits multiple possible
conclusions.
Yes
No

Purchase answer to see full
attachment