This page contains all of my notes related to scientific methods and research. My goal is currently to develop a strong foundation for scientific thinking.
Quantitative Methods (Course 1 Notes):
Source(s): Statistics and Research Methods: University of Amsterdam (Specialization) Course 1 Quantitative Methods
Vocab:
- Empirical: Verified by observation/experience rather than theory/logic
- Scientific Method Principles:
- Empirically Testability: Physical Evidence or Observations that either support or contradict hypothesis
- Replicability: Repeatedly confirmed
- Objectivity: A clear and explicit definition of procedure and methods should produce the same results.
- Transparency: All procedures, definitions, and assumptions are publicly shared.
- Falsifiability: There is a possibility for there to exist evidence that contradicts the hypothesis.
- Logic Consistency: The logical basis of a hypothesis and its subsequent conclusions are logically sound.
- Observation: General relation between properties without explanation
- Hypothesis: General relation between properties with explanation
- Law: Super precise description of relation between properties; typically mathematical formula
- Theory: Overarching explanation of many related phenomena
- Universals: Abstract and unobservable properties (e.g. love, gravity)
- Particulars: Specific examples/instances of universal (abstract properties) (e.g. object falling, cat that doesn’t like to be pet sitting next to you)
- Negative Case Analysis: Seeking contradictory evidence for a hypothesis.
- Summative: Focus on the outcome of a program
- Formative: Focus on the on the process of a program
- Construct: An abstract concept that cannot be directly measured.
- Variable: A specific aspect/instance of a construct.
- Operationalization: A specific and concrete way/method to measure a variable.
- Measurement: representation of the relations between things using numbers
Concepts:
Philosophical Realism: The belief that the external world exists independently of human thought.
Essentially schools of thought from which the scientific method emerged.
Epistemology: How can we know it? What is accessible and can be accessed? Study of knowledge
(1.) Rationalism: Physical world is an imperfect reflection of (ideal) reality. Knowledge about reality can be attained through reasoning. Innate knowledge. (Nature?)
(2.) Empiricism: Physical world is the reality. Knowledge about reality can be attained through observation via the senses. Sensory knowledge. (Nurture?)
(3.) Constructive Empiricism: Acknowledges unobservables and states that the truth/falsity cannot be determined as claims are provisional (evidence can always come up).
Ontology: What is there to know? What exists or is real?
(1.) Materialism: The world is made of matter, including our mental processes (brain interacting in physical world).
(2.) Idealism: The world is in our minds. The world is a mental constructions/representations.
(3.) Realism: The world is made of matter, but unobservables/universals do exist. Platonic Realism states they exist on a separate abstract plane. Scientific Realism states they can be used to support claims
(4.) Nominalism: The world is made of matter, but unobservables/universals do not exist. Gravity is simply a term that represents something falling.
Research Designs and Internal/External Validity
Threats to Internal Validity:
1. Maturation: Time can serve as an alternative explanation (for example, aging, healing, growth, development, etc.)
2. Mortality: Participant drop out can skew results (or affect outcomes).
3. History: Something could happen in the world that affects the participants in the study (a study on health is affected by the covid outbreak) and therefore, affecting the outcome of the study.
4. Instrumentation: The instrument could be modified (different pre-test vs. posttest) or have an error that affects the outcome.
5. Testing: Testing can change participant perception (sensitize them to the questions, accidentally reveal the group they are in, etc.) which could affect the outcome.
6. Regression to the mean: A statistical phenomena where previously extreme values tend to be closer to the mean/avg in subsequent tests (skewing differences between pre-test and post-test).
7. Bracketing:
8. First Order Constructs:
9. Second Order Constructs:
Designs
Purpose of a research design is to prove that we are not only measuring what we intended to, but we “prove” (or provisionally prove) our hypothesis.
Three important components are:
1. Manipulation: Being able to control the cause implementation allows us to study it.
2. Comparison: Being able to compare the cause implementation allows us to understand when it’s not there or when circumstances change (relativity).
3. Randomization: Being able to apply it to a randomized sample allows us to reduce any biases/confounding variables that would serve as alternative explanations.
a. Random Selection vs. Random Assignment: Random Selection pertains to the process of choosing participants randomly (choosing participants randomly). Random Assignment pertains to the process of choosing participant’s group randomly (organizing already chosen participants randomly).
Types of Designs
- Pretest Posttest Control Group:
a. Randomly Assigned.
b. Two Groups:
i) One group gets pretest, treatment, and posttest.
ii) One group gets pretest and posttest.
Purpose: To determine if the treatment had an effect. - Solomon Four Group Design:
a. Randomly Assigned.
b. Four Groups:
i) One group gets pretest, treatment, and posttest.
ii) One group gets pretest and posttest.
iii) One group gets treatment and posttest.
iv) One group gets posttest.
Purpose: To determine if the treatment and testing had any effects. - Placebo Control Group Design:
a. Randomly Assigned.
b. Pretest and posttest optional.
c. Three Groups:
i) One group gets pretest, treatment, and posttest.
ii) One group gets pretest and posttest.
iii) One group gets pretest, placebo, and posttest.
Purpose: To determine if the treatment and perception had any effects. - Repeated Measures:
a. Only one group of participants; order of treatments is randomly assigned.
b. All participants receive same set of treatments in randomized order.
c. Can be compared within subjects.
Typically:
i) One group gets pretest, treatment, posttest, another treatment, posttest, . . .etc.
ii) One group gets pretest, treatment, posttest, another treatment, posttest, . . .etc.
iii) etc. . .
Purpose: To determine if the treatments had any effects within the same group and compare treatments within group. - Switching Replications Design:
a. Randomly Assigned.
b. Two Groups:
i) One group gets pretest, treatment, and posttest.
ii) One group gets pretest and posttest.
c. Then roles are switched.
i) Control then gets treatment, and posttest.
ii) Experimental then gets posttest.
Purpose: To determine if the treatment had any effects compared to control, replicate treatment, and determine long term treatment effects. - Multiple Experimental Group with One Control Group Design:
a. Randomly Assigned.
b. More than 2 Groups:
i) One group gets pretest, treatment, and posttest.
ii) One group gets pretest, treatment 2, and posttest.
iii) etc. . .
ii) One group gets pretest and posttest.
Purpose: To determine if the treatment had any effects and compare different treatments. - Factorial Design:
a. Randomly Assigned.
b. Multiple independent variables, each with levels.
c. Treatments are the number of levels of each independent variable multiplied.
d. Groups:
i) One group gets pretest, combination (treatment), and posttest.
ii) One group gets pretest, combination 2, and posttest.
iii) One group gets pretest, combination 3, and posttest.
iv) etc. . .
Purpose: To determine if the treatments had any effects and compare the combined effects of each group. - Quasi-experimental Design:
a. NOT Randomly Assigned.
b. Non-equal groups.
c. Sometimes no manipulation/comparison.
d. Investigates causal relations.
Purpose: To determine if the treatments had any effects within the constraints of the groups. - Non-equivalent Group Design:
a. Naturally occurring groups; not randomly assigned.
b. Two Groups:
i) One group gets pretest, treatment, and posttest.
ii) One group gets pretest and posttest.
Purpose: To determine if the treatment had any effect. - Double Pre-test Design:
a. Naturally occurring groups; not randomly assigned.
b. Two Groups:
i) One group gets pretest, posttest, treatment, and posttest.
ii) One groups gets pretest, posttest, treatment, and posttest.
Purpose: To determine if the treatment and testing had any effect. To also determine maturation. - Proxy Pre-test Design:
a. Different measures for pre-test/posttest; used another measure.
b. Two Groups:
i) One group gets pretest, treatment, and posttest.
ii) One groups gets pretest and posttest.
Purpose: To determine if the treatment had any effect and save resources. - Multiple Time-Series Design
a. Multiple tests over a long Period of time.
b. Two Groups:
i) One group gets test, test, test, test, treatment, test, test, test, test.
ii) One groups gets test, test, test, test, test, test, test, test.
Purpose: To determine if the treatment had any effect long term and also determine maturation if any. - Switching Replications Quasi-Experimental Design
a. Naturally occurring groups; not randomly assigned.
b. Two Groups:
i) One group gets pretest, treatment, and posttest.
ii) One group gets pretest and posttest.
c. Then roles are switched.
i) Control then gets treatment, and posttest.
ii) Experimental then gets posttest.
Purpose: To determine if the treatment had any effects compared to control, replicate treatment, and determine long term treatment effects.
Correlational Designs
- Cross Sectional: Measures a section of the population at a given time
- Time Series: Measures one individual at several points in time
- Panel: Measures a group of individuals at several points in time
Other Designs
- Case Study: One group or one individual
- Evaluation Research: A type of applied research that evaluates the effectiveness of a policy/program
- Intervention Studies: A type of experimental research that evaluates the effectives of a treatment for individuals
- Validation Research: A type of research that evaluates quality of an instrument
Levels of Measurement
- Nominal: Whether or not things are the same. (Categorical)
- Ordinal: There is an order for things.
- Interval: No meaningful zero value and differences between values that are interpreted.
- Ratio: A meaningful zero value.
Qualitative Methods (Course 2 Notes):
Vocab:
- Naturalism: Research in natural settings.
- Normative: Opinion based.
- Verstehen: Deep understanding.
- Gestalt perception: The whole (of actions) is more than the sum of its parts (acts).
Ontology in Social Settings:
1. Objectivism/Realism: There is a social reality independent of human thought that does not rely on people for it to exist.
2. Constructivism: There is a social reality that is a not independent of human thought that is constructed by people. While people individually interact with one another, these interactions become shared meanings across groups.
3. Pragmatism: Social reality is best understood by its usefulness.
4. Subtle Realism: The observer is part of the social reality they observe, but the social reality exists independently of human thought. Therefore, the knowledge acquired is limited/error-prone by the fact that the observer is a part of it.
Epistemology in Qualitative Research:
1. Positivism: Gain knowledge through senses (empiricism), deductive and inductive (theory making and testing), and objectivity. Scientific vs normative statements (uses scientific ones).
2. Interpretivism: Meaning is subjective and interpretation of the world varies from individual to individual.
Hermeneutics:
The process of understanding the interpretation/meaning of a text as well as the text itself. In the social sense of the idea, this means understanding the cultural/social context surrounding an individual to understand better the individual and the world in which they exist. The main goal of hermeneutics is to interpret meaning. It can be circular (Hermeneutics Circle) where understanding the individual informs the understanding of the world and vice versa.
Phenomenology:
The study of phenomenon as the subjective and conscious experience that it is for people. It requires a description of the actual experience, an interpretation of this experience in relation to other experiences, and an analysis on its meaning. First order constructs are the phenomenon as seen and experienced by the participant first hand. Second order constructs are the phenomenon and participant experiences as seen by the researcher (interpretation).
Pragmatism
Knowledge is valued if it is useful and has effects/consequences in the world. Knowledge should serve as a solution for problems that serve a purpose for society. Truth is what best works in practice consistently and is relevant to the real world.
Types of Observation:
– Ethnography: Researcher is part of a social setting for a long period of time
Main Method: Participant Observation
Other Methods: Direct Observation, Unobtrusive Observation, Open interviews, Biographical Interviews, Focus Group Interviews, Document Analysis
Objective: To understand the people in the study
Content: Both method and written products
A kind of fieldwork.
Participant Observation:
1. Types of Data:
– Field notes, Pictures, Audio, Written, Documents
– Naturally Occurring data (excluding small amount of reactivity)
– In-depth data
– Contextualized data -> Broad within a context (social and historical)
– Also provides context to survey results.
2. Important Elements:
a. Participant:
– Background
– Skills
– Impression Management -> depends on the research and situation. Defined role that helps navigate social situations. Affects access and reactivity. Be more naive/dumber than you are in order to learn.
b. Observation:
– Watching
– Listening
– Smelling
– Feeling
– Tasting
– Posing Questions
Other:
– Writing
To conduct participant observation, personal background matters in the case of reactivity. Personal skills also come into play which ties into both access and reactivity.
Observing participant vs. Participating observant
Different data when you are simply observing the social situation vs. when you are personally interacting with it.
Observation
Using your senses to gain information?
Focus is essential but it could mean selection and thus, exclusion.
Paradigms:
1. Simmel -> Focuses on forms of interaction and social types
2. Goffman ->Focuses on how people act, interact, and form relationships to determine meaning
3. Lofland -> Focuses on city and public/private
4. Crapanzano -> Focuses on self-observation and ethnography (the self as a member of society)
5. Garfinkel, Sacks -> Focuses on how individuals’ interactions shape everyday life (how individuals create social order)
Jevaria Onymous 