KIN 605/Research Methods 
 (Last Updated On: 11-18-01) 

Texts: Research Methods in Physical Activity by Jerry Thomas & Jack Nelson; Statistics in Kinesiology by William J. Vincent

Instructor: Dr. Ann Maliszewski

Website As: www.ronjones.org/csun605notes.htm

Abstract:  Paragraph of concise overview with summary of research question.

·        Summarizes the study, especially the question asked, the method, and a brief account of the findings.

Analytical Research:  Research that involves in-depth study and evaluation of available information in an attempt to explain complex phenomena; can be categorized in the following way: historical, philosophic, review, and meta-analysis.

Annotated Bibliography:  List of resources that provides a brief description of the nature and scope of each article or book. 

Applied Research:  Addresses immediate problems for improving practice.  The worth of applied research must be evaluated on the basis of its contribution to the solution of some immediate problem.  Has direct value to practioners but in the researcher has limited control over the research setting

Assumptions:  (p. 56) Assumption that certain conditions exist and that the particular behaviors in question can be observed and measured.

·        Without certain fundamental assumptions or premises, a study could not proceed or take place. 

Basic Science Research:  Does not have immediate social significance.  May have limited direct application but the researcher has careful control of the conditions.

Categorical Variable:  Kind of IV that cannot be manipulated, such as: age, race, sex, and so on; also called moderator variable. 

Conclusion:  Includes “new” research question and hypothesis #1 & #2

Control Group:  Group of participants who are in the experiment but do not undergo the experimental procedures. 

Control Variable:  Factor that could possibly influence the results and that is kept out of the study.

Critique:  The “discussion”

Deductive Reasoning:  (p. 29) Moves from a theoretical explanation of events to specific hypotheses that are tested against (or compared with) reality to evaluate whether the hypotheses are correct.

Delimitations:  (p. 56) Limitation imposed by the researcher in the scope of the study; a choice the researcher makes to define a workable research problem.

·        Limits of gender, age, etc.

Dependent Variable: Depends on something you are manipulating i.e. depends on the independent variable.

·        The effect or consequent of the IV; also called the yield.

·        Variable whose value is partially determined by the effects of other variables.  It is not free to assume any value.  It is usually the variable that is measured in the research design. 

Diction: (p. 44)

Discussion Section of Research Paper:  Chapter or section of a research report that explains what the results mean (also the critique) that is the most important section of paper.

·        Discusses general findings of the study and whether the findings support the previous literature on the topic. 

·        Interprets meaning of findings and compares to previous studies and with hypothesis stated in the intro.

·        Can give opinion or speculation on topic and how findings may have practical use.  May also discuss ways to conduct future studies. 

Ecological Validity:  Extent to which research emulates the real world. 

Empirical:  Describes data or a study that is based on objective observations.

Epidemiology Defined: (p. 293) The study of the distribution and determinants of health related states or events in specified populations and the application of this study to the control of health problems.

Experimental Research:  Research that involves the manipulation of treatments in an attempt to establish cause and effect relationships. 

External Criticism: (p. 216) Establishes the authenticity of a primary source. 

External Validity:  The generalizability of the results of a study i.e. to what extent the results apply to the “real world.”

Extraneous Variable:  Factor that could affect the relationship between the IV and DV but that is not included or controlled. 

Generalizability: (p. 351) When reader of study evaluates the descriptions and analysis and determines what things apply to his or her situation. 

Human Subjects Committee:  (p. 76) Sets ethical standards and guidelines for research participants.

Hypothesis:  The anticipated outcome or expected results of a study or experiment.

• When a person sets out to do a study, they generally have an idea what the outcome or anticipated solution will be based on some theoretical construct, results of previous studies, or past observations or experience.

·        Hypothesis must be testable and designed so it can be either supported or refuted. 

·        Deduced statement always in present tense that specifies the direction the data will take when the study is conducted. 

·        Researcher’s best guess based on reading and thinking at specifying the direction that he or she believes the data will go. 

·        Deduced from theory or induced from other empirical studies and real world observations. 

·        Based on logical reasoning and, when predictive of the study’s outcome, are labeled research hypotheses. 

Independent Variable:  The part of the experiment that the researcher is manipulating; also called the experimental or treatment variable.

·        The one the researcher is trying to understand.

·        A Categorical Variable (also called a Moderator Variable) is a kind of independent variable except that it cannot be manipulated, for example, age, race, or sex. 

·        Variable that is free to vary and that is not dependent on the influence of another variable; a variable in the research design that is permitted to exert influence over other variables (i.e. the DV) in the study.  The IV is usually controlled by the research design. 

Inductive Reasoning:  (p. 27-29) Individual observations are tied together into specific hypotheses that are grouped into more general explanations that are then united into theory.

·        Many individual studies are needed to move from the specific hypotheses.

·        Deductive Reasoning moves down from the theory to specific hypotheses.

Informed Consent:  (p. 85)  Basic elements are:

·        Fair explanation of procedure to be followed, including an ID of those that are experimental.

·        Description of the attendant discomforts and risks.

·        Description of the benefits to be expected.

·        Disclosure of appropriate alternative procedures that would be advantageous for the participant.

·        An offer to answer any inquires concerning the procedures.

·        An instruction that the participant is free to withdraw consent and to discontinue participation in the project or activity at any time.

·        No exculpatory language through which the participant is made to waive, or appear to waive, any legal right or to release the institution or its agents from liability or negligence. 

Internal Criticism: (p. 216) Deals with trustworthiness of the information found in primary source of historical research.

Internal Validity:  The extent to which the results of a study can be attributed to the treatments used in the study.

Introduction:  (p. 50) Includes specific thesis statement i.e. “purpose” that is a declarative sentence but is written to be forceful, simple with direct vocabulary and not too technical.  Problem statement follows the intro unless a literature review is done in the intro.  In that case, a brief problem statement must soon appear before the actual literature review in the introduction. 

·        Intro paragraphs must create interest in the study.

·        Paragraph Sections: general intro, background info, lead in.

·        Provides reader with RV of literature and discussion of problem we will deal with.

·        Usually begins with general problem and moves to the specific research question the study asks.

·        Defines key terms and conceptual definition to IV and DV and other variables. 

·        Anchored in theory but don’t stretch out.  Use subheadings.

·        Should always end with statement of problem or question you will address in the study and the specific hypothesis.

Limitations:  (p. 56) Possible shortcoming or influence that either cannot be controlled or is the result of the delimitations imposed by the investigator. 

·        Can’t control these i.e. weather, attitudes, truthfulness of N, etc.

·        Researcher tries to reduce limitations that might stem from faulty procedures. 

Literature Review: (p. 42) 3 basic parts: intro, body, summary and conclusions that demonstrate that your problem needs investigation and that you have considered the value of relevant past research in developing your hypothesis and methods.

·        Don’t isolate the studies; put all individual studies together for a central finding or position.

·        Major part of developing research problem is reading what has already been published about the problem.  This helps to “identify” the specific problem.

·        Introduction: Should explain the purpose of the RV and how and why of its organization.

·        Body: Should be organized around important topics.

·        Summary: Summarize important implications and suggest directions for future research.

Measurement:  All measures have true score plus error. 

·        Accuracy:  Use methods valid and reliable.  Use pilot tests to evaluate validity and reliability of new instruments.  Go to original sources.

o       If a measurement is accurate, then it is both valid and reliable i.e. it is “consistently on target.” 

·        Systematic Error or Bias:  Due to something in the environment that alters performance in a given or systematic direction e.g. elevated temperature will lead to increased HR.

·        Random Error or “Noise”:  Mood of the participants e.g. some will be good and some will be bad.

o       Affect on the group mean will NOT tend to be in one direction therefore cancels out. 

·        Minimize Error: Conduct pilot tests (ask for feedback about environment and difficulty; evaluate accuracy of equipment).  Train testers/technicians.  Double check data at initial recording, when entered into computer, at every opportunity!

Measurement & Affective Behavior: (p. 194)

·        Likert Scale: Type of closed question that requires subject to respond by choosing one of several scaled responses; the intervals between items are assumed to be equal.

·        Semantic Differential Scale: Used to measure affective behavior in which the respondent is asked to make judgments about certain concepts by choosing one of seven intervals between bipolar adjectives i.e. the coach is “creative to unoriginal” on a 7-point scale.

Measurement Error: (p. 185) ME results from four sources.

1.      Participant: (mood, motivation, fatigue, health, fluctuations in memory and performance, previous practice, specific knowledge, familiarity with test items)

2.      Testing:  (lack of clarity or completeness in directions, how rigidly directions are followed, whether supplementary directions or motivation is applied)

3.      Scoring:  (competence, experience, dedication of the scorers and to the nature of scoring itself)

4.      Instrumentation:  (inaccuracy and lack of calibration of mechanical and electronic equipment, inadequacy of a test to discriminate between abilities and to the difficulty of scoring some tests)

Measurement Error & Rating: (p.195)

·        Central Tendency Errors: Inclination of the rater to give an inordinate number of ratings in the middle of the scale and thus avoiding the extremes of the scale.

·        Halo Effect: Threat to internal validity where raters allow previous impressions or knowledge about a certain individual to influence all ratings of that individual’s behaviors. 

·        Leniency: Tendency for observers to be overly generous in rating.

·        Proximity Error: Inclination of rater to consider behaviors to be more nearly the same when they are listed close together on a scale than when they are separated by some distance i.e. the different phases of behavior are rated the same. 

·        Observer Bias Error: Inclination of a rater to be influenced by his or her own characteristics and prejudices. 

·        Observer Bias Error: Inclination of rater to see evidence of certain expected behaviors and interpret observations in the expected direction.

Measurement & Standard Error: (p. 190-192) Every test yields only “observed” scores.  We can obtain only estimates of a person’s “true” score.  It is much better to think of test scores as falling within a range that contains the true score.

·        But how do we compare one score of one test to another score to a different test?  The scores must be converted into “standard scores” and expressed in terms of standard deviations from the mean.  You can determine standard scores by using Z scores or T scales. 

o       Z Score: The basic standard score that converts raw scores into units of standard deviation were the mean is 0 and the SD=1.0.

o       T Scale: Type of standard score that sets the mean at 50 and the SD at 10 to remove the decimal found in Z score and to make all scores “positive” i.e. Z=1.0 is T of 60 and a Z=-1.0 is T of 40.

Ø     Because 99.73% of scores fall between +/-3s, it is rare to have T scores below 20 (Z= -3.0) or above 80 (Z= +3.0).

Method Section of Research Report:  The recipe for conducting the study that lists all the ingredients.  It is desirable to include the procedure by which you will analyze the data. 

Moderator Variable:  See categorical variable.

Null Hypothesis:  Hypothesis used primarily in the statistical test for the reliability of the results that says that there are no differences among treatments (or no relationships among variables). 

Operational Definition/Operationalize Variables:  Must reduce the research question to measurable units i.e. operationalized the variables. 

Paradigm:  Outstandingly clear or typical example or archetype; framework of a scientific school or discipline within which theories, laws, and generalizations and the experiments performed in support of them are formulated.  The literature assists researcher to shape paradigm that directs or constrains the research. 

Paradigm Crisis Phenomenon:  Development of discrepancies in a paradigm leading to proposals of a new paradigm that better explains the data. 

Parsimony:  Keep the study simple.

Poster Sessions:  Method of presenting research at a conference in which the author places summaries of his or her research on the wall or on a poster stand and answers questions from passerby.

Preliminary Sources:  Consist of abstracts and indexes. 

Primary Sources:  Firsthand source of data in research; the original study and usually refers to journal articles.

Problem Statement of Research: (p. 52) Follows the introduction unless a literature review is included in the introduction.  Then a brief problem statement should appear soon in the intro section.

·        Try to specify what research questions you are asking.

·        ID variables in study including the IV, DV, and categorical variables (if any).  Some control variables can also be identified here.

Recording the Literature Reviewed:

·         Statement of the problem (and maybe hypotheses)

·         Characteristics of the participants

·         Instruments and tests used (including reliability and validity information if provided)

·         Testing procedures

·         IV and DV

·         Treatments applied to participants (if an experimental study)

·         Design and statistical analyses

·         Findings

·         Questions raised for further study

·         Citations to other relevant studies not located.

Reductionism:  Characteristic of normal science that assumes that complex behavior can be reduced, analyzed, and explained as parts that can then be put back together to understand the whole. 

References:

Reliability: Integral part of validity which pertains to consistency or repeatability of a measure; a measure of the consistency of the data when measurements are taken more than once under the same conditions.

• The study is repeatable i.e. consistent values when measured over and over.  Test cannot be considered “valid” if it is not reliable i.e. if test is not consistent, cannot depend on successive trials to yield the same results—the test simply cannot be trusted. 
• Can check reliability with a pilot study.
• Validity and Reliability must be specific to your population of interest i.e. just because it works with adults doesn’t mean it will work with children.

Reliability Expression:  Expressed by a correlation coefficient ranging from 0.00 to 1.00. The closer to 1.00, the less error variance it reflects and the more the true score is assessed.  Techniques for computing the reliability coefficient are:

1.      Interclass Correlation (Pearson r): This coefficient is a “bivariate” statistic meaning that it is used to correlate two different variables.  The most common used method of computing correlation between two variables.

·        Computations of Pearson r are limited to only two scores of X and Y. 

2.      Intraclass Correlation:  ANOVA used to obtain reliability coefficient.  

·        Test performance can be examined from trial to trial and then the most reliable testing schedule can be selected i.e. the last trials may differ significantly from the first trials because of learning curve or fatigue effect. 

·        ANOVA yields an F score that tells significance.

Reliability Scores:  Test reliability sometimes discussed in terms of scores.

·        Observed Score: Obtained score that comprises a person’s true score and error score.

·        True Score:  Part of observed score that represents the individual’s real score and does not contain measurement error.

·        Error Score:  Part of an observed score that is attributed to measurement error (from participant, testing, scoring, and instrumentation). 

Reliability Types:

·        Inter-Rater or Inter-Observer Reliability:  Used to assess the degree to which different raters/observers give consistent estimates of the same phenomenon or on the same participants e.g., if more than one technician is used, they should score things in the same manner.

o       Need specific instructions and raters have to practice to get their scores close.

·        Test-Retest Reliability:  Used to assess the consistency of a measure from one time or trial to another e.g., if tested for multiple trials or across multiple days, the scores should be the same.

o       Might have to toss the first few tests before the test results “level out” i.e. need to see the scores “flat line” more.  The first few trials/days could account for this “learning curve” so must build this into your model or allow for this learning curve. 

·        Parallel-Forms Reliability:  Used to assess consistency of results of two tests constructed in the same way from the same content domain e.g. comparing two tests of anaerobic power. 

·        Internal Consistency Reliability:  Used to assess the consistency of results across items within a test e.g. two questions that assess the same concept should elicit the same result. 

Reliability Analyzing:

• Consistency: (different than validity)
• Correlation & SEE
• Same day test-retest (usually physical performance)
• Split-Half Technique
• Stability:
• Repeated Measures ANOVA
• Test-Retest on separate days

Research Articles:  Major segments are:

·         Abstract

·         Intro

·         Method

·         Results

·         Discussion

·         References

Research Participants:  (p. 84) Can expect the right to: privacy or nonparticipation; remain anonymous; confidentiality; expect experimenter responsibility. 

Research Proposal:  A formal preparation that includes the introduction, review of literature, and proposed method for conducting a study. 

Results Section of Research Paper:  Contains description of types of analyses you used and results of these analyses as they pertain to the hypotheses of the study along with illustrations that are necessary to clearly represent data to readers.

Review:  Research paper that is a critical evaluation of research on a particular topic. 

Science:  Process of careful and systematic inquiry.

Scientific Method of Problem Solving:  Method that uses the following steps:

·         Defining and delimiting the problem

·         Forming a hypothesis

·         Gathering data

·         Analyzing data

·         Interpreting the results

Secondary Study:  Source of data in research in which an author has evaluated and summarized previous research. 

Survey: (p. 261) Technique of descriptive research that seeks to determine present practices or opinions of a specified population; can take the form of a questionnaire, interview, or normative survey. 

Tenacity:  Unscientific method of problem solving in which people cling to certain beliefs regardless of the lack of supporting evidence.

Thesis or Dissertation:  Two important aspects are criticism and completeness.

·         Title

·         Introduction

·         Problem statement

·         Hypothesis

·         Definitions

·         Assumptions and limitations

·         Significance

Thesis Statement:  The purpose.

Title:  Stating a position; many times they are in essence the statement of the problem. 

Topic:  Should look for causes and not effects.

Qualitative Research:  Method that involves intensive, long time observation in a natural setting; precise and detailed recording of what happens in the setting; interpretation and analysis of the data using description, narratives, quotes, charts, and tables.  Also called: ethnographic, naturalistic, interpretive, grounded, phenomenological, subjective, and participant observational.

Quantitative Research:

Validity: The soundness or correctness of a test or instrument in measuring what it is designed to measure i.e. the truthfulness of the test or instrument.

• Means you measuring what you think you are measuring.

• Validity and Reliability must be specific to your population of interest i.e. just because it works with adults doesn’t mean it will work with children.

Validity Analyzing:

·        Root Mean Square Error:  Amount of error around line of identity (x=criterion method vs. y=alternative measure method).

o       Best choice because this is compared to “the true.”

·        Standard Error of the Estimate (SEE): Amount of error around regression line (assume significant correlation).

·        Bland-Altman Technique:  Number of cases that fall within the 95% CI of true value.

Validity Types or Categories: 

·        Construct Validity: Degree to which a measure reflects the associated characteristic or to which a test measures a hypothetical construct; usually established by relating the test results to some behavior e.g. someone who scores high on a test for “cooperation” acts cooperatively in a “real-life” setting. 

o       “Usually” a psych-type test but “can be” applied to a physical test.  Does it really reflect the personality it reports to measure in “real life?”

o       Can be tested by “known group difference method” e.g. a skill critical to basketball performance can be performed better by successful basketball players than by downhill skiers. 

o       All other forms of validity are used for evidence of construct-related validity.  It is usually necessary to use evidence from all the other forms to provide strong support for the validity of a particular instrument and the use of its results. 

·        Content Validity: Measurement instrument reflects training i.e. can’t use isokinetic machine to test if you trained subjects on free weights. 

o       Usually educational settings i.e. did the test adequately sample what was covered in the course? Are there a corresponding number of questions in each area?

·        Criterion Validity:  Degree to which scores on a test are related to some recognized standard or criterion. 

o       Concurrent: “Gold Standard” method and the alternative method used simultaneously or near the same time should yield the same results e.g. underwater weighing should provide about the same estimate of body fat as bio impedance. 

Ø      Type of criterion validity that involves correlating an instrument with some criterion that is administered at about the same time i.e. “concurrently.”

Ø      Usually employed when the researcher wishes to substitute a shorter, more easily administered test for a criterion that is more difficult to measure. 

o       Predictive: Measure can accurately predict some future outcome e.g. GRE scores predict success in graduate school. 

Ø      Degree to which scores of predictor variables can accurately predict criterion scores. 

Ø      Need to determine a “base rate” before you can predict. 

Ø      May have little value if base rate is very low or high.

Ø      Multiple regression used because several predictors=greater validity coefficient. 

Ø      Shrinkage occurs which is when validity decreases after prediction formula is used with “new” sample.  Cross validation must then be used to minimize shrinkage. 

·        Logical/Face Validity:  Appears to test what it intends to test.  Looks like it is testing what it is supposed to so probably is.

o       What it’s measuring is pretty obvious e.g. measuring BP with a thermometer lacks face validity.

o       Degree to which measure obviously involves the performance being measured. 

Variable:  Any factor whose change the researcher is interested.

 RonJones.Org | Back to Top | Back to CSUN 605 | Site Map

                      Get Fit.  Be Strong.
                                 Corporate Wellness · Consulting · Health Promotion