Transforming raw data into ipsatized data

Reducing Response Bias with Reducing Response Bias with IpsativeIpsative Measurement in the Context of Measurement in the Context of Confirmatory Factor AnalysisConfirmatory Factor Analysis Mike W.L. CHEUNGMike W.L. CHEUNG WaiWai CHANCHAN Department of Psychology,Department of Psychology, the Chinese University of Hong Kongthe Chinese University of Hong Kong Paper presented at the International Association for Cross-Cultural Psychology XVth Congress, Pultusk 2000, Poland. A revised paper of the presentation was published as Cheung, M. W. L., & Chan, W. (2002). Reducing uniform response bias with ipsative measurement in multiple group confirmatory factor analysis. Structural Equation Modeling, 9, 55-77. What is Response Bias?What is Response Bias? � A systematic tendency to respond to a range of questionnaire items on some basis other than the specific content (Paulhus, 1991). � Generally, there are two categories (Paulhus, 1986; Lanyon, 1982): – Response Style – Response Set Response StyleResponse Style � The tendency to distort responses in a particular direction more or less regardless of the content of the stimulus (Paulhus, 1986; Rorer, 1965) � e.g., Uniform response bias, yeasayers and naysayers � Methods to minimize: – +ve and -ve wordings Response SetResponse Set � Conscious or unconscious desire on the part of the respondent to answer in such a way as to produce a certain picture of himself � e.g., Social Desirability � Methods to minimize: – forced choices, anonymous responses or – unthreatening questions before sensitive issues – randomized response method Examples of Response Bias in Examples of Response Bias in Different AreasDifferent Areas � Sensitive issues research – drug usage and sexual practices are underestimated (Bradburn, Sudman, Blair & Stocking, 1979) – the reliabilities and validities of self-reported drinking behavior are questionable because of response bias (Embree & Whitehead, 1991) � Cross-cultural studies – Stronger tendency for extreme checking in Hispanic than non-Hispanic (Hui & Triandis, 1989) – Different tendencies to use extreme scores in Japanese, British, American, Hong Kong, etc. managers (Stening & Everrett, 1984) Estimated Accuracy of Estimated Accuracy of SelfSelf--report Informationreport Information Adopted from Marquis, Marquis and Polich (1986) Estimated Reliabilities of Estimated Reliabilities of SelfSelf--report Informationreport Information Implications forImplications for Factor AnalysisFactor Analysis � Factor analyses (EFA, CFA or SEM) are always used in validation study. � Using simulation, Cheung (1997) showed that the statistical performance of CFA was poor when response bias was present. � In responding to this problem , ipsative measurement was proposed to analyze data when response bias was present. IpsativeIpsative MeasurementMeasurement � Definition: 1Tx = constant (within individual) – That is x1+x2+…+xp= constant for every individual – It is similar to within-subject centering � Two ways to achieve ipsative data: – i) ask participants response in relative values; – ii) transform the raw data into ipsative data. An ExampleAn Example � Participant 1 without response style: – Raw scores: 16, 19, 20, 22 and 23 – Average score: (16+19+…+23)/5 = 20 � Participant 1 with response style (+3): – Raw scores: 19, 22, 23, 25 and 26 – Average score: (19+22+…+26)/5 = 23 An Example: CalculationsAn Example: Calculations � Ipsative scores: – Par 1 without bias Par 1 with bias – 16-20 = -4 19-23 = -4 – 19-20 = -1 22-23 = -1 – 20-20 = 0 23-23 = 0 – 22-20 = +2 25-23 = +2 – 23-20 = +3 26-23 = +3 An Example: RemarksAn Example: Remarks � When there are response bias, the “true” scores and the observed scores are different. � However, they are the same after ipsatization � ⇒ remove uniform response bias � Note: if we only have the ipsative scores, we can never know their original raw scores. How can How can ipsativeipsative data be collected?data be collected? � For Response Style, e.g., uniform response bias: � Steps: – Collect data in raw scores – Ipsatize the raw scores • Uniform response bias is removed – Analyze the ipsative scores � For Response Set, e.g., Social Desirability: � Steps: – Collect data in the ipsative format, i.e., asking the relative values of respondants. • Social desirability is minimized because the actual attitudes or frequencies are unknown – Analyze the ipsative scores Can Can ipsativeipsative data be factorized?data be factorized? � Mathematical and empirical evidences: NO! – (e.g., Chan & Bentler, 1993; Dunlap & Cornwell, 1994; Jackson & Alwin, 1980) � Remember that: x1+x2+…+xp= constant – COV(xi, x1+x2+…+xp) = 0 for every i – Then COV(xi, x1) +…+ COV(xi, xp) = 0 � ⇒ at least one of the COV is negative even though they are positive in their raw scores Problems in CFA (SEM)Problems in CFA (SEM) � Sum of COV equal “0” in every row/column of the COV matrix – i.e., � ⇒ the sample and population COV matrices are singular � ⇒ EQS and LISREL cannot routinely be used Proposed MethodProposed Method � Chan & Bentler (1993, 1996 and 1998): – methods to analyze ipsative data in SEM; – statistical theory for these methods; and – conditions for these methods work. Proposed MethodProposed Method � They showed that – the proposed method was identified in general conditions – the goodness-of-fit indices were good – the “recovered” parameter estimates were near the true values � Cheung (1997) showed that this method could be applied in analyzing data with response bias. A Real Example: CPAIA Real Example: CPAI � A Dataset of Chinese Personality Assessment Inventory (CPAI) was borrowed from Prof. Fanny Cheung, CUHK � It consists of 4 personalities and 2 clinical factors (Cheung, Leung, Fan, Song, Zhang & Zhang, 1996) � Two personalities factors were chosen here CPAI: Interpersonal RelatednessCPAI: Interpersonal Relatedness (Chinese Tradition)(Chinese Tradition) � Six scales:- – HAR- Harmony – FAC- Face – REN- Ren Qing (Relationship Orientation) – FLE- Flexibility – MOD- Modernization – T_E- Thrift-Extravagance CPAI: IndividualismCPAI: Individualism � Three scales: – S_S- Self vs. Social Orientation – L_A- Logical vs. Affective Orientation – DEF- Defensiveness (Ah-Q Mentality) CPAI: Method of AnalysisCPAI: Method of Analysis � General steps (Chan & Bentler, 1993 for details): – a) Propose the pre-ipsative factor structures I N T E R I N D H A R F A C R E N F L E M O D T _ E S _ S L _ A D E F 0 0 0 0 0 0 αααα αααα αααα αααα αααα αααα ββββ ββββ ββββ 1 2 3 4 5 6 0 7 0 8 0 9                             – b) Calculate the ipsative factor structures I N T E R I N D H A R F A C R E N F L E M O D T _ E S _ S L _ A D E F αααα αααα ββββ αααα αααα ββββ αααα αααα ββββ αααα αααα ββββ αααα αααα ββββ αααα αααα ββββ αααα ββββ ββββ αααα ββββ ββββ αααα ββββ ββββ 1 2 3 4 5 6 7 8 9 −−−− −−−− −−−− −−−− −−−− −−−− −−−− −−−− −−−− −−−− −−−− −−−− −−−− −−−− −−−− −−−− −−−− −−−−                             αααα αααα αααα αααα αααα αααα αααα ββββ ββββ ββββ ββββ = + + + + + + + + = + + + + + + + + ( ) / ( ) / 1 2 3 4 5 6 0 0 0 9 0 0 0 0 0 0 7 8 9 9 – c) Constrains some loadings to be the same • Before ipsatization: Zero loadings • After ipsatization: Equal loadings – d) Delete one variable from analysis – e) Calculate the matrix of error terms A=I-p-111T since they are systematically correlated after ipsatization � Before ipsatized (A) After ipsatized (A) 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1                             0 888 111 111 111 111 111 111 111 111 111 0 888 111 111 111 111 111 111 111 111 111 0 888 111 111 111 111 111 111 111 111 111 0 888 111 111 111 111 111 111 111 111 111 0 888 111 111 111 111 111 111 111 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − 111 111 0 888 111 111 111 111 111 111 111 111 111 0 888 111 111 111 111 111 111 111 111 111 0 888 111 111 111 111 111 111 111 111 111 0 888 − − − − − − − − − − − − − − − − − − − − − − − − − − − −                             . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . – g) Fit the proposed model – h) Calculate or “recover” the original factor loadings, standard errors of parameter estimates CPAI Results: CPAI Results: GoodnessGoodness--ofof--fit Summariesfit Summaries � Common CFA ignoring response bias: – χχχχ2 (N=1585) = 783.62, df=26, p<.001 – CFI=0.780, NNFI=0.695, RMSEA=0.136, AIC=731.62 � Ipsative method (bias was treated as response set): – χχχχ2 (N=1585) = 269.12, df=17, p<.001 – CFI=0.905, NNFI=0.844, RMSEA=0.097, AIC=235.12 CPAI Results: Factor LoadingsCPAI Results: Factor Loadings � Common Method: Ipsative Method:                             −−−− −−−− 97.10 60.10 37.10 026.1 011.1 027.2 018.1 084.1 056.1 Def L_A S_S T_E Mod Fle Ren Fac Har INDINTER                             −−−− −−−− 17.00 98.20 42.10 057.2 094.1 058.1 007.2 008.1 070.2 Def L_A S_S T_E Mod Fle Ren Fac Har INDINTER CPAI Results: Factor Correlation CPAI Results: Factor Correlation and Error Variancesand Error Variances � Common Method: – Factor correlation = .824 – Error Variances = • [3.45 8.60 2.59 3.68 5.86 7.07 6.05 6.06 5.54] � Ipsative Method: – Factor correlation = .732 – Error Variances = • [2.21 8.06 2.62 4.49 6.30 6.60 6.06 2.60 3.08] CPAI Results: ImplicationsCPAI Results: Implications � The ipsative method fits the data better! � The directions are consistent in both methods � Ipsative method can give us a clearer picture of the factor structures. ConclusionsConclusions � When there is response bias, ipsative method can be used to minimize its effects. � When there is NO response bias, ipsative method works as BUT not optimal. � Does response bias be the causes for the non- replicated factor structures across cultures? Questions Are Welcomed! � Selected References � Chan, W., & Bentler, P.M. (1993). The covariance structure analysis of ipsative data. Sociological Methods & Research, 22, 214-247. � Cheung, M.W.L. (1997). Covariance structure analysis of data with response set bias: A comparison of the ipsative and ordinary approaches. Unpublished Thesis. � Clemans, W.V. (1966). An analytical and empirical examination of some properties of ipsative measures. Psychmetric Monographs, 14, 1- 56. � Paulhus, D. L. (1991). Measures of personality and social psychological attitudes. In J. P. Robinson, & R. P. Shaver (Eds.), Measures of Social Psychological Attitudes Series (Vol 1, p.17-59). San Diego: Academic Press Inc.