The Predictive Validity of the Academic Self-Regulated ... - SSRN papers

48 The International Journal of Educational and Psychological Assessment December 2011, Vol. 9(1)

The Predictive Validity of the Academic Self-Regulated Learning Scale Carlo Magno

De La Salle University, Manila Abstract The predictive validity of the Academic Self-regulated Learning Scale (A-SRL-S) was determined using Grade Point Average (GPA) as the criterion. This prediction is supported by the social cognitive framework where the use of such learning strategies results to the improvement of students’ grades. Previous studies are also consistent in establishing the notion that self-regulation strategies account largely on the increase of students’ grades. The A-SRL-S was administered to 2052 Filipino college students from five different private universities in the National Capital Region in the Philippines. The results indicate that GPA was significantly related to each of the A-SRL-S factors. All the factors of the A-SRL-S significantly predicted GPA using a path analysis. Specifically, the memory strategies, goal setting, and self-evaluation largely increased the variance in explaining GPA. The overall model also attained a good fit ( =6671.40, df=21, NFI=92. RFI=.99, IFI=.92, TLI=.93, CFI=.92, and RMSEA=.03). Implications on learning and teaching are discussed. 2

Keywords: Academic Self-regulation, GPA, Predictive validity

Introduction The use of self-regulation strategy is deemed effective when it results to students’ achievement such as increase in their grades. Zimmerman (2002) noted that several studies consistently showed self-regulatory processes lead to success in school. Success in school as an outcome of self-regulation in several studies is indicated by students’ grades (e. g., Kitsantas, Winsler, & Hui, 2008; Stumpf & Standley, 2002; Tuckman, 2003; Zwick & Sklar, 2005). The relationship between self-regulation and grades explains that performance in school assessed by course outcomes or final grades is achieved when students use a repertoire of selfregulation skills such as memory strategy, goal-setting, self-evaluation, seeking assistance, environmental structuring, learning responsibility, and planning and organizing. The prediction of grades using self-regulation as a predictor is explained by the social cognitive framework. The cognitive and behavioral dimensions involve the use of self-regulation while the outcome is the academic achievement as measured by students Grade Point Average (GPA). Academic Self-Regulation Self-regulation in the present study is anchored in the original conceptualization of Zimmerman (2000) as self-generated thoughts, feelings, and behaviors that are oriented to attaining goals. These specific thoughts, feelings, and behavior are specified in the studies of Zimmerman and Martinez Pons (1986, 1988, 2002). Moreover, these factors were studied by Magno (2010a) using a sample of Filipino college students. Magno (2010a, 2011) found that these self© 2011 Time Taylor Academic Journals ISSN 2094-0734


regulation strategies fall under seven factors namely memory strategy, goal-setting, self-evaluation, seeking assistance, environmental structuring, learning responsibility, and planning and organizing. These factors were initially explored using principal components analysis that explains 42.52% of the total variance and the items had high factor loadings (loadings >.40). These seven factors were then confirmed using Confirmatory Factor Analysis (CFA) with adequate fit (χ2=332.07, df=1409, RMS=.07, RMSEA=.06, GFI=.91, NFI=.89). In a subsequent study, the seven factors of self-regulation were again confirmed. This time, the construct validity was further established by correlating the scale with the Motivated Strategies for Learning Questionnaire (MSLQ) and Learning and Study Strategies Inventory (LASSI). A CFA was structured and the three latent constructs (A-SRL-S, MSLQ, and LASSI) significantly correlated with adequate fit in a three factor model (χ2=473.47, df=87, RMSEA=.08, SRMR=.04, AIC=.71, SBC=.92, BCCVI=.71) as compared to a one-factor and two factor models. Self-Regulation and Grades The relationship between self-regulation and grades showed to have high and consistent effect size as revealed in the metanalysis conducted by Dignath and Buttner (2008). Using self-regulation as predictors of grades was further established by Kitstantas, Winsler, and Hui (2008). They explained that academic performance is closely linked with individuals’ level of self-regulation. They found that selfregulation and grades were significantly correlated. When they predicted grades using multiple regression, the addition of self-regulation in predicting grades increased the variance from 37% to 45%. As compared with other predictors in the model (i. e., motivation, self-efficacy, and prior achievement), the entry of selfregulation in the prediction of GPA had the largest contribution. The contribution of self-regulation on grades was also investigated in past studies. For example, the study by Bell (2007) it was found that self-regulation contributed the most (B=.32) among other predictors of GPA. There are also other studies that support the contribution of self-regulation on grades. The study by Hall, Smith, and Chia (2008) found that executive processes involved in selfregulation such as planning, effort expended, and metacognitive applications has the largest contribution in predicting GPA than interpersonal self-efficacy and academic engagement. Consistent results were found in the study by Briley, Thomson, and Iran-Nejad (2009) where both dynamic and active self-regulation predicted GPA accounting for 46.4% of the total variance in GPA as compared to other variances when GPA was predicted. Moroever, the study by Magno (2010b) also predicted students’ grades and found that work method which includes selfregulation strategies had the highest contribution on grades on both mathematics and English. Aside from non-experimental designs, the effect of self-regulation on performance was assessed in some studies by training students some self-regulatory techniques and then students’ performance was measured. For example, the study by Ramdass and Zimmerman (2008) trained students with self-correction and selfevaluation strategies that resulted to significant gains in performance in © 2011 Time Taylor Academic Journals ISSN 2094-0734


mathematics. Likewise, the study by Tuckman (2003) trained students with strategy techniques that controlled for procrastination, building self-confidence, and managing one’s thinking and learning. A significant effect on GPA was obtained with a large effect size (d=.64). The Present Study Since the effect of self-regulation on grades were already established in reviews as supported by metanalsysis and consistent results from experiments and non-experiments (see Dignath & Buttener, 2008), the predictive validity of the Academic Self-regulated Learning Scale (A-SRL-S) was established by using GPA at the end of the term as a criterion. Previous validation of the A-SRL-S was already established such as its factorial validity, concurrent validity, convergent validity, and construct validity (see Magno, 2010a, 2011). The next step in developing the scale is to establish its predictive power on grades as the outcome. The prediction of grades is limited to the use of self-regulation strategies due to the following reasons: (1) Self-regulation showed to have the strongest contribution on GPA which was consistent in studies, (2) the study only aims to determine the predictive validity of the A-SRL-S. Method Participants The participants in the study were 2052 Filipino college students in the Philippines. The age range of the participants is from 16 to 20 years. They are from five different private universities in the National Capital Region (NCR) in the Philippines. All the participants volunteered to participate in the study. Instruments Academic Self-Regulated Learning Scale (A-SRL-S). The A-SRL-S was developed by Magno (2010a) to measure self-regulation of college students within the context of learning in higher education. Each item is responded by a four-point Lickert scale (Strongly agree, agree, disagree, and strongly disagree). The scale is composed of seven factors: Memory strategy (14 items), goal-setting (5 items), selfevaluation (12 items), seeking assistance (8 items), environmental structuring (5 items), learning responsibility (5 items), and planning and organizing (5 items). The seven factors were uncovered using an initial principal components analysis with varimax rotation. Using another sample, the seven factor structure was confirmed using Confirmatory Factor Analysis (CFA) and adequate fit was achieved (χ2=332.07, df=1409, RMS=.07, RMSEA=.06, GFI=.91, and NFI=.89). There is evidence of convergent validity where all seven factors were highly intercorrelated. High internal consistencies were also attained for each factor (.73 to .87). Using an IRT Graded Response Model, the scale showed appropriate step calibration where the responses are monotonically increasing. The Test Information Function curve © 2011 Time Taylor Academic Journals ISSN 2094-0734


showed precision for the overall instrument. Almost all items showed to have good fit and the few items that did not fit the GRM were revised. The scale’s construct validity was also established (see Magno, 2011). The A-SRL-S significantly correlated with the MSLQ and the LASSI and a three-factor structure that showed to have the best fit (χ2=473.47, df=87, RMSEA=.08, SRMR=.04, AIC=.71, SBC=.92, BCCVI=.71) than a two- or one-factor model. Grade Point Average (GPA). The students GPA were used as a criterion for the A-SRL-S. The students’ GPA at the end of the term was obtained. The possible GPAs range from 0.0 (failing grade) to 4.0 (highest grade). The grading system of the different universities was standardized. The grades were reversed for those universities which had lower grades representing higher marks. Procedure The A-SRL-S was administered to college students in five universities in the NCR. At the onset of administering the questionnaire, they were asked if they are willing to participate in the study. They were requested to sign a consent form if they agree to participate. They were instructed to answer the items in the questionnaire and were reminded not to leave any item unanswered. The participants can answer at their own pace within 30 to 40 minutes. They are also reminded that there are no right or wrong answers so they need to respond as honestly as possible. Once completed, the questionnaire is returned and the participants were debriefed about the purpose of the study. The grades of the students who participated in the study were requested from the registrar’s office towards the end of the term. The grades were converted into standard scores to equalize the range of different grade values across colleges and universities. Data Analysis The path analysis was used to determine the predictive validity of each selfregulation factor on GPA. The path analysis provides estimates of magnitude and significance of hypothesized casual connections between sets of directly measured variables. The relative sizes of path coefficients in the analysis indicate which effect is better supported by the data. Aside from the parameter estimates, the entire model is tested using goodness of fit: Chi-square, NFI, CFI, and RMSEA. Independence mode Chi-square and df assumes that the population covariances are all zero. The Bentler-Bennet Normed Fit Index measures the relative decrease in discrepancy function caused by switching from a “Null Model” or baseline model to a more complex model. This index approaches 1 in values as fit becomes perfect. The CFI in a noncentrality fit index the represents one approach to transforming the population noncentrality index F into range 0 to 1. And the Steiger-Lind RMSEA compensate for model parsimony by dividing the estimate of the population noncentrality parameter by degrees of freedom. This ratio in sense represent a “mean square badness-of-fit”.

© 2011 Time Taylor Academic Journals ISSN 2094-0734


Results The means, standard deviation, and Cronbach’s alpha of each subscale in the instruments were obtained. The Cronbach’s alpha is used to test the internal reliability among the items. The relationships among variables were established using Pearson’s r. The means obtained for the factors of self-regulation are quite high (which is close to 4.0). The mean of the GPA obtained is within the median region. The range of scores is not so dispersed as indicated by the low standard deviation values. All of the A-SRL-S factors had very high internal consistencies. When the factors of the A-SRL-S were intercorrelated, moderate to strong correlation coefficients were obtained (r=.49 to r=.60) and all are significant (p