Abstract
Introduction
Over the past three decades there has been increasing attention given to measuring the extent to which schools achieve their mandate of educating children, and an expectation of continual improvement in this regard. Collection of data on school improvement has not been limited to achievement, in recognition of the range of factors that constitute the experience of school for students, and ultimately impact on learning outcomes. Beyond achievement, indicators of attendance, disciplinary exclusions (e.g., suspension and expulsion), and school climate have been used to benchmark schools both against each other, and to track school performance over time. The current evidence base draws largely on studies using students as the unit of analysis, and improvements in any one of attendance, exclusions, or school climate are assumed to represent important indices of school improvement in their own right, and, to enhance student achievement over time. However, little work has analyzed longitudinal data using schools as the unit of analysis (rather than inferring from student level data), to understand how these indicators interact with each other over time at the larger school system level, and whether changes over time at the school level make a difference to other indicators over time. The current study aimed to understand the bidirectional associations among school-level indicators of achievement, attendance, suspension, and school climate over a 10-year period, through analyzing school-level administrative data from over 896 elementary schools in Queensland, Australia. To our knowledge, this is the first study to do so. The aim was to identify the most salient modifiable factors within the school context likely to aide enhanced achievement levels and support non-achievement measures of school improvement over time.
Attendance and Achievement
Poorer attendance measured at the student level is consistently associated with poor achievement over time (Ansari & Pianta, 2019; Hancock et al., 2017; Smerillo et al., 2018). Individual absenteeism tends to be largely stable across the schooling years, with early absenteeism predicting later absenteeism and poorer achievement (Ansari & Pianta, 2019), and chronic absenteeism in elementary school linked with consequences as far reaching as likelihood of secondary education completion (Smerillo et al., 2018). Importantly, a large Australian study of over 89,000 students demonstrated that absenteeism has an equally negative impact on achievement for students attending higher and lower socioeconomic schools (Hancock et al., 2017).
While it is clear that, at the student level, absenteeism has a negative impact on individual achievement, less is known about how school-levels of attendance interact with school level achievement over time. A cross-sectional study found strong associations between attendance and achievement, and reported that attendance mediated the relation between the school building condition and achievement (Maxwell, 2016). The present study explores school level attendance data and its associations with a range of other school improvement indicators over the period of a decade.
Suspension and Achievement
At the student level, multiple studies have linked the disciplinary practice of both in-school and out-of-school suspension with poorer achievement over time (Welsh & Little, 2018). For example, a meta-analysis of 34 studies linked school suspension of either type with both poorer achievement and school drop-out (Noltemeyer et al., 2015). Since then, evidence has continued to amass, including findings that even a single in-school suspension (often claimed as a less harmful approach than out-of-school suspensions) conferred a 25% risk of failure in standardized testing for Texan students (D. Smith et al., 2020). Further, associations between suspension and poorer achievement in the short term hold even when sociodemographic differences of students are accounted for (Hwang, 2018).
Suspensions are more frequently used as a behavior management strategy in schools in low socio-economic areas (Hemphill et al., 2014) and are disproportionately experienced by students from racial minorities and those from low socio-economic homes, and those receiving learning support (Hwang, 2018; Morris & Perry, 2016). This disproportionate experience means that students already vulnerable to poor academic achievement have an enhanced risk of suspension, conferring additional risk. In fact, there is evidence to suggest that the achievement risk conferred by suspensions is elevated for students in vulnerable groups compared to others, with suspension accounting for one-fifth of black-white differences in achievement in one American study (Morris & Perry, 2016).
Suspensions during elementary school are also more likely for Australian students who enter school showing developmental vulnerability relative to peers on early reading and numeracy skills, with these relations holding even after accounting for many other sociodemographic, child, and family factors that are also associated with early suspensions (Laurens et al., 2021). These data speak to the utility of being able to examine bidirectional relations among suspension and achievement over time, with consideration also for whether these associations can be demonstrated at a school level, not solely at the student level.
While evidence for the negative impact of suspension on student outcomes at the individual level is relatively clear, far less is known about school level rates of suspension and how these influence school achievement and other indicators over time. In Australia, where this study is situated, the use of disciplinary absences is at the discretion of the principal (leader) of each school, rather than mandated through policy, thus the practice varies widely across schools. Analysis of school-level longitudinal data in this study allows for examination of the ways in which rates of suspension are associated over time with other indicators of school improvement including achievement, attendance, disciplinary exclusions, and school climate.
School Climate and Achievement
School climate has been defined as “the relatively enduring quality of the school environment that is experienced by participants, affects their behavior, and is based on their collective perceptions of behavior in schools” (Hoy, 1990, p. 152). Dimensions of school climate identified across various models in the literature include: safety; relationships; teaching and learning; institutional environment; and school improvement processes (Lewno-Dumdie et al., 2020). School climate is an increasingly common metric in educational accountability and school improvement efforts, though the rigor and quality of measurement varies (Osher et al., 2020).
School climate is of increasing interest due to evidence linking positive climate with teacher motivation (Dickhäuser et al., 2020), more effective implementation of school-wide teaching and learning strategies (De Smul et al., 2020), enhanced attendance and engagement by students (Daily, Smith, et al., 2020), positive student self-concept (Coelho et al., 2020), improved academic outcomes (Daily, Mann, et al., 2020), and even increased global cortical thickness in children (Piccolo et al., 2019). Aspects of school climate appear to be a key mediating pathway via which principal leadership approaches ultimately translate to student outcomes (Sebastian et al., 2017; P. A. Smith et al., 2020). For example, teachers’ ratings of school climate have been found to depend on the specific principal leading the school, over and above all other school factors (Burkhauser, 2017), and in turn, teachers’ reports of school climate are significantly related to student achievement (Dicke et al., 2020; Granvik Saminathen et al., 2018). There is also evidence that where enhancements in school climate can be made (e.g., in perceptions of school safety and academic expectations), then achievement gains will follow (Kraft et al., 2016). In this study school climate is indexed by responses on teacher, parent, and student opinion surveys routinely collected by schools.
The Current Study
Taken together, it is clear that attendance, suspensions, and school climate are important constructs within understandings of school improvement, each of which is associated with achievement, most commonly studied at the individual child level. However, much less is known about how these indicators interact over time, particularly at the larger systems level of the school. This study aimed to understand the bidirectional associations among school headline indicators over a 10-year period by analyzing school-level administrative data from over 896 elementary schools in Queensland, Australia. These specific indicators were selected due to their empirical relevance as shown in prior literature, and their policy and practical relevance in the context of the jurisdiction in which this study was undertaken. These indicators are widely used and collected within this jurisdiction and nationally (Australian Government Productivity Commission, 2023) and internationally (Appels et al., 2022) to assess and benchmark the performance of schools and engage in school improvement agendas. In this study we aim to provide findings of practical relevance and meaning to school improvement departments and education policy makers, by using routinely collected administrative data available through education authorities, rather than research-specific measures.
Data were used from four measurement waves across the decade (2009, 2012, 2015, 2018). Using cross-lagged modelling to examine how the various school improvement indices related to each other over time, the aim was to identify the most salient modifiable factors within the school context likely to support enhanced achievement levels and vice versa. This modelling approach (see below), where prior levels of all outcomes are accounted for, provides strong evidence for the influence of all indicators on ongoing school achievement data.
Method
Sample
This study draws on school-level administrative data for the decade spanning 2009 to 2018, provided by the Queensland Government Department of Education data custodians, to the research team upon application. In 2011, government data reports there were 920 government elementary schools in this jurisdiction (Australian Bureau of Statistics, 2011). We were provided with data for 896 (97.4% of total schools reported in 2011) as schools were removed if they were not open for the whole decade of data we used. Approximately 79% of elementary age children in Queensland attend government schools (Australian Curriculum, Assessment and Reporting Authority, 2020). For all schools, the lowest year level was Grade Preparatory and highest level was Grade 6. Data were used from four measurement waves across the decade (2009, 2012, 2015, 2018).
In Australia, while there is a national curriculum and national assessment program, education is governed at the state/territory level with each jurisdiction implementing different policies regarding attendance and suspension. In the jurisdiction that this study is conducted in (Queensland), schooling is compulsory for children from age 6 and a half years (at the time study data was collected) to 16 years or the completion of year 10 (whichever is soonest). Parents have a legal obligation to ensure children attend school every day unless the parent provides a reasonable excuse. Schools are required to monitor attendance and immediately notify parents if children do not attend, and a reasonable excuse has not been provided by the parent (Queensland Government, 2023). Regarding suspensions in Queensland, school principals are the only staff members in school with the authority to make a suspension decision for an enrolled student which are made in accordance with the
Measures
Indicators of School Improvement
Statistical Analysis
Two longitudinal cross-lagged panel models (CLPM) were developed to examine the reciprocal and longitudinal relations among the six indicators of school improvement across the four timepoints of 2009, 2012, 2015, and 2018. A baseline autoregressive model (see Figure 1) estimated the cross-sectional correlations among the six indicators of school improvement and the autoregressive paths that represent the continuity of each measure over the four timepoints. A transactional model was then estimated that included all potential cross-lagged paths among the six indicators of school improvement across the four timepoints (see Figure 2). This modelling approach (see below), in which prior levels of all outcomes are accounted for, provides conservative evidence for the associations among school-level constructs over time. Specifically, we use recently published guidelines for cross-lagged panel models that suggest .03 be considered a small effect, .07 a medium effect, and .12 a large effect for these models (Orth et al., 2024). While cross-lagged panel models (CLPM) have been a dominant statistical model to investigate reciprocal associations between two (or more) constructs over time, newer analytic approaches have been proposed to strengthen causal claims from cross-lagged data analysis in observational studies (e.g., Hamaker et al., 2015). This is because traditional CLPM is not able to disentangle within-school fluctuations and between-school differences. Thus, in the case of our model, paths should be interpreted as aggregate temporal associations between schools. That is, whether schools relatively higher in one construct at an earlier time point tended to also be higher on another construct later. Our path estimates should not be considered evidence for within-school causal processes.
Autoregressive and cross-sectional correlation model.
Cross-lag model.
Model fit was assessed by four indices: Chi-squared (χ2) test, root mean squared error of approximation (RMSEA), the comparative fit index (CFI), and standardized root mean square residual (SRMR). Good fit was indicated by a non-significant χ2 test, an RMSEA value below .05, CFI value above .95 and SRMR below .08 (Kline, 2005); and adequate model fit by a CFI between .90 and .95 or an RMSEA between .05 and .08. Given the sample size, fit was assessed as adequate irrespective of a significant chi-square test, given the sensitivity of this index to large sample sizes. Models were estimated in Mplus Version 7 using the MLR estimator. Missing data was handles with maximum likelihood estimation (Enders & Bandalos, 2001). Standardized regression coefficients are presented.
Results
Descriptive Statistics
Descriptive statistics for the six indicators of school improvement at each timepoint are presented in Table 1. On average, the percentage of Grade 5 students scoring in the upper two bands of achievement on the NAPLAN reading domain increased over time from 24.62% in 2009 to 32.51% in 2018. The proportion of unexplained absences decreased over time from 25.10% in 2009 to 16.78% in 2018, while suspensions as a proportion of total enrolments increased over time from 7.35% in 2009 to 10.25% in 2018. The percentage of parents/caregivers, students and teaching staff indicating agreement on the school opinion survey over time was similar, that is school climate indicators did not change substantially across time. Table 2 represents the bivariate correlations among the six indicators of school improvement at each timepoint. Correlations were in the expected directions, and almost all were significant due to the large sample size. Academic achievement had small to moderate negative correlations with unexplained absences and suspensions at all timepoints and small positive correlations with parent, student, and teaching staff school opinion. Unexplained absences and suspensions were moderately positively correlated, and both had small negative correlations with parent, student, and teaching staff school opinion. Parent, student, and teaching staff opinion had small to moderate positive correlations with each other.
Mean and Standard Deviations of the Six School Improvement Indicators in 2009, 2012, 2015, and 2018.
Bivariate Correlations Among the Six Indicators of School Improvement at Each Timepoint.
Baseline Autoregressive Model
Table 3 presents the autoregressive paths for the six indicators of school improvement across the four timepoints from 2009 to 2018. There was moderate stability over time within each of the six indicators of school improvement, with large and significant autoregressive paths across all timepoints. The model accounted for 12.9% of variance in Grade 5 NAPLAN Reading (upper two achievement bands) in 2018, 25% of variance in unexplained absences in 2018, 40.4% of variance in suspensions in 2018, 3.6% of variance in parent opinion in 2018, 5.1% of variance in student opinion in 2018 and 3.6% of variance in teaching staff opinion in 2018. This model was a poor fit to the data, χ2 (122) = 570.08,
Baseline Autoregressive Paths for the Six Indicators of School Improvement.
Transactional Model of School Improvement
Table 4 presents the autoregressive and transactional paths between each of the six indicators of school improvement across the four timepoints from 2009 to 2018. Overall, this model showed adequate fit to the data, χ2 (66) = 235.11,
Transactional Paths Between the Six Indicators of School Improvement Across 2009 to 2018.
Standardized estimates for statistically significant (
Academic Achievement (Grade 5 NAPLAN Reading Upper Two Achievement Bands)
Schools with higher earlier academic achievement in 2012 and 2015 tended to show partial correlations with fewer unexplained absences in 2015 (medium effect size) and 2018 (large effect size), respectively. Earlier academic achievement in 2009 at the school level was associated with relatively fewer suspensions in 2012 only (large effect size). Earlier academic achievement was not associated with parent, student or teaching staff school opinion three years later, across the period 2009 to 2018.
Unexplained Absences
Schools with relatively higher earlier unexplained absences showed a partial correlation with relatively poorer academic achievement 3 years later (medium to large effect sizes), across the period 2009 to 2018. Earlier unexplained absences at the school level in 2009 were associated with increased suspensions in 2012 with a large effect size. Earlier unexplained absences in 2009 were associated with less positive parent school opinion three years later in 2015 (large effect). Earlier unexplained absences were consistently associated with less positive student school opinion three years later across the period 2009 to 2018, and associated with less positive teaching staff opinion three years later across the period 2012 to 2018 with large effect sizes.
Suspensions
Schools with relatively higher earlier suspensions showed a partial correlation with relatively poorer academic achievement (large effect sizes) and increased unexplained absences (moderate to large effect sizes) three years later, across the period 2009 to 2018. Earlier suspensions in 2009 were associated with less positive parent opinion three years later in 2012 (large effect size). Earlier suspensions in 2015 were associated with less positive teaching staff opinion 3 years later in 2018 with a large effect. Early suspensions were not associated with student opinion three years later, across the period 2009 to 2018.
Parent Opinion
Earlier parent opinion was partially correlated with student opinion three years later, across the period 2012 to 2018 with moderate to large effect sizes. Earlier parent opinion was not associated with any other later school improvement indicators across the period 2012 to 2018.
Student Opinion
Earlier student opinion was not associated with any later indicators of school improvement three years later, across the period 2012 to 2018.
Teaching Staff Opinion
Earlier teaching staff opinion in 2015 was associated with fewer unexplained absences three years later in 2018 with a large effect size. Earlier teaching staff opinion was not associated with any other later indicators of school improvement three years later, across the period 2015 to 2018.
Discussion
This study used a decade of administrative data from almost 900 elementary schools in one jurisdiction in Australia to document school improvement over time and quantify bidirectional associations among indicators across this period. Overall, there was a net improvement over the decade on indicators of academic achievement and student attendance, but an increase in the use of student suspension. Indicators of school climate remained largely stable.
In the transactional longitudinal model, relative school levels of unexplained absences were consistently associated with poorer academic achievement three years later, increased suspensions for some time periods, and less positive student, parent, and staff opinions. School levels of suspensions were consistently associated with poorer academic achievement levels three years later, along with increased unexplained absences and poorer parent and staff opinion levels. Academic achievement was associated with a lower level of unexplained absences in subsequent years, and in one period to fewer suspensions. School climate, as indexed through community opinion, showed limited associations with other headline indicators in subsequent years, apart from high teaching staff opinion which was associated with fewer later unexplained absences. Early parent opinion was associated with students’ later opinions. These unique findings provide some important insights into school improvement and the levers that might be employed at the school and policy level.
Taken together, addressing attendance and the use of suspensions, and their antecedents at a school level have the potential to make a downstream impact on enhanced academic achievement over time at the school level. Attendance levels have long been associated with improved academic outcomes when students are the unit of analysis (Ansari & Pianta, 2019; Hancock et al., 2017; Mills et al., 2021; Smerillo et al., 2018), and more recently with schools as the unit of analysis (Maxwell, 2016; Mills et al., 2021). Strategies used to improve attendance have included: monitoring student attendance data; communicating regularly with students and parents regarding the importance of attendance; setting and communicating high expectations for attendance, behavior, and academic achievement; involving school support staff and external agencies in providing case management of students; having liaison staff for particular cultural groups; having a reward system; having extra-curricular activities; educating and supporting parents; and using sanctions; with these found to be effective in supporting academic achievement over time (Mills et al., 2021; Department of Education, Training and Employment, Queensland Government, 2013). Of note for future studies will be the extent to which the experience of the 2020 global pandemic, including the closure of schools and more strict protocol regarding students staying at home with mild symptoms, continues to have an overall net negative impact on student attendance in the future. Schools and educational researchers should remain curious and aware of the pushes and pulls of attendance, which may have changed significantly in some settings during the pandemic, and which may require innovative approaches to address in the future.
The use of school exclusion as a disciplinary action in response to student behavior has raised many concerns over the last two decades with multiple calls for a reduction in the use of this approach (Ritter, 2018). Research shows exclusion is disproportionately used for students from disadvantaged and minority backgrounds (Welsh & Little, 2018), school exclusion is linked with longer term adolescent behavior problems (Hemphill et al., 2017), and even when complex socio-demographic characteristics are accounted for, the link between suspension and poor academic achievement at the student level persists (Chu & Ready, 2018). Findings in the current study suggest that beyond the individual child level, school levels of suspension are an important consideration in understanding longitudinal school improvement. Reduction in use of suspensions has been found with the implementation of school-wide positive behavior interventions (James et al., 2019; Kim et al., 2018). While associated immediate academic improvements were not found in one study (James et al., 2019), another found mathematics achievement improvements after a three-year implementation period of behavior support and associated reductions in suspensions (Kim et al., 2018). Taken together, it is suggested that mandating a reduction in use of suspension, with this being a school-based decision in the jurisdiction of this study, is likely to have an important impact on school level achievement data over time, but that for change to be evidenced in achievement data will likely take some time. Further, this approach is likely to be most effective when paired with additional training and support for educational leaders and teachers in relation to school-wide positive behavioral supports and social-emotional learning delivered within integrated Multi-Tiered Systems of Support (McIntosh & Goodman, 2016; Majeika et al., 2024).
While other studies have found school climate linked to improved student attendance and achievement (Daily, Mann, et al., 2020; Daily, Smith et al., 2020), this association was largely not replicated here. However, the school opinion survey data used in the current study, while a useful indicator of satisfaction for school reporting purposes, may not provide a robust measure of school climate. Here, only a small selection of items for each of the parent, student, and staff opinion surveys was made available to the researchers (and not at all time points across the decade) for use in these analyses, in line with typical reporting practices for schools in this jurisdiction. There are a much larger number of items collected within each survey and it is possible that a more nuanced and useful index of school climate could be constructed from these in the future given greater data access, or alternative routine collections of school climate indices be considered, with careful selection from the available student-report (Marraccini et al., 2020), parent-report (Aldridge & McChesney, 2020), and staff-report measures.
Limitations and Future Directions
While this study has many strengths, including a large sample of schools, a decade of longitudinal data, and a modelling approach whereby earlier levels of each indicator are controlled for, it is not without its limitations. Our cross-lagged panel modelling approach produces partial correlations among variables which do not allow for causal inference, and do not separate between-school and within-school effects (Hamaker et al., 2015). While our findings may be relevant at a whole-of-sector level in terms of between-school variation, it is possible that the relations among variables shown here differ
Results should be interpreted bearing in mind that school suspension is a school-based decision (rather than government) in this jurisdiction, thus, its implementation varies widely across schools. The school opinion survey data used to tap school climate was not consistently available for the whole decade, has varying response rates across the jurisdiction, and as noted earlier, may not have adequately represented school climate given we have no validity evidence for this measure. Further, there is a level of specificity and variance lacking in the school opinion survey data given collapsed responses resulting in an overall percentage agreement figure were provided for analysis.
While the study included several typical school effectiveness indicators, not all indicators were available at all time points across the decade, and several important factors likely to influence student outcomes were not included. For example, there have been calls for student subjective wellbeing to be included as a headline indicator of school effectiveness and a focus of school improvement plans (Cleveland & Sink, 2017). Although not routinely measured in the jurisdiction of the current study and so not included in analyses, student wellbeing is a crucial consideration for future studies, with schools highlighted as playing an important role in population mental health (Australian Government Productivity Commission, 2020). Further, we do not control in our analyses for any school level socio-demographic characteristics including socio-economic status, or cultural make-up of the school. Future analyses could seek to discover more detail on the variations among associations that may be apparent for schools of diverse socio-demographic nature.
This study presented a unique longitudinal analyses of school level data for most government elementary schools in one jurisdiction of Queensland, Australia, which is a large state with a range of urban, rural, and very remote schools, a full spectrum of socio-economic advantage levels, and a multicultural and Aboriginal and Torres Strait Islander population. While this allows confidence that the findings are largely representative at the jurisdiction level, it should be recognized that associations among the variables examined here are likely to differ across school contexts, including non-government schools. Further, students within different year levels and from different cultural backgrounds will experience school significantly differently (Buckley et al., 2019) and so future studies should consider within school sub-group analyses. We also only included academic achievement from one elementary year level. Findings might differ should achievement at younger or older grades be examined. Further, at both the school and individual student level there will be many mediating and moderating mechanisms that vary the extent to which the mechanisms presented here operate. For example, schools in low socio-economic areas tend to have poorer attendance and higher use of suspension (Mills et al., 2021; Department of Education, Training and Employment, Queensland Government, 2013), likely to compound the risk for poor achievement for students. There is limited research on specific pathways at the student level (Noltemeyer et al., 2015) which future research should seek to address.
Conclusion
This study examined the transactional relations among achievement, suspension, attendance, and school climate over a 10-year period for almost 900 elementary schools in one jurisdiction in Australia. Overall, there were consistent associations between unexplained absences and suspensions and later
Footnotes
Acknowledgements
The authors thank the Queensland Department of Education for their assistance in providing the administrative data for analysis for this article.
ORCID iDs
Ethical Considerations
This study was a secondary data analysis of existing de-identified school-level administrative data provided by an education jurisdiction. Ethical approval was not required, rather an administrative review was provided by the Queensland University of Technology.
Author Contributions
KEW led all aspects of study design, development of datasets, supervised the data analysis, led the write up of this article. NH conducted all analyses and wrote the methodology and results sections of this article. KRL provided critical contributions and revision to the article. MK provided critical contributions and revision to the article. SC provided critical contributions and revision to the article. NS provided critical contributions and revision to the article. RS-L provided critical contributions and revision to the article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was part of a larger project funded by a Queensland Government Department Education Horizon grant.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
The data used for the study was provided by the Queensland Department of Education for this specific study and cannot be shared.