Schools and school districts are complex, dynamic systems affected by numerous factors, specific to the particular environment. These factors, which range from the stability of the home life of the enrolled children, to the interpersonal relationships of the school staff, to the funding decisions of the school board, to the laws passed by the U.S. Congress (and innumerable additional factors in between), all interact in sometimes predictable but often completely surprising ways. Educational initiatives and interventions that work well in one environment can prove completely ineffective (or un-implementable) in a different school setting, for a myriad of reasons. For university faculty and STEM professionals who partner with K-12 schools to implement and assess STEM educational reform initiatives, particularly for those who choose to work or scale up projects in non-charter or non-specialized lab school settings, the complexity of the system of K-12 education makes it difficult to identify all the potential barriers that can impact the proposed project. Unexpected factors can easily derail an otherwise well thought-out project, both in terms of project implementation and also in the success of assessing student outcomes.
Educational researchers have long studied school reform and the issues of what facilitates and hinders success in curricular and other interventions 1,2 . Experts in educational policy and public policy also have studied the interaction of policies and practices of reform agendas within social and organizational contexts 3,4,5 . Industrial engineering, which had its origins in studying manufacturing systems, is a field where researchers have made great contributions towards understanding complex systems including transportation systems, financial systems, health care, and even recently humanitarian support systems 6 .
The Advanced Manufacturing and Prototyping Integrated to Unlock Potential (AMP-IT-UP) NSF Math/Science Partnership at the Georgia Institute of Technology is creating an innovative framework, which is both conceptual and theoretical and rooted within the field of industrial and systems engineering, to examine barriers and enablers to school change and reform. The framework describes the system in terms of both agents and the attributes of those agents and will become the foundation for identifying a subset of attribute combinations that allow for successful change in the system. In this paper we describe the first step in creating this framework, namely identifying the agents within K-12 education and the attributes of these agents that are critical to educational change. The paper also presents a sample scale for describing these attributes.
Industrial engineering is concerned with the design, improvement and installation of integrated systems of people, materials, information, equipment and energy. It draws upon specialized knowledge and skill in the mathematical, physical, and social sciences together with the principles and methods of engineering analysis and design, to specify, predict, and evaluate the results to be obtained from such systems. 7 Historically, industrial engineering was concerned with manufacturing processes; however, in more recent times it has been applied to many other contexts including transportation and logistics systems, financial systems, and health systems. Systems engineering, on the other hand, is a rapidly evolving field for managing, designing, and optimizing complex systems involving interactions between multiple interdisciplinary subsystems 8,9 . Considering the system as a whole leads to more informed decision-making, even at the subsystem or component levels. The educational system is clearly complex; it is an integrated, multilayer system of people, money, knowledge, and information as outlined above and hence it is ripe for the tools that industrial and systems engineering provide.
There have been very few systematic applications of industrial and systems engineering principles to model education systems. Nicholls et al 10 use hard and soft modeling techniques to develop a methodology for diagnosis and facilitation of organizational change management programs in an Australian university, and Figueiredo et al 11 use data envelopment analysis to develop a decision support methodology to increase school efficiency in Bolivia's low income community. However, a systems approach in which interactions between the different agents affecting the school (e.g. students, teachers, administrators, community etc.) is missing in these papers. There is an attempt at modeling education using systems engineering by Pedamallu et al 12,13,14 . In this work, system dynamics are used to study the factors that affect the academic performance of primary school students in the inner squatter and outer squatter districts of Turkish cities. However, in this study, survey data is used to formulate causal relationships, but there is no mechanism for distinguishing correlation from causation. In addition, the effect of policy variables on the attributes of the agents is excluded.
According to a recent editorial in the International Journal of Production Economics 15 , there exists a need to apply more rigorous systems engineering and operations research techniques to model the system of education.
The model will be defined by a collection of Agents and their Attributes. Agents are considered to be independent entities that make decisions, and their attributes affect what decisions are made. For simplicity, the agents will be populations of agents that will be assumed to have basic population attributes rather than individual attributes (for example the Student Body of a school rather than each of the individual students). The model will then be described by a vector of attributes for each of the agent populations. This will be called the State of the system. The state of the system when we begin studying it is called the Initial State. The state of the system at the end of the period of study will be called the End State. The space of all possible states is called the State Space. In general, we are interested in studying how the system changes over time. These changes can be described by indicating how the attributes of some or all of the agent populations change. It is important to note that movement from one state to another requires resources (time, money, political will, effort, etc.). We call these movements State Transitions. We will use the term Acceptable Zone to indicate the collection of states where the desired intervention or implementation is considered to be successful.
Below are the basic definitions of the agent populations within K-12 education and their attributes:
Entities (Agent Populations): Students, Teachers, School Leadership, School System Administration, Community, Government
These are the parts of the model that act and have the potential for change. While each group is made up of many individuals with different characteristics, to simplify the work (as mentioned above), each group is considered as a population that has a collective description and movement.
The first two of these groups, "Students" and "Teachers", are self-explanatory. "School Leadership" refers to the Principal, Assistant Principals, Department Chairs and any other staff member who helps to set the policies and culture of the school; the actual set of individuals in this population will vary from school to school. "School System Administration" refers to the Superintendent, Deputy Superintendents, Curriculum Directors, School Board members and any other personnel involved in setting policy and procedures for the school system of the school under study. "Community" refers to the local community of residents (including parents), and agents of that community such as local newspapers and civic associations, in the vicinity of the school being studied. While in general "Government" refers to all levels of government -from local to county to state to federal--different case studies will most likely concentrate on the limited subset of these levels that set educational policies that directly impact the schools.
These are the dimensions that we use to describe each of the agent populations. As mentioned above, this is a collective description rather than a large set of individual descriptions. The first three attributes are common ways to divide up the parts of the mind and how people react to new situations. The affective domain refers to emotions, cognitive ability refers to intelligence in multiple dimensions, and conative is related to drive and striving. Intra-group Relationships is used to describe how the population works and acts together, while Inter-group Relationship describes how the particular population works and acts with the other agent populations.
Figure 1 is a generic high-level diagram of movement in the system to a successful End State:
There are two implicit questions that arise when looking at this diagram. First, what is necessary for there to be a non-empty acceptable zone? Second, given a non-empty acceptable zone, what is necessary for there to be a feasible path from the initial state to an end state in the acceptable zone?
The first question depends on the intervention planned. For example, it is clear that if the educational intervention is intended to ensure that every first grader is reading on grade level, then it is possible to have a non-empty acceptable zone. However, if the intention is for every third grader to understand calculus, then it is highly unlikely that there will be any acceptable end states. For most cases, the answer to this question will be determined by how well the intervention matches the given context of the school being studied.
The second question is also dependent upon the context but it is also highly dependent on the available resources. Going back to the first example, if the context is one of highly skilled and motivated teachers in a high SES community, then given a reasonable intervention, most likely there will be a feasible path with an acceptable end state (every first grader reading on grade level). However, for the same intervention in a high needs school with a high proportion of students with disabilities and/or coming from homes in poverty, with a contentious or disengaged In general, the model allows for an analytical approach to answering these two questions. A means for describing the state of the system at any point in time based on a set of attributes of the agents in the system must first be provided. Then, given any particular planned intervention, one can analyze the available feasible paths through the state space to reach solutions to these two questions.
To build a model to answer the questions mentioned above we have been investigating systems engineering methods. Our approach is to first develop a model framework from a meta-model standpoint. This meta-model can then be applied to different case studies to build a specific model for that particular case. For this meta-model, a model boundary chart 16 Exogenous variables, or design variables, are inputs to the model. Excluded variables are assumed to be beyond the scope of the model. For a specific case study, it is likely that not all of the agent attributes will be included as endogenous variables but rather will be considered exogenous variables.
The challenge now is to formulate quantitative relationships between the different agents and the attributes of these agents so that quantitative analysis can be performed. We hypothesize that a combination of system dynamics and agent-based modeling methodologies can be applied to this problem 17 . System dynamics allows for the depiction of causal relationships at the attribute level and assumes that each attribute of each agent is an independent variable. However, this is an assumption that is not always true in reality. Since the agents are interacting with each other, the attributes of each agent should be assumed to be correlated or coupled, as this would be more aligned with the reality. By contrast, agent-based modeling assumes that the different agents are the agents in that they each make independent decisions, and the school is the environment where they interact. However, forming quantitative relationships to analyze the state change of the agents where each state is defined by the five attributes combined together is actually more difficult than forming quantitative relationships as one would do in a systems dynamics model. So, a hybrid approach between system dynamics and agent-based modeling might be more feasible and applicable. Other techniques that are more popular in operations research such as hidden Markov models 18 will also be investigated.
The education system's agents and their attributes are introduced above. In this section, a preliminary selection of relevant attributes is presented for each of the agents in the system. Table 1, on the following page, gives a more detailed view of what will be measured as the system state. Each agent will be analyzed on the basis of the five different broad attributes, so there are 30 different components of the matrix that make up the system state. Each of these components, for example the conative characteristic of the teacher population at the school, will be described using a rubric created through discussions with educational domain experts and drawing upon the educational literature 19,20,21 . The rubric describes each component on the following 4-level scale:
1. Destructive 2. Absent 3. Situational 4. Constructive
In general, this can be interpreted in the following way:
Destructive implies that the attribute is present in a negative quality that harms the agent's ability to succeed. Absent means that the attribute is not present at all, or present in a neutral way. Situational means that under certain contexts, the agent exhibits this attribute in a positive manner (but only in those contexts); while Constructive is used to indicate that the agent exhibits this attribute in a positive way independent of the surrounding context. Clearly, these terms need to be fleshed out in more detail for each of the agent populations. A sample rubric for rating teacher characteristics is shown in Table 2.
As the model is developed, screening methods may be applied to determine the most important attributes and reduce the effective size of the state space. It is also possible that additional attributes could be added or substituted for those defined here as new influences are discovered or taken into consideration. Assessment of these variables is an additional consideration; the accuracy and sensitivity of the data collected must be taken into account and factored in to the reliability of the model's predictive capabilities.