The case of Nqatoqi^[1]

Maria Sophia Aguirre^[2]

Introduction

When one thinks of poverty, often issues such as malnourishment, scarce resources, illiteracy, violence, vulnerability, lack of access to basic needs or financial systems, corruption, migration, lack of security, crime and terrorism, are some of the realities that come to mind. It is rear, in discussions of poverty, to adress topics such as hope, selfishness and autonomy, individualism, and passivity. It is even more infrequent for people to consider hope, relationality, participation, and a proactive disposition, as pathways to reduce poverty^[3]. In this paper we attempt to address these infrequent but fundamental aspects of the reduction of poverty, by presenting the outcomes of a randomized control trial experiment (RCT) carried out in Guatemala during the school year 2014.

The intervention consisted in the implementation of a modified version of the Educational Program on Civic Values Nqatoqi´ (Nqatoqi’I), an educational intervention that seeks to promote the living of civic and social values among educators, parents, and students; i.e., the educational triangle.  Following an integral economic development methodology, the RCT modified the existing Nqatoqi’s design, by introducing proactive and participatory interventions (PPI). The RCT was implemented in primary public schools located in zonas rojas, high crime zones, of the municipality of Mixco, Guatemala.

The relevance of human capital as well as of healthy social and civic institutions for sustainable development and for the prevention of poverty is a well-documented fact.^[4] Loening (2002 and 2004) evaluate the role that human capital has had on Guatemala’s growth process. Findings confirm its importance for the growth process of the country as well as for the development of solid institutions. They also confirm that when weakened, it leads to the generation of poverty. Rivera and Lavarreda (2008), using official data from the Department of Education, study the salary levels and incentives necessary to obtain better teachers’ performance, as this was identified as an obstacle to secure quality educators, especially in rural areas. Findings suggest that to attain a higher teaching effectiveness, a salary increase is not enough. Other conditions that range from basic infrastructure to security are also necessary. ASIES has carried out numerous socio-economic studies dealing with education, the impact of crime in communities, and the role of civic education in generating political stability.  ASIES (2012), one of these studies, presents a series of recommendations for immediately improving the educational system both at municipal levels and nationwide. CIEN also has carried out several studies addressing educational and security challenges in Guatemala. Relevant for this RCT are CIEN (2008), which evaluates the progress of the Programa de Promoción de la Reforma Educativa en América Latina y el Caribe (PREAL) in Guatemala and CIEN (2011), which analyzes the economic impact, risk factors, and networks that foster organized crime and it addresses measures to control them. Three last three studies mentioned the need for school and community level interventions to strengthen social and civic institutions.

Azpuru (2002) studies youth’s attitudes towards civic engagement in Guatemala. He highlights the importance of civic education in values, in order to ensure the future of the democratic process in the country. Also relevant is the study of the University of Istmo (2012) on domestic violence in Guatemala. Consistent with finding in other countries, the study reports a higher probability of domestic violence in other than married family structures, when there is alcoholism and/or drug abuse in the household, when the level of female education is significantly low, and when there is more than one woman in the life of the abuser. Supporting the mentioned findings, some RCTs studies provide evidence suggesting that participation as well as a richer community life can assist in achieving more efficient economic outcomes, and on the prevention and reduction of poverty^[5].

The experiment presented in this paper, sought to test the effect of introducing PPIs as a means to overcome problem found in the design of the Nqatoqi’I, thus improving its impact. Specifically, it incorporated proactive and participatory interventions in the existing Nqatoqi’s designed (Nqatoqi’II). The educational initiative was developed by the faculty of Education of the University of Istmo. This educational initiative, offers educational training in civic and social values to school principals and teachers in rural and urban areas. While fulfilling the requirements of the Curriculum Nacional Base (CNB, Base National Curriculum) on values, Nqatoqi’ seeks to meet an urgent need to promote a different way of living in Guatemala, where pervasive selfishness, violence, consumerism and materialism is highly prevalent and they undermine the country stability. The presence of organized crime related to drugs as well as armaments and human trafficking in the country, are putting the survival of the entire society at risk. It also affects the security of families, communities, and the whole country. The program begun in 2003 and has been imparted in several communities across the country, most of which are located in “red zones” (high crime areas).   

Utilizing an integral approach, we evaluated Nqatoqi’s impact between 2012-2013. The results of the study evidenced the positive and statistically significantly impact that the program was generating on teachers, parents, students, families, and the immediate communities where Nqatoqi’ was implemented. This positive influence was found not only on the immediate outcomes, i.e. acquisition of the knowledge and practice of civic and social values, and on the pedagogical techniques imparted during the intervention. It also found evidence of positive and statistically significant impact on the manner in which the direct beneficiaries of Nqatoqi’, i.e. teachers, parents and students, lived according with their dignity; as well as on the manner subjects lived their social and civic responsibilities. We also found that the program could be strengthened by introducing three modifications: 1) a leadership training to teachers and directors based on participatory leadership; 2) a proactive and participatory approach to the design and implementation of the escuela de padres (parents’ association) in each school; and 3) the introduction of a module on education of affectivity for students enrolled in 5th and 6th grades that directly engaged parents, so they in turn, work with their children in a participatory manner.^[6] These adjustments to the program were introduced in the Nqatoqi’s implementation of 2014, in three districts of Mixco.  The sample collected included 2,787 students from third to sixth grade, 2,174 parents, and 142 educators (Supervisors, Principals, and teachers).

RCT results indicate that the introduction of PPIs significantly improve Nqatoqi’s outcomes. Specifically, Principals and teachers having a more participatory approach to leadership increases the collaboration among educators in the teaching of values across the curriculum as well as its effectiveness. In addition, parents become more engaged in the escuela de padres and in the education of their children, when proactive participation is elicited in the creation and running of the escuela de padres. Parents also become actively engaged in helping their children learn to manage their affectivity, and do so in a respectful manner. Finally, we find a significant increase in the way beneficiaries live according with their dignity at home, as well as in the way they live their social and civic responsibility. At the same time, we find a statistically significant decrease in the beneficiaries’ exposure to social ills in their communities. Students attending schools where Nqatoqi’II was implemented, report being more motivated to live well their affectivity because of their parents’ encouragement as well as being more self-motivated to do so. Finally, students’ academic outcomes also significantly improve.

The paper is structured as follows. Section II presents the methodology. In section III the program highlights and the experimental design are laid out. A description of the data collected and utilized for the experiment follows. Section V reports the outcomes. The paper ends with the conclusion.

Methodology

This RCT was design with an integral approach as proposed by (Aguirre 2011 and 2013). At the core of the methodology, is the acknowledgement of an experiential reality. A person exits, lives and acts together with others, not in isolation. A consequence of this reality is that the way persons interact with one another can help or jeopardize sustainable development. Two other consequences follow. First, they maximized not as self-utility maximizers but rather as social persons; second, considering the interpersonal-relational dimension of economic actions as important for it can affect outcomes. Indeed, the economic agent can be an agent of change when carrying out economic decisions. Thus, the approach offers a more complex understanding of maximization in the decision-making process, as it understands the economic agent as a person who is social by nature and maximizes as such. With this perspective, the integral economic approach seeks to identify effective channels of relationships which make economic development sustainable. To do this, in designing and implementing any intervention, focuses on the direct beneficiaries of an intervention as well as on the interpersonal relationships around the beneficiaries, and it gives primacy to “action” when measuring. 

A direct consequence of this expansion of the maximization process, in the design of this RCT, is that we do not rely on monetarized incentives to stimulate a change in behavior. Rather, the integral approach introduces proactive participatory components (PPI), to encourage changes in behavior that seeks to improve outcomes. Drawing from the seminal work of Schein (1999, 2009, and 2010) and other organizational sociologist/psychologists, the incentive provided in this case, is the opportunity for Principals, teachers, parents, and students to engage in the teaching and/or living of civic and social values in a participatory manner. Within this framework, as it is the case in standard experimental design, the subjects in the experiments are motivated by their desire to better their lives, yet in seeking to do this, thy maximize as social person and they do not receive cash transfers nor any monetary compensation for either their participation and/or as an incentive. Instead, incentives are generated by providing participatory alternatives that seek to foster proactive engagement in the living of values at the school, in their family life, and in their communities. At the same time, as it follows the cannons for a standard RCT, it requires a baseline for control and treatment groups as well as post-treatment and control data collection.

Program Highlights and Experiment Design

As previously mentioned, Nqatoqi’ offers educational training in civic and social values to school principals and teachers in rural and urban areas. The training teaches participants to understand, incorporate, and teach social and civic values across the curriculum.  The program has three objectives: 1) to strengthen the formation of values capable of generating changes in the educational institutions and in the community towards a more peaceful and secured environment as well as socially responsible citizens; 2) identify mechanisms to strengthen the communication between parents and children; and 3) foster habits in students and teachers to improve themselves as persons and thus be agents of change in their families, community and the society at large.  To do so uses methodological guides, which are imparted in training sessions given to principal of schools and to an additional two teachers per school. After the training, participants receive educational guides and class preparation materials for students between 6 and 16 years of age enrolled in K to 6th grade.  These materials develop in more depth the content covered during training sessions.  After the training, participants are expected to pass on its context to all the teachers in their school, and all together then, work with the students and their parents in the teaching of civic and social values. Teachers pass on the Nqatoqi’s content to parents, through escuelas de padres and other means of parental involvement in the school. The training lasted 30 hours. Six months after the training finishes, there is a “follow up seminar.”  In it, Nqatoqi’s personnel together with teachers evaluate the “implementation plan” progress. The latter includes the introduction of civic and social values transversally in the curriculum as well as a plan to develop and implement the escuela de padres. Educators that receive Nqatoqi’s training do so without pay and in response to the educational district supervisor’s invitation, who works in conjunction with school principals so to ensure they facilitate completion of the training by both the principals themselves and the selected teachers.   

Before imparting the Nqatoqi’, a diagnostic survey is distributed to principals and teachers. Its objective is to identify the educators’ main needs and problems they face in the community when teaching social and civic values. This information is used for the design of the training, so to tailor it to the specific needs identified by the educators. 

The original Nqatoqi’s design (Nqatoqi I) included some leadership training as well as design of escuela de padres, but neither included a participatory approach. Neither the original design included any module addressing the teaching of affectivity. In the 2014 design (Nqatoqi’ II), the training module on leadership was modified so to introduce a participatory management style, which sought to elicit initiative and proactive participation on the part of teachers and parents in the values curriculum design and implementation. The training module addressing the escuela de padres, also was significantly modified. Under Nqatoqi’I, principals dictated the time and dates for the escuela de padres as well as its content. They did so base on what seemed most convenient for them and for the teachers in the respective schools. This led to low parents’ participation because of scheduling problems, job conflicts, and or lack of interest in the content imparted during the meetings of the escuela de padres. Instead, in the Nqatoqi’II version of this module, educators were trained to elicit parents input and engagement as well as collaboration in the organization, content, implementation, and running of the escuela de padres. This new approach was expected to increase parents’ participation in the school meetings, in their children’s value education, and in the living of values in their families and communities.

During the impact evaluation carried out in 2012-2013, we encountered significant levels of personal and social ills in the communities. We found evidence of violence and maras’ (gangs) presence as well as of addictions among children in the schools included in the sample.  Table 1 reports these findings. Specifically, 60% and 52% of children and parents respectively reported feeling insecure in the communities where they live. Similarly, we encountered high level of drugs and maras activity in their communities, that both parents and children witnessed on a daily basis (between 20%-60% of children included in the sample, reported knowing children involved in crime.) Some of the maras’ operated from within the communities where they live, others operated base on other communities. The degree of influence of this groups was reflected in children reporting a wish to become a member of a mara (mareros) when they grew up (16% of all students. In some schools, this percentage reached 37%.)  Of these children, 50% of them are girls and 50% are boys.

Likewise, we encountered high levels of domestic violence –physical and/or verbal abuse as well as abuse of alcohol on the part of parents- within the communities (ranging between 25-43%). Finally, a significant number of children were exposed to or sexually active from an early age (26.32% of second graders reported knowing a friend who was sexually active.) Because of these last findings, Nqatoqi’II included a module addressing the education of affectivity by engaging the parents in discussing and helping children channel their affectivity in a healthy manner, while ensuring a safe environment in their home.

The Nqatoqi’II training took place at the beginning of January 2014.  It included principals and selected teachers from all the schools included in the sample with the exceptions of those that were trained under the Nqatoqi’I modality. By the time classes began all teachers in the selected schools had received the training and were ready to begin implementing the methodology. The six-month followed up took place at the end of June 2014. 

Table 1: Violence, Influence of Maras, and Addictions among Children as Reported by Children and Parents. (% of total for each group)

Sample and experiment implementation

To determine our sample size (n), we used the standard sample distribution equation, expanded to increase its volume. The standard sample size formula estimated was:

where n is the sample size, za/2 is the critical value, σ is the population’s standard deviation, and E is the margin of error.  At a 95% confidence interval and with a 5% margin of error, the recommended minimum sample size is 384. We expanded our sample by a factor of three to achieve volume as well as by adding a 20% clustering effect and a 10% collection error. A total estimated sample of 3,042 students distributed between treatment and control groups. We stratified by school size. A corresponding parent for each student included in the study and all 142 principals and teachers in the selected schools were also included in the sample. A total of 6,226 subjects. Students, parents, and teachers’ data collected, was mapped so to be able to carry out a 360 analysis. The latter facilitated the inclusion of interpersonal dimensions in the analysis.

All zones considered for the study, were defined as “zonas rojas” (high crime zones). Randomization took place at the zone level for the treatment groups. The control group included 3rd to 6th grade primary school students in public schools from zones 2 and 10 of Mixco, together with their parents and educators. The latter received Nqatoqi’ I training and implemented the intervention based on this training. The treatment group also included students from 3rd to 6th grade public primary schools, but they proceeded from zones 1 and 3, also located in the municipality of Mixco. All 3rd to 6th grade students attending the schools chosen in each zone, were included in the sample.

To collect the information, three different surveys were distributed in Spanish: students, parents, and educators. Students also were tested in analytical and critical skills, by completing a short exercise. The majority of the surveys were completed online using Qualtrics software over a period of three weeks. When access to computers were not possible, surveys were filled on paper and then uploaded.^[7] Baseline was collected before the training for the teachers, and at the beginning of the school year, the first three weeks of February 2014, for students and parents. Post data collection took place in October 2014, before the school year ended. After data cleaning, the final sample size collected was 5,103. This included 2,787 students between 3rd and 6th grade, 2,174 parents, and 142 educators (Principals, and teachers). This constituted 82% of the expected sample. As the collected sample size falls within the confidence interval of the sample design, analysis of outcomes are statistically representative.

Table 2 presents the characteristics of treatment and control groups at baseline, as well as the corresponding test for equality. Comparing the demographic characteristics of the treatment and control group, support statistical comparability between them. On average, educators are women of Ladino origin, married, and mothers of one or two children.  The average age of the educators is 40-44 years of age and they hold a tertiary teaching degree. Similarly, and on average, the parents of the children that participated in the research are Ladino, married, and are between 30-34 years of age.  For the most part, they have completed Básico (junior high) education and hold unskilled labor jobs. The average monthly income level per households is very low, between 2,501-4,000 Quetzales (here on Q).  Their children hold at least one job, and sometimes have two.^[8] Students that work have a higher probability of being enrolled in afternoon schools (jornadas vespertinas). The distribution between girls and boys is 50/50 and on average they are Ladinos.

Comparability also was found at baseline among the variables utilized to measure behavioral change in leadership style, participation in the escuela de padres, and parent’s engagement in their children’s education of affectivity.

These are reported in the next section together with the reporting of post data outcomes. Outcomes’ analysis was conducted by comparing subjects’ behavior before the interventions were implemented and at the end of the academic year.

Table 2: Treatment vs control characteristics

Estimations and results

As previously mentioned, three surveys (teachers, parents, and students) were utilized to collect the data at baseline and post data collection.  We first focused on three sets of variables. The first set of variables, measures proactive and participatory behavior, in the implementation of social and civic values education across the curriculum, in each school, as reported by teachers. Table 3 reports baseline and post mean values for each of the variables as well as the results for the test for equality of means. Measurements in this set include whether teachers were trained or not on the teaching of values, whether the principal and the other teachers worked together in the implementation, and whether they engaged each other in a proactive and participatory manner. In all cases there is a significant increase in participatory behavior and in implementation effectiveness in treatment schools.

Table 3: Changes in proactive and participatory behavior in teaching values  (Average % of total relevant population)

The second group of variables included in the analysis measure parents and teachers’ participatory behavior in the creation and implementation of the escuela de padres (Table 4). Question regarding principals’ leadership style, parents’ engagement in the design, organization, and implementation of the escuela de padres as well as engagement in the school at large are included in this section. A test for different of proportions of the variables examine, evidence significant and large changes in proactive participatory behavior among treated teachers and parents. Two significant behavioral change are also found in the control group, but these are not as large or as significant as the changes of behavior generated in the treatment group. Teachers are more engaged in the teaching of values to students and in the development of the escuela de padres. Same outcomes are found among parents.  More importantly, parents increase their active participation in the escuela de padres, and they find that doing so, is helpful and relevant for the education of their children. This is also the case in helping their children managed their affectivity and in them understanding, in a constructive and positive light, their sexuality.

Finally, the third group of variables measures parent’s engagement in their children’s education of affectivity and sexuality as reported by the children (Table 5). The variables included measures children behavior towards their parents and colleagues regarding their affectivity and sexuality. In all cases we find significant proactive and participatory changes in behavior. Overall, treatment teachers and parents report significant and positive changes in behavior. The proportion of teachers and parents across treatment schools that exhibit higher proactivity and participatory behavior is between 1.5 and 5 times higher than the corresponding reported behavioral changes in the control group.  Significant proactive and participatory change in behavior is also found among students in treatment groups. As both groups received some version of the Nqatoqi’ training, outcomes suggest that the introduction of PPI have been effective in eliciting change of behavior in treated subjects.

Table 4: Changes in proactive and participatory behavior in “escuela de padres” (Average % of total relevant population)

Table 5: Changes in proactive and participatory behavior with parents education of affectivity (Average % of total relevant population)

The impact of the experiment on behavioral change can be also measured by means of a cross section simple regression framework with fixed effects. We estimated equation (2) by means of Ordinary Least Squares (OLS) with fixed effects for zonas.  

where Post-Pre is the difference in the variable of interest (behavior) for a given individual I, between the post-experiment (collected at the end of the school year) minus baseline (collected at the beginning of the school year, and before the intervention was introduced). Treat is the dummy capturing the treatment received (treatment of Nqatoqi’II=1, 0 if Nqatoqi’I);  Xi  encompasses other explanatory variables of importance that we wish to analyze (typically interpersonal relational behavior); Ci encompasses all control variables, including demographics (to mitigate omitted variable bias), vi  is the fix effect, and εi is the error term. ^[9] 

Every regression was tested for violations of OLS and, when present, corrected. Heteroscedasticity, was corrected by means of White’s method of computing heteroscedasticity-consistent standard errors for cross section data.  

Nqatoqi’ does not only seek to teach civic and social values. Rather, it wants to generate changes in educational institutions and communities by fostering habits in students, parents, and teachers, to improve themselves as persons and thus be agents of change in their families, their communities and the society at large. Because of this, it is expected that beneficiaries of the Nqatoqi’ II, would be more motivated to make positive changes in their behavior and, in doing so, help others to do the same (proactive and participatory behavior). At a minimum, their behavioral change should be reflected in a better quality of life at home as well as on a better fulfillment of their social and civic responsibilities. A consequence of these behavioral changes, there should be a reduction of the prevalence of personal and social ills in the communities where they live. It could also be expected that these overall improvements in the subjects’ lives, would lead to students reporting better academic outcomes as well as higher levels of motivation to live the values learned.

To measure these factors, we constructed indexes using principal component analysis (PCA) when appropriate. The indices were created to fully measure the effect of the program on a variety of aspects of the subjects’ lives. These include: 1) Academic outcomes (measured by a short critical and analytical thinking exercise applied only to 5th and 6th graders, and also by students’ Grade Point Average (GPA) collected at the beginning of the school year and at the end); 2) motivation (measured by self-motivation and by the motivation students received from parents to live the values learned); and 3) socioeconomic aspects (measured by the quality of life, and the living of their social and civic responsibilities, as well as their exposure to personal and social ills.) A description of the variables included in the construction of each index can be found in the Appendix.

PCA calculates certain values – called “eigenvectors” –based on correlations between a given set of variables.  These eigenvectors are used as weights for each variable in the index so as to give prominence to the variables which held the strongest influence over the direction of the index.  We did not drop any variables, as is often the result of PCA, because we wanted to capture as much of reality as possible. 

Academic engagement looks at the student’s relationships with teachers, interest in academic studies, innovation in assignments, and participation in academic life. The critical and analytical thinking is based on an exercise distributed to students. Measurements capture the level of expected reasoning for the fifth and sixth grade students included in our evaluation.

The quality of life index paints an overall picture of the participant’s relationships with friends and family members, time for leisure activities and personal care, the living of their faith, and engagement in group activities. Specifically, it measures the frequency with which subjects engage in these activities. As the way time is allocated to each of the activities, reflects the value given to building relationships and caring for others at different levels, and also to caring for oneself, the index captures whether or not subjects live according with his or her dignity.  The social responsibility index focuses on whether subjects are looking to help his friends, family, or community and or whether the parents and teachers live up to their social and civic responsibilities such as fulfilling their civic duties, volunteer work, etc. Finally, personal and social ills, asks questions about the behavior of others capturing if subjects have heard of or knows anyone who participates in drugs or violent related activities, or suffer abuse, among others. These four groups of indexes capture a more holistic view of the impact that introducing PPIs in Nqatoqi’ has had on students, teachers, and parents.

We first analyze the change in participatory behavior on all subjects. Table 6 reports the regressions’ estimation outcomes. The dependent variables are indices constructed utilizing the variables reported in Tables 3-5. The higher the number of the index, the more proactive participatory is the  behavior.  All PPI components introduced in Nqatoqi’ II generate statistically significant and positive changes on the participatory behavior of teachers, parents and children. If parents perceive the school as a support to their parental responsibilities, they are also more engaged. The same occurs when there is good communication between parents and teachers. These two factors highlight important pathways to generate effective channels of relationships. The participatory approach to the escuela de padres’s designed and implementation seems to effectively address the typical conflict parents faced between the time at which escuela de padres were schedule and their job hours.  The higher the number of jobs parents have, the lower is their participation in the escuela de padres, but in schools where Nqatoqi’II was introduced, this is no longer the case. In these schools, typically escuela de padres’ meetings are scheduled on Saturdays. As a consequence, the possibility of scheduling conflict with working hours is diminished. Parents and teachers volunteer in such a way that all the organizational burthen of the parent’s meetings happening on Saturday, do not falls on the Principal or the teachers of the schools. The impact of the PPI on children’s change of behavior related to their affectivity, is similar to the other findings. Indigenous parents exhibit a larger change in participatory behavior than Ladinos. Finally, the higher the levels of education, the greater is the change in participatory behavior.

Table 6: Change in proactive and participatory behavior

We also analyze the impact of introducing PPIs on the teaching values across the curriculum implementation’s effectiveness. Results confirm expected outcomes, i.e., the introduction of the different PPIs in the Nqatoqi’II increases effectiveness. The analysis also highlights relational pathways (effective channels of relations), that increase the intervention’s effect on effectiveness. These include working in a collaborative manner with the Principals, with other teaches, and with parents in the implementation of teaching values. It also includes teachers’ commitment to values, and parents being engaged in forming their children in affectivity. Results are reported in the Table 7.

Table 7:  Effectiveness in the Implementation of values in the curriculum 

Similar analysis was carried out on three specific variables that measure students’ behavioral changes regarding their affectivity. The variables included are: whether or not they speak with their parents about issues with friends (boys and girls) that bother them; whether or not they encouraged friends to wait until they get married to engage in sex; and whether or not they behave in a respectful manner with their boyfriend/girlfriend. Results are presented in Table 8.

Table 8: Students’ change in proactive and participatory behavior in the area of affectivity

All three PPIs impacted children’s behavior regarding their affectivity in a positive and statistically significant manner. If parents that were/are engaged in the development of the escuela of padres and in the education of their children’s affectivity, children report a larger and positive change in behavior. Additional relational pathways, that help the generation behavioral change, include the commitment of teachers and parents to teaching values, and the support parents receive from the school in their efforts to educate their children. Girls report a larger change in behavior regarding speaking with their parents when something worried them and in treating with respect their boyfriends. If the parent is a single mother (mother is head of household), the child reports a larger change in the behavior than if both parents are present (father head of household). This is especially relevant, as the probability of girls being sexually active, is higher among those living with single parents’ households. ^[10] 

We next measure the impact of the PPIs on the students’ quality of life, i.e., living their personal lives according with their dignity, and on the living of their social and civic responsibility.  Table 9 reports the regression outcomes. All three PPIs introduced, as part of the experiment, have a positive and significant impact on students’ behavioral change. This is the case for behaviors directly related to the living of values (such as being respectful), or for behaviors that facilitate them living values (speak with parents when concerned about something). Also there is evidence that parents perceiving schools as supporting them in their parenting responsibilities; and children and parents receiving a consistent message regarding the importance of living values, are positive influences in children’s behavioral change. Thus, both are pathways (effective channels of relations), that help children live their social and civic values by fulfilling their respective responsibilities in these two areas. Parents having many jobs, even if they participate in the escuela de padres, impact negatively their children ability to fulfill their civic and social responsibilities.  This is not the case for quality of life. An important reason for this discrepancy, is the lack of time the parents face due to them having more than one job. This leaves them with little or not time available for them to be involved in social and civic activities, and/or to accompany their children to these types of activities.

Similar results are found when analyzing the impact of Nqatoqi’II on social and personal ills as well as on students’ self-motivation, students’ perception of parents support on straving to live by values, and on academic outcomes.  For the sake of brevity, we report in this paper the students’ treatment and control distributions for the mentioned behavioral changes.^[11] 

Figures 1-3 present the mentioned indices’ distributions. Figure 1 reflect the change of children’s exposure to personal and social ills in the communities where they live. In both control and treatment groups, there is a decline in students’ exposure to personal and social ills but the decline is larger among children in treatment groups. On average, the gap between treatment and control groups is three and two times respectively. The existence of maras’ networks operating in multiple communities, including those which the intervention does not reach, is one of the factors explaining this difference on the size of the impact. 

Table 9: Change in proactive and participatory behavior

Figure 1: Distribution of change in children exposure to Personal and Social Ills 

Figure 2, depicts students’ behavior changes regarding self-motivation and motivation students received from parents to live values. A description of the variables included in the construction of these indexes can be found in Tables A.3 and A.4. Similar to the findings on social ills, control and treatment students shown higher levels of motivation in both indices, but the group that received the PPI treatment (Nqatoqi’II) report higher levels of motivation (two and eight times higher respectively). Team work between parents and teachers, as well as consistency in the message receive and the example given regarding the importance of living values at school and at home are significant factors contributing to the larger impact found among treatment students.

Figure 2: Distribution of change in children motivational behavior

Figure 3: Distribution of change in children academic performance

The positive impact of behavioral change, spills over academics. Utilizing two measure, GPA and the critical and analytical skills, we find evidence that students’ improvement in the living of values leads to a statistically significant increase in academic outcomes. Description of the critical and analytical skill test can be found in Table A.1 and A.2 of the Appendix. The GPA increase among treatment students is six times higher than the control group. The gap in the critical and analytical thinking skills is significantly lower, only 33%. The poor preparation of teachers’ in the country partially explains this low impact.

Overall, findings underline the effectiveness of utilizing PPIs to generate positive behavioral changes regarding the teaching and living on values. This in turn contributes toward safer communities and better academic outcomes.  Results also indicate, that collaboration between teachers and parents, as well as among parents, and a good communication between parents and children are effective pathways – effective channels of relations-, that reinforces the intervention’s impact on the living of values.

Analyzing the impact of the PPIs on teachers and parents’ behavioral change in the areas of civic and social responsibilities, we find similar results. ^[12]  For the sake of brevity these outcomes are not reported.

Conclusions

The RCT carried out in the municipality of Mixco, Guatemala, during the academic year 2014, sought to test the impact of introducing participatory and proactive interventions in the design of the educational program Nqatoqi’. The initiative, developed by the Department of Education in the University of Istmo, seeks to improve the learning and living of values among educators, parents and students. Following the directives of the Curriculum Nacional Base (CNB, Base National Curriculum) on values, the program trains teachers on the introduction of the teaching of values across the curriculum. After an impact evaluation carried out in 2012-2013, three modifications were recommended to the existing program. These included: 1) a leadership training to teachers and directors based on participatory leadership; 2) A proactive and participatory approach to the design and implementation of the escuela de padres (parents’ association) in each school; and 3) the introduction of a module on education of affectivity for students enrolled in 5th and 6th grades that directly engaged parents, so they in turn, work with their children in a participatory manner

Three proactive and participatory interventions (PPIs) were introduced in the original Nqatoqi’s design (Nqatoqi’I), which in this paper we called Nqatoqi’II.  Both interventions were implemented in four districts of Mixco’s public schools to children enrolled in third to sixth grades.  Outcomes indicate that the introduction of PPIs significantly improve the living of values in teachers, students and parents. Findings also highlight important pathways, effective channels of relations, that help reinforce PPIs’ impact. These include collaborative efforts between parents and teachers in teaching values, teacher’s teamwork as well as teamwork between principals and teachers. Consistency of message and example from teachers and parents in the teaching of values, i.e., consistency of message at school and at home regarding the importance of living values are also effective channels of relations.

Improvement in the living of values, also leads to a decrease of personal and social ills in the communities where they live; it elicits self-motivation to live values better and it also increases parent’s motivation to children to live values. We also find a statistically significant improvement on quality of life of all subjects as well as an improvement in the subject’s fulfillment of their social and civic responsibilities. Furthermore, this change of behavior in turn leads to improvement in academic outcomes.  Students in treatment schools report higher GPAs and higher scores in the critical and analytical skills exercise that was imparted to them.  In summary, all expected outcomes of the intervention were fulfilled.

Bibliography

Aguirre, Maria Sophia (2013a). An impact evaluation of the educational program on civic values in Nqatoqi’, IED Report #1, School of Business and Economics, The Catholic University of America.

Aguirre, Maria Sophia (2013b). An integral approach to an economic perspective:  consequences for measuring impact,” Journal of Market and Morality, 16(1), 53-67.

Akerlof, George, and Yellen, Janet (1994). Gang behavior, law enforcement, and community values. Washington D.C.: Brookings Institution Press.

Alesina, Alberto/Giuliano, Paola (2016). Culture and institutions. Journal of Economics Literature, 53 (4), 894-944.

ASIES (2012). Agenda de educación municipal con calidad, Suplemento ASIES.

Azpuru, Dinorah (2002). La cultura democrática de los jóvenes y los maestros en Guatemala, ASIES.

Banerjee, Abhijit and Sendhil Mullainathan (2010). The shape of temptation: implications for the economic lives of the poor, NBER Working Paper No. 15973.

Borgatti, S. P., Everett, M. G., & Johnson, J. C. (2013). Analyzing social networks, Thousands Oaks, C.A.: Sage.

Centro de Investigaciones Económicas Nacionales (CIEN) (2008).  Segundo informe del progreso educativo de Guatemala 2008, Guatemala: CIEN.

Centro de Investigaciones Económicas Nacionales (CIEN) (2011).  Magnitud del costo de la violencia, Guatemala: CIEN.

Duflo, Esther (2012).  Human values and the design of the fight against poverty, The Tanner Lectures on Human Values, Harvard University.

Duflo, Esther and Emmanuel Saez (2002). The Role of Information and Social Interactions in Retirement Plan Decisions: Evidence from a Randomized Experiment, Quarterly Journal of Economics 118(2), 815-842.

Duflo, Esther, Pascaline Dupas, and Michael Kremer (2010). Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya, American Economic Review, 101(5),1739-51.

Duflo, Esther and Abhijit Banrjee (2012). Poor economics: A radical rethinking of the way to fight global poverty, Ney York, N.Y.: Public Affairs.

Duflo, Esther, Pascaline Dupas, and Michael Kremer (2015).  Education, HIV, and early fertility: experimental evidence from Kenya. American Economic Review 105(9), 2757–2797.

Gayen, Avijit and Joydeep Chandra (2010), Role of trust in evolution of scientific collaboration networks, Physical Review, 82(2), 026112.

Loening, Josef (2002). The impact of education on economic growth in Guatemala: A time-Series analysis applying an error-correction methodology, Diskussionsbeitrag, 87 des Ibero-Amerika Instituts, Göttingen.

Loening, Josef (2004).  Time series evidence on education and growth:  The case of Guatemala, 1951-2002,  Revista de Análisis Econámicos, 19(2), 3-40.

Feigenberg, Benjamin, Erica field, and Rohini Pande (2013).  The economic returns to social interactions: experimental evidence from microfinance, Review of Economic Studies, 80(4), 1459-1483.

Rivas, Jorge Lavardel and Jorge Lavarreda (2008). Análisis de las remuneraciones de los doscentes del sector público en Guatemala, Ministry of Education of Guatemala.

Rodrik, Dani (2000). Institutions for high-quality growth: What they are and how to acquire them, Studies in Contemporary International Development, 35(3), 3-31.

Schein, Edgar (1999). Process consultation revised, Englewood Cliffs, NJ: Printice Hall.

Schein, Edgar (2009). The corporate culture survival guide, San Francisco, C.A.: John Willey & Sons.

Schein, Edgar (2010). Organizational culture and leadership, San Francisco, C.A.: John Willey & Sons.

Scott, John (2008).  Social network analysis: a handbook. 2nd ed. CA: Sge. 2nd Edition

Universidad del Istmo (2012). Estudio de opinión pública sobre maltrato a la mujer por parte de la pareja, Guatemala: Universidad del Istmo.

Valente, Thomas W, 2010. Social networks and health: models, methods, and application. Oxford: Oxford University Press

Zwane, Alix Peterson, Jonathan Zinman, Eric Van Dusen, William Pariente, Clair Null, Edward Miguel, Michael Kremer, Dean S. Karlan, Richard Hornbeck, Xavier Giné, Esther Duflo, Florencia Devoto, Bruno Crepon, and Abhijit Banerjee (2011), Being surveyed can change later behavior and related parameter estimates, Proceedings of the National Academy of Science, 108(5), 1821-1826.

Appendix

Table A.1: Critical Thinking Measurements

Description:	Answers:
Common letter between Murcia and Madrid	(1) M (0) Other
Has a beak and does not eat…	(1) Shovel (0) Other
How close is the brother of my father’s son to me?	(1) All the above (0) Older brother (0) Younger brother (0) Brother
Today I went to buy oranges, the lady of the store has given me 6, I have eaten 1 and my father 2, another has fallen and it has spoiled. How many oranges do I have left?	(1) 3 (0) 2 (0) 4 (0) None
DIDIIDID is to 49499494 as DIIDIIDD is to…	(1) 49949944 (0) 94494499 (0) 94944949 (0) 49944949
We have three boxes of equal size. Inside each of the three boxes there are two smaller ones and in each one of these four still smaller ones. How many boxes are there in total?	(1) 24 (0) 9 (0) 33 (0) 30
The plant is to the seed as the human is to the…	(1) Ovum (0) Ovary (0) Spermatozoon (0) Embryo (0) Uterus
If an electric train goes from North to South, which side will the smoke emit?	(1) No smoke is emitted from an electronic train (0) Other
Juan is faster than Sara and Eva is slower than Juan. Which of the following statements is true?	(1) We cannot know if Sara is faster than Eva (0) Eva is faster than Sara (0) Eva is slower than Sara (0) Eva is as fast as Sara
How many quarters are 6 halves?	(1) 12 quarters (0) 8 quarters (0) 10 quarters (0) 14 quarters
A person who went to Seville on the road overtook a man with 7 women, each woman had 7 sacks, each bag 7 cats and each cat 7 kittens. Kittens, cats, men, women and sacks how many were going to Seville?	(0) 2800 (1) 402 (0) 2401 (0) 2801 (0) 2802
A TV is sold in Bolivianos 600 at the corner store, last week I got a message to my cell with the announcement of the same television but at 450 Bolivianos, what is the discount percentage?	(1) 25% (0) 20% (0) 10% (0) 35%
What is 14% of 28?	(1) 3.9 (0) 2 (0) 3 (0) 12
How many continents are there?	(1) 5 (0) 1 (0) 3 (0) 4
What is the largest country in South America?	(1) Brazil (0) United States (0) Mexico (0) Colombia
How many countries do you have in Latin America?	(1) 20 (0) 8 (0)10 (0) 16
If a triangle has an inner 90 ° angle is it called?	(1) Rectangle Triangle (0) Acute Triangle (0) Triangle Equilateral
Luis is training for a 10-kilometer race, every day he must run 5km, how many miles he is running weekly if a mile is 1.6 km	(1) 21.7 miles (0) 12 miles (0) 10.6 miles
José’s trouser was worth $ 4000 Bolivianos. Yes, in the sales of the month of July they make a discount of 30% How much must pay?	(1) 2800 Bolivianos (0) 3700 Bolivianos (0) 1500 Bolivianos (0) 2800 Bolivianos
My uncle lives in NY and asked us to call him on Thursday, February 10 at 2:00 p.m. If I live in Bolivia at what time should I call him?	(1) 2:00 a.m. (2) 3:00 pm (3) 4:00 p.m. (4) 2:00 p.m.

Table A.2: Analytical Reasoning Measurements

Table A.3 Motivation to improve in school: Parents Index

Table A.4. Motivation to Improve in School: Self-Motivation Index

 Table A.5 Motivation to Improve in School: Educator and Peers’ Index

Table A.6 Academic Engagement Index

 Table A.7 Quality of Life Index

 Table A.8: Civic Responsibility Index

Table A.9: Social Responsibility Index

Table A.10: Social Ills Index

---