PILOT EVALUATION OF THE
WISCONSIN CHARTER SCHOOL PROGRAM
Introduction
Charter schools have overthe last ten years become an important education reform. During the 1990's charter schools wereestablished in every area of the country. Wisconsin's charter school law, initially passed in 1993 and revised in1995, 1997, and 1998, has resulted in the creation of 62 charter schools by1999. By fall 2000, the Department ofPublic Instruction estimates that Wisconsin will have close to one hundredcharter schools.
Supporters argue thatbecause they are freed from many regulations that govern the typical publicschool and they have the autonomy to create innovative programs that betterserve their students and parents, charter schools have great potential toimprove education. Thus, proponentsbelieve the practices charter schools develop can function as effective andefficient models for other schools and thereby lead to broader educationalreform.
Most observers agree thatfor charter schools to justify the elimination of normal district and stateregulations and to fulfill their promise, accountability is essential. Although individual charter schools as wellas their chartering agencies may conduct their own evaluations of theirprograms, a statewide, longitudinal evaluation of Wisconsin's charter schoolsis essential if policy makers are to understand the overall impact of thereform.
Purpose
The purpose of this pilot evaluationis to develop a model for a statewide evaluation of charter schools. As such, it should not be considered anevaluation of the three charter schools used to pilot the design.
EvaluationQuestions
The questions that guided the evaluation are thefollowing:
1. What is the nature of theinnovative program of the charter school?
a. What type of innovativeprogram has been identified?
b. What is the innovativeprogram's philosophical or empirical support?
c. How is the innovativeprogram being implemented?
2. How effective is theinnovative program in terms of academic and other types of achievement andsuccess?
a. How do the 3rd, 4th,8th, and 10th grade state achievement test scores ofstudents in pilot evaluation charter schools compare with students in othercharter and non-charter schools?
b. To what extent do otherindicators demonstrate student growth?
c. How satisfied are teachersand parents with the charter school innovative program and its effects?
3. Is the school environmentconducive to student health and growth?
4. Are the teachers preparedand committed?
5. Does the school governanceinclude parent voice and does it facilitate innovative program success?
6. Does the charter contractensure that the goals of the charter school reform will be achieved?
7. Has the charter schoolinfluenced practice in public schools?
The term "innovative program" used in thisevaluation refers to a non-typical, unusual program that is not generally usedin similar contexts.
SelectedCharter Schools
The charter schools selected forthis pilot evaluation were Khamit Institute, Walker International MiddleSchool, and Horizon Academy. Theseschools were chosen because they represent a range of school levels, types,sizes, and chartering agencies, and they are located in the Milwaukeearea. Khamit Institute is a grade K4-8elementary school enrolling 55 students. It is chartered by the city of Milwaukee and located on the near westside. It has been a functioning privateschool since 1996, and a charter school since 1998. Walker International Middle School is a grade6-8 middle school on the south side of the city. It is chartered by the Milwaukee PublicSchool and enrolls 790 students. Walkerbecame a charter school in 1999. HorizonAcademy is a grade 9-12 school-within-a-school for at-risk students. It is chartered by the St. Francis Schoolsand located in St. Francis High School on Milwaukee's far southeast side. Horizon Academy became a charter school in1998 and currently enrolls about 40 students.
DataCollection
Data were collected from the threecharter schools through an administrator questionnaire; administrator, teacher,and parent interviews; school and classroom observations; and charter schoolcontract analysis. These data werecollected from January through May, 2000. Achievement test scores for the three schools and comparison schoolswere obtained from the Wisconsin Department of Public Instruction.
Administratorquestionnaire. The administratorof each charter school was asked to complete a questionnaire that dealt withthe innovative program features, implementation, goals, assessment processes,and effects. It also dealt with student discipline policy, school environment,teacher preparation and commitment, parent involvement, school governance,chartering agency accountability, and impact on public schools.
Administratorinterview. Each charter schooladministrator was interviewed regarding his or her views on the adequacy of theAdministrator Questionnaire questions and format and to clarify or extendcomments made on the administrator questionnaire. These interviews were tape recorded andtranscribed.
Schooland classroom observations. Thenumber of classroom observations varied with the size of the school, as did thenumber of teacher interviews. At bothKhamit and Walker, variety in grade levels, subject areas, and teachers wassought. At Khamit eight observationswere made, including all four teachers. At Walker 12 observations were made in the classrooms of one of theschool's interdisciplinary teams. AtHorizon, which has a single teacher, two observations were made. The observations were of complete lessons,using a general observation guide. Observers took notes and developed narrative accounts of theobservations.
Teacherinterviews. Four teachers atKhamit, six at Walker, and one at Horizon were interviewed once regarding theirperceptions of the innovative program including teaching, assessment, andeffects. Also, their perceptions ofautonomy, parent involvement, and other matters were elicited. These interviews were tape recorded andtranscribed verbatim.
Parentinterviews. Two parents at eachcharter school were interviewed to determine their knowledge of andsatisfaction with their charter school and the effects they perceived on theirchild or children.
Charterschool contract. Each of thethree charter school contracts was examined to determine the intended andimplemented accountability measures and other intended features.
Achievement test scores. Students in grades 3, 4, 8, and 10 of Wisconsincharter schools are required by law to take state achievement tests. The scores attained by students in the threecharter schools were obtained from the Wisconsin Department of PublicInstruction (DPI) and used to create models for analyzing charter schoolperformance.
Structureof This Report
This evaluation is divided intothree parts. Part I: Program Analysisfocuses on the nature of the innovative program but also describes otherimportant characteristics of the selected schools. PartII: Contract Accountability Analysisprimarily examines the accountability provisions of the charter contracts ofthe selected schools. Part III: Achievement Test Analysis provides a structure by which to comparestudent achievement in the selected charter schools with student achievement inother schools matched on school size, racial makeup, exceptionality, andgender. The report concludes with asynthesis and discussion of findings.
Part I: Program Analysis
Khamit Institute
IntendedInnovative Program
Khamit Institute serves children fromfour-year-old kindergarten through 8th grade. The purpose of the school is to have students"learn about and appreciate their cultural history and forms whileobtaining an exemplary academic education" (Khamit Institute StaffHandbook, 1988). The culture in whichstudents are immersed is based on the Khamitic culture of ancient Egypt. Guiding principles that are said to be drawnfrom that culture include knowledge of oneself, that all people have genius andworth, and that harmony and balance pervade existence.
The educational program of KhamitInstitute has both a curricular focus and an instructional focus. The stated curricular focus is to preparestudents to successfully interact in a world society without being overwhelmed. It includes the development of culturalappreciation, academic ability, life skills including entrepreneurialexperience, self awareness, a range of intelligences, and globalunderstanding. The specific academiccurriculum goals of Khamit Institute as stated in their charter school contractare the following:
1. Global awareness. Students will be able to demonstrate andarticulate an understanding of themselves and the world around them throughhumanities, social studies, and foreign language. They will develop personal growth and globalidentity. School-wide Khamitic themesbased on the range of intelligences are studied.
2. Reading mastery. Students will become strategic, independent,and confident readers. They will becomecompetent in decoding strategies, comprehension skills, literacy skills, andstudy skills.
3. Mathematics and science. Students will be able to make informed decision and use masteredscience and mathematics concepts to solve problems. They will become capable problem solvers,think and communicate mathematically and scientifically, develop a depth ofunderstanding needed for advanced study, and make connections betweenmathematics and science.
4. Reasoning and writing. Students will be able to think, express themselves, and write clearly.
5. Physical education. Students will be able to make mind-body connections through mastery ofmartial arts and health and wellness knowledge.
Thecurriculum, according to the administrator, uses themes "to integratesubjects, grades, and the school activities with the vision, mission, andphilosophy of Khamit Institute."
The instructional focus is directinstruction. This method of teaching,heavily researched in the 1970's in reading and mathematics at the elementaryschool level, is a teacher-centered way of the teaching. Although there are various forms of directinstruction, characteristics common to all forms are that the teacher sets thelearning agenda, provides and models knowledge and skills, controls participation,elicits and critiques students' understandings, provides practice experiences,reteaches when necessary, and reviews prior learnings. Student interests, choices, problem solving,critical thinking, and creativity are typically not associated with directinstruction.
The key elements of directinstruction specified in the Khamit Institute charter school contract are thefollowing:
1. Explicit teacher-ledlearning that develops students' thinking and independent problem solving.
2. Homogeneous instructionalgroups based on academic level rather than grade using a common curriculum withno separate tracks.
3. Mastery of sequenced contentand skills prior to advancement to a succeeding level.
4. Polished, scripted lessons.
5. Whole class engagementincluding frequent choral responses as well as individual responses.
Inaddition to direct instruction, accelerated learning was mentioned as a featureof the school's instructional program by the administrator and teachers. It was selected because it "is mostclosely aligned with Khamit principles," the administrator said. Accelerated learning is a method of teachingthat is not dissimilar from direct instruction. It includes the use of behavioral objectives, preparing students toreceive instruction, presenting and memorizing information, practice orperformance activities, and assessment. It differs from direct instruction in that it incorporates the sevenintelligences (described by Howard Gardner) and, as used at Khamit, employsmusic to enhance learning.
ImplementedInnovative Program
Both the identified curriculuminnovations and instructional innovations are being implemented at KhamitInstitute to some degree.
Innovativecurriculum. Rather than being an aspect of thecurriculum, Khamitic culture, according to the administrator, serves as "acultural base from which our children can learn. It's a system that has been successful indeveloping the potential of people. Andwe want to practice it and to teach it to our children." Curriculum is viewed as the core subjectssuch as reading, mathematics, and language. Nevertheless, Khamitic culture, principles, and values are a centralfeature of what is taught at Khamit Institute.
Although the authenticity of characterizingindividual aspects of the program as Khamitic was not investigated for thisevaluation, it appears that Khamitic and African culture is rich and pervasiveat Khamit Institute. The first thing onenotices upon entering the school at about 8:00 a.m. is the rhythmic beat ofAfrican drums. Each school day is begunwith a total school gathering in an assembly room, an entry room on the groundfloor brightly decorated with African posters, murals, symbols, and artwork. Four or more boys trained inAfrican drumming beat the drums as children and teachers gather for daily openingactivities. It appears that all thechildren and teachers are dressed in African-style "uniforms" of thesame blue and pink (on this day) tie-dyed colors. The session, led by an older student and ateacher, consists of African songs, including the School Pledge to the tune of"Oklahoma;" the use of African terms and phrases such as "HeruNefer" (Good Morning), "Ser-t" (Female Elder), and"Ur-Auat" (Queen Mother); the honoring of all adults in the room;unison pledge and ritual chants relating to Khamit principles; recognition ofstudent behavior consistent with Khamitic ideals; and preparation for theschool day by having students visualize themselves engaged in appropriatelearning behavior. Throughout the sessionthe drums spontaneously erupt to emphasize and applaud comments andevents. They also strike up at thesession's conclusion to accompany the flow of students to their classrooms.
The suffusion of Khamitic andAfrican culture continues throughout the day. African maps and batik hangings are displayed on classroom walls. In an observed lesson in global studies forgrades 4-6, the subject matter was countries of Africa. The students were to memorize the countriesand regions of Africa as well as to acquire information about Ethiopia and Nigeria. During the lesson Khamitic terms were usedsuch as "As" (attention), African songs were sung, and studentsdanced African dances. The teacherreminded students of a raffle for a free trip to Ghana. In a lesson in entrepreneurship class forgrades 1-3, the business enterprise being established was named the Men NeferMagic Shop.
All curriculum, then, is augmentedby Khamitic and African learnings. Asone teacher remarked, "we weave it in because we are a Khamitic immersionschool. We weave in Africa, African,whenever we can throughout the day."
What the administrator referred toas the core curriculum does not appear to differ substantially from what onemight find in any elementary school. Reading, language, mathematics, science, and social studies all occur,but with the Khamitic overlay where appropriate. Music and languages, also, clearly reflectthe cultural emphasis. Songs, dance,drums, and recorded jazz and other music seem to play an important role, as dothe foreign languages of Swahili and French. The administrator said, in reference to the curriculum, "I didn'tthink we were an innovative school, per se."
Innovativeinstruction. Direct instruction,the type of instruction identified for use at Khamit, was observed in part inmost classrooms. Teachers have receivedinservice training in direct instruction and appear to know its theory. Some of the teaching practices observed areconsistent with direct instruction, but some are not. Homogeneous, multi-age grouping as is done inreading and mathematics fits direct instruction, although it is not requiredfor direct instruction to be used. Helping students to memorize facts, asking recall questions, lecturing,unison responding, and practice worksheets are all direct instructiontechniques and were observed in the global studies lesson, the entrepreneuriallesson, and other lessons. Othercharacteristics of direct instruction such as stating a purpose for the lesson,modeling procedures and products, checking for understanding, and re-teachingwere observed infrequently.
A number of practices that were observed, althougheffective if done well, were in opposition to direct instructionprinciples. These include assigning workrather then explaining it and guiding students through it, group work withoutteacher leadership, and structuring a lesson inductively. In addition, there seemed to be a wide rangeof competence among teachers and assistants when doing direct instruction: from unsure of content, stiff, and unaware ofwhat students were actually doing in one class, to well-versed in the content,comfortable, and constantly cognizant of students' actions and needs inanother.
In one classroom, students were given a booklet ofworksheets to complete individually, apparently without previous directions orexplicit substantive input. In anotherclassroom students were told to open their textbooks and begin "Lesson 41,Section A." The teacher read thedirections to the class, but content was not explained and demonstrated. In still another classroom, reasons why theclass was engaging in a business enterprise were elicited from students andwritten on the chalkboard by the teachers. Students were required to copy the completed list. Again, no direct instruction about reasonsfor the business enterprise occurred. The activity previously mentioned about interviewing a person born inAfrica could be consistent with direct instruction but only after students hadobtained a substantial body of knowledge about Africa and the activity wascarefully structured and controlled, not simply assigned.
In regard to accelerated learning,the use of song as a mnemonic device was observed in several classrooms, butother features of this method such as whole-to-part and a focus on auditory,visual, and kinesthetic learning features mentioned by the administrator, werenot observed.
Some instructional and some classroom managementpractices that were not developmentally appropriate were observed. In one lesson, for example, it was doubtfulthat the six- and seven-year-olds were able to grasp the meaning and role ofcorporate officers. In another, a youngchild who was not following the teaching assistant’s complex entrepreneurialpresentation but not disturbing others was nagged repeatedly by anotherassistant and physically removed, crying, from the room.
Innovativeorganization. Khamit students are grouped into fourclasses--K4-K5, 1st-3rd grade, 4th-6thgrade, and 7th-8th grade. Multi-age grouping at Khamit, according to the administrator, is mainlya practical choice resulting from the small size of the student body ratherthan a theoretical decision. Generally,the daily schedule for all students is this: school opens with the assembly and breakfast, followed by reading andspelling, math, reasoning and writing, and Khamitic studies. Students generally stay with their homeroomteacher all morning. After lunch andrecess, they might be with different teachers or teaching assistants for socialstudies, computer lab, French, or Swahili.
Effects ofthe Innovative Program
According to the Khamit Institutecharter school contract, student progress will be measured in five ways:
1. Mastery tests at regularintervals associated with direct instruction.
2. Portfolios used innon-direct instruction areas.
3. "Constructs/projects"displaying students' intellectual strengths.
4. Journal writing providing"avenues for personal insight."
5. Self assessment as well aspeer input and constructive teacher feedback.
The administrator reported thatjournals, portfolios, projects, and mastery tests were used, but results werenot shared with CERAI evaluators. Regarding Khamit Institute effectiveness, the administrator said that“33% of the graduates have been in the top 10% of their high school classes;84.1% of students advanced at least one grade level in reading, math, andlanguage arts.” She also remarked,“We’ve seen students make phenomenal progress since they’ve been here….Wehaven’t done a scientific documentation of it.”
The administrator said that results were “reportedto authorities via monitoring agency.” The KhamiticInstitute Programmatic Profile and Educational Performance – 1998-99 AcademicYear, the evaluation report conducted forthe chartering agency, contains data from pre and post placement tests in math,reading, and writing. The report statesresults from the students who took pre-tests in September 1998 and post-testsin June 1999. Of the 37 students takingthe math tests, 80% “advanced at least one grade level.” Of the 27 student who took the reading tests,85% advanced at least one grade level. Of the 31 students tested in “reasoning and writing,” 87% similarlyadvanced at least one grade level.
That evaluation report contains figuresindicating the percent of students (N=35) who attained specified benchmarks inthree areas. In the areas of healthhabits 100% passed the exam. In the areaof class project completion, 100% “completed [the] appropriate number. In “multiple intelligence knowledge” based on“role playing scenarios,” 80% demonstrated average or above average competency,about 11% outstanding competency, and about 9% “some” competency.
That report also includes results onthe required state examinations: for the3rd graders (N=6) who took the Wisconsin Reading Comprehension Test(WRCT), and for 4th graders (N=7) and 8th graders (N=3)who took the Wisconsin Student Assessment System (WSAS) exams. Finally, the report summarizes the results ofsurveys of parents and teachers. Thereport concludes that areas of focus should include facilities improvement,parental involvement, data collection for teacher-based assessment, anddevelopment of pre- and post-tests in science and social studies.
Other Features of the Charter School
Additional aspects of the charterschools examined in this evaluation are school environment, teacherpreparation, school governance and parent involvement, the role of thechartering agency, and charter school’s influence on other public schools.
Schoolenvironment. Khamit Institute,housed in two connected storefronts, downstairs and up, is situated adjacent tothe sidewalk of a busy thoroughfare. Apaved parking lot for the closed business next door serves as theplayground. Inside Khamit, as notedabove, bright, African decor is pervasive. Some classrooms are formed by semi-permanent dividers.
The school and classroom environment at Khamitreflect the Khamitic philosophy and practices combined with assertivediscipline procedures. Students arereferred to as “geniuses” because of the belief that genius of one kind oranother resides in everyone. Lunches andbreakfasts, prepared on-site, are vegetarian. The modified assertive discipline practiced at Khamit is based on a setof rules which carries a “hierarchy” of consequences if a rule is broken. The rules, posted in each classroom, are thefollowing:
1. Follow directions at alltimes.
2. Keep hands, feet, objects toyourself.
3. Get permission to speak orleave seat.
4. Complete work plan on time.
Whena student violates the rules or procedures, according to the administrator, “wegive them a tape, a meditation tape. Andon the tape there are affirmations about the behavior that they should havehad. And, it’s done with music in thebackground that relaxes them. So, that’sjust an example of how Khamitics use discipline.”
Use of the meditation tape was notobserved, but numerous instances of acknowledging positive classroom behaviorwere observed. These include, “I likethe way Marcus is sitting tall,” “I like the way you’re sitting tall like ayoung scholar,” Y’all were wonderful,” “I like the way people are facingforward,” and “You have such neat work, Carla.” Rarely were reproofs or negative comments of any kind heard. When students misbehave, the teachertypically says “Ahs” (attention), which the students repeat and then quietdown.
Students with severe behavior problems are referredto a school discipline committee. Thecommittee charts and implements an intervention plan for referred students.
The administrator described theschool environment as being serene. Aserene environment, she said, would be “one in which number one, the staff isserene and where they have techniques for helping students to achieve a serene[state]…. They use music, we have a waterfall, we have plants, for theaesthetic part of it.”
Teacherpreparation. The teachers atKhamit appear to be committed and caring. Clearly, they embrace the curriculum and instructional thrusts of Khamitand are dedicated to Khamitic principles. They are knowledgeable about Khamit’s philosophy, direct instructiontheory, and assertive discipline. As oneteacher said, “We have ongoing training for everything. You name it, we’ve got training for it.” However, none of the four teachers possessesa Wisconsin teaching license. One holdsa temporary teaching permit. The otherthree are unlicensed. (See Table 1.) Several teaching assistants were observed inclassrooms throughout the day, including in teaching roles, and as substituteteachers. The academic director alsoteaches when needed. The extent of theirpreparation was not investigated for this report.
Table1. Wisconsin licenses held by teachersat Khamit Institute.
| Type of License | regular license; teaching in area and grade level of license | regular license; expired or teaching out of area or level | charter license | temporary permit | short-term sub license | unlicensed |
| Number of Teachers | 0 | 0 | 0 | 1 | 0 | 3 |
Source: Staff listprovided by Khamit administrator. Licensure information provided by DPI as of August 21, 2000.
Schoolgovernance and par