Thursday, November 21, 2013

SFUSD LYING ABOUT SUCCESS OF SCHOOL IMPROVEMENT GRANTS



SFUSD has been hyping the success of SIG since the program ended in 2012-2013, last school year.  To be sure I was skeptical about this $45M grant from the get-go - what I considered to be an absolutely obscene amount of money to throw at nine schools over a 3-year period, especially with so little preparation time. That amount of money is more than the annual budgets of over 30 elementary schools combined. After having spent considerable time researching SIG during the first two years of the program,I commented repeatedly  on another blog, the SF K Files, about the waste, fraud and abuse that I discovered.  So, while I acknowledge I am not the most objective observer to comment on this $45M program, I also did a lot of homework on the subject.  I put this caveat up front so readers can review these results with that in mind. Of course, I believe that any bias I may have is not reflected in the numerical results or comparisons drawn. If anyone thinks otherwise I would like to know and please feel free to comment with any concerns. I'm not a statistician and is this not an in-depth analysis nor does this simple bread and butter statistical comparison require a higher level of  scientific procedure. We are simply reviewing the numbers based on information gathered from California Department of Education API reports.

Results

In short, what I gleaned from the API data was that 5 of the 9 schools did little to no better than the District API achievement averages  and three did worse. Nevertheless SFUSD's  review is misleadingly positive. SFUSD  pats itself on the back for a job well done though $45,000,000 in grant money for approximately 3,860 student participants did not raise achievement numbers in any significant way for most of the schools.


Method

The method of analysis I use  is simple. I compare the API of the last year before the start of SIG (2009-10) with the final year API (2012-13). Many of you know that the now defunct API index is a statistically adjusted achievement metric.  It is different from using STAR, an absolute metric, though individual STAR results provide the raw data for API.  That data is adjusted for statistical variants that would cause averaging errors. As examples, and there are many, new students are not included in an API because they don't figure in a school's growth and Students With Disabilities are not included as the kind and number of SWDs varies considerably between schools  and creates an apples to oranges comparison. Also, it should be noted that API has a subjective quality that gives additional weight to growth at the lowest quintile because that is where schools are expected to target remediation efforts and are statistically rewarded for doing so which handicaps the results at low performing schools for the better. This gives low performing schools a leg up and leans in favor of increased success when large grants are employed with the intent to focus on that lowest performing quintile. Also, schools in the lowest quintile typically show a potential for more upward mobility as there's more room to grow. Conversely, high performing students show less upward mobility in a sort of law of diminishing returns, if you will. In any case,  API results yield more favorable numbers than STAR test results. Since STAR is only assigned by individuals and grades, not to schools overall, and because of the adjusted quality of API for school-wide analysis and comparison,  API is a better tool than STAR and really the only tool, despite the preferential statistical treatment of lower performing schools.  

I would be remiss not to point out one problem with the STAR data used to create API. It has a Federally mandated test participation rate of 95% to be considered scientifically viable and  legal. Over the years few underperforming schools have met that target participation rate and the Feds  have looked the other way. Those of you who have reviewed STAR results may be aware of this participation rate issue. If you look at the results on the CDE's STAR site of high performing schools in SFUSD, invariably you will notice that participation rates are well over 95%. However, low performing schools often have much lower rates, sometimes as low as 80%. This is a variable that is hard to factor in, but the general rule is that test no-shows are low-performers. In effect, if rates were higher at low performing schools, it is likely that the results would be lower than they are. It's something to keep in mind and bears mentioning since it also handicaps results to favor low performing schools.

SIG was in effect for the school years 2010-11, 2011-12 and 2012-13.  I compared the year (09-10) before the grant started with the year of the grant ended (12-13). This allowed for a comprehensive review since individual yearly statistics varied and the important numbers were how the schools started out versus how they ended up. Also, SIG got a late start in the first year and growth that year can't be compared correctly with the next in a year-over-year comparison.  Also, Chavez ES had its first-year tests invalidated due to some funny business during test administration.

The nine SIG schools are Bryant, Chavez, Carver, Muir, Buena Vista-Horace Mann, Revere, Everett. Mission and O'Connell. I will not be including Buena Vista Horace Mann in this analysis because that school became a K-8 and adjusted API data is not available.  Willie Brown received some SIG funding for school closure, though it is reopening soon. SFUSD favorably compared the SIG schools  to the other five schools in the two Superintendent Zones, Bret Hart, Drew, Malcolm X, Flynn and Marshall HS. Three of those schools actually dropped over the 3-year period which meant that SFUSD's comparison was really a joke. That is to say, it is easy to compare favorably to schools that lost ground (and rather ridiculous).  Below I have listed the nine SIG schools and the  five other schools included in the SZ. The names are followed by the 2009-10 API, 10-11,11-12 and the 2012-13 API. The BVHM numbers are not accurate apples-to-apples comparisons due to the merger of BV and HM.


DATA

 Name                  09,   10,   11,   12

Bryant               696, 701, 731, 703

Chavez              685, NA , 661, 690

Muir                  635, 689, 715, 731

BVHM*...........653,  682, 727, 748

Carver.............701, 701, 740, 755

Revere              655, 683, 753, 772

Everett              607, 638, 693, 728

Mission........... 625, 642, 639, 641

O'Connell........603, 594. 667, 656

****************************

Bret Hart           627, 650, 655, 648

Drew                  710, 609, 677, 665

Malcolm X         800, 790, 724, 711

Flynn                  706, 710, 738, 694

Marshall HS       758, 774 ,768, 783

Summary of Data

The obvious first conclusion that can be drawn is that the Bryant, Chavez and Mission results are absolutely awful with or without an investment in millions. Underperforming the District, their 7 and 5, and 16  respective point increases over the  3-year period are negligible and statistically insignificant. Small variations are statistical noise as school APIs experience normal random fluctuation year over year. Remember that these numbers do not represent a one year change, but the total change over three years making these results very disappointing indeed. These lowest performers I term Tier 3 schools.

In the second tier, are the very mediocre results of Carver and O'Connell which rose 54 and 53 points over the period. This is mildly above the overall district results, but only when the low quintile weighting is not factored in. These results also do not speak particularly well to the efficacy of the SIG grants, though the results are better than the abysmal improvement of three Tier 3 schools.

In Tier 1, are Muir (96 pts.), Revere (117pts.), and Everett (121 pts.). These schools far outperformed the other SIG schools with the possible exception of BVHM which was not included in this comparison and they represent the high point of the SIG program.  These schools showed steady gains over the period. Bryant, Mission and O'Connell did not manage to post increases in all three years and Bryant and O'Connell scores dropped significantly in the final year, which is very disappointing considering that the implementation should have shown the best results in that third and final year.

In regards to the rest of the five schools also in the Superintendent Zones that were not SIG recipients, their results were terrible in the main, especially considering that these schools were also given many additional resources, though not nearly so much as the SIG schools. Of the five schools three of them, Drew, Malcom X and Flynn actually dropped over the same 3-year period. The other two, Bret Hart and Marshall saw district- average size increases.  What does this say about the benefit to SFUSD of Carlos Garcia's and Richard Carranza's Superintendent Zones - a program that has been the District's highest priority?
Conclusions

The School Improvement Grant program over the 3-year period  produced very different achievement results among recipient schools. Three schools did very well, over-performing district averages, and three did very poorly, underperforming district averages. Two schools showed mediocre but around average  compared to non-SIG schools, thereby under-performing as SIG recipients.  One school was excluded from analysis. Approximately $12,000 -$13,000 was spent per student on average in the nine schools over the 3-year period, factoring in attendance. It is difficult to concur with SFUSD that the grant program was successful since at least half the schools didn't do any better than the district as a whole and some did considerably worse, especially considering the statistical benefits enjoyed by low performing schools, as  previously mentioned. Despite the extraordinarily large and unprecedented windfall funding that was SIG,  the costs versus results point to a very uneven intervention with more poor results than good ones. The fact that some schools outperformed while others underperformed speaks to poor implementation at the district level where  money totaling in the millions was spent on administration.

I would be remiss not to point out that statistical comparisons are difficult for many reasons. Besides the bottom quintile advantage, the highest quintile sees the smallest gains year over year as it becomes more and more difficult to improve over-performance. So comparing districtwide averages gives a great  advantage point for point to the lowest schools. For example, a school above 900 is hard-pressed to raise a score by 30 points in a year but a low performing school  could easily see a similar point improvement.  The real gains of the SIG schools, especially the five Tier 2 and 3 schools was bad, indeed, considering the unprecedented funding made available to them during the last three years. So why is SFUSD gloating?

Saturday, November 16, 2013

LOWELL HIGH SCHOOL AND THE DIVERSITY AGENDA


As pressure mounts to do away with tracking in SFUSD middle schools, it seems only a matter of time before the Board of Education decides to apply their "equity"  standards to Lowell.  Granted, the college admissions process makes high school an entirely different animal than middle school. High schools without rigorous honors and AP classes put students at a distinct disadvantage when it comes to college applications and  testing. Spreading out high achievers from Lowell would likely increase the total number of district students in advanced classes due to the upsurge in interest, leaving SFUSD better positioned financially to increase the quality and number of advanced classes at every high school.  High schools would likely see a boost in their scores according to whatever the new standard will be now that API no longer exists, an achievement District leaders see as the holy grail in their quest to lower the achievement gap as a function of statistics, even if those average performance numbers belie real academic progress.  Spreading out student as well as teacher talent would have a leveling effect, no doubt,  but would it raise real achievement across socio-economic lines?

At present Lowell creams off 2,700 students out of the approximately 13,000  high school students.  The district effect of this concentration of high-performing students at Lowell  is a no-brainer - all the other schools have a much lower percentage of high performing students than they would otherwise.  For a district obsessed with diversity above even student achievement, a lack of academic diversity at SFUSD high schools - the result of Lowell's creaming process - makes Lowell's status as a merit-based school a political football.  If SFUSD could spread the top performers around, the District would look much better on paper and it would provide certain financial advantages in staffing and economies of scale. But it's unlikely that greater academic diversity will be a tonic for overall improvement across ethnic lines, partially because the loss of middle school honors does little to prepare students for rigorous high school classes, but also because separate honors and AP high school classrooms would still function as a brick and mortar barrier that separates  honors students from  general ed students within schools. That is to say, achievement gap statistics are fairly consistent across racial lines within diversified schools as compared to overall district achievement statistics.  The benefits of diversification  aren't entirely refuted by the disturbing prevalence of the achievement gap because there are social gains associated multiculturalism,  a more open and tolerant society. However, as schooling and achievement are concerned, diversity isn't all it's cracked up to be as a tool of cross-cultural academic progress.

The idea of diversity as achievement is a scam of politicians perpetrated on a trusting and radicalized SF school public that accepts all liberal tenants as appropriate policy objectives even when they aren't. Education experts have never substantiated  diversity is an educationally beneficial school reform, but that hasn't stop the social justice zealots on the BOE from applying it as policy and squelching lack of diversity also called elitism and its bastard child, Lowell High School. Lowell represents  a smoking gun to this failure of policy - the living proof that low socioeconomic status is not a perfect predictor of student outcome (far from it) - that poor students not only can outperform, but excel. That inconvenient truth exposes SFUSD's skeleton in the closet - the strategic plan uses diversity as a political tool, not an educational tool.  This is true for the student assignment system and for the district's funding policies. Lowell is SFUSD's fly in the ointment, that ointment being  a magical balm with which SFUSD soothes lingering racial fears and promotes its social justice agenda - diversity - propaganda as panacea for the achievement gap  which remains SFUSD's own illegitimate child. 

Common sense tells us schools are about student achievement, not redistributing communities for a social good that has little to do with education. And the high courts have ruled that racial diversity is not the purpose of public education when it is not part of a court ordered desegregation plan. But to the extent that SFUSD can hype the idea that diversity increases achievement even when it doesn't, the district can continue down the path to remove "inequities" like honors classes and honors schools (read: lower achievement gap), thereby promoting a mediocrity that is analogous with lower overall performance but smaller differences between performance. (It just isn't fair that some students do better than others, even if they work their asses off and any school that does better will be punished with less funding. Lowell was the only school  I know of that had a per pupil decrease in funding for the current 2013-14 school year. (That's sarcasm just in case you weren't sure.)
 
Predicting what a Lowell-no-more scenario would do to enrollment is tricky as the current SAS has proven. No doubt some higher performers would bolt the district, but it is also possible that more students would attend city high schools with each school providing greater academic opportunity. Given Lowell's stringent Band 1 entry requirements, many high caliber students who didn't quite make the grade would be encouraged by the citywide expansion of honors and AP classes and the increased number of high performing students, as would high-performing students who chose not to apply to Lowell for any number of reasons. It is likely that private middle school applicants to public high school would drop, at least initially, due to the loss of the exclusivity and overall cache that is Lowell's mantle currently.  But it's also possible that public middle school applicants to public high school could rise.  Cost of private school and pressure on the middle class has spurred an increase in public school admissions and that phenomenon could drive greater high school enrollment as well, particularly if high schools provide more attractive prerequisites and college bound opportunities. 

I would be remiss if I didn't add that Lowell is a school with a very long and very proud tradition. It has many influential backers and I suspect that this has prevented (think political clout) the BOE from making any changes despite its predilection  to do so.  My older son attends Lowell and loves it, so I'm not an impartial observer.  For my part I would not be inclined to want to see Lowell closed unless SFUSD went to some extraordinary lengths to provide the kind of quality education for high performers districtwide that Lowell, despite lacking in equitable district support,  provides now. What I see is SFUSD going to extraordinary lengths to help certain low-performing schools,  and the rest, including a majority of low-performing students at better schools, have to make due with less - hence the small class sizes at the so-called underserved schools while most other district schools burst  at the seams, particularly Lowell, even at a plus-950 API. 
I agree with many who say Lowell's creaming effect on the rest of San Francisco high schools is not a good thing. But until SFUSD can come up with a way to honor its honors students, to nurture excellence while tackling failure, and to give every child a fair share of the services he or she deserves, let's celebrate Lowell's real year-over-year successes.

One thing is for sure: Cardinals kick ass!



 

Saturday, November 2, 2013

GOV. BROWN AXES 2013-14 SCHOOL TESTING FOR K-12


 
I'm not prepared to write a post on the subject of MAPP so, in the interest of moving the discussion  forward,  I have copied an article from Calwatchdog.com.  Regarding the question of federal funding, whether the Feds would actually withhold funding as a result of AB484 is uncertain considering that they have never done so before (if you don't include RTTT, only Title), but the risk is great considering the amount at stake, anywhere from $1.5B to upward of $3B. It well may be that California blinks. In any case, the move to MAPP is certain with Common Core in effect. It is hard to comprehend why the Feds would require obsolete STAR testing in an era of the Common Core Curriculum. I don't get it given that the content and tests don't match. I suspect this is what happens when the Federal Government, which has no constitutional mandate in education, gets involved in the business of the states.


October 15, 2013
By Katy Grimes

Most California’s K-12 public schools will go through the 2013-14 school year without standardized testing after Gov. Jerry Brown just signed Assembly Bill 484 into law. The following school year, 2014-15, less rigorous federal Common Core tests will begin.

AB484 is by Assemblywoman Susan Bonilla, D-Contra Costa, and coauthored by Senate President pro Tem Darrell Steinberg, D-Sacramento. The bill was sponsored and pushed hard by Superintendent of Public Instruction Tom Torlakson, also a Democrat. Torlekson claimed AB484 would overhaul California’s assessment system “to address the deeper learning called for by the Common Core State Standards.”
 
AB484 authorizes new Common Core-aligned assessments known as the Measurement of Academic Performance and Progress – MAPP Testing.
 
Additionally, AB484 would suspend the public release of student data performance for 2013, and possibly in years to come.
 
“This legislation will continue to be guided by what’s right for California’s children — moving forward with instruction and assessments reflecting the deeper learning and critical thinking our students need to compete and win in a changing world,” Torlakson said. “Our goals for 21st century learning, and the road ahead, are clear. We won’t reach them by continuing to look in the rear-view mirror with outdated tests, no matter how it sits with officials in Washington.”
 
No love of testing

“Brown never liked testing, or research data,” Lance Izumi told me; he is the Senior Director of Education Studies at the Pacific Research Institute, CalWatchdog.com’s parent think tank. He’s also the author of “Not as good as you think: Why the Middle Class Needs School Choice,” Obama’s Education Takeover,” and “Short Circuited: The Challenges facing the Online Learning Revolution in California.”
 
California students had been tested once each year in math and English under the state’s K-12 Standardized Testing and Reporting system, known as STAR testing.

 Rather than waiting until the Common Core curriculum goes into effect in 2014, the governor saw this as an opportunity to rewrite state standards, and toss out STAR testing, Izumi said.

Bonilla’s bill now replaces the standardized testing with a system that has yet to be vetted, according to Izumi. Students will take field tests, and only be tested in math or English. Tests will no longer be given in both math and English.
 
“The MAPP testing program will be made up of assessments being designed by the Smarter Balanced Assessment Consortium, one of two multi-state organizations formed to create the next generation of assessments aligned to the Common Core,” a press statement from Torlakson explained. “Field tests of the new assessments, set for the spring of 2014, are designed as ‘tests of the tests.’”

Risking federal education funding

In addition to the disregard for the importance of standardized testing, the bill puts California at odds with the federal government’s education requirements, and the upcoming Common Core curriculum.
 
U.S. Secretary of Education, Arne Duncan, opposed AB484, and warned state officials that if AB484 passed, California could lose $1.5 billion in federal funding for the current school year.

And California schools will no longer have to report students’ test scores to parents. “If you don’t respect individual student outcome data, parents and students don’t know how they are doing,” Izumi said. “And you can’t hold schools and teachers accountable.”

Common Core vs. STAR

Izumi said administering STAR testing costs California $25 million a year. During the committee process, Torlekson and Bonilla advocated dumping STAR testing, ostensibly to save the cost, and said the state should instead put the money toward Common Core.

 But according to Izumi, the governor has already allocated $1.25 billion for Common Core curriculum implementation and administration.

 The state also will spend an additional $2.1 billion on schools to implement the Local Control Funding Formula, Brown’s plan to spend more money on low-income school districts, minority and immigrant student populations, and non-English speaking students. To do that, Brown said he will take some of the current education funding from the better-performing schools.

I wrote about this in “Gov. Brown calls for redistribution of school funding,” when Brown cited a lack of civil rights and social inequities as what is wrong with California public schools.

Race to the bottom

According to Izumi, President Obama “strong-armed the states into adopting these Common Core standards through a number of devices, principally through the Race to the Top competition for federal grants.”

 The Race to the Top $4 billion grant scheme was awarded to states “leading coherent, compelling, and comprehensive education reform.” The grant was authorized under the federal “stimulus” program, the American Recovery and Reinvestment Act of 2009.

According to Izumi, the federal and state Departments of Education had developed numerous experimental educational programs promising to be reform breakthroughs in education. But the reforms were later discarded as failures, wasting large amounts of the taxpayers’ money.

Not all schools can play

Izumi said all tests have to be given by computer. But not all schools districts will be able to participate in the new MAPP testing. And Torlekson’s California Department of Education has no idea how many school districts don’t have computer systems.

Izumi warned, “Some kids may not get any testing.”

 - See more at: http://calwatchdog.com/2013/10/15/gov-brown-axes-2013-14-school-testing-for-k-12/#sthash.xMp2g3ZP.dpuf