Are Chinese students smarter or is testing system rigged in their favour?

Gary Sands points out that while Shanghai 15-year-olds topped global exams in reading, science and maths, their results are not representative of China and should not be meaningfully compared with those of other countries. Dan Steinbock argues, however, that the overall test scores still show students in the US and Europe are falling behind those in emerging economies

Gary Sands

Dan Steinbock



The results of the most recent Programme for International Student Assessment (Pisa) again reveal Shanghai's 15-year-old students as the smartest in the world in reading, maths and science, coming out ahead of Hong Kong and Singapore.

The results of the assessment, run by the Paris-based Organisation for Economic Co-operation and Development, seem to confirm the widely held notion that Asian students excel in maths and sciences due to the region's commitment to education and the number of hours spent studying, given that students in Singapore, Hong Kong, Taiwan, South Korea, Macau and Japan all placed in the top 10.

For Western countries, concerns over declining educational standards seem to be confirmed, too. Students in France ranked 25th, Britain 26th, Italy 32nd and Spain 33rd. The US, the world's favourite destination for students, fell in all three categories, placing 24th, 36th and 28th, respectively in reading, maths and science.

Some argue this may herald a new era of creative thinking in Asia. But before we accept the results at face value, we should delve more deeply into Pisa's problematic methodology. The problems are two-fold.

First, instead of choosing one consistent geographical entity, Pisa selects a sample that "represents the full population of 15-year-old students in each participating country or education system", which can include cities, countries or economies. The top three are in fact cities (or city states). Most results came from a sample of scores around nations, but China was allowed to take its sample from select cities.

This is equivalent to allowing London to be the sole representative of Britain or for Boston and its suburbs to represent the US.

And this is how Shanghai, arguably China's smartest city, became the sole representative of education in China, although it only contains a mere 2 per cent of China's population. Shanghai probably became China's smartest city due to several factors, including a per-capita gross domestic product that's twice the national average and by allowing its teachers a high degree of autonomy. And with some of the highest incomes nationally, Shanghai's rich parents can afford the best education as well as expensive after-hours tutoring.

That leads us to another potential problem with the Pisa testing - that the sampling done in Shanghai and other cities was not taken from a wide variety of schools; rather, that the very best schools and students were cherry-picked.

The Pisa sampling process, according to the website, states that each country or education system submits a sampling frame to a research firm, containing all age-eligible students for each of its schools. An independent research firm then draws a scientific random sample of a minimum of 150 schools with two potential replacements for each original school.

Since each country or education system is responsible for recruiting the sampled schools, if one of the randomly chosen schools refuses to participate, for any reason, the country or educational system can choose from up to two neighbouring schools. Replacement schools can represent up to 35 per cent of the sampling frame. Once the schools are chosen, each country or education system submits student listing forms.

On test day, student participation must be at least 80 per cent. Did some of the weaker students not take the test? Could some of the weaker schools have refused to participate? And, despite having tested students in 12 handpicked provinces, China chose to submit only the Shanghai results. Only an independent audit of the results could prove conclusively whether the very best fruit was picked from the tree, but the sampling process does not preclude the potential for manipulation.

At face value, the Pisa results appear to be a huge propaganda victory for the Chinese and their educational system. But the real danger in publishing the Pisa results lies not in the fooling of thousands of headline readers around the world, but in the complicit cover-up of the huge disparities in education among Chinese provinces. Almost two-thirds of all children live in rural areas, where school attendance rates can be as low as 40 per cent.

A survey by the China Association for Science and Technology showed only 3 per cent of Chinese have basic scientific literacy, a level 20 years behind developed countries.

By allowing China to potentially rig the test, the OECD is failing in its mandate to help governments foster prosperity by providing information. Perhaps it made an exception for China - just as many foreign businesses have been forced to do - in the hope that the nation will eventually play by the rules. But if the OECD's intention in allowing cities to compete is to coax China into eventually releasing nationwide results, that argument seems a bit far-fetched, given the amount of time and effort China needs before its nationwide educational standards catch up.

The OECD is merely kowtowing to Beijing and teaching our children that bending the rules is acceptable. The final grades are in - a well-deserved "A+" for a select group of students in Shanghai, and an "F" for the headline grabbers of the OECD.

Gary Sands has run his own private equity financial advisory in Shanghai since 2006


A recent issue of Time magazine lamented that "China is cheating the world student rankings system". The 2012 results of the Organisation for Economic Co-operation and Development' global Programme for International Student Assessment (Pisa) exams, held every three years, showed the Chinese cities of Shanghai and Hong Kong came first and third, respectively.

In the global exam that evaluates 15-year-old students' reading, science and maths skills, Chinese students appear to be superior while American students continued to underperform.

"Enough is enough," complained the venerable Time. "Beijing must supply national data to assessors and not simply the results of a small minority of elite students."

Shanghai is not China, just as New York is not the entire United States. However, the Pisa results do indicate that students in the US and Europe are falling behind their counterparts in China and emerging Asia.

What's even more intriguing is that old prejudices persist - even against Pisa's hard-data evidence.

"Whenever an American or European wins an Olympic gold medal, we cheer them as heroes," OECD's education chief and special adviser Andreas Schleicher said recently. "When a Chinese does, the first reflex seems to be that they must have been doping; or if that's taking it too far, that it must have been the result of inhumane training."

As Schleicher and his colleagues have patiently explained in public, the Chinese have not been cheating in Pisa. Instead, it seems that many educational and economic policy authorities in the West are cheating themselves.

However, other critics in the West claim that perhaps resident internal migrants might not have been covered by Shanghai's Pisa sample. In other words, perhaps the urban Chinese did not cheat, but surely their migrant counterparts did!

True, Shanghai, like many other emerging mega cities, is still developing its education system and the Pisa 2012 covered only 79 per cent of the 15-year-olds in Shanghai. That, however, is not exceptional. Despite its long record in high-school education, even the US sample covered less than 90 per cent of its 15-year-olds. And the same goes for other advanced nations. Further, it is also possible that that these excluded in the West might have pulled down the average performance.

Then there are those who argue that, well, maybe the Chinese don't cheat, and maybe even the migrant kids don't cheat, but basic reading, maths or sciences are not so relevant in an era when it's the advanced, specialised knowledge that really matters. Nevertheless, globalisation spares the least those whose education and skills are poor - and rewards most those who excel in these basic categories.

In many advanced nations, students usually blamed everyone else but themselves, as the Pisa results also suggest. In contrast, the students in Shanghai believed that if they work hard and trust their teachers, they will succeed.

Pisa is a worldwide study by the OECD in member and non-member nations. Developed from 1997, the first Pisa assessments were started in 2000. These global exams, however, rest on a history of international school studies that were first launched in the late 1950s.

In 2000, Finland excelled in all three categories, followed by other Nordic countries, the Oceania duo Australia and New Zealand, Germany and the Netherlands, while Asia was represented by South Korea and Japan. By 2009, the top-five ranks in maths were dominated by Asia: Shanghai, Singapore, Hong Kong, South Korea and Taiwan. The rise of Asian education systems was also reflected in sciences and reading in which Shanghai excelled.

In the most recent Pisa assessment, Shanghai dominates maths, sciences and reading. It is followed by Singapore, Hong Kong, Taiwan, South Korea and Macau. Of the old advanced countries, Finland, Japan, Canada and Ireland remain in the top-10 lists.

Meanwhile, new transitional and emerging players have joined in these rankings, including Vietnam, Poland and Estonia. Today Vietnamese students excel in reading over French and the American students.

Prosperity is now seen to be based on productivity, which rests on innovation. The rise of China and emerging Asia in education and skills is an inherent part of the far broader new deal in global innovation.

Only a few years ago, leading policymakers and senior executives in the West believed that China, along with other emerging markets, would remain the "factory of the world" for years to come, and India its "back office". It was a convenient fantasy - but that's all it ever was.

Innovation builds on specialised knowledge, which rests on development, applied and basic research. In turn, the latter build on reading, maths and sciences.

What we are witnessing is the steady, relative erosion of student skills in advanced economies, coupled with a gradual ascent of students' skills in emerging and developing economies.

Dr Dan Steinbock is research director of international business at India China and America Institute (US) and visiting fellow at Shanghai Institutes for International Studies (China) and the EU Centre (Singapore). See also

Gary Sands is a senior analyst at Wikistrat, a crowdsourced consultancy, and a director at Highway West Capital advisers. A former diplomat with the US Overseas Private Investment Corporation, he is now based in Taipei.