Thursday, December 30, 2010

What’s on your mind? | Facebook

Facebook used R to analyze how, when, and what users do with their updates.

For more details, visit Facebook blog at:

What’s on your mind? | Facebook

Saturday, December 25, 2010

Using statistics to understand armed conflict

Drew Conway, a doctoral student in New York, uses statistical analysis to make sense of armed conflicts. Pasted below is his graphic that he developed from analyzing Wikileaks data about Afghanistan in July 2010. He used R software to generate the graphic.

The gold standard in newsroom graphics: The New York Times


Watch Amanda Cox explain how The New York Times uses the graphics in the print and online edition. The New York Times  has been at the cutting edge of using data and graphics. The hour-long video is worth watching for any one interested in using data to communicate.

Forbes magazine recognizes R

R is on a role. First it was the New York Times, than the Facebook, and now Forbes.


Friday, December 24, 2010

Data journalism at its best

Watch how journalists at Guardian used the data to paint a hitherto hidden picture of the Afghan war.


Is the economy really recovering?

The for-hire motor carrier freight services price index has maintained its upward trend since the first quarter of 2009. The price index collapsed in the middle of 2008 indicating the slow-down in the economy and the resulting lower demand for freight services.

The moderate, yet sustained, increase in the price of freight services since early 2009 suggest that the demand for such services has been increasing in Canada, albeit at a slower pace. The couriers and messengers services price index also exhibited a moderate increase of 0.2% in November over October. The year-over-year increase was 2.7%, again a moderate increase in the cost of doing business in Canada.

This should be welcome news because the price pressures are moderate in Canada and conducive for sustained economic growth.

Source: Statistics Canada

Value Added Tax set to increase in UK


The Value Added Tax (VAT)  is set to increase in UK from 17.5% to 20% as of January 4. The increase in tax is likely to induce higher demand for retail because consumers will try to beta the tax by purchasing before January 4. This bodes well for retailers, at least in the short run.

Thursday, December 23, 2010

Air Cargo slowdown suggests lower electronics demand in North America

Earlier this week I had blogged about expected slow-down in the electronics sales this Christmas season. Other reports now are suggesting the same. See the report from Bloomberg pointing out that U-bound air cargo out of Asia has been slackening as of late, suggesting lower demand for air cargo which often comprise electronics exports from Asia to America.

By Steve Matthews and Anthony Feld

Dec. 23 (Bloomberg) -- A slackening in Asian air cargo bound for the U.S. may signal that American consumers are slowing purchases of electronics and appliances after a post- recession spending spree.

Shipments from Asia to the U.S. rose at a 15 percent pace in October, less than half the January-to-June rate, the International Air Transport Association said Dec. 14. Electronics and appliances make up about half of Asian imports shipped by air. Global air-cargo demand growth may slow to 5.5 percent next year from 18.5 percent this year, IATA said.

Cooling sales of discretionary items such as flat-screen TVs and personal computers might indicate that household purchases won’t expand as fast as many economists are assuming, said Chris Rupkey, chief financial economist at Bank of Tokyo- Mitsubishi UFJ Ltd. in New York. Retailers such as Best Buy Co. and Whirlpool Corp. have said consumers, whose spending accounts for 70 percent of the U.S. economy, are shying away from purchases of big-ticket items.

“Asia freight volumes to the U.S. are slowing, and this raises the risks to the outlook for electronic retail sales here,” said Rupkey. “There may be a little bit of caution still given the unemployment rate is nearly 10 percent and the consumer is trying to get the debt-laden balance sheets back under control.”

Consumer spending may expand 3 percent next year, Rupkey forecasts, less than the 4 percent projected by Neal Soss, chief economist at Credit Suisse Holdings USA Inc. in New York, and Jim Paulsen, chief investment strategist at Wells Capital Management in Minneapolis. Consumer spending gains have averaged 2.9 percent in the past 15 years.

Selective Investors

Investors may have to be more selective in their choice of consumer stocks in 2011 after the companies outperformed the past two years, said Colin McGranahan, a retail analyst at Sanford C. Bernstein & Co. in New York.

The Standard & Poor’s Retail exchange-traded fund, which includes Best Buy and Fort Worth, Texas-based RadioShack Corp., has risen 136 percent since the start of 2009, compared with 39 percent for the S&P 500 exchange-traded fund.

“Selectivity is key,” said McGranahan, who recommends shares of office-supplies retailer Staples Inc., based in Framingham, Massachusetts, and Minneapolis-based discount retailer Target Corp. “The group overall will struggle to outperform. The stocks fully embed the recovery. The opportunity for outperformance is far more limited.”

Sales of electronics fell 0.6 percent last month for a second straight decline, Commerce Department figures showed Dec. 14. Overall, retail sales gained 0.8 percent in November after a 1.7 percent jump in October.

Consumer Bellwethers

Electronics and appliances are bellwethers of consumer spending because they are usually discretionary purchases and can be postponed, said Britt Beemer, chief executive of America’s Research Group, a retail industry tracking group in Charleston, South Carolina.

“Everyone’s budget is tight,” he said.

Rupkey says sales of electronics may slow after rising 3.7 percent over a year earlier through November on a three-month moving-average basis. Sales growth of electronics and appliances averaged 2.4 percent over the past 10 years.

“Retail sales have been on fire for electronics goods the last few months and may be cooling a little,” he said.

Soss and Paulsen say they are confident that a payroll-tax cut and higher employment will boost spending above its long- term average.

Fiscal Stimulus

“The job gains, mediocre though they have been, the income gains, the low interest rates and now the prospect of a significant fiscal stimulus have gotten the consumer back in the game,” Soss said.

Growth will be “led by the consumer,” Paulsen said, as private payroll gains pick up next year.

Freight tonnage growth in Asia is giving way to more sustainable growth, said Andrew Herdman, director-general of the Association of Asia Pacific Airlines.

“An extraordinarily strong recovery” has boosted Asian cargo, Herdman said in a Bloomberg Television interview Nov. 25. “We don’t see signs of double-dip” recession. “We are going to see a sustained recovery. That makes us optimistic about next year.”

International Priority freight pounds increased 29 percent at FedEx Corp. in the three months through November, down from gains of about 41 and 68 percent the previous two quarters, the second-largest U.S. package-shipping company said Dec. 16.

‘Steady Pace’

“Overall, the global economic picture is increasingly more positive as recovery continues at a steady pace,” Chief Executive Officer Fred Smith said on a conference call with analysts, while “Asia is moderating toward more normal high growth rates.”

UTI Worldwide Inc., an international freight-forwarding and logistics company, said Dec. 2 that fourth-quarter volumes in air freight would be about the same as last year after a 20 percent increase in the third quarter from a year earlier.

“The trend has clearly been a growth rate deceleration,” said Benjamin Hartford, transportation analyst at Milwaukee- based Robert W. Baird & Co. Inc.

Best Buy, based in Richfield, Minnesota, cut its annual profit forecast to $3.20 to $3.40 a share, from $3.55 to $3.70, because of weaker-than-expected sales of televisions, computers and video-game software.

‘Too Aggressive’

“Our top-line growth assumptions earlier in the year turned out to be too aggressive based on the environment that we see for demand, specifically in the TV industry and the computing industry overall,” said James Muehlbauer, chief financial officer, in a conference call with investors.

Best Buy’s shortfall resulted in part from not being “aggressive enough on pricing,” said David Strasser, an analyst at Janney Capital Markets in New York.

Competitor hhgregg Inc., based in Indianapolis, lowered its annual earnings estimate in November to $1.30 to $1.45 a share from $1.35 to $1.45 a share.

“Clearly, there continue to be some structural challenges in the overall economy that are creating headwinds in our industry,” said Dennis May, chief executive officer, in a Nov. 9 conference call with investors. “However, we are also firmly believe that the consumer is both healthier and more opportunistic than they were a few quarters ago.”

Whirlpool said Oct. 27 that demand was moderating in North America. The Benton Harbor, Michigan-based company cut its 2010 forecast for North American industry shipments to 3 percent from 5 percent.

“We do think we’ll see a return to what I call normal demand levels in the developed markets in the U.S. and Europe,” said Jeff Fettig, chairman and chief executive officer, in at an analyst meeting with investors.

--Editors: Christopher Wellisz, James Tyson

Anthony Feld in New York at

To contact the reporters on this story: Steve Matthews in Atlanta at;

To contact the editor responsible for this story: Christopher Wellisz at

Good news for large urban transit in Canada

Statistics Canada is reporting a 5.3% y-o-y increase in transit ridership in October 2010. The fare box revenue, excluding subsidies rose to $232.7 million in October 2010, suggesting that the transit ridership increased mostly in 2010, which is consistent with a recovering economy.
Combined ridership and revenue levels for 10 of Canada's largest urban transit properties rose in October compared with October 2009.

These 10 companies represent about 80% of total urban transit across the country.

Ridership levels rose to 133.6 million passenger trips in October, a 2.5% increase from the same month a year earlier.

The rise in ridership pushed revenue (excluding subsidies), up 5.3% from October 2009 to $232.7 million in October.

The Daily, Wednesday, December 22, 2010. Large urban transit

Income inequality in Toronto

The discussion about income polarization in Toronto assumes that it would adversely affect the welfare of those living in low-income neighbourhoods. This may not necessarily be true.

While it is true that income inequality has increased in Toronto since 1970s, however one cannot automatically assume that those who currently live in low-income neighbourhoods are relatively more disadvantaged than those who lived in low-income neighbourhoods in the 70’s, when income inequality in Toronto was much less.

Professor Nancy Ross of McGill University has in fact found no association between income inequality and mortality in Canada and concluded that effective distribution of social and economic resources in Canada could be the reason behind blunting the possible adverse effects of income inequality.There are however other studies that have found association between morbidity and income inequality. For instance, some studies have found that life expectancy in Canada for the rich is higher than those who are poor.

As long as we have in place a resource distribution arrangement that affords the same access to health-care, public education, and other fundamental services, such as waste collection and water supply, to all irrespective of their incomes, we should not get too concerned about some of our fellow citizens getting too rich.
Enhanced by Zemanta

Wednesday, December 22, 2010

Executions fall due to cost, lack of lethal drug | Reuters

Reuters is reporting a slowdown in executions in the United Sates because of shortage of sodium thiopental, an anesthetic used in executions. Apparently, the chemical is imported from other countries for use in the United States. It's shortage has forced American executioners to search for alternatives, especially after British groups have threatened a law suite against those involved in exporting the chemical from England to the United States. In Oklahoma, for instance, the State used pentobarbital, a drug commonly used to euthanize animals.

Also reported in the same story is the fact that of the 1,234 executions conducted in the United States since 1976, Texas and Virginia accounted for half.

It is extremely repugnant to even entertain the idea that the government has the mandate to take away human life. Even more disturbing is the fact that some States in the United States are more execution-happy than others. Are Texans and Virginians more violent than the rest of the Americans or that the State Legislature and the Courts are more self-righteous and do not think much of killing human beings.

It is however neither Texas nor Virginia that lead the world in state-sponsored executions. On a per capita basis, Iran and Saudi Arabia lead the world in killing their own. China, on the other, hand conducts the most executions.

I oppose death penalty not because I am soft on crime, but because the death penalty can be abused by the State, as is the case in Iran and Saudi Arabia.  Make no mistake, the US is not much better. Statistical analysis has repeatedly shown that in the United States, a black convict is multiple times more likely to be executed if found guilty of murdering a white person than vice versa. It is better that as humans we acknowledge the limits of our rationality and our vulnerability to biases and prejudices.

(Reuters) - The United States executed fewer people this year, in part because there is a shortage of the drug used in lethal injections and because executions are too expensive in tough economic times, a report released on Tuesday said.The Death Penalty Information Center said in its annual report that executions decreased 12 percent this year and new death sentences stayed near the lowest level since capital punishment was reinstated in 1976.
Texas led the nation with 17 of the 46 executions carried out this year in the United States. The total is down from 52 in 2009 and less than half the number put to death in 1999.
"Whether it's concerns about the high costs of the death penalty at a time when budgets are being slashed, the risks of executing the innocent, unfairness, or other reasons, the nation continued to move away from the death penalty in 2010," Richard Dieter, the center's executive director and author of the report, said in a statement.
One factor reducing or delaying executions is difficulty obtaining sodium thiopental, one of the drugs used in lethal injection executions, the Washington-based group said.
Executions were postponed or canceled in five states due to a shortage of the drug, it said. Arizona imported some from Britain, where executions have been abolished, but Britain is now restricting the drug's exportation.
New death sentences in 2010 will total 114, near the lowest level since 1976 when executions were authorized by the U.S. Supreme Court, and down two-thirds from their peak in 1996.
There have been 1,234 executions in the United States since 1976, nearly half of those carried out in Texas and Virginia.

Executions fall due to cost, lack of lethal drug | Reuters

Tuesday, December 21, 2010

Visualizing Facebook Friends: A map is worth a thousand lines!

R, the computing language of choice for number crunchers, got a big boost recently when it turned out that the now famous image of facebook’s global friendly connections was in fact drawn with R. Paul Butler, the intern at facebook, who created the graphic, explains in a blog why he chose R for the task.


Sunday, December 19, 2010

Electronics sales may not be electrifying this time around

While the print and electronic media remains infatuated with the latest gadgetry, predicting higher electronic sales every December, this year is likely to be different. There has been a significant decline in the searches conducted on Google by shoppers in Canada. It is hard to tell the magnitude of reduction in the sales of electronics during this Christmas. However, chances are that the overall electronic sales in Canada may experience a decline in Christmas sales.

The electronics retailing in Canada is more of a pseudo competition between the two leading retailers, Best Buy and the Future Shop (also owned by Best Buy Canada). The latest Internet searches by the shoppers in Canada suggest that Best Buy has taken over Future Shop in popularity. In 2008 and 2009, Future Shop lead Best Buy in Canada amongst shoppers’ searches on the Internet. In 2010, Best Buy has left Future Shop behind.

December 2008 December 2009 Nov-Dec 2010
image image image

Also worth noting is the return of The Bay in the top 10 shopping searches in Canada. In the past, Bay has been bleeding as low-cost retailers had taken significant market share away from Canada’s flagship retailer. Also listed is Amazon amongst the top rising searches for 2010 and not Indigo/Chapters. This suggests that online book retailing is still a forte of Amazon, even in Canada.

Further evidence to my above-mentioned assertion is found the actual sales data (as shown in the graph below as seasonally adjusted monthly retail sales) which shows that sales of electronics and appliance stores (red colour line) and furniture and home furnishing stores took a tumble during recession in 2008 and that electronics sales have not recovered to the peak observed in mid-2008.

Sales at general merchandise stores (depicted as green line), on the other hand, suggest almost no impact of the recession. The upward trend in sales for such stores has continued unabated since 1996. Furthermore, the gap between the sales volume of general merchandise trade and the sales of electronics and furniture stores has continued to widen, despite the hyped media coverage of bumper sales of uber gadgets.


Housing starts as leading indicator for recessions

While I was working on my doctoral thesis on housing starts in Canada, I was intrigued by the fact that housing starts always served as leading indicators for recessions, and times, for recovery. The following graph from St. Louis Federal Reserve presents an explicit picture of the same phenomenon:


The role of housing was unique in the last recession. Consider the following graph, also from the same source, which shows that unlike previous recessions, the last recession in the United States saw a significant decline in housing prices.


Does gender matter for academic promotion? Evidence from a randomised natural experiment | vox - Research-based policy analysis and commentary from leading economists

"Several countries have recently introduced gender quotas in hiring and promotion committees at universities. Evidence from promotions in the Spanish university system suggests that quotas are only effective at increasing the number of successful female applicants in promotions to top positions. This column argues that, given that sitting on committees reduces the available time for research, gender quotas should be implemented only for more senior academic positions."
Does gender matter for academic promotion? Evidence from a randomised natural experiment | vox - Research-based policy analysis and commentary from leading economists

Friday, December 17, 2010

Would income inequality result in higher mortality rates for the poor?

The answer for Canada is not necessarily. A study by McGill University’s Nancy Ross found no association between income polarization and mortality rates.

Relation between income inequality and mortality in Canada and in the United States: cross sectional assessment using census data and vital statistics


Objective: To compare the relation between mortality and income inequality in Canada with that in the United States.

Design: The degree of income inequality, defined as the percentage of total household income received by the less well off 50% of households, was calculated and these measures were examined in relation to all cause mortality, grouped by and adjusted for age.

Setting: The 10 Canadian provinces, the 50 US states, and 53 Canadian and 282 US metropolitan areas.

Results: Canadian provinces and metropolitan areas generally had both lower income inequality and lower mortality than US states and metropolitan areas. In age grouped regression models that combined Canadian and US metropolitan areas, income inequality was a significant explanatory variable for all age groupings except for elderly people. The effect was largest for working age populations, in which a hypothetical 1% increase in the share of income to the poorer half of households would reduce mortality by 21 deaths per 100 000. Within Canada, however, income inequality was not significantly associated with mortality.

Conclusions: Canada seems to counter the increasingly noted association at the societal level between income inequality and mortality. The lack of a significant association between income inequality and mortality in Canada may indicate that the effects of income inequality on health are not automatic and may be blunted by the different ways in which social and economic resources are distributed in Canada and in the United States.

Where is Toronto’s missing middle class? It has suburbanized out of Toronto

The recent study by the University of Toronto’s (U of T) urban researchers has raised alarms about the increase in income polarization in the City of Toronto where the very rich and the very poor have expanded and the middle class has shrunk over the years. The study raises concerns about the implications of the rising income inequalities in the City of Toronto.

The report is indeed an important contribution to the ongoing dialogue on how to create socially just societies and thus serves as a scorecard on how the City performs on income inequality. The report also mentioned that the  growing number of the very poor in the City of Toronto are immigrants and visible minorities.

While I agree with the general findings of the report, and consider it a must read for those interested in cities, I have though some concerns about the methodology and some implied, yet unstated, conclusions. First the methodology. The authors have focussed on the City of Toronto and have only tangentially mentioned the possibility that the middle class may have migrated to the outer suburbs. I call this phenomenon of suburbanizing middle classes ‘the brown flight’, since immigrants from South and South-East Asia constitute a large number of the new middle class, which has settled in Toronto’s outer suburbs.

Second is the implied assumption that growing income inequality by default is at odds with social cohesion and long-term economic and social viability of the region. This assumes that the social safety nests, which are predominantly supported by the taxes imposed on high and very-high income earners, are incapable of providing short- and long-term services for the very poor in the City. Furthermore, the breakdown of neighbourhoods by income also gives the impression that there may be a socially-optimum breakdown of neighbourhoods by income categories.

I will address below only the first concern in this blog.

I have repeated the analysis done by the authors of the U of T study with two differences. Instead of using individual income, I have based my analysis on household incomes. This does not fundamentally change the distribution  of income profiles mentioned in the U of T study, however it is theoretically more consistent because income manifests itself at the household level more than it does at the individual level. For instance, a poor household is unlikely to have both poor and not poor members of the household. The pooling of income within a household  can help the household deal with poverty better than a single-person, and therefore a single-income, household. My own research on household dynamics in the Peel region suggests that single-person households are more vulnerable to poverty than the rest because of lack of monetary support from other household members. Thus using household income as a metric to study income polarization is better than using individual income.

The City of Toronto is part of the larger urban system or conurbation known as the Greater Toronto Area. Statistics Canada defines Toronto’s conurbation as the Toronto Census Metropolitan Area (population 5.1 million, Census 2006) and includes the City of Toronto (population 2.5 million, Census 2006) and the neighbouring suburban municipalities. Some even include Hamilton in the same conurbation.

imageIn an earlier blog on the topic I had suggested that Toronto’s middle class has migrated out of the City of Toronto and has settled in the outer suburbs of Toronto, which are part of the Toronto conurbation. The reason for outward migration of the middle class are rooted in demographics and affordable housing. The middle class comprises of growing families with children who need more shelter space at affordable prices, which is abundantly available in the outer suburbs. Furthermore, financially stable and multigenerational immigrant households abandon Toronto at an increasing rate than the rest since they crave affordable shelter space more than the rest. This creates the spatial mismatch between the shelter needs of the middle class and the small-sized, yet prohibitively expensive, housing stock available in the City of Toronto.

See the map to the right (click on it to see the larger version), which shows the percentage of households with 4 or more persons. The brown and red-coloured census tracts in the suburbs act as proof for the claim that the growing households have left for the suburbs. Similarly, the blue-coloured census tracts in Toronto along the waterfront and in areas of higher income suggest small-sized households comprising either young professionals or empty-nesters, who outbid households with children for shelter space in the choice neighbourhoods.

imageIn this blog I will present evidence to my claim that income polarization in the City of Toronto is a result of the arbitrary boundaries used to define Toronto. These boundaries conform to the political and administrative boundaries of the City, but not to the housing and employment markets observed in the City. The evidence for this argument lies in the commuting patterns observed in the City. My own calculations from the 2006 place of work data from Statistics Canada suggest that almost 600,000 work trips cross City of Toronto’s boundary every day. These are the individuals who either live in Toronto (186,555 trips) and work in the suburbs, or the suburbanites (407,535) who work in Toronto and cross the arbitrary boundaries, which delineate the City, twice every day.

See the map to the right (click on it to see the larger version),  where I have depicted municipal boundaries in grey colour. Also shown are the pre-amalgamation boundaries of Toronto’s former inner suburbs. The green colour neighbourhoods in the map depict the middle class, which is primarily concentrated in Toronto’s outer suburbs. The blue colour low-income cohorts are increasingly concentrated in the amalgamated City of Toronto making a U-shaped poverty concentration around the high-income neighbourhoods concentrated along the Yonge Street.

imageThe correlation between the location of rail tracks in the City of Toronto and the spread of poverty in proximity to rail tracks has not been discussed in detail in the past. The U-shaped poverty visible in the above map is along the rail/subway tracks. Furthermore, I also noticed a significantly large number of homicides committed in the vicinity of rail racks, which are also neighbourhoods depicting poverty (see the map to the right).

The egalitarian suburbs

Toronto’s outer suburbs are far more egalitarian than City of Toronto itself. When I reviewed the income distribution, I was able to replicate the findings from the U of T’s report. Consider the following histogram of household income that illustrates more than 60% residents in the City of Toronto (2.5 million) earn at least 20% less than the average household income ($91,579 for 2005) observed for Toronto CMA (5.1 million people).


The graph also shows that almost 8% of the population lives in Census Tracts (approximate population 4,000) where the average household income is more than 40% of the average household income in Toronto CMA. The numbers presented here are slightly different from what is reported in the U of T study because the U of T study computed the percentage of the census tracts (i.e., land area) and not the percent of population, which I have presented in the above-mentioned graph. The following table is based on the data presented in Map 3 on Page 5 of the U of T study.


The middle class found in suburbia

The most interesting finding emerges when one plots the income distribution for the outer suburbs of the  Toronto CMA, i.e.,  after excluding the City of Toronto from the data set. The resulting histogram is presented below, which is computed for the 2.6 million suburbanites who live in the “905 municipalities”.


The missing middle class is right in the middle of the graph where 60% of the suburbanites live in neighbourhoods where the average household income is within the +/- 20% of the average household income for Toronto CMA. The symmetrical distribution presented in the above graph suggests that the extreme income disparities seen in the City of Toronto do not exist in the outer suburbs of Toronto.

The other main concern in methodology adopted by the U of  T study is the presence of high extreme values in the income variable. The literature on income inequality discusses the issues related with extreme values. The histogram below confirms the existence of the problem where a small number of census tracts report very high incomes in the range of $250,000 and $800,000. A naive method of correcting this problem is to use a trimmed data set that excludes extreme values. However, more statistically robust methods are readily available and have already been programmed in some econometrics software. I will address this issue later in another blog.


There exists in fact a greater heterogeneity in the income distributions within the region. I have plotted income distributions for the nine main municipalities, which are home to 4.6 million of the 5.1 million residents of Toronto CMA. The following graph has been truncated at a maximum household income of $250,000. The graphs below show that the peaks observed for the suburban municipalities are concentrated around the average household income observed in the region ($91,579), whereas the results are skewed for Toronto with a large number of low income concentration as well as a small number of very large income households.


Another way of looking at this issue is to examine the share of population for each income category distributed amongst the select 9 municipalities, including Toronto. The following table illustrates the concept. Notice that 95% of those whose incomes are at least 40% less than the regional average, and 76% of those whose incomes are at least 20% less than the regional average  reside in the City of Toronto. Similarly, 53% of those whose incomes are more than 40% above the regional average also reside in the City of Toronto. This confirms the findings of income equality in the U of T study.

However, more interesting results are reported for the middle income categories. 68% of those whose income falls within +/- 20% of the regional average income, i.e., the middle class, lives in Toronto’s external suburbs. Furthermore, 77% of the slightly well-off households whose incomes are between 20% to 40% higher than the regional average also reside in the outer suburbs.

Also note that the very low income category is non-existent in  Markham, Milton, Oakville, Richmond Hill, and Vaughan.


Lastly, consider the Lorenz curves for the City of Toronto (bottom left) and the 905 municipalities (bottom right).  The Lorenz curve for the City of Toronto reveals a unique distribution, confirming the trends mentioned above where a small number of households earn very large incomes. On the other hand, the Lorenz curve for the outer suburbs suggest mild income inequality.


Thursday, December 16, 2010

Tories continue to hurt Census's prospects for getting reliable data

Tony Clement, the minister of industries, has recently revealed that the government's claims of spending millions more to attract sufficient response for the voluntary household survey are no longer true. The government made those claims after unanimous opposition from industry, academia, and informed members of the society to the decision to scrap the mandatory long form census. Since Census is no longer grabbing the headlines, the government has revealed its original plan that never had serious financial commitment to promote the voluntary survey, which is set to replace the mandatory long form census in 2011.

The Canadian Press

Updated: Wed. Dec. 15 2010 8:45 PM ET

OTTAWA — A campaign to encourage people to fill out the census was slashed because government decided to add two language questions to the form, Industry Minister Tony Clement said Wednesday.

Clement had promised to spend up to $30 million on a campaign to boost response rates to the 2011 census and the new voluntary National Household Survey, which is the government's replacement for the mandatory long-form census.

He told a Commons committee in July that it was important to "pay a price" to get reliable data and protect the privacy rights of Canadians.

But Statistics Canada revealed earlier this week that only half that amount would be set aside in contingency funds for promoting the census and survey.

Now, $10 million will be spent to add two questions on official languages usage to the mandatory short-form census. The government made the decision at the 11th hour after pressure from francophone groups over the summer.

Another $5 million will be spent on extra printing and postage costs associated with sending the new voluntary long questionnaire to more households.

Clement said something had to give once the language questions were added.

"We had to act within a budget, that's part of what we do as cabinet ministers, and when we wanted to add questions and we wanted to reprint, all of those things cost money so I wasn't going to blow the budget to make those changes, so we had to take it out of somewhere," Clement said.

He insists that the government can still do a good job encouraging people to do their civic duty.

"The upshot of it is, can we mount a successful publicity campaign with the money and the answer is yes," Clement said.

The Opposition criticized the rejigging of the census budget.

"Experts have expressed serious concern that the government's incessant fear-mongering on the long-form census will have disastrous effects on Canadians being willing to fill out the short-form census," said Liberal MP Carolyn Bennett, co-author of a private member's bill to revive the long census.

"Now we learn that the government has slashed and reallocated the very budget to persuade Canadians to participate in the census. Will the government stop misleading Canadians about the privacy of the census data, start telling the truth, and restore the mandatory long-form census?"

The Conservative government eliminated the long-form census over the summer, saying it was trying to achieve a balance between the need for reliable data and the right to privacy.

Critics, including the former chief statistician, the Federation of Canadian Municipalities, charities, academic organizations and provincial ministers have all warned that reliable data will be lost by making the long questionnaire voluntary.

Internally, Statistics Canada officials have said they expect only a 50 per cent response rate to the National Household Survey after a first mailout. The agency has said it cannot predict how much information it will be able ultimately to release from the survey. - Campaign to promote census cut to fit budget: Clement- CTV News, Shows and Sports -- Canadian Television

WikiLeaks inspires sanitary pad ad in Pakistan

KARACHI: Pakistani advertisers in the feminine hygiene business have harnessed the political notoriety of WikiLeaks to tell women that while the US State Department might leak, they don't have to.

Advertising sanitary pads on selected billboards in Pakistan's financial capital Karachi, the latest catchphrase is: "WikiLeaks... Butterfly doesn't".

Leaked American diplomatic cables turned the Internet whistleblower into a household name in Pakistan, fascinating and appalling members of the public over reported inner dealings of their political and military elite.

Pakistani advertisers usually avoid the divisive world of politics but advertisers said the commercial had attracted considerable attention.

"We have received a huge response from the public and everyone has commended us on it," said Syed Amjad Hussain, head of business development at RG Blue Communications, which pitched the advertisement to manufacturers Butterfly.

"It could have been yet another ad showing a girl promoting the sanitary pad, but we made it different, completely different," art director Munir Bhatti told AFP.

Hussain said the response had been "great" although a fully-fledged media campaign had yet to start.

Coverage of the leaked American cables highlighted Western concerns about the safety of Pakistan's nuclear arms and politicians' private support for US drone attacks on al-Qaida and the Taliban in the country's tribal belt.

They also revealed that the Pakistani army considered forcing out President Asif Ali Zardari, who made contingency plans for his assassination.

Read more: WikiLeaks inspires sanitary pad ad in Pakistan - The Times of India

WikiLeaks inspires sanitary pad ad in Pakistan - The Times of India

Wednesday, December 15, 2010

The good, the bad, and the ugly in Toronto

What happened to the middle class in Toronto? University of Toronto's urban/social geographer, David Hulchanski, and colleagues argue that the middle class has disappeared from Toronto in a recently released update on the growing income disparity in the City of Toronto. They argue that the income disparities have reached new heights where wealth and poverty in extremes define the City.

The report also raises alarm about the urban poor, which in Toronto's case appear to be visible minorities and immigrants who do not posses adequate education or job skills required by the Toronto's specialized job market.

I have a hypothesis, which may help locate Toronto’s missing middle-class. I believe demographics and affordable housing are behind the trends  that the authors have highlighted in their report. The missing middle-class from Toronto, which comprise to a large extent of  immigrants from South Asia and the Asia-Pacific, has perhaps migrated to the outer suburbs in the Durham, Peel, and York regions of the Greater Toronto Area. Furthermore, non-immigrant households with children have also abandoned Toronto in search of affordable housing in the suburbia.

The question to ponder is why financially “successful” immigrants (well-educated professionals who have migrated in the past 20 years to Canada from South Asia and Asia-Pacific) not settle in droves in Toronto after they have gained some economic traction in Canada. And why those immigrants who struggle in the job market stay behind in Toronto?

It appears that the financially “successful” immigrants leave Toronto for demographic reasons. Multi-generational households with children and grandparents need larger homes, which are available in plenty in the outer suburbs and at affordable prices. A 4-bedroom house in a reasonable neighbourhood in Toronto costs in excess of $550,000. The same-size housing unit in a newer neighbourhood in the suburbs costs at least $100,000 less.

Larger household size of the new middle class immigrant households, which cannot be readily fit into the small-sized housing units in the transit accessible, and prohibitively expensive, neighbourhoods in Toronto, may partially explain why the “brown flight” has taken place.

Source: Statistics Canada, 2006 Census

The missing middle class has abandoned Toronto for the outer suburbs where affordable housing and new infrastructure is in plenty. The housing markets in Mississauga and Brampton, one of the fastest growing places in Canada with a combined population of almost a million people, are to a large extent driven by the immigrants from South Asia. Chinese immigrants drive the housing markets in the York Region, which is situated to the north of the City of Toronto.

The following graphic from the report reveals almost non-existence of extreme poverty in the outer suburbs in the greater Toronto Area, which is now home to the missing middle class from Toronto.

The City of Toronto's housing stock is in fact more conducive for smaller-sized households. Unlike the recent immigrants from South Asia, who have larger-sized families, former immigrants and their descendants from Western and Eastern Europe have small-sized families, which could be accommodated with ease in the available housing stock in the City of Toronto. 

So what about the immigrants who stay behind? The answer lies in the availability of social services, e.g., shelters for homeless, subsidized housing and other similar facilities, which are more prevalent in Toronto than they are in the outer suburbs. It would make sense for households that rely on state's assistance to stay in Toronto to avail readily available social services.

Thus, Toronto's inner suburbs are now acting as the staging grounds or intermediate neighbourhoods for the new immigrants who save enough while living their to buy a home in the outer suburbs and eventually relocating their to start a new suburban life as the missing middle class of Toronto.

While the University of Toronto's study does not explain if the income polarization is causing further harm by limiting or preventing access to health-care, education, or the like for the low income households, it is still a very important contribution to the on-going debate on how to create  a socially just and egalitarian society.

Shrinking middle class makes Toronto a city of socioeconomic extremes - The Globe and Mail

Students Know Good Teaching When They Get It, Survey Finds -

Teachers whose students described them as skillful at maintaining classroom order, at focusing their instruction and at helping their charges learn from their mistakes are often the same teachers whose students learn the most in the course of a year, as measured by gains on standardized test scores, according to a progress report on the research.

Students Know Good Teaching When They Get It, Survey Finds -

Graduate Degrees in Real Estate Are Attracting Students -

"In 2008, when most Americans were fleeing the real estate market, David Mann was charging toward it. After selling a clothing manufacturing company in Manhattan that he had helped run, Mr. Mann, 32, decided to pursue a master’s degree in real estate development. "
Graduate Degrees in Real Estate Are Attracting Students -

Tuesday, December 14, 2010

The return of the non-creative class: wealthy suburbs

The recent release of census data in the United States shows that suburban communities in Virginia lead the United States in wealth generation.  At the same time, the highest incidence of poverty was concentrated in American Indian reservations in South Dakota.

The above findings regarding suburban communities should be of interest to urban planners who often believe the suburbs to be undesirable landscapes inhabited by obese and lifeless individuals, whose lives are soon to be buried under the unsustainable mortgage debt.  Not only the latest data show an increase in wealth in the suburban communities, but also a concentration of highly educated individuals.  According to the data, seven counties out of a total of 17 where more than 50% of the adults possessed a a bachelors degree were located in suburban communities.

The data from the American Community Survey also revealed that immigrant populations are fast spreading in the United States into communities where they had not set foot in the past.  The immigrants have started to move into suburban and rural communities whereas in the past, immigrants concentrated primarily in urban communities.


From the New York Times,


WASHINGTON — The three places in the country with the highest median household income are all in Virginia, according to census datareleased on Tuesday, while those with the highest rates of poverty are in four American Indian reservations, all in South Dakota.

The Virginia counties of Fairfax and Loudoun and the city of Falls Church had the highest median income, according to the data, which spans 2005 to 2009. Falls Church was the highest at $113,313, up by 17 percent from 2000. The lowest median income was in Owsley County, Ky., at $18,869.

Of the five counties with poverty rates higher than 39 percent, four contain or are in American Indian reservations in South Dakota. The fifth, Willacy County, Tex., is on the Gulf Coast.

The data is from the Census Bureau’s American Community Survey, which samples 1 in 10 Americans on a variety of social, economic and demographic topics. It is the single largest release of data in the bureau’s history, with 11 billion individual estimates covering 670,000 geographic locations. It gives details on the characteristics of American society based on surveys, and is separate from the 2010 Census, which will provide a precise count of all Americans.

In another finding, the immigrant population has spread out to areas that previously had none. The number of immigrants increased by 40 percent in the nearly 2,500 counties where they had comprised less than 5 percent of the population in 2000. Immigrants have traditionally settled in cities, but in the last decade, they have followed jobs to rural and suburban areas, particularly ones with housing booms.

Among the sharpest increases were in Frederick County, Md., where the foreign-born population nearly tripled to almost 10 percent of the population. Other fast risers were Stafford County, Va., where the foreign-born population also tripled, and Newton County, Ga., where it jumped fourfold.

Meanwhile, the nation’s traditional immigrant centers — the 16 counties where immigrants made up more than 30 percent of the population, according to the data — saw their foreign-born population grow by only a quarter million.

The suburbs of Washington also contained the highest concentration of people with higher education. Seven of the 17 counties where more than half of people 25 and older had a bachelor’s degree were in the city’s suburbs. Three counties were in Colorado (Boulder, Douglas and Pitkin) and two were in California (Marin and San Francisco) and Maryland (Howard and Montgomery).

There were 62 counties where less than 10 percent of the adult population had a bachelor’s degree. Fourteen of these counties were in Georgia, nine in Tennessee, eight in Kentucky and five each in Florida and West Virginia.

Sabrina Tavernise reported from Washington, and Robert Gebeloff from New York.

A $60 million surplus in transit operations: Wishful thinking or creative accounting

As someone who researches and teaches transit planning, I was simply shocked to see TTC projecting a $60 million surplus for 2010. For this surplus to realize either the principles of accounting have to be held in abeyance or the dictionary’s definition of surplus has to be updated.

TTC’s own data for 2009 showed operating expenses at $1.33 billion and operating revenues of $886.4 million. Thus the operating deficit (not surplus) for 2009 equalled $442.3 million.

TTC is projecting 480 million trips (riders) for 2010, roughly 9 million more riders than it carried in 2009. Even with the fare-hike in January 2010 and the additional 9 million riders, a $442.3 million deficit cannot be turned into a $60 million surplus simply because TTC’s operating revenue per trip will still be around $2, which is not sufficient to generate a surplus. Consider that the operating revenue per trip in 2009 was merely $1.88. This is because not everyone pays the full fare, thanks to volume discounts and lower fares for adults, students and others. In addition, TTC employees make millions of trips a year for work, which contribute to the total annual ridership, but not a dime to the revenue.

Furthermore, in 2009 wages, salaries, and benefits accounted for over a billion dollars in operating expenses, representing  over 70% of the total expenses. Because of wage pressures, TTC’s operating expenses will be higher in 2010. The savings in fuel costs and increase in ad revenue are too miniscule to turn TTC’s fortunes. 

What is of interest is that the price elasticity of transit demand has been turned on its head. I remember Richard Soberman, the guru of public transportation, mentioning in his lecture at the University of Toronto in 1997 that the price elasticity of transit demand is around –0.3. This suggests that a 10% increase in transit fare may result in a 3% decline in transit ridership, at least in the short run. Well, that’s no longer true for Toronto. A significant fare hike in January 2010 has not been accompanied with a decline in ridership.

One should, however, note that 2008 and 2009 witnessed a slowdown in economic activity, which often coincides with low transit ridership. The projected increase in transit ridership in 2010 could simply be the result of economic growth, as well as population growth, in the GTA.

Also of interest is the fact that TTC is suggesting that the increase in ridership has been realized on bus routes in the suburbs. I have long argued for a greater recognition of buses as the lynchpin of transit systems. Without the feeder bus networks, subways would simply collapse.

Globe and Mail, December 14, 2010

John Lorinc: The TTC is cruising toward an unprecedented $60-million surplus for 2010 thanks to a combination of increased ridership, stronger ad sales, cheaper fuel and the surprising resilience of Toronto commuters in the face of last winter’s controversial fare hikes.

“We've had surpluses [in the past], but nothing this big,” said TTC general secretary Vince Rodo.

The results are part of TTC chief general manager Gary Webster’s third-quarter report, to be presented at the commission’s inaugural meeting Wednesday.

What’s less clear, however, is whether Mayor Rob Ford and his budget chair, Mike Del Grande, will now ask the TTC to make do with a smaller subsidy for 2011 in light of the agency’s surprisingly robust performance. Mr. Del Grande couldn’t be reached for comment Monday.

The TTC last year received a $430-million operating subsidy from council, and a further $83-million for Wheel-Trans. “We’re going to be asking for the same subsidy level we asked for in 2010,” TTC spokesperson Brad Ross said.

But new TTC chair Karen Stintz appears to be leaving the door open for a rollback. “The commission will consider the options presented by the surplus at the January meeting,” said a spokesperson for Ms. Stintz.

Mr. Rodo says the TTC’s fare revenues were $41-million ahead of projections for 2010, with a record-setting 480 million riders expected this year or about four per cent more than the agency had estimated. High unemployment and fare hikes, such as the one imposed earlier in the year, generally result in fewer riders. But Mr. Rodo said the TTC benefited because many Torontonians are looking to save money, either by taking transit or purchasing Metropasses.

Increased traffic on bus routes accounted for the bulk of the ridership growth, he added. To meet demand, TTC officials cancelled two rounds of service reductions planned for March and September.

The TTC also saw a $3-million bump in its advertising revenues because of the ridership growth.

On the expense side of the ledger, Mr. Rodo said TTC officials have moved to purchasing diesel on the so-called spot market rather than locking into longer-term futures contracts. Typically, futures contracts allow the TTC to hedge against rising energy prices. But in recent months, the agency has found it more economical to buy short-term supplies.

The autumn of Larry Summers by The Washington Post

An unnecessarily harsh farewell by the Washington Post
The autumn of Larry Summers

Tuesday, December 14, 2010; A02

It was the final question from the audience following Larry Summers's final speech as President Obama's chief economic adviser. "What," asked Caren Bohan of Reuters, "will you miss most about being in the White House?"

Summers could have taken the chance to wax eloquent about the virtues of government service, but instead he glared at the questioner. "Reporters like you," he said. Awkward laughter followed. Bohan's eyes widened, and Summers chuckled at his little joke.

For a man delivering his valedictory, with TV cameras rolling, it was oddly petulant. Bohan had written a profile of Summers this year that emphasized his "rougher edges" and his role in making debates among Obama's economic advisers resemble a "World Wrestling Federation smackdown."

But the parting shot was vintage Summers: a man who rose to national prominence because of his intellect but is now leaving government known more for his dyspepsia. Over the past two years, there has been no shortage of things to put him in a sour mood.

On his last tour in government, he was the wunderkind who, at age 28, had become one of the youngest tenured professors in Harvard's history. Before he turned 40, he joined the Clinton administration, where as economics adviser and Treasury secretary he presided over boom times - earning a stature that propelled him to the Harvard presidency.

But Summers peaked too soon. First came his conflicts with women on the Harvard faculty, leading to his unceremonious departure and his quip Monday that he "came to Washington to get out of politics." Then, returning to government, he was passed over for his old job at Treasury and for the chairmanship of the Federal Reserve. This time, the 56-year-old Summers will leave office with an unemployment rate near 10 percent, and Democrats and Republicans talking about the administration's "failed" economics team.

On Monday morning, he went to the Economic Policy Institute, a liberal think tank, to give his "perspectives on the past two years." But in his remarks, he spoke of not a single wrong decision he made.

In the question time, the Huffington Post's Dan Froomkin prompted Summers to list some regrets. The economist would say only that "one would like to have seen more rapid progress," but that "relative to what was really very widely feared, the outcome has been a good deal better."

Summers's final performance was very much in character. He arrived 10 minutes late for the speech, his suit jacket open, his shirt pulling tightly at the buttons, his suitpants stained on one of the knees. His hair showed signs of bedhead, but it could have been mussed by Summers during one of his morning meetings. He jiggled his legs while listening to the introduction by EPI President Larry Mishel, who had some edgy words for his guest. Although both men grew up in Philly, Mishel said, "when he moved to Boston, he adopted the Boston Red Sox as his baseball team. Me? I'm still a fan of the Fightin' Phils."

Summers rushed to rebut this point - by insulting the home team. "If I lived in Washington, I might still be a Phillies fan, too." There were groans in the audience.

After that, the valedictory was appropriately scholarly. Summers spoke about "global fora" and "the canonical good" and "Moynihan's corollary to Baumol's disease." But more than brainy, he was bullish.

"Had it not been for President Obama's willingness to support a sufficiently aggressive response from the late stage of the presidential campaign to his first days and months in office, I have little doubt that we would be looking at a vastly different world today," he said.

He recalled his glory days during the Clinton years - "The United States improved its structural deficit by nearly four percentage points of GDP between 1993 and 2000" - and maintained that Obama "has already taken several important steps" to lessen the deficit. "I am not one who sees financial collapse on the imminent horizon," Summers said.

I asked Summers whether, in retrospect, there was anything he would have done differently. He rephrased the question. "Would I like the results to be even better than they have been on a number of different dimensions? Of course. But I think the president is right to take pride in what has been averted. And that is not easy always for people to understand, but is something that I think is very, very important."

Americans don't know how good they've got it - because they aren't as smart as Larry Summers.

The autumn of Larry Summers

Sunday, December 12, 2010

Globalization of trade and rights: Bangladeshi workers demand decent wages

The secret behind the success of rapid increase in the Bangladeshi textile exports is fast unfolding. Subjected to undignified wages, the textile workers suffered as the North American and European consumers enjoyed their clothes delivered by the retailers who outsourced not just manufacturing, but the exploitation of workers.

Inadequate labour laws in Bangladesh and other similar places has resulted in the continued suffering of workers who are now on the street campaigning for their rights or jumping to their death off the roofs of the same factories that make designer labels and technical gizmos for companies as reputed as Apple.

The following is from BBC:

At least three people were killed and dozens more injured when police in Bangladesh clashed with garment factory workers demanding better pay, police have said.Security forces beating garment workers in Chittagong, Bangladesh, 12 December 2010

Police used batons and tear gas to disperse thousands of protesters in Dhaka and Chittagong.

The unrest came a day after demonstrations shut down factories in southern Bangladesh.

The workers say wages have not gone up, even though rises were due last month.

Police tried to disperse protesters attacking factories and smashing vehicles in Chittagong Export Procession Zone (CEPZ) on Sunday, police official Reza al-Hasan said.

Local media said one of the men killed was a 35-year-old rickshaw puller in Chittagong, but it was unclear how he had died.

All three victims were reportedly shot dead. At least 50 people were injured during the unrest in Chittagong.  Almost all factories in the CEPZ are closed because of the protests, officials said.

The BBC's Anbarasan Ethirajan, in Dhaka, says demonstrations on Saturday forced a South Korean company to shut down all its 11 factories there.

Scores more were injured in Dhaka, where thousands of workers gathered outside their factory in an industrial area of the city early in the morning in protest against its closure as a result of the disturbances.

Roads in Dhaka were blocked and at least two vehicles set on fire, police said.

Labour unions say many of the factories are not implementing the new salary scale announced by a government wage board earlier this year.

From November, the factories should have been paying a wage of at least $43 (£27) a month.

Around Dhaka, workers in some factories have been protesting for a number of days, demanding increased pay.

More than three million people, mostly women, work in Bangladesh's garment industry, which supplies many major Western stores and is a key sector of the country's economy.

Friday, December 10, 2010

Shanghai teenagers beat the rest of the world in learning

Further to my earlier blog about the xenophobic coverage of the same development where students in China ended up performing better in learning than the rest, the coverage in the prestigious Science Journal has been more objective.


Wednesday, December 8, 2010

Top Scores From Shanghai Stun Experts

The following is from The New York Times, which reports that some “experts” are surprised to see 15-year old Chinese students from Shanghai beat the rest of the world in reading, science, and math skills.

I am frankly surprised at the “stunned educators.” Why is it hard to imagine that students from developing economies can outperform the rest?


December 7, 2010

Top Test Scores From Shanghai Stun Educators


With China’s debut in international standardized testing, students in Shanghai have surprised experts by outscoring their counterparts in dozens of other countries, in reading as well as in math and science, according to the results of a respected exam.

American officials and Europeans involved in administering the test in about 65 countries acknowledged that the scores from Shanghai — an industrial powerhouse with some 20 million residents and scores of modern universities that is a magnet for the best students in the country — are by no means representative of all of China.

About 5,100 15-year-olds in Shanghai were chosen as a representative cross-section of students in that city. In the United States, a similar number of students from across the country were selected as a representative sample for the test.

Experts noted the obvious difficulty of using a standardized test to compare countries and cities of vastly different sizes. Even so, they said the stellar academic performance of students in Shanghai was noteworthy, and another sign of China’s rapid modernization.

The results also appeared to reflect the culture of education there, including greater emphasis on teacher training and more time spent on studying rather than extracurricular activities like sports.

“Wow, I’m kind of stunned, I’m thinking Sputnik,” said Chester E. Finn Jr., who served in President Ronald Reagan’s Department of Education, referring to the groundbreaking Soviet satellite launching. Mr. Finn, who has visited schools all across China, said, “I’ve seen how relentless the Chinese are at accomplishing goals, and if they can do this in Shanghai in 2009, they can do it in 10 cities in 2019, and in 50 cities by 2029.”

The test, the Program for International Student Assessment, known as PISA, was given to 15-year-old students by the Organization for Economic Cooperation and Development, a Paris-based group that includes the world’s major industrial powers.

The results are to be released officially on Tuesday, but advance copies were provided to the news media a day early.

“We have to see this as a wake-up call,” Secretary of Education Arne Duncan said in an interview on Monday.

“I know skeptics will want to argue with the results, but we consider them to be accurate and reliable, and we have to see them as a challenge to get better,” he added. “The United States came in 23rd or 24th in most subjects. We can quibble, or we can face the brutal truth that we’re being out-educated.”

In math, the Shanghai students performed in a class by themselves, outperforming second-place Singapore, which has been seen as an educational superstar in recent years. The average math scores of American students put them below 30 other countries.

PISA scores are on a scale, with 500 as the average. Two-thirds of students in participating countries score between 400 and 600. On the math test last year, students in Shanghai scored 600, in Singapore 562, in Germany 513, and in the United States 487.

In reading, Shanghai students scored 556, ahead of second-place Korea with 539. The United States scored 500 and came in 17th, putting it on par with students in the Netherlands, Belgium, Norway, Germany, France, the United Kingdom and several other countries.

In science, Shanghai students scored 575. In second place was Finland, where the average score was 554. The United States scored 502 — in 23rd place — with a performance indistinguishable from Poland, Ireland, Norway, France and several other countries.

The testing in Shanghai was carried out by an international contractor, working with Chinese authorities, and overseen by the Australian Council for Educational Research, a nonprofit testing group, said Andreas Schleicher, who directs the Organization for Economic Cooperation and Development’s international educational testing program.

Mark Schneider, a commissioner of the Department of Education’s research arm in the George W. Bush administration, who returned from an educational research visit to China on Friday, said he had been skeptical about some PISA results in the past. But Mr. Schneider said he considered the accuracy of these results to be unassailable.

“The technical side of this was well regulated, the sampling was O.K., and there was no evidence of cheating,” he said.

Mr. Schneider, however, noted some factors that may have influenced the outcome.

For one thing, Shanghai is a huge migration hub within China. Students are supposed to return to their home provinces to attend high school, but the Shanghai authorities could increase scores by allowing stellar students to stay in the city, he said. And Shanghai students apparently were told the test was important for China’s image and thus were more motivated to do well, he said.

“Can you imagine the reaction if we told the students of Chicago that the PISA was an important international test and that America’s reputation depended on them performing well?” Mr. Schneider said. “That said, China is taking education very seriously. The work ethic is amazingly strong.”

In a speech to a college audience in North Carolina, President Obama recalled how the Soviet Union’s 1957 launching of Sputnik provoked the United States to increase investment in math and science education, helping America win the space race.

“Fifty years later, our generation’s Sputnik moment is back,” Mr. Obama said. With billions of people in India and China “suddenly plugged into the world economy,” he said, nations with the most educated workers will prevail. “As it stands right now,” he said, “America is in danger of falling behind.”

If Shanghai is a showcase of Chinese educational progress, America’s showcase would be Massachusetts, which has routinely scored higher than all other states on America’s main federal math test in recent years.

But in a 2007 study that correlated the results of that test with the results of an international math exam, Massachusetts students scored behind Singapore, Hong Kong, South Korea, Taiwan, and Japan. Shanghai did not participate in the test.

A 259-page Organization for Economic Cooperation and Development report on the latest Pisa results notes that throughout its history, China has been organized around competitive examinations. “Schools work their students long hours every day, and the work weeks extend into the weekends,” it said.

Chinese students spend less time than American students on athletics, music and other activities not geared toward success on exams in core subjects. Also, in recent years, teaching has rapidly climbed up the ladder of preferred occupations in China, and salaries have risen. In Shanghai, the authorities have undertaken important curricular reforms, and educators have been given more freedom to experiment.

Ever since his organization received the Shanghai test scores last year, Mr. Schleicher said, international testing experts have investigated them to vouch for their accuracy, expecting that they would produce astonishment in many Western countries.

“This is the first time that we have internationally comparable data on learning outcomes in China,” Mr. Schleicher said. “While that’s important, for me the real significance of these results is that they refute the commonly held hypothesis that China just produces rote learning.”

“Large fractions of these students demonstrate their ability to extrapolate from what they know and apply their knowledge very creatively in novel situations,” he said.

In PISA Test, Top Scores From Shanghai Stun Experts -