An uphill battle

I took this photo in the NYC subway a few days ago. My apologies for the quality, but I thought it's a great juxtaposition:

In the top of the photo is an ad from the NYC Department of Health, advising you to choose food with less sodium. (Here's an AP story about the ads.) But to the bottom right is an ad for McDonald's dollar menu, and those are everywhere. While it doesn't mean we shouldn't run such ads, it's worth remembering that the sheer volume of food advertising will always dwarf opposing health messages. 

Group vs. individual uses of data

Andrew Gelman notes that, on the subject of value-added assessments of teachers, "a skeptical consensus seems to have arisen..." How did we get here? Value-added assessments grew out of the push for more emphasis on measuring success through standardized tests in education -- simply looking at test scores isn't OK because some teachers are teaching in better schools or are teaching better-prepared students. The solution was to look at how teachers' students improve in comparison to other teachers' students. Wikipedia has a fairly good summary here.

Back in February New York City released (over the opposition of teachers' unions) the value-added scores of some 18,000 teachers. Here's coverage from the Times on the release and reactions.

Gary Rubinstein, an education blogger, has done some analysis of the data contained in the reports and published five posts so far: part 1, part 2, part 3, part 4, and part 5. He writes:

For sure the 'reformers' have won a battle and have unfairly humiliated thousands of teachers who got inaccurate poor ratings. But I am optimistic that this will be be looked at as one of the turning points in this fight. Up until now, independent researchers like me were unable to support all our claims about how crude a tool value-added metrics still are, though they have been around for nearly 20 years. But with the release of the data, I have been able to test many of my suspicions about value-added.

I suggest reading his analysis in full, or at least the first two parts.

For me one early take-away from this -- building off comments from Gelman and others -- is that an assessment might be a useful tool for improving education quality overall, while simultaneously being a very poor metric for individual performance. When you're looking at 18,000 teachers you might be able to learn what factors lead to test score improvement on average, and use that information to improve policies for teacher education, recruitment, training, and retention. But that doesn't mean one can necessarily use the same data to make high-stakes decisions about individual teachers.

1WTC

After years of delays, the new One World Trade Center is going up fast. I took this picture just one block from my office:

Business Week has an interesting account of how the redevelopment of the site was debated and negotiated over the last 10 years. And of course the next month will see a flurry of writing and coverage related to the ten-year anniversary of 9/11. Ten years ago I was a high school student in Arkansas and New York City felt very far away. My mom had been visiting her sister in DC and was flying home that morning. I needed to see the orthodontist in Little Rock so my dad and I drove down together to pick her up at the airport, and heard the first reports on the radio on the drive down. Nobody in my family flew very often so everyone knew she'd be flying -- my parents spent the rest of the day contacting people to assure them my mom hadn't been on one of the flights. I missed all the emotional reactions of learning at school that many people recount, but that afternoon I worked my normal shift at my town's Kroger grocery store as we had a small rush of people buying supplies, and listened to people talking about lining up to get gas, just in case.

New books

... that I wish I had time to read: 1) Laurie Garrett's first book since Betrayal of Trust (2000) is I Heard the Sirens Scream, which takes on 9/11, the anthrax attacks, and the US response to both. I'm most interested in the discussion of "the bizarre chemistry of The Plume that rose from the burning crushed World Trade center for four months." Alanna Shaikh interviews Garrett about the book in UN Dispatch.

2) The Other Barack, though it sounds depressing.

3) The Invention of Brownstone Brooklyn: Gentrification and the Search for Authenticity in Postwar New York. A review:

By 1982, Osman writes, the number of hardware stores in Park Slope was more than three times the per-capita average in the rest of the city, and surveys indicated that a majority of Park Slope residents were undertaking most improvements themselves. In the current age of multimillion-dollar brownstone sales, it’s easy to forget the more modest roots of these neighborhoods....New, politically savvy residents sometimes found common cause with local residents in lobbying for services and opposing large-scale development. In 1975, the Fort Greene Non-Profit Improvement Council was powerful enough to obtain a court injunction halting study of the construction of a new Giants Stadium on the Atlantic Terminal site. Such coalitions, however, don’t always hold together....

Sounds interesting throughout, especially now that I've been to Brooklyn. Yes, before this summer I had never really ventured outside of Manhattan on my few visits to New York. Also, the question of gentrification is one of those things I used to think was simple, when I first read of it. Surprisingly (or not?) it was talking through those issues in DC with several friends who are urban planners that made me realize that there generally aren't easy answers to any question involving old and new residents and changing economic fortunes in a neighborhood.

Not #4: After reading J's review, I think I'll pass on Inside the Everyday Lives of Development Workers.

NYC mystery of the day: trash collection

I keep hearing complaints from both New York residents and visitors alike that the city smells. You get used to it pretty fast, but it's true -- especially during the summer. In the two other cities I've lived in (Washington, DC and Baltimore) I would put the trash out in a specially marked bin or garbage can for pick-up. My apartment in the East Village has a designated bin on the sidewalk, but we seem to be an exception rather than the rule. Most people just stack their trash bags on the sidewalks, like so:

This contributes to the smell, and probably to the rat problem as well. So why doesn't New York require trash to be placed in bins like at least some other American cities?

This brief history of trash collection in NYC is fascinating, but it doesn't really offer an answer. So I'm stumped for now, but my best guess is that sidewalk and building entryway space are at such a premium that space-consuming trash bins have never been popular. If you have another explanation I'd be happy to hear it.

Equality in NY

I had this post saved as a draft for the last week or so -- oops: ------

It's a great summer to be in New York City. I was watching the news on same sex marriage pretty closely, and as soon as the religious exemptions amendment passed -- signalling that passage of the bill itself was just a matter of time -- bloggers started noting that crowds were gathering in front of the Stonewall Inn in Greenwich Village. I live about a mile east of there in the East Village, so I headed out immediately to be there at the historic moment.

This may come as a surprise for gay rights advocates -- or for pretty much anyone who didn't go to an extremely conservative university -- but I hadn't heard of the Stonewall Riots until a year or two ago. The Stonewall has been on my long list of historical sights to see in New York but I hadn't been there yet, and what better time to visit than on this historic occasion?

Sure enough, there was a big crowd gathered and quite a few media outlets on hand. I snapped this shot of an endearing older couple being interviewed:

And here are two NYPD officers doing crowd control, chatting amiably with the celebrants:

While the pace of change can often seem glacial for those eagerly advocating (as they rightly should) for justice now, it struck me that on a grander scale this progress has come impressively fast. Just a little over 40 years ago -- half a lifetime -- the police were systematically oppressing and raiding the few gay establishments in New York. Their actions were hardly inconsistent with popular will either, as there really was no gay rights movement yet. And now, in 2011, there the officers were, guarding a peaceful and spontaneous celebration by New Yorkers -- male and female, gay and straight -- of marriage equality, something that was probably inconceivable to the Stonewall rioters. Yes, the law is not yet perfect and we still have far to go, but for that night it felt right to pause and reflect on just how far we've come.

Advice not to trust

Yesterday morning I came across the gentleman pictured below in New York's Union Square. I've been meaning to take more pictures of "things you only see in NYC" -- a category which generally consists of extremes of pets and fashion -- but I think this deserves its own post:

He was either selling or trying to give away copies of a book titled Uropathy: The Most Powerful Holistic Therapy by one Martin Lara. Since the review of Uropathy on Amazon.com is from the Village Voice, I assume that the evangelist was either Lara or one of his disciples. The review:

Vitamin Pee! Urine is a natural remedy, so raise a glass! That's what alternative therapist Martin Lara wants everyone to do. In his Uropathy: The Most Powerful Holistic Therapy, pee's the ultimate cure-all. Gagging aside, it's not so unconventional: former Indian prime minister Morarji Desai guzzled ounces each morning, observing an ancient Hindu practice. Lara learned about it 11 years ago, when the self-taught therapist he's never studied traditional medicine became disenchanted with science's inability to cure his ailments. Since then he's lectured to thousands. Not any pee will do it must be your own, which Lara says is a nontoxic biofeedback stimulator that boosts immunity by activating the lymphatic system, thus restoring the body to an internally balanced state of health. Dosages range from a few drops of Lara's "Ultimate Universal Remedy" an elixir of water, urine, and white rum to several ounces for serious conditions like cancer, dysentery, or Alzheimer's. Of course, not everyone is ready for this leap of faith. On his Web site Lara argues against obsessing over taste and smell: "Urine is a sample of what is flowing through your veins and repulsive urine should be a motivation to improve the internal conditions, rather than an excuse for not using Uropathy." -- The Village Voice

He was quite earnest. I didn't engage him in conversation because two other passersby were already talking to him. A girl was explaining that urine is what your kidneys decide your body doesn't need. But she wasn't just explaining it, she was disgusted, and angry. His response was similar to a major defense of homeopathic medicine, that the "toxin makes the remedy" (or something like that). The girl got exasperated and left with her friend, and you could hear her ranting as she walked away. I chose not to continue the conversation because I was on my way to meet friends, but in hindsight I wish I had stayed because there are some questions I don't have the answers to:

  • How often does he talk publicly about this? What does he do for a living? Ie, is this it, or does he have a boring day job and this is his true passion?
  • Does he feel that drinking urine has cured whatever health problems that he originally sought help for? (I would guess so.)
  • Why does he think drinking urine has not been more widely adopted?
  • Does he think that his approach (especially the t-shirt and public 'evangelism') is the most effective way to spread his message? I would guess he enjoys the attention on some level, but also promotes his beliefs through other, more effective channels.
  • What are the typical reactions he gets? How many people stay and talk with him at length, and of those how many eventually adopt his therapy?
  • I'd like to talk a bit about Western medicine. Not necessarily the biomedical interventions we favor, but the scientific process by which we (ideally) establish that a practice is beneficial. Does he think urine therapy could be tested by a randomized controlled trial? If not, why not?
  • If the passerby had stuck around: why did she choose to argue with him? Did she really think that a guy wearing a bright yellow "Drink Urine" t-shirt in Union Square was likely to change his mind? And for the man himself: how common is her argumentative reaction?

I think a natural first reaction to something this out of the ordinary is laughter or mockery, or the assumption that he's clinically insane. On further thought, what he believes -- in factual support and argumentative method, if not in substance -- isn't that different from much of alternative medicine, and his methods have been widely adopted by many mainstream religions and social movements as well as less-respected 'fringe' beliefs. If those are both true, why isn't his belief more widely adopted? Is it just too taboo?

I think I could have learned valuable things about the mixture of reason and emotion and belief that guide human choices if I had stayed and asked some of these questions. I don't think I'll change his mind, but I plan to look for him if I'm ever strolling through Union Square on a weekend again.

(Note: evidently "urine therapy" is a thing. The Wikipedia page starts with "In alternative medicine..." -- never a good sign.)

Monday Miscellany: NYC edition

Two weeks ago I moved to New York City for the summer, so today's links from around the interwebs are focused on the Big Apple:

Miscellany: Epidemic City and life expectancy

In 8 days I'll be done with my first year of graduate studies and will have a chance to write a bit more. I've been keeping notes all year on things to write about when I have more time, so I should have no shortage of material! In the meantime, two links to share: 1) Just in time for my summer working with the New York City Department of Health comes Epidemic City: The Politics of Public Health in New York. The Amazon / publisher's blurb:

The first permanent Board of Health in the United States was created in response to a cholera outbreak in New York City in 1866. By the mid-twentieth century, thanks to landmark achievements in vaccinations, medical data collection, and community health, the NYC Department of Health had become the nation's gold standard for public health. However, as the city's population grew in number and diversity, new epidemics emerged, and the department struggled to balance its efforts between the treatment of diseases such as AIDS, multi-drug resistant tuberculosis, and West Nile Virus and the prevention of illness-causing factors like lead paint, heroin addiction, homelessness, smoking, and unhealthy foods. In Epidemic City, historian of public health James Colgrove chronicles the challenges faced by the health department in the four decades following New York City's mid-twentieth-century peak in public health provision.

This insightful volume draws on archival research and oral histories to examine how the provision of public health has adapted to the competing demands of diverse public needs, public perceptions, and political pressure.

Epidemic City delves beyond a simple narrative of the NYC Department of Health's decline and rebirth to analyze the perspectives and efforts of the people responsible for the city's public health from the 1960s to the present. The second half of the twentieth century brought new challenges, such as budget and staffing shortages, and new threats like bioterrorism. Faced with controversies such as needle exchange programs and AIDS reporting, the health department struggled to maintain a delicate balance between its primary focus on illness prevention and the need to ensure public and political support for its activities.

In the past decade, after the 9/11 attacks and bioterrorism scares partially diverted public health efforts from illness prevention to threat response, Mayor Michael Bloomberg and Department of Health Commissioner Thomas Frieden were still able to work together to pass New York's Clean Indoor Air Act restricting smoking and significant regulations on trans-fats used by restaurants. Because of Bloomberg's willingness to exert his political clout, both laws passed despite opposition from business owners fearing reduced revenues and activist groups who decried the laws' infringement upon personal freedoms. This legislation preventative in nature much like the 1960s lead paint laws and the department's original sanitary code reflects a return to the 19th century roots of public health, when public health measures were often overtly paternalistic. The assertive laws conceived by Frieden and executed by Bloomberg demonstrate how far the mandate of public health can extend when backed by committed government officials.

Epidemic City provides a compelling historical analysis of the individuals and groups tasked with negotiating the fine line between public health and political considerations during the latter half of the twentieth century. By examining the department's successes and failures during the ambitious social programs of the 1960s, the fiscal crisis of the 1970s, the struggles with poverty and homelessness in the 1980s and 1990s, and in the post-9/11 era, Epidemic City shows how the NYC Department of Health has defined the role and scope of public health services, not only in New York, but for the entire nation.

2) Aaron Carroll at the Incidental Economist writes about the subtleties of life expectancy. His main point is that infant mortality skews life expectancy figures so much that if you're talking about end-of-life expectations for adults who have already passed those (historically) most perilous times as a youngster, you really need to look at different data altogether.

The blue points on the graph below show life expectancy for all races in the US at birth, while the red line shows life expectancy amongst those who have reached the age of 65. Ie, if you're a 65-year-old who wants to know your chances of dying (on average!) in a certain period of time, it's best to consult a more complete life table rather than life expectancy at birth, because you've already dodged the bullet for 65 years.

(from the Incidental Economist)

Summer plans

My first (and only) full year of coursework for my masters program is drawing to a close. Finals are in mid-May, and comprehensive exams are in early June. Then it's off to New York City! On June 8 I will be joining the 5th class of the Epi Scholars program with the New York City Department of Health and Mental Hygiene. Each scholar (read: grad student in epidemiology) is paired with a staff mentor and given a specific project to work on throughout the summer. The research work is augmented by training on SAS and GIS, educational sessions on health disparities and other topics that touch on the work of the Department. The project I will be working on involves characterizing children with severe lead poisoning in New York to help clinicians better screen for them. I've only heard amazing things about this program from the Hopkins students and alumni who have gone before, so I'm quite excited.

In the fall I will move to an as-yet-undisclosed location and spend the majority of my second year doing 'field work' in global health. One of the reasons I chose this program is that it gives me the chance to get a substantial chunk (6-12 months) of work experience abroad in one place before I complete my degree, and I plan to take full advantage of that. More on where I'll be once things are finalized.

Randomizing in the USA, ctd

[Update: There's quite a bit of new material on this controversy if you're interested. Here's a PDF of Seth Diamond's testimony in support of (and extensive description of) the evaluation at a recent hearing, along with letters of support from a number of social scientists and public health researchers. Also, here's a separate article on the City Council hearing at which Diamond testified, and an NPR story that basically rehashes the Times one. Michael Gechter argues that the testing is wrong because there isn't doubt about whether the program works, but, as noted in the comments there, doesn't note that denial-of-service was already part of the program because it was underfunded.] A couple weeks ago I posted a link to this NYTimes article on a program of assistance for the homeless that's currently being evaluated by a randomized trial. The Poverty Action Lab blog had some discussion on the subject that you should check out too.

The short version is that New York City has a housing assistance program that is supposed to keep people from becoming homeless, but they never gave it a truly rigorous evaluation. It would have been better to evaluate it up front (before the full program was rolled out) but they didn't do that, and now they are.  The policy isn't proven to work, and they don't have resources to give it to everyone anyway, so instead of using a waiting list (arguably a fair system) they're randomizing people into receiving the assistance or not, and then tracking whether they end up homeless. If that makes you a little uncomfortable, that's probably a good thing -- it's a sticky issue, and one that might wrongly be easier to brush aside when working in a different culture. But I think on balance it's still a good idea to evaluate programs when we don't know if they actually do what they're supposed to do.

The thing I want to highlight for now is the impact that the tone and presentation of the article impacts your reactions to the issue being discussed. There's obviously an effect, but I thought this would be a good example because I noticed that the Times article contains both valid criticisms of the program and a good defense of why it makes sense to test it.

I reworked the article by rearranging the presentation of those sections. Mostly I just shifted paragraphs, but in a few cases I rearranged some clauses as well. I changed the headline, but otherwise I didn't change a single word, other than clarifying some names when they were introduced in a different order than in the original. And by leading with the rationale for the policy instead of with the emotional appeal against it, I think the article gives a much different impression. Let me know what you think:

City Department Innovates to Test Policy Solutions

By CARA BUCKLEY with some unauthorized edits by BRETT KELLER

It has long been the standard practice in medical testing: Give drug treatment to one group while another, the control group, goes without.

Now, New York City is applying the same methodology to assess one of its programs to prevent homelessness. Half of the test subjects — people who are behind on rent and in danger of being evicted — are being denied assistance from the program for two years, with researchers tracking them to see if they end up homeless.

New York City is among a number of governments, philanthropies and research groups turning to so-called randomized controlled trials to evaluate social welfare programs.

The federal Department of Housing and Urban Development recently started an 18-month study in 10 cities and counties to track up to 3,000 families who land in homeless shelters. Families will be randomly assigned to programs that put them in homes, give them housing subsidies or allow them to stay in shelters. The goal, a HUD spokesman, Brian Sullivan, said, is to find out which approach most effectively ushered people into permanent homes.

The New York study involves monitoring 400 households that sought Homebase help between June and August. Two hundred were given the program’s services, and 200 were not. Those denied help by Homebase were given the names of other agencies — among them H.R.A. Job Centers, Housing Court Answers and Eviction Intervention Services — from which they could seek assistance.

The city’s Department of Homeless Services said the study was necessary to determine whether the $23 million program, called Homebase, helped the people for whom it was intended. Homebase, begun in 2004, offers job training, counseling services and emergency money to help people stay in their homes.

The department, added commissioner Seth Diamond, had to cut $20 million from its budget in November, and federal stimulus money for Homebase will end in July 2012.

Such trials, while not new, are becoming especially popular in developing countries. In India, for example, researchers using a controlled trial found that installing cameras in classrooms reduced teacher absenteeism at rural schools. Children given deworming treatment in Kenya ended up having better attendance at school and growing taller.

“It’s a very effective way to find out what works and what doesn’t,” said Esther Duflo, an economist at the Massachusetts Institute of Technology who has advanced the testing of social programs in the third world. “Everybody, every country, has a limited budget and wants to find out what programs are effective.”

The department is paying $577,000 for the study, which is being administered by the City University of New York along with the research firm Abt Associates, based in Cambridge, Mass. The firm’s institutional review board concluded that the study was ethical for several reasons, said Mary Maguire, a spokeswoman for Abt: because it was not an entitlement, meaning it was not available to everyone; because it could not serve all of the people who applied for it; and because the control group had access to other services.

The firm also believed, she said, that such tests offered the “most compelling evidence” about how well a program worked.

Dennis P. Culhane, a professor of social welfare policy at the University of Pennsylvania, said the New York test was particularly valuable because there was widespread doubt about whether eviction-prevention programs really worked.

Professor Culhane, who is working as a consultant on both the New York and HUD studies, added that people were routinely denied Homebase help anyway, and that the study was merely reorganizing who ended up in that pool. According to the city, 5,500 households receive full Homebase help each year, and an additional 1,500 are denied case management and rental assistance because money runs out.

But some public officials and legal aid groups have denounced the study as unethical and cruel, and have called on the city to stop the study and to grant help to all the test subjects who had been denied assistance.

“They should immediately stop this experiment,” said the Manhattan borough president, Scott M. Stringer. “The city shouldn’t be making guinea pigs out of its most vulnerable.”

But, as controversial as the experiment has become, Mr. Diamond said that just because 90 percent of the families helped by Homebase stayed out of shelters did not mean it was Homebase that kept families in their homes. People who sought out Homebase might be resourceful to begin with, he said, and adept at patching together various means of housing help.

Advocates for the homeless said they were puzzled about why the trial was necessary, since the city proclaimed the Homebase program as “highly successful” in the September 2010 Mayor’s Management Report, saying that over 90 percent of families that received help from Homebase did not end up in homeless shelters. One critic of the trial, Councilwoman Annabel Palma, is holding a General Welfare Committee hearing about the program on Thursday.

“I don’t think homeless people in our time, or in any time, should be treated like lab rats,” Ms. Palma said.

“This is about putting emotions aside,” [Mr. Diamond] said. “When you’re making decisions about millions of dollars and thousands of people’s lives, you have to do this on data, and that is what this is about.”

Still, legal aid lawyers in New York said that apart from their opposition to the study’s ethics, its timing was troubling because nowadays, there were fewer resources to go around.

Ian Davie, a lawyer with Legal Services NYC in the Bronx, said Homebase was often a family’s last resort before eviction. One of his clients, Angie Almodovar, 27, a single mother who is pregnant with her third child, ended up in the study group denied Homebase assistance. “I wanted to cry, honestly speaking,” Ms. Almodovar said. “Homebase at the time was my only hope.”

Ms. Almodovar said she was told when she sought help from Homebase that in order to apply, she had to enter a lottery that could result in her being denied assistance. She said she signed a letter indicating she understood. Five minutes after a caseworker typed her information into a computer, she learned she would not receive assistance from the program.

With Mr. Davie’s help, she cobbled together money from the Coalition for the Homeless and a public-assistance grant to stay in her apartment. But Mr. Davie wondered what would become of those less able to navigate the system. “She was the person who didn’t fall through the cracks,” Mr. Davie said of Ms. Almodovar. “It’s the people who don’t have assistance that are the ones we really worry about.”

Professor Culhane said, “There’s no doubt you can find poor people in need, but there’s no evidence that people who get this program’s help would end up homeless without it.”