Insights on Change Blog

By Sheila Wilcox

Virtually everyone asks you to take a survey—your job, restaurants, stores, schools, airports and the list goes on. It’s been reported that American adults are invited to take surveys 7 billion times a year. There have even been surveys about what people think of surveys (they’re not too crazy about them)!

Not too long ago, surveys were generally reserved for use by researchers and evaluators like us. Then, organizations of all types realized the benefits of getting large amounts of quantitative data quickly and began surveying their customers, stakeholders, employees and more. Many of these are marketing surveys—shorter surveys, asking simply about satisfaction with or use of very specific things. In contrast, evaluation surveys tend to be longer and more complex since we are interested in the impact of a program. Because we’re trying to assess deeper levels than satisfaction and use, our surveys may include more detailed and personal questions that ask information related to respondent’s background or how their attitudes or behaviors have changed.

In today’s over-surveyed world, people may click on our evaluation survey expecting a customer satisfaction survey—something they can finish in a couple minutes without needing to think too deeply about it (or possibly even profit from). That is not the mindset we want them in. It can lead to respondents quitting in the middle of the survey or feeling disgruntled afterwards and less likely to take another survey (such as that post-program survey we’re hoping they complete in a couple months).

What can we do to still get the data we need in this era of customer satisfaction surveys?

• Stay as focused as possible – we must push ourselves (and our clients) to avoid adding more than is needed for the key evaluation outcomes.
• Explain why we’re asking questions, particularly the more personal ones (this could even come at the end of the survey if you are worried about biasing them).
• Emphasize how their data helps the program or organization they have been involved with keep receiving funding or improving (possibly even encourage clients to share findings with their respondents).
• Provide accurate time estimates – don’t say it is a short 5-10 minute survey when it’s really 15-20 minutes or people won’t be mentally prepared for what we’re asking them.
• Consider other types of data collection – can you shorten the survey and use interview follow-ups or other existing data to get the information needed?

We can’t control how many other survey requests our respondents receive, so we need to provide appropriate and accurate information about the survey we’re asking them to take.

By Kris Helé

I recently attended a wonderful lecture through UC Berkeley Extension by UCB Public Health Professor William Dow. “Health Care Reform Update: Post-Affordable Care Act (ACA) Progress and Challenges” was an unexpectedly satisfying way to spend a sunny Saturday afternoon. It was a quick, digestible run-down of what health care reform entails, what’s been implemented to date and what’s coming in 2014 with full implementation.

I left the session with a number of key take-aways to inform not only BTW’s health practice but my own perspectives about health care reform. For instance:

• By 2016, after a couple years of full implementation, an estimated 3 million people in California will still be without coverage, mostly immigrants. Though we knew the ACA would not provide universal coverage, this is a sobering statistic for a policy whose primary purpose is to expand health insurance.
• Public opinion is split on so-called “Obamacare.” However, more than half of individuals polled say they really don’t have enough information about the ACA to back up their strong opinions. Fortunately, many organizations—Kaiser Family Foundation, California HealthCare Foundation and the Department of Managed HealthCare, for example—are trying mightily to get the word out to address the information gap.
• If we were to start over and initiate truly transformational change in the health care system, many would recommend scrapping the dysfunctional employer-based system that dominates health insurance pre- and post-ACA. Still, the ACA is what we’ve got, and what many fought hard for, so we better make the most of it.

The most provocative point Dow made was during the Q&A session, when he dispelled the myth that health prevention efforts equal cost savings. I couldn’t believe my ears—a public health professor really just said that? Sacrilege! With his economist hat on, Dow says that only a handful of preventive interventions are truly cost saving—smoking cessation and flu vaccines are among them. Other prevention strategies require a large investment of money in the system, and so they ultimately may not save costs, but may still be the right and worthwhile thing to do.

Dow’s recommendations for cost-effective, impactful societal investments? 1) Education. 2) Education. 3) Education. This is a reminder that it’s not only important to think about what we’re spending our money on—the ACA comes with a $1.7 trillion estimated price tag over 10 years—but what we aren’t spending it on instead.

I left the lecture reaffirmed, with an abiding sense that the ACA is the right thing to do, if not the perfect way to do it. But in the coming years, we must see the policy fulfill its promise of reducing the number of uninsured, as well as deliver on other goals of improving health, increasing equity and slowing health care costs. May the provisions of the policy stand strong until these outcomes are borne out, and until the ethos of quality, equity and affordability become forever embedded in our approach to health care.

Note: Any errors in my recollection or interpretation of Dow’s lecture are entirely my own.

By Amanda Philips

This week marks the 19^th National Public Health Week, a week dedicated to recognizing the contributions of public health and ways to improve our nation’s health. This year’s theme, “Public Health is ROI: Save Lives, Save Money,” emphasizes the value of prevention and the importance of well-supported public health systems in preventing disease, saving lives and curbing health care spending.

National Public Health Week prompts us to reflect on the current state of our nation’s health and how we are responding to the changing landscape. Over the past three decades, childhood obesity rates in America have tripled, with obesity now affecting 18% of children. Obesity rates are even higher in African American and Hispanic communities, where nearly 40% of children are overweight or obese. Research shows that children and adolescents who are obese are likely to be obese as adults and are at increased risk for health problems such as heart disease, type 2 diabetes, strokes and several types of cancer.

So, what can we do to improve our nation’s public health? How can we provide a healthier, more vibrant future for our children? How can we improve neighborhoods, schools and homes to create living spaces that promote healthy living?

BTW is proud to work with many organizations engaged in improving community health. These organizations are trying to “get to the roots” of health problems in their communities by identifying and addressing causes of ill health and focusing on prevention. Through our work, we have seen these strategies in action as community members:

• Promote physical activity in safe spaces
• Create community health networks
• Advocate for environmental and land use improvements in neighborhoods
• Train adults and youth to provide health education to their peers
• Develop gardens and farmers markets in community centers or schools

While these organizations are working hard to improve the health of Americans and create a healthy future for all children, they can’t do it alone. Everyone has a role to play in improving our public health, including parents, elected officials, schools, health care professionals, faith-based and community-based organizations, and private sector companies. What will you do to promote a healthy future?

By Nadai Salibi

Ida, you’re my hero! At 97 years young you are doing things that I can only hope to do in the prime of my life. As Lisa Ling chronicles your life in Our America I watched in awe as you ran the 100 meter dash alongside 20 year olds while breaking your own record. Your ability to stay mobile and live independently sets you apart from the general population, and most certainly your age group. After learning about the tragedies you have suffered—your husband’s death when your daughter was just a child and the death of your two sons to drug-related incidents—I wondered how you found resilience. Most of all, I wondered what set your health and physical endurance apart from other seniors.

After seeing your story, I couldn’t help but be reminded of a case study in Malcolm Gladwell’s Outliers that examines a community of Italian descent in Pennsylvania. Intrigued by an astounding low-level of disease, baffled researchers conducted a thorough analysis of the community. What they found led to even more confusion. This community didn’t follow the conventional healthy lifestyle patterns known to breed such a disease-free life. In fact, healthy diets weren’t apparent, obesity was prevalent, exercise was non-existent, smokers were common, and they ruled out any genetic rationale. What could make this community an outlier? Well, the researchers came to a peculiar discovery. This community’s unique health composition resulted from their physical and social environment which mirrored the village they descended from in Italy. Strong community bonds and extended family interactions translated into a level of happiness, health and well-being that is not as common in other communities.

As I think back to Lisa Ling’s Our America and recall the other seniors she profiles, I noticed how most of the seniors are not like Ida. On the contrary, many are in fragile states, some with dementia, others bedridden. Although some of these seniors are being cared for by their family members, I see something different in Ida’s life. I see the village of this Italian community. Shelly, Ida’s daughter, fulfilled a crucial role by introducing her mother to running, while Ida’s extended family and community serve as constants in her life.

At BTW, through our experience evaluating community health initiatives, we too have seen the value of building community to collectively improve health outcomes that go beyond regular doctor visits and exams. In this case, Shelly did that for Ida. Shelly designed a healthy environment for Ida where her age and physical location didn’t matter. As proven in a small town in Pennsylvania, this can be the healthiest attribute for any age.

Secret Lives of Seniors: 97-Year-Old Breaks Track Record!

By Naomi Orensten

David Brooks piece on the limitations of data in The New York Times hits on many of the real challenges and limitations of quantitative data collection and analysis that evaluators and consultants, like us here at BTW informing change, face regularly. For example, numbers can miss nuance, big data are messy and sometimes interpreters of data look for what they want to see rather than the overall picture that the data actually paint. I want to remind data developers and consumers —that is, most of us!—of three critical data collection practices that address some of Brook’s concerns.

1. Collect data that tell you what you need to know. Data become distracting and messy often because we collect information arbitrarily, because it happens to be available or because we mistakenly think more is better. It is important to thoughtfully choose data points that will identify whether you’re moving toward your objective.

2. Use a mix of data types to fully understand a program, issue or organization. First let’s define data. Data are pieces of information. Quantitative data— numbers and statistics—typically explain ‘what.’ For example, 50% of students improved their test scores. Qualitative data—information collected from observations, interviews, focus groups or open-ended survey questions—explain why. Was the improvement due to increased instruction, parent involvement and/or a more supportive learning environment? Together, these data weave a much fuller understanding of any given issue, providing decision-makers with more complete story than was otherwise possible.

3. Account for the complexity of the real world in data collection and analysis. Programs and organizations don’t operate in a vacuum, and neither should data. Every study has limitations, which must be clearly communicated with stakeholders. But a good analyst and evaluation partner approaches the data to answer contextual questions, equipped to engage in the messiness. Tools such as regression and statistical modeling, for example, are designed to help us understand the cause and sequence of things as well as the relationships among variables.

In the article’s final paragraph, Brooks reminds the reader that “like any tool,” data are, “good at some things and not at others.” Sure, fair enough. But let’s not let that be an excuse for poor practice. The combination of good data and good process will go a long way to help consultants, researchers and decision-makers understand our work, make better decisions and pursue solid strategy.

By Jay Sherwin

A recent New York Times story by Jason DeParle offers a moving portrait of three smart and ambitious young women, friends from Galveston, Texas, with dreams of a better life. All three enrolled in college but, five years later, none has graduated. All three are back in Galveston, working, paying off their debts and no doubt wondering how things might have turned out differently.

In telling the stories of these three women, DeParle makes this powerful statement about contemporary American life: “Education, a force meant to erode class barriers, appears to be fortifying them.” He’s right, and it’s a deeply disturbing trend.

Thirty years ago, 19 percent of students from the poorest quartile of American families enrolled in four-year colleges. That number has increased to 29 percent. But in the same time, the number of students from the richest quartile entering college has increased from 58 percent to 80 percent. The difference in college graduation rates is equally dramatic: 9 percent of poor students complete college, compared to 54 percent of rich students.

Poverty prevents many young people from pursuing and persisting in higher education, but poverty is only part of the explanation. Other social and structural factors conspire to keep even high achieving low income students from learning their way into the middle class.

Two recently published studies illustrate this point.

An analysis by Harvard’s Center on Education Policy Research found that many low income high school students with strong academic potential did not pursue higher education or enrolled in less-selective colleges or community colleges, where they are much less likely to earn a diploma than students who attend more academically demanding institutions.

A second recent study, conducted by Christopher Avery of Harvard and Caroline Hoxby of Stanford, found that high-achieving, low income students often fail to apply to college because they lack the information and encouragement that more privileged students receive. Few of these students have mentors or role models who have successfully pursued higher education. Most attend high schools with little or no college advising.

At BTW, we work with foundations and other clients that are deeply invested in educational opportunity, and we are part of a national conversation about how to help under-represented students reach college and succeed there. There are some positive trends and signs of change, but we need more programs to help low-income high school students make well-informed college choices and more resources to help those students persist and graduate.

The three young women from Galveston all faced a variety of academic, social and personal challenges— no single strategy or intervention would have helped all of them to succeed. But something bigger is broken. When poor students don’t receive the support, advice or academic foundation they need, educational opportunity is a false promise. In a world where achievement is the gateway to social mobility, we must find ways for our educational system to tear down the barriers to equality instead of fortifying them.