By Dr Carol Graham

America can appear to be sharply divided by politics, incomes and even life expectancy. But why are poor black Americans so much more optimistic than their white counterparts?

America has seen a dramatic increase in the number of so-called “deaths of despair”.

Caused by opioid addiction, alcohol or drug overdose and suicide, these deaths have hit middle-aged white people without a college education particularly hard.

The trend is extensive enough to have driven up the overall mortality rate, with the US in the unusual position of being a rich country where life expectancy is falling rather than going up.

Amid all this is a perhaps unexpected reality – the people who are most optimistic about their future are the most disadvantaged: poor black Americans, who are even slightly more optimistic than rich black people.

And by poor, we mean a household of four earning less than $24,000 per year, which is effectively the official US poverty line.

Very different outlooks are revealed when black and white people are asked about their wellbeing.

To measure the differences, we use an 11-point scale, where 0 is desperate and 10 is very optimistic.

On this scale, black people are, on average, 1.4 points higher on the optimism scale than white people are. Black people on average score above 8, while the average for white people is 7.

The gap across poor black people and poor white people is even larger, at 1.4 points.

That is a significant gulf – greater than the difference between rich and poor people, which is 0.6 points. The average score for rich people is 7.5, while for poor people it is 6.9.

The starkest disparity is found in the detail that poor black people are almost three times as likely to be a point higher on the optimism scale than poor white people.

And poor black people are half as likely to report experiencing stress the previous day than poor white people.

Desperation, stress and worry are closely linked to a greater risk of premature death, regardless of income or race.

People with low levels of optimism and high levels of stress and worry are more likely to die from deaths of despair, or to live in areas with high levels of such deaths.

The link is strongest for poor white people without a college education, especially those who live in rural areas.

Of course, this relationship can run both ways.

Desperation can lead to premature death, but living in an area where many such deaths occur can itself be a potential cause of lack of hope, stress, and worry.

The link between optimism and longevity is strong.

Among Americans born between 1935 and 1945, those who reported higher levels of optimism as young adults were much more likely to be alive in 2015 than less optimistic people. We found that 86% of pessimists died by 2015, while only 77% of the optimists did.

The explanations for these trends are complex, but economics and ambition play a part.

For many years, blue-collar white people had some advantages over minorities, but they are now among the most disaffected parts of society and the most vulnerable to deaths of despair.

Crucially, they are much more likely to report that their lives are worse than those of their parents and, in terms of stable employment and status, that is often the case.

In contrast, black and Hispanic people are more likely to report that their lives are better than their parents’ lives.

While disadvantage and discrimination still exist, minorities have been making gradual progress in narrowing gaps with white people in terms of their education, wages, and life expectancy.

Some of this is because of concrete gains made by those communities.

But much of the improvement is due to the relative decline in the incomes and status of poor white people – a trend associated with the hollowing-out of blue-collar jobs, which are decreasing in both number and stability at the same time as the market for high-skilled labour continues to prosper and grow.

The health implications of this have been particularly serious.

White people without a college education are much more likely to die prematurely of preventable deaths than any other group.

They also report more pain, stress, and anger than other parts of society.

Another factor appears to be higher levels of resilience among poor minorities.

For example, black and Hispanic people are much less likely than white people to report depression or commit suicide.

Some of this may be due to resilience built up over generations of hardship – as seen in many poor countries.

Informal safety nets may also play a role – such as the community surrounding black churches and extended Hispanic families.

However, this does not mean that poor black people are entirely happy with their situation.

Despite reporting high levels of optimism, this group was much less likely to report feeling safe in their neighbourhoods, or to be satisfied with their financial situations, than their white counterparts.

There are also gaps in what we know about resilience – in particular how optimism varies between different places.

For example, after adjusting for differences such as education and employment, the minority groups with the highest levels of optimism and the lowest levels of stress in the US are in the southern states.

This may seem surprising, as these same places have a historical association with racism and still have extensive poverty and poor health.

Yet they also have a strong sense of black culture and community: factors that are difficult to measure but undoubtedly important.

It is a picture similar to the strong sense of community and culture that helps explain why Latin Americans report higher levels of happiness than those in regions with comparable or higher incomes.

A better understanding of resilience may be as important as a greater insight into the causes of desperation.

It might also be that we could learn a great deal from better monitoring of wellbeing and the health of society.

Other countries, including the UK, regularly track wellbeing, with life satisfaction, smiling, stress, and a sense of purpose in life all included among annual statistics.

Routinely collecting such figures in the US could be a simple first step towards better understanding where things have gone wrong for some parts of society.

And why some people are so much more optimistic than others.