Recent political and popular discourses in the UK have drawn upon a range of different concepts and powerful and easily recalled sound bites to describe groups who are disadvantaged and who are portrayed as undeserving. The labelling of disadvantaged groups in negative terms and in order to support punitive policies has a long history and not just in the UK. From the racialized ‘underclass’ discourses popular in the US to the recent discourse around ‘Troubled Families’ in the UK, there is a long tradition of labelling disadvantaged groups in such ways that they are alleged to be poor because of their dysfunctional cultures, anti-social behaviours and destructive family life-styles. Drawing on interviews collected with different generations of deeply disadvantaged families we offer one of the first, empirical, sociological accounts of the problems and troubles that some families can face – over decades and over generations. We use this empirical case study by way of illustrating how these negative discourses successfully pave the way for punitive policy interventions and how they also have implications for how disadvantaged groups are treated and for personal well-being.