Hello Darkness, My Old Friend:

How Streaming Services Could Stop Us Wallowing With Algorithmic Music Recommendation

Words by

To preface this article on music streaming and mental health, I want to first acknowledge that in the period between first completing the research for this piece and its publication Spotify have released a new feature for their platform known as Daily Wellness. These weekly playlists contain a “mix of music and wellness to guide you through your day” and while this is certainly a neat idea that I commend them for implementing, it differs from what I will go on to discuss in its formatting and execution. Critically, it is not a purely music-based tool, including some bite-sized mindfulness podcasts that, to my mind, interrupt the flow of some well personalised feel-good music. While some may well find this useful, listening to the same hackneyed tropes about self-care that I’ve heard repeated ad nauseum makes my eyes roll so hard that my retinas almost detach (see: “activating your purpose”, “creating stress breakthroughs”, “finding your resonant archetype”, etc.).

Though music tends to travel via the collective experience, it remains an immensely personal aspect of our lives that has, for many of us, become inextricable from our memories, aspirations and identities. It forms a crucial part of the individual, acting as a window into how a person views themselves and their position in the world, through expression in music performance and through reflection in music listening. Operating on the same reward pathways as food, drugs and sexual pleasure, music listening is often reported as a form of self-care used to regulate or improve a low mood. 

Music and mood have a long-established and multifaceted relationship where mood can impact music listening decisions and music can express or induce emotional states: ideas which have been harnessed in some successful music therapies. Musical elements like tempo, rhythm, timbre and harmony may be used to either reflect or counteract one’s mood; for example, you may choose slower, more harmonic music to relax when feeling anxious, or choose faster, more upbeat music to maintain a positive mood. These kinds of associations are cross-cultural with a fairly universal understanding of cues like vocal intensity as an indication of anger. There are of course some more culturally specified representations of emotion in music, with minor keys and dissonance being more affiliated with sadness in Western music.

One interaction of particular consequence and complexity is that of sad music and mood – the intersection of which is portrayed in starkly different terms by the scientific community and the mainstream media. The benefits of listening to sad music, containing negative lyrical themes and/or acoustic cues associated with sadness, have been talked about since the days of Aristotle, who suggested it could be overwhelming to a point of catharsis. Since then, those ideas have propagated to a point where societies broadly accept the curative properties of wallowing in one’s own sadness through music and portray the idea unchallenged in mainstream film and television. Take, for instance, the 2001 hit Bridget Jones’s Diary which contains the well-remembered and well-loved scene where the protagonist sing-sobs along to Celine Dion’s “All by Myself” while sinking glasses of red wine. Although done with comedic intention, tropes like this go to show that using sad music is a culturally entrenched and commonly understood method of trying to reduce distress or anger. This is mirrored in evidence that finds mood regulation as one of the primary motives behind listening to sad music.

However, recent data suggests that following these media tropes too stringently can be counterproductive, leading to a deterioration in listener mood. Though plenty of reports have shown that sad music is a legitimate and useful tool for cognitively processing negative emotions and safely exploring deeper feelings, findings regarding mood regulation specifically have been much less consistent. Where the evidence appears more clear-cut is when looking outside of the neurotypical population and focusing on those who struggle with depression or tend towards rumination. These overlapping circles on a Venn diagram show the strongest and most consistently negative impacts on their mood from listening to sad music. This is thought to be the result of negative attentional biases – interpreting, for instance, neutral facial expressions more negatively than neurotypicals – as well as a lower motivation to engage in mood enhancing behaviours. Such a combination results in a propensity to interpret even emotionally neutral music more negatively, whilst also engaging with this mood-congruent music more than the neurotypical population. This heightened sensitivity to, and overconsumption of, negativity in music has resulted in many parallel findings that sad music worsens feelings of dysphoria and social isolation in people with unhealthy thinking patterns. As the late, great Ron Taylor put it in his role as Bleeding Gums Murphy in The Simpsons, “[playing] the blues isn’t about feeling better, it’s about making other people feel worse.” 

Though little investigation has yet been done concerning the long-term mood and mental health impacts of listening to more upbeat music, there is evidence to suggest that depressed individuals and ruminators are also atypically sensitive to the short-term mood enhancing effects of happier music. This information is valuable because even momentary distractions from depressive episodes can be of paramount importance to wellbeing, providing some respite from feelings of emptiness and helplessness. Attempts to instil a more conscious approach to music listening in adolescents accessing mental health care have already found decreases in psychological distress along with improved feelings of personal agency and hopefulness in their participants. The most successful and long-lasting mood improvements in music therapies appear to follow a tenet known as the iso-principle – a process in which music selections begin as mood-congruent (i.e. negative in mood) and gradually become more positive over the listening duration by controlling elements such as the pitch and tempo. This principle is notably absent from Spotify’s Daily Wellness playlists.

Assimilating these ideas into a tool for effective music-based mood regulation appears viable now more than ever, given the increasingly impressive algorithms platforms like Spotify and Soundcloud are able to generate music recommendations with. An automated, personalised form of iso-principle music therapy could provide listeners an easy means of passive mood regulation. This also fits well with both the demographics of streaming service users and the epidemiology of depression sufferers, with each of these groups skewing towards adolescence and young adulthood.

As listeners or viewers, we often arrive on entertainment platforms with no specific desire for our media consumption, which is shown in the fact that 70% of YouTube clicks from the homepage are onto recommended videos and 30% of Spotify listening is recommended tracks. These statistics will likely only increase further as more data is collected and algorithms improve, allowing a recommender system like Discover Weekly to understand your taste in music on an even more fundamental level. Personalising media consumption to this degree began primarily as a response to the vast overload of online content that continues to grow every day on the myriad of streaming, journalistic and commercial platforms that are now available to us. This has required the development of ever more efficient pathways to accurate preference prediction and relevant content discovery that can effectively facilitate mood regulation.

Recommender systems have justifiably received significant flak in recent years, with a lot of excellent investigative reporting that has shown the tendency of Youtube algorithms to lead viewers of political content down rabbit holes of increasingly extreme ideas. While Spotify or Soundcloud may have less chance of radicalising your grandma, there is certainly the potential for reinforcement of destructive behaviours through the consistent recommendation of negatively affecting music. To borrow a metaphor from Tim Quirk, Head of Global Content Programming at Google, the role of music platform designers is now that of a park ranger who must tend to a vast musical landscape and ensure their visitors enjoy their excursions by promoting healthier interactions with the elements of that landscape. As the flora and fauna continue to grow in quantity, the park rangers must provide some way of navigating the ever-expanding terrain, not by suggesting their own preferred paths, but by providing options that will allow the visitor to maximise the benefits they gain from their experience in the park. In practice, this guiding role is of greatest importance to the more vulnerable among us – depressed individuals, and those at a higher risk of diagnosis, who are significantly more sensitive to negative emotional values in music, leading to mood dysregulation and the potential exacerbation of pathological symptoms. Therefore, using the most advanced technologies available, the powerful emotional force of music could be harnessed to steer these listeners towards music habits that promote recovery from depressive disorders, rather than passively enabling damaging behaviours by only recommending more sad songs.

Modern recommender systems used by music streaming platforms tend to combine three separate mechanisms for their track suggestions. The first, collaborative filtering, uses data such as streaming numbers, additions to user playlists, and visits to an artist’s discography to create a taste profile for the listener.  This data is then used to locate other users with similar taste profiles and suggest content based on what one individual has enjoyed, but what the other has not yet come across. The second technique involves analysis of the raw audio files in order to extract the musical features of a track and provide recommendations on the basis of similarities between the musical elements like tempo, loudness, key and time signature. For instance, the system could recognise if you’ve been listening to lots of music containing drones and suggest previously unheard tracks that also use them. The third and final pillar of modern music recommendation is known as latent semantic analysis which collates written data from all over the internet (forums, reviews, etc.) to provide a language-based representation of each track or artist. The most frequently used terms written in reference to a track or artist are then used to locate others that are defined similarly, providing a broader cultural context than the audio alone. 

While those mechanisms provide a reasonable approximation of our general musical preferences, they assume that such preferences to remain static and immutable regardless of any contextual factors. An ideal recommender system might, for instance, recommend more energetic, high tempo music to an athlete training at the gym but present them mellow and acoustic sounds late at night in their own home. Without an understanding of context, the system may provide recommendations that aren’t context appropriate, such as speedcore over breakfast, or ska at any time of day. Along with factors of place, time, weather and motives for listening, recommender systems could be modelling the user’s emotional state to further improve their predictions and enable successful mood regulation. Passive collection of context data is now easier than ever before, considering that a large proportion of media consumption occurs on mobile devices which regularly track information such as time, GPS location, weather, level of activity, and even traffic conditions. Smartphone sensors have already begun to show promise in other domains such as tourism, where location tracking has been harnessed to improve recommender precision. You may have already noticed that TripAdvisor can now occasionally send you suggestions for where to eat and explore when you arrive in a new city, despite you never actively notifying it of your travel plans.

Mood appears much less straightforward to capture in comparison, given that current smartphones lack any kind of inbuilt mood tracker. However, this may not be the case for much longer, as recent developments in machine learning indicate that it is possible to identify mood indirectly using algorithms that pick up on facial micro-expressions or features of our speech patterns that are indicative of an emotional state. In the future, these techniques could be harnessed by integrating mood recognition technology into smartphone cameras and digital assistants (Siri, Alexa, etc.), enabling continuous and individualised data collection for both implicit user mood and emotional response to musical cues during streaming. There have already been reports of companies like Huawei and Apple developing assistants that are capable of mood recognition, so the possibility of applying this data to music recommendations from streaming services appears to be on the horizon. 

The elephant in the room here is of course the invasiveness of measures like these and the ability it could provide massive multinational corporations to track and potentially exploit data pertaining to mental health. It’s creepy enough having TripAdvisor subtly hinting that it might know where you are or where you’ll be at any given moment, but personal health feels considerably more private and has already begun to be capitalised on for targeted advertising. The tin foil hat I sometimes don has led me to the distinct possibility of companies using music to manipulate our emotions and behaviour in other, more underhanded ways. Then I remember that McDonald’s has already veering into this territory for years, using classical music as a sonic weapon against anti-social behaviour. So, this leaves us at a strange crossroads where, should this become a reality, we would each have to weigh up our approach to data privacy and whether mood tracking is a step too far, leaving us with a few different paths:

  1. Adopt a scorched earth policy, scraping every last trace of yourself from the internet so that Mark Fuckerberg and his goons will never find your new home in the woods, away from prying eyes.
  2. Slide into apathy and surrender your entire digital identity to any and all takers – upload your soul to the cloud.
  3. Recognise that many of the intimate details of our lives have already been recorded and sold on, deciding on a case by case basis whether exposing another side of yourself to a faceless corporation will lead to a net gain in quality of life.

Personally, as someone who often struggles to articulate or even understand how they’re feeling, any mechanism for streamlining that process of emotional awareness would be welcomed with open arms. 

Regardless, the tools are now available for major music streaming services to provide highly accurate mood-based track suggestions. These recommendations could then be formatted in accordance with the iso-principle, slowly shifting from negative to positive emotional values, thus providing a gentle push towards mood enhancing behaviour that could provide longer sustaining improvements which still allow for enjoyment and cognitive benefits to initially be obtained from sad music. Offering recommendation of this type as an option on streaming platforms bypasses the issue of decreased motivation and could provide well individualised healthcare on a large scale through mainstream platforms like Spotify and Soundcloud. This, in turn, could allow for better short-term mood levels, if not long-term mental health outcomes, for those who struggle with mood dysregulation. 

None of this is meant to try and invalidate the importance of engaging with art that tackles upsetting or challenging ideas, as that is of similar significance to emotional development and the processing of hardships that we encounter or see in the experiences of others. As much as we like to mock the histrionics of an emo phase, it feels like a rite of passage for a reason. And as someone who recently unearthed an folder of music downloads from 2012, featuring such gems as “Avenged sevenfold- Nightmare(clean 1080p hd).mp3” and “Black Veil Brides New Religion Lyrics(1).mp3”, an emo phase renaissance may not be entirely off the cards. I would also never want to try and deny people the joys of an occasional sulk – to quote James Acaster, “I don’t know what chemical your brain releases when you have a sulk, but if they sold that chemical, I would rub it on my gums.” The point here is that there is a fine line between reflection and rumination that is not always completely clear until we scrutinise the content that we consume on a daily basis. And for those of us who sometimes stray too far into the latter pattern of behaviour, a simple tweak to a recommendation system has the potential to considerably improve outcomes regarding mood regulation and mental health.

If you would like to read a longer, fully referenced version of this article that explains in more detail how recommendation systems work and how they draw on the organisation of knowledge in the human brain to successfully generate predictions, the link is below.

PDF: Harnessing Music Recommendation Systems to Improve Mood Regulation Outcomes in Depressed Individuals