When I first saw the article, ‘Singaporeans among Asia’s happiest, says social media survey’ in the Today newspaper, I did not know which part of the article to pick on, because every part could be easily refuted. I can just imagine how Amanda Lee must have felt when she was told that she had to write this article based on a survey that is so weak in its basis.
So, according to Today, “a survey based on what people are saying on social media claims that Singaporeans are among Asia’s happiest people”. Today also reported that, “The survey is conducted by Eden Strategy Institute (ESI), a consultancy on social innovation.”
It was also reported that when “asked how this survey gels with previous ones, such as the Gallup poll, Mr Chu said he felt that it was “more scientific” and “looks at the drivers of happiness on a much wider population”.”
As a reminder, the Gallup survey, “measured positive emotions in 148 countries and areas in 2011 using five questions. These questions ask people whether they experienced a lot of enjoyment the day before the survey and whether they felt respected, well-rested, laughed and smiled a lot, and did or learned something interesting … Singaporeans, Armenians, and Iraqis are least likely worldwide to report feeling positive emotions.”
In contrast, the Asia Happiness Index was conducted “in five countries, namely Singapore, Malaysia, the Philippines, India and Indonesia. It calculates the Happiness Index score by taking into account the social media population in each country and the number of hits generated by a search engine when the researchers key in the predetermined words.”
So, according to Mr Chu, the Asia Happiness Index is “more scientific” and “looks at the drivers of happiness on a much wider population”. Let’s look at a quick comparison between the two surveys.
What Is The Methodology?
The Gallup survey was actually done in 148 countries whereas the Asia Happiness Index is done only in 5 countries.
Also, if you look at the Gallup survey’s description of its methodology, it says that, “Results are based on telephone and face-to-face interviews with 1,000 adults, aged 15 and older, conducted in 2011 in 148 countries and areas. For results based on the total sample of national adults, one can say with 95% confidence that the maximum margin of sampling error ranged from ±3.4 percentage points to ±3.9 percentage points. The margin of error reflects the influence of data weighting. In addition to sampling error, question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of public opinion polls.”
ESI claims on their website that, “The Asia Happiness Index is backed by scientifically-reviewed principles of true happiness, powered by an online intelligence tracking engine, and comprehensively covers over 200 million social media accounts in our initial set of five countries.” In their press release, they took great pains to discredit the methodologies of the other survey instruments by saying that, “Traditionally, Happiness studies have relied on the self-reporting method, yet the accuracy of self-reporting methods is known to be limited by memory bias, central tendency bias, acquiescence bias, and social desirability bias. It can further be misrepresented by the way survey questions are framed by the researchers.” Yet I wasn’t able to find out further how ESI had developed the index’s methodology or what their methodology was, other than what they had stated that the index “is backed by scientifically-reviewed principles of true happiness, powered by an online intelligence tracking engine, and comprehensively covers over 200 million social media accounts in our initial set of five countries.” This doesn’t say much.
On top of that, for the Gallup survey, you are able to download a “more complete methodology and specific survey dates” at the Gallup’s Country Data Set details.
Who Is This Company – Eden Strategy Institute?
Finally, and most importantly, do you know that the Eden Strategy Institute is actually based in Singapore. You can see their address here.
So, what we know by now is that a Singapore-based company has created a happiness index which they surveyed in only 5 countries where they ranked Singapore as the happiest country in Asia. In contrast, Gallup which is based in the United States surveyed 148 countries and ranked the United States down at 35th on the list.
- First, only 5 countries were surveyed in this index and they are all in the South and Southeast Asian region.
- Second, the countries selected are of different socio-economic strata. It is hardly possible to have an accurate comparison as to how the socio-economic status of the country has an effect on the index scores.
- Third, the methodology doesn’t describe or compare the usage patterns of the different populations in these countries – how do their patterns of usage affect their terminology use?
- Fourth, does the methodology also discern the amount of usage and population size of the country to distill a more representative picture of the countries?
Finally, does ESI understand what the implications of using a broad survey of online social media tools are? According to a study conducted by The University of Gothenburg in Sweden which “surveyed 1011 Swedish Facebook users with our questionnaires, measuring respondents’ Facebook usage patterns, well-being and self-esteem,” the researchers found that “people on Facebook are likely to only report the events that are worthy of reporting. If everyone would report only those events that are worthy of reporting, the end result would be an illusion of people in general being more happy and successful than may be the case. When we then compare our own lives with others’ seemingly more successful careers and happy relationships, we may feel that our own lives are less successful in comparison.”
The study also says that, “because Facebook users often don’t have access to a full range of information about a person, they may erroneously infer that if a person is happy on virtually every picture, this person must be a very happy person … The correspondence bias is closely related to the fundamental attribution error – our tendency to ascribe a certain trait (e.g. happiness) to an individuals’ personality (happy kind of person), rather than to external circumstances (the party caused the happiness). Thus, on Facebook, it is easy to disregard that nearly all pictures are taken under happy circumstances (parties, vacations, interaction with friends and family) and thus erroneously conclude that other people are more happy than they may be … If Facebook users overestimate the happiness of their peers (which seem to be high and consistent), they may fall into the trap of comparing their own happiness (which consists of highs and lows and is constantly ﬂuctuating) and consequently end up feeling less happy.” You can read more about the study here.
Does ESI know this? ESI’s Mr Chu had said that, ““Singaporeans love to send funny pictures or pictures of food, as a means of reaching out and sharing.” Well, there you have it, Mr Chu. Are Singaporeans really that happy? Your survey tool and methodology is hardly the scientific that you call it that it can adequately measure the happiness of Singaporeans. As to claim that other happiness studies are “limited by memory bias, central tendency bias, acquiescence bias, and social desirability bias,” ESI’s very own Asia Happiness Index are mired in these same biases as well. Now that we have concluded that ESI’s remarks are simply a matter of the pot calling the kettle black, if there are at least 3 different happiness studies which rank Singapore much lowly on our levels of happiness and each of these studies have a much wider country comparison, surely that should come to something?
The Gallup survey in 2012 ranked Singapore last out of 148 countries. The Happy Planet Index in 2012 ranks Singapore 90th out of 151 countries. The World Happiness Report in 2012 ranks Singapore 33rd out of more than 150 countries.
And in two of these (Gallup and the Happy Planet Index), Malaysia, Philippines, Indonesia and India were all ranked higher than Singapore. Let me give ESI a tip – in future, if you want to come out with a new happiness tool, even if it’s just for show, at least come out with a stronger methodology or make it much harder to dissect. The Ministry of Finance does a fantastic job at that, so you should learn from them.
If you remember, this smacks of what happened when the Lee Kuan Yew School of Public Policy (Asia Competitiveness Institute) developed a new ranking of world cities last year – the Global Liveable Cities Index – which ranked Singapore 3rd, as compared to all other indexes which Singapore wasn’t even ranked in the top 10 in. I had previously written an article about this. You can read it here.
Clearly, the Singapore government is desperate to counter any surveys which present Singapore in a negative light. But instead of ensuring that the concerns of the people are met, so that this will reflect more positively in the surveys, the government’s strategy is to release their own surveys which rank Singapore highly to counter the countless surveys that are already in existence!
Wouldn’t it be more sustainable, and actually logical, that the government look into why Singaporeans are unhappy and find ways to promote the happiness within them, rather than to push out a survey to lie to the people, hoping that this PR move would sufficiently placate the people? Singaporeans are unhappy. Period. No survey which marks up how happy Singaporeans are will be able to pretend that away.
So, instead of fulminate the truth, perhaps the government needs to have the boldness to start looking at these issues with immediate attention to resolve them, well, with resolve.