UK insurance firmAdmiral had intended to launch an app this week offering discounted car insurance premiums to first time drivers based on an algorithmic assessment of their Facebook posts.
All drivers would have had to do is sign in with their Facebook logintogrant permission to the company to scan their Facebook posts in order to get a potential discount on their car insurance premiums.
However the experimenthas fallen foul of Facebooks platform policy, which puts strict limits on how developers on theplatform can use the information usersshare with them.
Clause 3.15 of thepolicy also specifically prohibitsuse of data obtained from Facebook to
make decisions about eligibility, including whether to approve or reject an application or how much interest to charge on a loan.
In an interview with The Guardianabout the opt-in firstcarquote app, project lead Dan Mines described itas a test, saying:We are doing our best to build a product that allows young people to identify themselves as safe drivers This is innovative, it is the first time anyone has done this.
The algorithms that Admiralhad developed for the app apparently aimed togleanpersonality traitsfromusers Facebook posts by analyzing how posts were written with individuals whoscored well forqualities such as conscientiousness and organization more likely to be offered discountsvs those who cameacross as overconfident/less well organized, as judged by their Facebook postings.
Photos were not intended to beused to assess drivers the analysis was purely based on text updates to Facebook.
Our analysis is not based on any one specific model, but rather on thousands of different combinations of likes, words and phrases and is constantly changing with new evidence that we obtain from the data, Yossi Borenstein, the principal data scientist on the project, told the paper. As such our calculations reflect how drivers generally behave on social media, and how predictive that is, as opposed to fixed assumptions about what a safe driver may look like.
Giving a more specificexample of how Admirals appwould beassessing a Facebook users attitudebehind the wheel, The Guardian suggested overuse of exclamation marks in Facebook posts might count against a first time driver, whileposting lists and writing in short, concrete sentences containingspecific detail would be seen as a plus.
Admiral said no price rises would be incurred as a result of using the app but discounts of up to 350 were set to be offered although the company was also not ruling out expanding the projectin future to loop in additional social media services and, potentially, to also increase premiums for some drivers.
The future is unknown, said Mines. We dont know if people are prepared to share their data. If we find people arent sharing their data, then we wont ever get to consider that [expanding firstcarquote].
As it turns out, the apps future is unknown as Facebook is not prepared to share user data with Admiral for this eligibility assessmentuse-case. Which, if the team had readFacebooks platform policy, should have been immediately clear.
PresumablyAdmiral has beenworking on the app for multiplemonths at the very least. Yetagain, any Facebook platform developer should be aware that all apps are subject tofinal review by the company before they can golive to ensure compliance with itsplatformpolicy. Even test apps.
Admiral now says the firstcarquotelaunch has been delayed noting on the website that: We were really hoping to have our sparkling new product ready for you, but theres a hitch: we still have to sort a few final details.
It also touts other use cases for the app such as being able to see what some other new drivers have paid forcar insurance and somedetails of the cars they drive. Although thats a far cry from offering first time drivers discounts based on how many exclamations marks they typically deployin their Facebook posts.
We tried to contact the company with questions but atthe time of writing Admiral had not responded, and its press office had professed itself too busy to speak withan outside PR firm being engaged to fence queries. Well update this story with any response.
In a statement provided to TechCrunch a Facebook spokesperson confirmedAdmiral will only be able to use Facebook accounts for login and identity verification so not for scanning post data. The companyfurther suggests the insurerintends to rework the app to createan alternative data source toassess drivers eligibility.
The Facebook spokesperson said:
We have clear guidelines that prevent information being obtained from Facebook from being used to make decisions about eligibility.
We have made sure anyone using this app is protected by our guidelines and that no Facebook user data is used to assess their eligibility. Facebook accounts will only be used for login and verification purposes.
Our understanding is that Admiral will then ask users who sign up to answer questions which will be used to assess their eligibility.
Its worthnoting that Facebook has itself patented using social graph for assessing eligibility of creditworthiness, as the Atlantic reported last year.
US patent 9,100,400, granted to Facebook in August 2015, includes a specific method for authenticating an individual for access to information or service based on that individuals social network with oneof the examples given using the scenario of aservice provider beinga lender who assesses an individuals creditworthiness based on the average credit rating of the people the individual is connected to on their social network
In a fourth embodiment of the invention, the service provider is a lender. When an individual applies for a loan, the lender examines the credit ratings of members of the individuals social network who are connected to the individual through authorized nodes. If the average credit rating of these members is at least a minimum credit score, the lender continues to process the loan application. Otherwise, the loan application is rejected.
Its unclear whether Facebook intends or intended to launch any such creditworthiness assessment service itself we asked and it did not respond. But many patents are filed defensively and/or speculatively. And,as the Atlantic notes,using a persons social graphto assess creditworthiness wouldrun huge risks ofattracting discrimination lawsuits. So the patent does not really read like a serious product proposal on Facebooks part.
Beyond that, if Facebooks platform were to become implicated in weightyexternalassessments of individuals, with the potential to have seriouslynegative impacts on their lives, the company would risk discouraging users fromsharing the sort ofpersonaldata its ad-targetingbusiness model relies on. Which is surely part of the reason its denyingAdmiralthe ability to scanFacebook posts to assess driving proficiency.
Facebook is already negatively implicated in state surveillance activity as ahoneypotof datautilized byintelligence and law enforcement agencies. And on privacy grounds, given its own business model relies on profiling users for ad targeting. But stepping into offering formal assessments of individuals creditworthiness, for example, would feel likea massive pivot for the social media giant although the temptation for it to try to unlock more worth fromthe mountain of data it sits on isonly setto grow, givenAIsrising star and growing appetite for data.
In a blog post welcoming Facebook blocking Admiral from scanning users posts, digital rights organization the Open Rights Grouppointsout the underlying biases that canmake any such algorithmic assessments problematic.
There are significant risks in allowing the financial or insurance industry to base assessments on our social media activity, writes Jim Killock. We might be penalised for our posts or denied benefits and discounts because we dont share enough or have interests that mark us out as different and somehow unreliable. Whether intentional or not, algorithms could perpetuate social biases that are based on race, gender, religion or sexuality. Without knowing the criteria for such decisions, how can we appeal against them?
Insurers and financial companies who are beginning to use social media data need engage in a public discussion about the ethics of these practices, which allow a very intense examination of factors that are entirely non-financial, he adds.
Facebooks data is rich, but often ambiguous, may lack context and presents many risks. It is not clear to us that social media information is an appropriate tool for financial decision making.
Asked for hisview on the risks of Facebook itself using its platform to sell assessments on the fitness of its users for accessing other products/services, such as financial products, Killock also told TechCrunch: Rules on profiling and use of data have to ensure that people are not disadvantaged, unfairly judged, or discriminated against. Facebooks data is rich, but often ambiguous, may lack context and presents many risks. It is not clear to us that social media information is an appropriate tool for financial decision making.
Also bloggingaboutAdmirals attempt to turn Facebook data into premium-affectingpersonality assessments, law professor Paul Bernalvoices similar concerns about what he dubs thevery significantrisks of such a system being discriminatory.
Algorithmic analysis, despite the best intentions of those creating the algorithms, are not neutral, but embed the biases and prejudices of those creating and using them, he writes. A very graphic example of this was unearthed recently, when the first international beauty contest judged by algorithms managed to produce remarkably prejudiced results almost all of the winners were white, despite there being no conscious mention of skin colour in the algorithms.
Bernalalso argues that the sort of linguistic analysis Admirals app was apparently intending would havevery likely favored Facebook users in command of what might be seen as educated language and make any kind of regional, ethnic or otherwise non-standard use of language put its user at a disadvantage.
The biases concerned could be racial, ethnic, cultural, regional, sexual, sexual-orientation, class-based but they will be present, and they will almost certainly be unfair, he adds.
Bernal goes on to suggest thatFacebookusers develop survival tactics as a short term fix fordefeatingany assessments being made of them based on their social graphs and footprints urging especially youngpeople(who are perhaps currently most at risk of being harmfully judged by their social media activity) tokeep the more creative sides of your social life off Facebook.
He also calls fora push by regulators towards developing a framework foralgorithmic accountability to control the autonomous technologies being increasingly deployedto control us.
Algorithms need to be monitored and tested, their impact assessed, and those who create and use them to be held accountable for that impact, he adds. Insurance is just one example but it is a pertinent one, where the impact is obvious. We need to be very careful here, and not walk blindly into something that has distinct problems.
Algorithmic accountability was alsoflagged as a concern bya UK science and technology parliamentary committeelast month, in a report considering the host of social, ethical and legal questions that arise from growing use of autonomous technologies, and given how quickly machine learning algorithms are being deployed towrangleinsights from data-sets.
The committee recommended the government establishes a standing Commission on Artificial Intelligence aimed at identifying principles to govern the development and application of AI, and to provide advice and encourage public dialogue about automationtechnologies.
While, in the US, arecentWhite House report also consideredthe riskof biases embedded in AI.