Apple Card’s Algorithm Problem

David Heinemeier Hansson (video):

Jamie Heinemeier Hansson:

I care about transparency and fairness. It’s why I was deeply annoyed to be told by AppleCard representatives, “It’s just the algorithm,” and “It’s just your credit score.” I have had credit in the US far longer than David. I have never had a single late payment. I do not have any debts. David and I share all financial accounts, and my very good credit score is higher than David’s. […] But AppleCard representatives did not want to hear any of this. I was given no explanation. No way to make my case.

[…]

I care about justice for all. It’s why, when the AppleCard manager told me she was aware of David’s tweets and that my credit limit would be raised to meet his, without any real explanation, I felt the weight and guilt of my ridiculous privilege.

David Heinemeier Hansson:

Sridhar Natarajan and Shahien Nasiripour:

A Wall Street regulator is opening a probe into Goldman Sachs Group Inc.’s credit card practices after a viral tweet from a tech entrepreneur alleged gender discrimination in the new Apple Card’s algorithms when determining credit limits.

Sonder Scheme:

We suspect that the Goldman algorithm was trained on data that included an important bias: that the husband is the primary card holder in traditional credit card approval. This biased the data so the algorithm assigned higher creditworthiness to the primary card holder. This meant that the primary card holder status became the proxy for gender.

The whole situation was made worse by a number of applications coming from a demographic or group that exposed this bias, an AI-enabled product which broke the mental model of Apple family sharing and a total lack of a “human-in-the-loop” recovery combined with unexplainable and non-intuitive AI.

David Heinemeier Hansson:

Steve Wozniak:

Daniel Vassallo:

David Heinemeier Hansson:

David Heinemeier Hansson:

Overall, Apple Card is not having a good week with this going viral. Also, sorry about all the twitter embeds, just shows the picture better that way.

Roger Stringer spends most of his time solving problems for people, and otherwise occupying himself with being a dad, cooking, speaking, learning, writing, reading, and the overall pursuit of life. He lives in Penticton, British Columbia, Canada