Is AI Bias Really an AI Problem?
Recorded in Chinese
Gender patterns in an image tool I built remind me of a résumé-screening example from my reading. I explore where bias comes from and possible connections among education, people’s environments, and AI ethics.
00:01 I've been reading a book recently,
00:03 and I came across an example in it.
00:06 That example
00:07 reminded me of a problem
00:08 I encountered while building
00:10 an AI tool in the past.
00:12 The problem was this:
00:14 I was using
00:16 an AI platform's API to generate images.
00:20 And I noticed
00:21 that most of the images it generated
00:26 had a masculine style.
00:30 For example, when it generated
00:36 an image of someone eating,
00:41 the person was always male.
00:43 Or in scenes involving sports,
00:48 the elements were also male-coded.
00:52 I found that very strange.
00:56 Why was it doing that?
00:58 To solve the problem,
01:00 I added some constraints to the prompt
01:04 so it would
01:07 generate images according to my needs.
01:11 If gender was mentioned,
01:15 it should generate an image
01:17 related to that gender.
01:19 If there were no clear gendered terms,
01:23 it should take a gender-neutral direction.
01:28 After I added those constraints
01:31 to the code,
01:33 the quality of the images improved considerably.
01:36 But there were still tiny details
01:41 I wasn't entirely satisfied with.
01:43 Yet those details
01:45 were perhaps so minor
01:46 that I could ignore them.
01:48 So I left them alone.
01:50 That experience
01:52 made me wonder:
01:54 how could merely calling an API
01:56 produce such a pattern?
02:04 Later I learned
02:07 that AI training data
02:09 comes from the human world.
02:11 And human-generated data
02:14 contains a great many biases.
02:18 That was also when
02:20 I gained a deeper understanding of AI.
02:24 Often, when I use AI,
02:27 I don't think about
02:29 where AI comes from,
02:30 where its training data comes from,
02:32 or how that training data
02:34 relates to human-generated material.
02:39 It's a little like
02:41 using something every day
02:43 without knowing
02:46 the principles behind it
02:50 or what it involves.
02:53 That was an entry point for me.
02:57 Returning to the book I mentioned,
03:02 I forget whether it was The Coming Wave
03:06 or The Alignment Problem.
03:10 Anyway, the example was about
03:12 between 2014 and 2018,
03:15 Amazon developed
03:19 an AI learning tool.
03:21 It was essentially a résumé scorer.
03:24 First, it took résumés from its database
03:28 from a certain range of years—
03:32 the résumés of candidates,
03:36 probably candidates
03:37 who had eventually been hired—
03:40 and fed them to the AI as training data.
03:43 Then it had the AI
03:46 score newly submitted résumés.
03:51 At first,
03:52 the résumés receiving high scores
03:55 were mostly from men,
03:57 the lower scorers were women.
03:59 After the team discovered this,
04:03 in the backend,
04:05 they added code-level constraints,
04:08 such as removing identifying labels
04:11 and removing gender-related labels,
04:14 then had the AI score them again.
04:17 But later they discovered
04:19 that the AI was rather clever.
04:22 When it encountered gender,
04:24 it no longer simply scored men higher
04:27 and women lower.
04:29 Its explicit gender scoring
04:30 may have become more balanced.
04:33 So where was the problem?
04:35 Men's and women's résumés
04:40 contained subtle differences,
04:43 for example in word choice.
04:44 They weren't quite the same.
04:45 From its original training data,
04:50 it learned that men's résumés
04:53 used more masculine language,
04:55 while words in women's résumés
04:56 tended to sound more feminine.
04:59 After learning this pattern,
05:01 it used it to score résumés again.
05:05 So even though the system
05:07 had been designed to remove gender labels,
05:10 it still couldn't avoid
05:13 gender bias affecting its scores.
05:18 Amazon later
05:19 abandoned the project.
05:23 Recently, haven't we seen
05:26 AI résumé platforms appear?
05:30 I wonder whether they
05:32 have considered this issue,
05:34 and if they have,
05:36 how they plan to solve it.
05:39 After all, even Amazon,
05:41 such a huge platform,
05:43 experienced this kind of failure.
05:46 As for today's technology,
05:49 I honestly don't know
05:51 how advanced it has become
05:52 or whether it can avoid such bias.
05:54 Even within
05:55 the tech and AI industries,
06:00 people have long discussed
06:02 how
06:04 we can reduce bias in AI.
06:09 But in my everyday use,
06:12 I've noticed that
06:14 I have almost never encountered
06:15 AI showing bias.
06:17 Nor have I seen AI
06:19 act in a deeply unethical way,
06:21 or anything like that.
06:23 So I'm very curious:
06:24 when AI shows that side,
06:27 what context
06:29 or what situation?
06:31 Sometimes I wonder
06:34 what kind of person,
06:35 using what approach
06:36 and what kind of
06:38 interaction,
06:40 or exchange
06:41 could force AI to reveal that side.
06:45 I'm very curious about that.
06:49 That also reminds me—
06:51 I recently wrote an article.
06:55 It was
06:58 a kind of submission.
07:00 A blogger from overseas
07:02 was looking for a researcher
07:05 and posed four questions.
07:08 He wanted applicants
07:13 to choose one of the four,
07:17 write
07:18 their view on that question,
07:21 or propose a solution.
07:23 I chose one of them.
07:26 It was roughly about—
07:30 I won't go into the details,
07:32 because the results aren't out yet.
07:36 Although I know there won't be
07:37 any major result,
07:39 I still want to keep it private for now.
07:41 But in that article,
07:44 I wrote that
07:46 to solve this enormous problem,
07:49 we shouldn't aim at AI itself
07:53 or artificial intelligence itself,
07:56 but at people
07:57 and the environments around them.
08:01 How do we change people?
08:03 How do we change their environments?
08:08 Both questions
08:11 point to what we often call education.
08:15 If education can shape someone well
08:18 and improve the environment
08:22 around that person,
08:23 then their
08:25 developmental path,
08:27 and who they grow into,
08:30 may become more ideal.
08:33 So I think,
08:35 many people call for AI ethics
08:38 and related regulation and limits.
08:41 Although,
08:44 that work certainly should be done,
08:46 I sometimes feel—how should I put it?
08:50 Commercial interests are—
08:54 they're like a mountain
08:56 that no one can move.
08:57 Especially given the position of
09:00 those commercial actors in tech,
09:03 and in international—
09:06 uh,
09:08 let me mute that part.
09:09 I mean international interests.
09:11 Those interests cannot be abandoned.
09:15 So we can only,
09:17 to use an idiom,
09:19 take an indirect path to save the nation.
09:25 The question I chose
09:27 asked how the OpenAI Foundation should
09:32 spend the portion of its funds
09:37 that had grown
09:39 with OpenAI's rising valuation.
09:46 My idea was
09:48 to invest those funds in education too.
09:52 But how should the money be allocated?
09:57 How should it enter the education system?
10:00 Those questions require thought.
10:01 They are the more concrete points
10:03 I discussed in the article.
10:07 Later,
10:10 after writing the article,
10:11 and submitting it,
10:12 I read another book.
10:16 Again, I forget whether it was
10:18 The Coming Wave or The Alignment Problem.
10:21 It also said
10:23 that shaping people and their environments
10:28 is the root issue in solving some problems—
10:33 the fundamental issue.
10:35 The author's view
10:37 may have resembled my own feeling:
10:39 when we encounter a problem,
10:42 we habitually
10:44 jump straight into the problem itself
10:47 instead of
10:49 asking where it came from.
10:51 So we identify its root cause
10:54 and start solving it at the source.
10:59 I also sometimes think
11:01 that solving the immediate problem
11:03 really is the most effective approach.
11:07 But that brings us to another phrase:
11:11 treating symptoms, not the root cause.
11:13 So I think
11:16 first-principles thinking is very useful.
11:18 We should always find the source.
11:21 If we design solutions from the source,
11:25 they're often most effective.
11:27 Although the—
11:29 mm,
11:31 results will come very slowly,
11:33 I still think
11:36 first-principles thinking
11:40 is extraordinarily useful and effective.