Intelligence
estimation from social network profile images
Xingjie Wei
xw323@cam.ac.uk
David Stillwell
ds617@cam.ac.uk
ABSTRACT
Profile pictures on social networks are users opportunity to
present themselves and to affect how others judge them. In
most social networks, profile pictures are public by default.
1,122 Facebook users completed a matrices intelligence test
and shared their current Facebook profile picture. Strangers
also rated the images for perceived intelligence. We use automatically extracted features from profile pictures to predict both measured and perceived intelligence. Intelligence
estimation from images is a difficult task even for humans,
but experimental results show that human accuracy can be
equalled using computing methods. We report the image
features that predict both measured and perceived intelligence, and highlight misleading features such as smiling
and wearing glasses that are correlated with perceived but
not measured intelligence. Our results give insights into inaccurate stereotyping from profile pictures and also have implications for privacy.
Keywords
Intelligence quotient, Measured intelligence, Perceived intelligence, Intelligence estimation, Computational aesthetics
1.
INTRODUCTION
ACM ISBN .
DOI: 1
Measured
intelligence (MI)
User
Feature extraction
by computers
Estimated
MI
IQ test
Correlation
Computer
Estimated
PI
?
Perceived
intelligence (PI)
Feature extraction
by humans
Feature analysis
Human rater
2.5
2
1.5
1
0.5
0
-0.5
-1
-1.5
-2
1
10
11
12
13
14
15
16
17
18
19
20
Image index
2.
PROPOSED FRAMEWORK
Colour
Composition
Texture
Local
Name
HSV statistics
Emotion-based
Colourfulness
Colour name
Dark channel
Colour sensitivity
Edge pixels
Regions
Symmetry
Entropy
Sharpness
Wavelet
Tamura
GLCM
GIST
Body
Skin
Face
Glasses
Pose
Colour histogram
LBP
GIST
SIFT
Len.
5
3
1
11
1
1
1
2
2
1
4
12
3
12
24
2
1
4
2
3
512
944
512
2048
Description
Circular variance of H channel, average of S, V (use of light), standard deviation of S, V
Valence, Arousal and Dominance in V and S channels
Colour diversity
% of black, blue, brown, grey, orange, pink, purple, red, white and yellow pixels
The minimum filter output on RGB channel, reflects image clarity, saturation and hue
The peak of a weighted colour histogram representing the sensitivity with respect to human eye
% of edge pixels to present the structure of an image
Number of regions, average size of regions
Horizontal symmetry and vertical symmetry
Gray distribution entropy
The average, variance, minimal and maximal value of sharpness
Wavelet textures (spatial smoothness/graininess) in 3 levels on each HSV channel
Coarseness, contrast and directionality of texture
Contrast, correlation, energy, homogeneousness for each HSV channel
Low dimensional representation of a scene, extracting from a whole image
The presence of body* and the proportion of the main body
% of skin pixels
# of faces*, the proportion of main face, the horizontal and vertical locations of main face
The presence of normal glasses* or sunglasses*
The pitch angle, roll angle and yaw angle of head
Histogram of colour from local blocks
u2 ) from local blocks
Local Binary Pattern (LBPi,2
GIST features extracting from local blocks
Dense SIFT features from local blocks
with manual check to make sure the automatic detection results are correct
as testing sample and the remaining images are the training samples on which regression models are trained. The
selected dimension K in section 2.3.2 is set as N/2 where N
is the number of training samples.
Len.
95
512
944
512
2048
p < 0.05
MI
34.3%
8.0%
15.4%
9.0%
7.4%
ratio
PI
56.3%
11.9%
44.6%
43.2%
11.7%
Yaw angle
Roll angle
Pitch angle
Glasses
Local
Sunglasses
Smile degree
Face centre Y
Face centre X
Face proportion
# faces
Skin proportion
Body proportion
Body
Vertical sym.
Horizontal sym.
# regions
Edge pixels
Colour sensitivity
Dark channel
Yellow pixels
White pixels
Red pixels
Purple pixels
Pink pixels
Orange pixels
Composition
Green pixels
Grey pixels
Brown pixels
Blue pixels
Black pixels
Colourfulness
Dominace
Arousal
Valence
Std. V
Std. S
Avg. S
H circular variance
Colour
Entropy
Avg. sharp.
Var. sharp.
Min. sharp.
Max. sharp
Wavelet H lev. 1
Wavelet H lev. 2
Wavelet H lev. 3
Wavelet S lev. 1
Wavelet S lev. 2
Wavelet S lev. 3
Wavelet V lev. 1
Wavelet V lev. 2
Wavelet V lev. 3
Wavelet H sum
Wavelet S sum
Wavelet V sum
Tamura coarseness
Tamura contrast
Tamura directionality
GLCM contrast H
GLCM correlation H
GLCM energy H
GLCM homogeneity H
GLCM contrast S
GLCM correlation S
GLCM energy S
GLCM homogeneity S
GLCM contrast V
GLCM correlation V
GLCM energy V
GLCM homogeneity V
GIST channel 1
GIST channel 2
GIST channel 3
GIST channel 4
GIST channel 5
GIST channel 6
GIST channel 7
GIST channel 8
GIST channel 9
GIST channel 10
GIST channel 11
GIST channel 12
GIST channel 13
GIST channel 14
GIST channel 15
GIST channel 16
GIST channel 17
GIST channel 18
GIST channel 19
GIST channel 20
GIST channel 21
GIST channel 22
GIST channel 23
GIST channel 24
0.2
0.15
0.1
0.05
0
-0.05
-0.1
-0.15
-0.2
0.2
0.15
0.1
0.05
0
-0.05
-0.1
-0.15
-0.2
Measured intelligence
Perceived intelligence
Texture
Figure 3: Correlations between image features and MI (orange bars) or PI (blue bars). Darker bars indicate correlations
which are statistically significant (p < 0.05).
Table 3: Estimation results
MI
Human
Computer
Baseline
PI
Computer
Baseline
Spearman
RMSE
NRMSE
0.24
0.27
14.50
14.49
0.20
0.20
0.36
0.54
0.56
0.15
0.16
: p < 0.001
4. CONCLUSION
We have proposed a framework of intelligence estimation
from profile images both by humans and computers. Intelligence estimation is a difficult task even for humans, but experimental results show that it is possible to equal humans
accuracy using computing methods, while also having the
potential to reduce biased judgements.
Our results found that people who are both measured and
perceived as intelligent avoid the colour pink, purple and
red in their profile images, and their images are usually less
diversified in colour, more clear in texture, and contain less
skin area. Besides that, intelligent people also like use the
colour green, and have fewer faces in their images, but this
does not affect how others judge them. Essentially, most
intelligent people in our dataset understand that a profile
picture is most effective with a single person, captured in
focus, and with an uncluttered background.
The following cues are inaccurate stereotypes - correlated
with perceptions of intelligence but not measured intelligence. Profile images containing more grey and white, but
less brown and green, with higher chromatic purity, smiling
and wearing glasses, and faces at a proper distance from the
camera, make people look intelligent no matter how smart
they really are.
Profile pictures are a ubiquitous way for users to present
themselves on social networks, and typically they are not
considered to be private data. Our results show that humans use inaccurate stereotypes to make biased judgements
about the perceived intelligence of the person who uses a
profile picture, which raises considerable concerns about the
common practice for hiring managers to search candidates
online before inviting them to interview. Our results also
indicate that the choices that users make of how to present
themselves in their profile picture are reflective of their measured intelligence, and that computer algorithms can extract
features and automatically make predictions.
5.
REFERENCES