13_Rafayel_Barseghyan.DVI


Mathematical Problems of Computer Science 39, 101{105, 2013.

Sear ching I mages by Using Color Str uctur e

R a fa ye l V . B a r s e g h ya n

Institute for Informatics and Automation Problems of NAS RA

e-mail: barseghyan@gmail.com

Abstract

In this paper a color-structured image search method is proposed which allows
to classify large image collections by color and quickly ¯nd the interested images.
According to the proposed method a software system is developed that demonstrates
the e±ciency and accurancy of the proposed method.

Keywords: image search, color structure, visual similarity.

1 . In t r o d u c t io n

In r e c e n t ye a r s im a g e s e a r c h e n g in e s , s u c h a s Go o g le Im a g e s [1 ], Y a n d e x Im a g e s [2 ], B in g
Im a g e s [3 ],Tin E ye [4 ], P ic S e a r c h [5 ] a n d m a n y o t h e r s , a r e b e in g r a p id ly d e ve lo p e d . Th e s e
s e a r c h e n g in e s a llo w t o ¯ n d t h e d e s ir e d im a g e s b y u s in g va r io u s c r it e r ia s u c h a s c r e a t io n
d a t e s , o r ie n t a t io n , s iz e , c o lo r a n d vis u a l s im ilia r it y ( d u p lic a t e s ) .
Me t h o d s fo r c o n t e n t b a s e d im a g e s e a r c h in g c a n b e d ivid e d in t o t wo m a jo r t yp e s : vis u a l r e -
r a n kin g [6 ], [7 ] a n d p r e d ic t io n -b a s e d r e ¯ n e m e n t [8 ]. In t e n t S e a r c h [8 ] p r o vid e s a m e c h a n is m
t o a llo w t h e u s e r s t o s e le c t a fe w im a g e s o f in t e r e s t , a n d a u t o m a t ic a lly m in e t h e u s e r 's in t e n t
t o r e o r d e r im a g e s e a r c h r e s u lt s . In t h is p a p e r , a s im p le m e t h o d fo r ¯ n d in g im a g e s b a s e d
o n e xp lic it in t e n t is p r e s e n t e d . Th e d e ve lo p e d s o ft wa r e a llo ws t o ¯ n d t h e d e s ir e d im a g e s b y
u s in g t h e p e r c e n t a g e o f t h e d e s ir e d c o lo r s .
In [9 ] a n d [1 0 ] yo u c a n s e e a s ys t e m wh ic h is s im ilia r t o t h e s u g g e s t e d m e t h o d . [9 ] wa s
d e ve lo p e d a n d s u p p o r t e d b y Id e e L a b s [1 1 ] Ca n a d ia n -b a s e d c o m p a n y wh ic h o wn e d Tin E ye
[4 ]. Mu lt ic o lo r S e a r c h L a b [9 ] m a ke s c o lo r -b a s e d im a g e s e a r c h wit h in m o r e t h e n 1 0 m illio n
im a g e s fr o m Flic kr [1 2 ] wh ic h h a s a lic e n s e o f Cr e a t ive Co m m o n s . [9 ] h a s a c o n t r o l o f
d e s ir e d c o lo r s c o n c e n t r a t io n in im a g e s . Th e u s e r s c a n s e le c t u p t o 5 c o lo r s a n d c o n t r o l c o lo r
c o n c e n t r a t io n b a la n c e . A s a c o n s e qu e n c e o f in c r e a s e in c o n c e n t r a t io n p e r c e n t o f o n e c o lo r
t h e o t h e r c o lo r s c o n c e n t r a t io n p e r c e n t d e c r e a s e s .
In [9 ] ( s e e Fig u r e 1 ) b a s e c o lo r s p a le t t e u s e d c o n t a in s 2 5 6 c o lo r s . In [1 3 ] u s e d p a le t t e c o n t a in s
2 8 c o lo r s , Go o g le 's ( s e e Fig u r e 2 ) , B in g 's a n d P ic s e a r c h 's c o lo r s p a le t t e c o n t a in s 1 2 c o lo r a n d
Y a n d e x's ( s e e Fig u r e 3 ) p a le t t e c o n t a in s 9 c o lo r s .

2 . Me t h o d

In t h e ¯ r s t s t e p o f o u r a lg o r it h m we n e e d t o c o r r e s p o n d e a c h p ixe l c o lo r o f e ve r y im a g e t o
o u r c o lo r fr o m o u r b a s e c o lo r s p a le t t e . A s b a s e c o lo r s t h e Go o g le 's c o lo r p a le t t e is u s e d . Fo r

1 0 1



1 0 2 Searching Images by Using Color Structure

t h is we n e e d t o g e t R GB va lu e s o f e a c h p ixe l in o u r c o lo r p a le t t e . A ft e r t h a t we n e e d t o ¯ n d
t h e c lo s e s t va lu e o f c o lo r in R GB s ys t e m fo r e a c h p ixe l. A s a d is t a n c e m e t r ic e m e a n s qu a r e
e r r o r is u s e d . Th e m a t h e m a t ic a l fo r m u la is p r e s e n t e d b e lo w:

D = ( pixelpal:rgb:r ¡ pixelimg:rgb:r ) 2 + ( pixelpal:rgb:g ¡ pixelimg:rgb:g ) 2 +
+ ( pixelpal:rgb:b ¡ pixelimg:rgb:b ) 2 ( 1 )

D¡ > min

wh e r e pixelpal:rgb - is a r g b va lu e o f p a le t t e c o lo r a n d pixelpal:rgb:r, pixelpal:rgb:g a n d
pixelpal:rgb:b a r e , r e s p e c t ive ly, t h e r e d , g r e e n a n d b lu e va lu e s o f r g b .

3 . R e a liz a t io n

Th is s e c t io n is d ivid e d in t o fo u r lo g ic a l p a r t s : Ge t t in g t h e im a g e s fr o m Flic kr , in d e xin g
( a n a lyz in g ) t h e im a g e s , s t o r in g t h e in d e xe d r e s u lt s a n d s e a r c h in g wit h in t h e s t o r e d d a t a .

² Th e ¯ r s t s t e p in c o r p o r a t e s m in in g Flic kr fo r im a g e s . Fo r t u n a t e ly, Flic kr p r o vid e s a wa y
o f d o in g t h is : it a llo ws t h e s o ft wa r e o f o u t s id e u s e r s t o in t e r a c t wit h t h e ir A P I [1 4 ].
W e h a ve c h o s e n a ll im a g e s wh ic h a r e t a g g e d a s " A r m e n ia " a n d lic e n s e d a s " Cr e a t ive
Co m m o n s " . Flic kr s e a r c h r e t u r n e d 1 6 2 5 0 im a g e s m e e t in g t h e s e c r it e r ia . Fo r d o in g
t h is wit h Flic kr A P I we n e e d t o s ig n u p a s a Flic kr d e ve lo p e r in o r d e r t o r e t r ie ve t h e
" A P I K E Y " . On e o f t h e lim it a t io n s o f t h e Flic kr A P I is t h a t o n e c a n o n ly r e t r ie ve a
m a xim u m o f 5 0 0 im a g e s p e r p a g e fo r a s p e c ī e d s e a r c h t e r m .
It is in t e r e s t in g fa c t t h a t t h e d o wn lo a d e d im a g e s c o n t a in o n ly 3 9 im a g e s in GIF fo r m a t
a n d 4 3 im a g e s in P N G fo r m a t , t h e o t h e r s a r e c o d e d b y JP E G fo r m a t .

² A ft e r d o wn lo a d in g t h e r e qu ir e d im a g e s we s t a r t t o a n a lys e t h e m . Fo r d o in g t h a t we
h a ve wr it t e n a n in d e xe r . Th e in d e xe r wa s wr it t e n b y u s in g Im a g e Ma g ic k [1 5 ]. Im -
a g e Ma g ic k is a wid e ly kn o wn s o ft wa r e lib r a r y t o c r e a t e , e d it , c o m p o s e , o r c o n ve r t
b it m a p im a g e s . It c a n r e a d a n d wr it e im a g e s in a va r ie t y o f fo r m a t s ( o ve r 1 0 0 ) ( t h e
fu ll lis t o f t h e s u p p o r t e d fo r m a t s o n e c a n s e e in [1 6 ]) .
W e c a lc u la t e t h e p e r c e n t s o f p a le t t e 's c o lo r s in g ive n im a g e . W e t a ke o n ly t h a t c o lo r s
wh ic h h a ve m o r e t h a n o n e p e r c e n t e xis t a n c e in t h e g ive n im a g e .

² Th e m o s t c r it ic a l p a r t o f o u r s ys t e m is s t o r in g in fo r m a t io n o f t h e a n a lys e d im a g e s .
W e c a n u s e c la s s ic S QL d a t a s t o r e s u c h a s S QL it e , MyS QL o r P o s t g r e S QL , b u t fo r
h a vin g m o r e s p e e d wh e n d o in g s e a r c h we c h o o s e n e w t yp e s o f d a t a s t o r e R e d is [1 7 ].
R e d is is a s c h e m e le s s , fa s t a n d r e lia b le d a t a b a s e e n g in e . R e d is s t a n d s o u t fr o m m a n y
o t h e r N o S QL d a t a b a s e s wit h it s p o we r fu ll d a t a t yp e s . Fo r o u r s ys t e m t h e b e s t c h o ic e
o f d a t a t yp e is t h e " S o r t e d S e t s " .
W e s t o r e t h e a n a lys e d d a t a a s

color id percent img id

wh e r e img id -is t h e id e n t ifa c t io n n u m b e r o f im a g e , color id - is t h e id e n t ifa c t io n
n u m b e r o f c o lo r fr o m p a le t t e a n d percent is t h e p e r c e n t o f color id in t h e g ive n im a g e .
If t h e g ive n c o lo r d o e s n 't e xis t in t h e g ive n im a g e we s t o r e 0 .



R. Barseghyan 1 0 3

² Fo r s e a r c h in g im a g e s wit h t h e r e qu ir e d c o n c e n t r a t io n o f c o lo r s we wr it e qu e r ie s t o o u r
d a t a b a s e . Fo r g e t t in g im a g e s wh ic h c o n t a in t h e r e qu ir e d p e r c e n t s o f t h e d e s ir e d c o lo r
we n e e d t o o p e r a t e o n s c o r e s . W e c a n d o t h is b y u s in g t h e " z r a n g e s c o r e " o p e r a t o r .
Th e o p e r a t o r " z r a n g e s c o r e " r e t u r n s a s e t .

zrangebyscore color id percent < input percent

A ft e r g e t t in g t h e r e qu ir e d s e t s fo r a ll d e s ir e d c o lo r s we d o s e t s in t e r s e c t io n .
To e xc lu d e c o lo r s we n e e d t o d o a d d it io n a l qu e r y like t h is

zrangebyscore color id percent ¡ inf 0

4 . Co n c lu s io n

Th e d e ve lo p e d s o ft wa r e s ys t e m c a n h e lp t h e o wn e r s o f la r g e im a g e a r c h ive s t o e a s ily ¯ n d
t h e d e s ir e d im a g e s . To g e t m o r e r e le va n t im a g e s t h e s ys t e m s u p p o r t s m u lt ic o lo r s e a r c h a n d
a llo ws t h e u s e r t o in c lu d e / e xc lu d e c o lo r s a n d c o n t r o l p e r c e n t a g e c o lo r o f t h e d e s ir e d im a g e .
Th e d e ve lo p e d s o ft wa r e s ys t e m wo r ks fa s t , fo r e xa m p le , t h e s e a r c h p r o c e s s in o u r c o lle c t io n
wit h 1 6 2 5 0 im a g e s t a ke s o n ly 0 ,0 7 s e c o n d s .

Refer ences

[1 ] h t t p :/ / im a g e s .g o o g le .c o m

[2 ] h t t p :/ / im a g e s .ya n d e x.r u

[3 ] h t t p :/ / b in g .c o m / im a g e s

[4 ] h t t p :/ / t in e ye .c o m

[5 ] h t t p :/ / p ic s e a r c h .c o m

[6 ] W . H . H s u , L . S . K e n n e d y, a n d S .-F. Ch a n g , \ R e r a n kin g m e t h o d s fo r vis u a l s e a r c h ,"
IE E E M ultiM edia, vo l. 1 4 , n o . 3 , p p .1 4 -2 2 , 2 0 0 7 .

[7 ] Y . Jin g a n d S . B a lu ja , " P a g e r a n k fo r p r o d u c t im a g e s e a r c h . In W W W ," W W W 2 0 0 8 ,
B e ijin g , Ch in a , p p . 3 0 7 - 3 1 6 , 2 0 0 8 .

[8 ] J. Cu i, F. W e n , a n d X . Ta n g , \ In t e n t S e a r c h : In t e r a c t ive o n -lin e im a g e s e a r c h r e -
r a n kin g ," P roc. 16th ACM Int'l Conf. M ultimedia, p p . 9 9 7 -9 9 8 , 2 0 0 8 .

[9 ] h t t p :/ / la b s .t in e ye .c o m / m u lt ic o lr

[1 0 ] h t t p :/ / la b s .t in e ye .c o m / m u lt ic o lo u r

[1 1 ] h t t p :/ / id e e in c .c o m

[1 2 ] h t t p :/ / ° ic kr .c o m

[1 3 ] h t t p :/ / d r ib b b le .c o m

[1 4 ] h t t p :/ / ° ic kr .c o m / s e r vic e s / a p i/

[1 5 ] h t t p :/ / im a g e m a g ic k.o r g

[1 6 ] h t t p :/ / im a g e m a g ic k.o r g / s c r ip t / fo r m a t s .p h p

[1 7 ] h t t p :/ / r e d is .io



1 0 4 Searching Images by Using Color Structure

Figure 1: Tineye Multicolr Search Lab interface.

Figure 2: Google's image search interface.

Figure 3: Yandex's image search interface.

Submitted 16.01.2013, accepted 25.02.2013.



R. Barseghyan 1 0 5

ä³ïÏ»ñÝ»ñÇ áñáÝáõÙ Áëï ·áõݳÛÇÝ Ñ³ïϳÝÇßÝ»ñÇ

è. ´³ñë»ÕÛ³Ý

²Ù÷á÷áõÙ

²ß˳ï³ÝùáõÙ ³é³ç³ñÏí³Í ¿ Ù»Ãá¹, áñÁ ÃáõÛÉ ¿ ï³ÉÇë áñáÝ»É å³ïÏ»ñÝ»ñª
û·ï³·áñÍ»Éáí ¹ñ³Ýó ·áõݳÛÇÝ Ñ³ïϳÝÇßÝ»ñÁ: Øß³Ïí³Í Ù»Ãá¹Á Ïû·ÝÇ å³ïÏ»ñÝ»ñÇ
Ù»Í Ñ³í³ù³Íáõ áõÝ»óáÕÝ»ñÇÝ Ñ»ßïáõÃÛ³Ùµ ·ïÝ»É å³Ñ³ÝçíáÕ å³ïÏ»ñÝ»ñÁ: Øß³Ïí»É
¿ Íñ³·ñ³ÛÇÝ Ñ³Ù³Ï³ñ·, áñÁª áñå»ë ÷ÝïñÙ³Ý å³ñ³Ù»ïñ, û·ï³·áñÍáõÙ ¿
ó³ÝϳÉÇ ·áõÛÝÇ (·áõÛÝ»ñÇ) ³éϳÛáõÃÛáõÝÁ/ µ³ó³Ï³ÛáõÃÛáõÝÁ: гٳϳñ·Á ÃáõÛÉ ¿
ï³ÉÇë ϳé³í³ñ»É å³Ñ³ÝçíáÕ ·áõÛÝÇ (·áõÛÝ»ñÇ) ³éϳÛáõÃÛ³Ý ÙÇç³Ï³ÛùÁ: гٳϳñ·Á
÷áñÓ³ñÏí»É ¿ û·ï³ï»ñ»ñÇ ÏáÕÙÇó Flickr ¿É»ÏïñáݳÛÇÝ é»ëáõñë í»ñµ»éÝí³Í “Armenia”
- åÇï³Ïáí Ýß³·ñí³Í ¨ “Creative Commons” ÉÇó»Ý½Ç³ áõÝ»óáÕ ÝϳñÝ»ñÇ µ³½ÙáõÃÛ³Ý
íñ³:

Ïîèñê èçîáðàæåíèÿ ïî öâåòîâûì õàðàêòåðèñòèêàì
Ð. Áàðñåãÿí

Àííîòàöèÿ

 ðàáîòå ïðåäëîæåí ìåòîä äëÿ ïîèñêà èçîáðàæåíèé ïî öâåòîâûì
õàðàêòåðèñòèêàì. Ðàçðàáîòàííûé ìåòîä ïðèçâàí ïîìî÷ü âëàäåëüöàì áîëüøèõ
àðõèâîâ èçîáðàæåíèé áûñòðî è âèçóàëüíî ïðîñòî íàéòè íóæíûå èçîáðàæåíèÿ.
 ñîîòâåòñòâèè ñ ïðåäëîæåííûì ìåòîäîì áûëî ðàçðàáîòàíî ïðîãðàììíîå
îáåñïå÷åíèå, êîòîðîå ïîçâîëÿåò èñêàòü íóæíûå èçîáðàæåíèÿ êàê ïî âõîæäåíèþ
îïðåäåëåííîãî öâåòà (îïðåäåëåííûõ öâåòîâ), òàê è èñêëþ÷åíèþ îïðåäåëåííîãî
öâåòà (îïðåäåëåííûõ öâåòîâ), à òàêæå ïîçâîëÿåò çàäàòü ïðîöåíò âõîæäåíèÿ öâåòà
(öâåòîâ). Ðàçðàáîòàííîå ïðîãðàììíîå îáåñïå÷åíèå áûëî òåñòèðîâàíî íà áàçå
èçîáðàæåíèé, êîòîðûå áûëè çàãðóæåíû ïîëüçîâàòåëÿìè â èíòåðíåò ñåðâèñ Flickr
è áûëè ïîìå÷åíû ìåòêîé ”Armenia” è èìåëè ëèöåíçèþ ”Creative Commons”.