This study explores the effects of various spatiotemporal dynamic texture characteristics on human emotions. The emotional experience of auditory (eg, music) and haptic repetitive patterns has been studied extensively. In contrast, the emotional experience of visual dynamic textures is still largely unknown, despite their natural ubiquity and increasing use in digital media. Participants watched a set of dynamic textures, representing either water or various different media, and self-reported their emotional experience. Motion complexity was found to have mildly relaxing and nondominant effects. In contrast, motion change complexity was found to be arousing and dominant. The speed of dynamics had arousing, dominant, and unpleasant effects. The amplitude of dynamics was also regarded as unpleasant. The regularity of the dynamics over the textures' area was found to be uninteresting, nondominant, mildly relaxing, and mildly pleasant. The spatial scale of the dynamics had an unpleasant, arousing, and dominant effect, which was larger for textures with diverse content than for water textures. For water textures, the effects of spatial contrast were arousing, dominant, interesting, and mildly unpleasant. None of these effects were observed for textures of diverse content. The current findings are relevant for the design and synthesis of affective multimedia content and for affective scene indexing and retrieval.
AkaikeH, 1974“A new look at statistical model identification”IEEE Transactions on Automatic Control19716–723 doi:10.1109/TAC.1974.1100705 ◂
3.
AkaikeH, 1987“Factor analysis and AIC”Psychometrika52317–332 doi:10.1007/BF02294359 ◂
4.
AmaralD GBehnieaHKellyJ L, 2003“Topographic organization of projections from the amygdala to the visual cortex in the macaque monkey”Neuroscience1181099–1120 doi:10.1016/S0306-4522(02)01001-1 ◂
ArifinSCheungP Y K, 2007“A novel video parsing algorithm utilizing the Pleasure-Arousal-Dominance emotional information”IEEE International Conference on Image Processing 2007 (Washington, DC: IEEE Press) ◂
7.
BarringtonLChanA BLanckrietG, 2010“Modeling music as a dynamic texture”IEEE Transactions on Audio, Speech, and Language Processing18602–612 doi:10.1109/TASL.2009.2036306 ◂
8.
BaumgartnerTLutzKSchmidtC FJanckeL, 2006“The emotional power of music: How music enhances the feeling of affective pictures”Brain Research1075151–164 doi:10.1016/j.brainres.200-5.12.065 ◂
9.
BentlerP M, 1990“Comparative fit indexes in structural models”Psychological Bulletin107238–246 doi:10.1037/0033-2909.107.2.238 ◂
10.
BigandEVieillardSMadurellFMarozeauJDacquetA, 2005“Multidimensional scaling of emotional responses to music: The effect of musical expertise and of the duration of the excerpts”Cognition & Emotion191113–1139 doi:10.1080/02699930500204250 ◂
11.
BillockV A, 2000“Neural acclimation to 1/f spatial frequency spectra in natural images transduced by the human visual system”Physica D: Nonlinear Phenomena137379–3910 doi:10.1016/S0167-2789(99)00197-9 ◂
12.
BillockV ACunninghamD WHavigP RTsouB H, 2001a“Perception of spatiotemporal random fractals: An extension of colorimetric methods to the study of dynamic texture”Journal of the Optical Society of America A182404–2413 doi:10.1364/JOSAA.18.002404 ◂
13.
BillockV Ade GuzmanG CKelsoJ A S, 2001b“Fractal time and 1/f spectra in dynamic images and human vision”Physica D: Nonlinear Phenomena148136–146 doi:10.1016/S0167-2789(00)00174-3 ◂
14.
BoltzM GEbendorfBFieldB, 2009“Audiovisual interactions: The impact of visual information on music perception and memory”Music Perception2743–59 doi:10.1525/mp.2009.27.1.43 ◂
15.
BresinRFribergA, 2000“Emotional coloring of computer-controlled music performances”Computer Music Journal2444–63 doi:10.1162/014892600559515 ◂
16.
CerfMClearyD RPetersR JEinhäuserWKochC, 2007“Observers are consistent when rating image conspicuity”Vision Research473052–3060 doi:10.1016/j.visres.2007.06.025 ◂
17.
ChanA BVasconcelosN, 2005“Layered dynamic textures”Proceedings of Neural Information Processing Systems18 ◂
18.
ChenT PChenC-WPoppPCooverB, 2011“Visual rhythm detection and its applications in interactive multimedia”IEEE Multimedia1888–95 doi:10.1109/MMUL.2011.19 ◂
19.
ChuangY-YGoldmanD BZhengK CCurlessBSalesinD HSzeliskiR, 2005“Animating pictures with stochastic motion textures”ACM Transactions on Graphics24853–860 doi:10.1145/1073204.1073273 ◂
20.
ConstantiniRSbaizLSüsstrunkS, 2008“Higher order SVD analysis for dynamic texture synthesis”IEEE Transactions on Image Processing1742–52 doi:10.1109/TIP.2007.910956 ◂
21.
DelplanqueSN extquotesinglediayeKSchererKGrandjeanD, 2007“Spatial frequencies or emotional effects? A systematic measure of spatial frequencies for IAPS pictures by a discrete wavelet analysis”Journal of Neuroscience Methods165144–150 doi:10.1016/j.jneumeth.2007.05.030 ◂
22.
DijkstraKPieterseMPruynA, 2006“Physical environmental stimuli that turn healthcare facilities into healing environments through psychologically mediated effects: Systematic review”Journal of Advanced Nursing56166–181 doi:10.1111/j.1365-2648.2006.03990.x ◂
23.
DorettoGChiusoAWuY NSoattoS, 2003“Dynamic textures”International Journal of Computer Vision5191–109 doi:10.1023/A:1021669406132 ◂
24.
FernandezDWilkinsA J, 2008“Uncomfortable images in art and nature”Perception371098–1113 doi:10.1068/p5814 ◂
25.
FieldD J, 1987“Relations between the statistics of natural images and the response properties of cortical cells”Journal of the Optical Society of America A42379–2394 doi:10.1364/JOSAA.4.002379 ◂
26.
FieldD J, 1994“What is the goal of sensory coding?”Neural Computation6559–601 doi:10.1162/neco.1994.6.4.559 ◂
27.
ForsytheANadalMSheehyNCela-CondeC JSaweyM, 2011“Predicting beauty: Fractal dimension and visual complexity in art”British Journal of Psychology10249–70 doi:10.1348/000712610-X498958 ◂
28.
FujiwaraMAonoSKuwanoS, 2006“Audio-visual interaction in the image evaluation of the environment—an on site investigation”Inter-Noise ◂
29.
GomezPDanuserB, 2007“Affective and physiological responses to environmental noises and music”International Journal of Psychophysiology5391–103 doi:10.1016/j.ijpsycho.2004.02.002 ◂
30.
GroissboeckWLughoferEThumfartS, 2010“Associating visual textures with human perceptions using genetic algorithms”Information Sciences1802065–2084 doi:10.1016/j.ins.2010.01.035 ◂
31.
GuttmanS EGilroyL ABlakeR, 2005“Hearing what the eyes see: Auditory encoding of visual temporal sequences”Psychological Science16228–235 doi:10.1111/j.0956-7976.2005.00808.x ◂
32.
HagerhallC MLaikeTTaylorR PKüllerMKüllerRMartinT P, 2008“Investigations of human EEG response to viewing fractal patterns”Perception371488–1494 doi:10.1068/p5918 ◂
33.
HanjalicA, 2006“Extracting moods from pictures and sounds: Towards truly personalized TV”IEEE Signal Processing Magazine2390–100 doi:10.1109/MSP.2006.1621452 ◂
34.
HanjalicAXuL-Q, 2005“Affective video content representation and modeling”IEEE Transactions on Multimedia7143–154 doi:10.1109/TMM.2004.840618 ◂
35.
HoutkampJ MSchuurinkE LToetA, 2008“Thunderstorms in my computer: The effect of visual dynamics and sound in a 3D environment”Proceedings of the International Conference on Visualisation in Built and Rural Environments BuiltViz'08 M Bannatyne and J Counsell (Los Alamitos, CA: IEEE Computer Society) ◂
36.
HuangJWaldvogelM, 2005“Interactive wallpaper”SIGGRAPH 2005 Electronic Art and Animation Catalog (New York: ACM) ◂
37.
HusainGThompsonW FSchellenbergE G, 2002“Effects of musical tempo and mode on arousal, mood, and spatial abilities”Music Perception20151–171 doi:10.1525/mp.2002.20.2.151 ◂
38.
JuricevicILandLWilkinsAWebsterM A, 2010“Visual discomfort and natural image statistics”Perception39884–899 doi:10.1068/p6656 ◂
39.
JuslinP NVästfjällD, 2008“Emotional responses to music: The need to consider underlying mechanisms”Behavioral and Brain Sciences31559–575 ◂
40.
KellarisJ JKentR J, 1993“An exploratory investigation of responses elicited by music varying in tempo, tonality, and texture”Journal of Consumer Psychology2381–401 doi:10.1016/S1057-7408(08)80068-X ◂
41.
KimE YKimS-JJeongKKimJ, 2005“Emotion-based textile indexing using colors and texture”Fuzzy Systems and Knowledge Discovery Lecture Notes in Computer Science (Berlin / Heidelberg: Springer) ◂
42.
KlineR B, 2010Principles and practice of structural equation modelling (New York: The Guilford Press) ◂
LaiC-HWuJ-L, 2007“Temporal texture synthesis by patch-based sampling and morphing interpolation”Computer Animation and Virtual Worlds18415–428 doi:10.1002/cav.195 ◂
45.
LinTImamiyaAHuWOmataM, 2007“Display characteristics affect users' emotional arousal in 3D games”Universal access in ambient intelligence environments Lecture Notes in Computer Science (Berlin / Heidelberg: Springer) ◂
46.
LucassenM PGeversTGijsenijA, 2011“Texture affects color emotion”Color Research & Application36426–436 doi:10.1002/col.20647 ◂
47.
MacCallumR CAustinJ T, 2000“Applications of structural equation modeling in psychological research”Annual Review of Psychology51201–226 doi:10.1146/annurev.psych.51.1.201 ◂
48.
MachajdikJHanburyA, 2010“Affective image classification using features inspired by psychology and art theory”Proceedings of the International Conference on Multimedia (MM'10) (New York: ACM) ◂
49.
MaoXChenBMutaI, 2003“Affective property of image and fractal dimension”Chaos, Solitons & Fractals15905–910 doi:10.1016/S0960-0779(02)00209-6 ◂
50.
MasakuraYNagaiMKumadaT, 2006“Effective visual cue for guiding peoples' attention to important information based on subjective and behavioral measures”Proceedings of The First International Workshop on Kansei ◂
51.
McKinneyC HTimsF C, 1995“Differential effects of selected classical music on the imagery of high versus low imagers: Two studies”Journal of Music Therapy3222–45 ◂
52.
MehrabianA, 1996“Pleasure-arousal-dominance: A general framework for describing and measuring individual”Current Psychology14261–292 doi:10.1007/BF02686918 ◂
53.
NasarJ LLinY-H, 2003“Evaluative responses to five kinds of water features”Landscape Research28441–450 doi:10.1080/0142639032000150167 ◂
54.
OextquotesingleHareLHibbardP B, 2011“Spatial frequency and visual discomfort”Vision Research511767–1777 doi:10.1016/j.visres.2011.06.002 ◂
55.
ParragaC ATrosciankoTTolhurstD J, 2000“The human visual system is optimised for processing the spatial information in natural visual images”Current Biology1035–38 doi:10.1016/S0960-9822(99)00262-6 ◂
56.
PéteriRChetverikovD, 2006“Qualitative characterization of dynamic textures for video retrieval”Computer Vision and Graphics. Proceedings of the International Conference on Computer Vision and Graphics (ICCVG 2004) Computational Imaging and Vision 32 (Berlin / Heidelberg: Springer) ◂
57.
PéteriRFazekasSHuiskesM J, 2010“DynTex: A comprehensive database of dynamic textures”Pattern Recognition Letters311627–1632 doi:10.1016/j.patrec.2010.05.009 ◂
58.
PostF HVrolijkBHauserHLarameeR SDoleischH, 2003“The state of the art in flow visualisation: Feature extraction and tracking”Computer Graphics Forum22775–792 doi:10.1111/j.1467-8659.2003.00723.x ◂
59.
RecanzoneG H, 2003“Auditory influences on visual temporal rate perception”Journal of Neurophysiology891078–1093 doi:10.1152/jn.00706.2002 ◂
60.
RussellJ A, 1980“A circumplex model of affect”Journal of Personality and Social Psychology391161–1178 doi:10.1037/h0077714 ◂
61.
RussellJ APrattG, 1980“A description of the affective quality attributed to environments”Journal of Personality and Social Psychology38311–322 doi:10.1037/0022-3514.38.2.311 ◂
62.
RussellJ AWardL MPrattG, 1981“Affective quality attributed to environments: A factor analytic study”Environment and Behavior13259–288 doi:10.1177/0013916581133001 ◂
63.
SalminenKSurakka VLylykangasJRaisamoRSaarinenRRaisamoRRantalaJEvreinovG, 2011“Emotional and behavioral responses to haptic stimulation”Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems (New York: ACM Press) ◂
64.
Schermelleh-EngelKMoosbruggerHMüllerH, 2003“Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures”Methods of Psychological Research Online823–74 ◂
65.
SimmonsD RRussellC L, 2008“Visual texture affects the perceived unpleasantness of colours”Perception37146–146 ◂
66.
SmithJ RLinC-YNaphadeM, 2002“Video texture indexing using spatio-temporal wavelets”Proceedings of the International Conference on Image Processing ◂
67.
SoleymaniMChanelGKierkelsJ J MPunT, 2008“Affective ranking of movie scenes using physiological signals and content analysis”Proceeding of the 2nd ACM workshop on Multimedia semantics (New York: ACM) ◂
68.
SteigerJ H, 1990“Structural model evaluation and modification: An interval estimation approach”Multivariate Behavioral Research25173–185 doi:10.1207/s15327906mbr2502_4 ◂
69.
SukH-JJeongS-HHangT-HKwonD-S, 2009“Tactile sensation as emotion elicitor”Kansei Engineering International8147–152 ◂
70.
TaylorR PSpeharBWiseJ ACliffordC WNewellB RHagerhallC MPurcellTMartinT P, 2005“Perceptual and physiological responses to the visual complexity of fractal patterns”Nonlinear Dynamics, Psychology and Life Sciences989–114 ◂
71.
Van HagenM, 2011Waiting experience at train stations (Delft, The Netherlands: Eburon) ◂
72.
WangK, 2009“Research of the affective responses to product's texture based on the Kansei evaluation”Proceedings of the 2009 Second International Symposium on Computational Intelligence and Design (Los Alamos, CA: IEEE Press) ◂
73.
WeiskopfDErlebacherGErtlT, 2003“A texture-based framework for spacetime-coherent visualization of time-dependent vector fields” Proceedings of the 14th IEEE Visualization Conference 2003 (VIS'03) (Washington, DC: IEEE Computer Society) ◂
74.
World Medical Association, 2000“World Medical Association Declaration of Helsinki: Ethical principles for medical research involving human subjects”JAMA: The Journal of the American Medical Association2843043–3045 doi:10.1001/jama.284.23.3043 ◂
75.
ZhangQWangboT, 2007“The dynamic textures for water synthesis based on statistical modeling”Technologies for E-Learning and Digital Entertainment LNCS-4469 Eds (Berlin/Heidelberg: Springer) ◂
76.
ZhaoGPietikainenM, 2007“Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions”IEEE Transactions on Pattern Analysis and Machine Intelligence29915–928 doi:10.1109/TPAMI.2007.1110 ◂
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
