Sage Journals: Discover world-class research

Abstract

The aim of the study is to examine the applicability of image segmentation—identification of objects/regions by partitioning images—to examine online social photography. We argue that the need for a meaning-independent reading of online social photography within social markers, such as hashtags, arises due to two characteristics of social photography: (1) internal incongruence resulting from user-driven construction and (2) variability of content in terms of visual attributes, such as color combinations, brightness, and details in backgrounds. We suggest visual affluence—plenitude of visual stimuli, such as objects and surfaces containing a variety of color regions, present in visual imagery—as a basis for classifying visual content and image segmentation as a technique to measure affluence. We demonstrate that images containing objects with complex texture and background patterns are more affluent, while images that include blurry backgrounds are less affluent than others. Moreover, images that contain letters and dark, single-color backgrounds are less affluent than images that include subtle shades. Mann–Whitney U test results for ten pairs of hashtags showed that eight pairs had significant differences in visual affluence. The proposed measure can be used to encourage a “visually oriented” turn in online social photography research that can benefit from hybrid methods that are able to extrapolate micro-level findings to macro-level effects.

Keywords

image segmentation visual affluence Instagram hashtags

Introduction

Emergence of platforms primarily driven by visual content, such as Instagram, has intensified the significance of visual communication. Social photography in Instagram has been examined from multiple perspectives, such as visibility labor in advertorials (Abidin, 2016), platform effects on street art and graffiti (MacDowall & de Souza, 2018), and promotion of body type and objectification (Tiggemann & Zaccardo, 2016), using different analytical approaches (e.g., Fardouly, 2018; Ging & Garvey, 2018; Guidry et al., 2018). As these studies indicate, a range of methodologies, including ethnography, semiotics, and content analysis, has been used to examine Instagram. Despite the significance of such work, diversity of content and the increasing volume of images pose challenges for researchers. As the relevance of images for a hashtag is defined by users and the quality of such images are not uniform, making generalized claims regarding the content in visual expressions within a hashtag and its effects is difficult at best. This demands meaning-independent analyses of online visual content that could benefit from measures that can be applied across different samples, including large volumes of images, as diversity of content may depend on sample size. Design features, such as “like buttons,” allow metrification of user affect and engagement by converting them into numbers (Gerlitz & Helmond, 2013). However, scholars who highlight the need for alternative approaches to study digital platforms have stressed the need to move beyond “vanity metrics” such as brute counting of views and likes (e.g., Rogers, 2018). Lack of accessible measures beyond vanity metrics can limit the scope of scholarly work that examines social photography. While measures afforded by platforms, such as the number of views and likes, help quantify user engagement with visual content to some extent, social science research can benefit from measures that capture the essence of visual communication while allowing automated analysis to quantify the nature of visual content in social photography.

This study aims to contribute to the growing body of Instagram research on two levels. First, our goal is to address the issue mentioned above by suggesting a meaning-independent measure of visual content that allows examining Instagram images using image segmentation—that is, partitioning images into multiple regions or objects (Rajinikanth & Couceiro, 2015)—and argue that the number of machine-readable regions in an image can be used as a measure of visual affluence (“richness” of visual content). As we demonstrate in the study, visual affluence can serve as a distant measure for examining online social photography. A large number of studies that examine Instagram images tend to focus on specific social contexts, such as “pro-ana” (pro-anorexia) and “thinspiration” communities (Ging & Garvey, 2018), body image concerns and objectification among young women (Fardouly, 2018), and political campaigns (Filimonov et al., 2016). This shows a “socially oriented” turn in visual social media research. While context-specific work is necessary to examine local-level effects, it is important to theorize social photography beyond contexts. Caliandro (2018, p. 551) claimed that

the main task for the ethnographer moving across social media environments should not be exclusively that of identifying an online community to delve into but of mapping the practices through which Internet users and digital devices structure social formations around a focal object (e.g., a brand).

However, analysis of social photography can be extended beyond specific social contexts and/or focal objects, as visual content can be examined beyond representation and meaning-making. For instance, although psychological and behavioral effects of color have been subject to substantial inquiry in various fields (e.g., Ab. Jalil et al., 2012; Spence, 2018; van Esch et al., 2019), there is only a limited number of studies that examine the role color plays in online images (e.g., Hyun & Kim, 2019; Kim & Hyun, 2018).

There are many approaches to analyzing images that exceed the boundary of meaning-making. For instance, notions such as visual clutter—“state in which excess items, or their representation or organization, lead to a degradation of performance at some task” (Rosenholtz et al., 2007, p. 3); visual complexity—presence of high amounts of information in a texture (Amadasun & King, 1989) and the difficulty in providing verbal descriptions (Heaps & Handel, 1999); and entropy—the amount of information needed to describe the behavior of a given system (Schieber & Gilland, 2008)—have been used to examine visual content. We aim to propose visual affluence as a related measure that can be applied across different types of visual content and sample sizes, from a few images to large volumes of images. From a technical point of view, visual affluence is a more accessible measure to automate quantification of visual content that does not require training algorithms as required by methods such as deep learning (see Wason, 2018). In this study, we use the R EBImage package (Pau et al., 2010), which provides general-purpose functionality for image analysis to examine the applicability of image segmentation as a measure of visual affluence. In the following section, we raise the need for “visually oriented” social media image research highlighting internal incongruence and variability in visual hashtags. Then we segment a series of images to examine the potential of image segmentation as measure of visual affluence. We also map visual affluence across a range of hashtags, identifying “levels of affluence” on Instagram to initiate application of visual affluence across hashtags.

Social Photography

For more than a century, photographic images have played an exceptional role in the way we see and think about the world, ourselves, and others (Lister, 1995). In a time of network connectivity, when mobile technologies abound, photography presents us with the appearance of things and promotes the consumption of a “great many more images around, claiming our attention” (Sontag, 1973, p. 1), urging closer attention to our relationship with the photographic image. The emergence of social network sites (SNSs), especially platforms such as Instagram that are driven by visual content, has amplified the role visual content plays in everyday life. The real-time dissemination and exchange of a vast volume of photographic imagery to a networked public has challenged and shifted photographic practices (Zappavigna, 2016) toward “ubiquitous photography” (Hand, 2012; Kember, 2013). The act of photography is therefore social, networked, user-based, amateur, personal, or even vernacular (Batchen, 2002; Rubinstein & Sluis, 2008; Van House, 2011). The social media photograph moves between platforms and devices, presented in collection with other images and media, and locational and temporal data. From a photography studies perspective, Rubinstein and Sluis (2013) provide an analysis of the current state of the photographic image as a networked and an algorithmic one. They note that “the image within the network is doing something other than showing us pictures, and it is doubtful if we have the right vocabulary to address this image economy” (p. 156). Highlighting the “undecidability” of online images, Rubinstein and Sluis argue that the networked image delivers an image of the multiplicity engendered by the network to the screen rather than identity. This suggests that online social photography should be read in a variety of modes, including levels of analysis that exceed meaning and representation. In the following section, we discuss the internal incongruence and variability of content as characteristics of online social photography and raise the need for a wave of “visually oriented” social media research.

Visual Hashtags, Internal Incongruence, and Variability

Central to our inquiry of online social photography is the sociotechnical affordance of hashtags. Coined by Gibson (1986), the notion of affordances captures what an environment affords or offers to an animal. Gibson suggests that affordances in a given environment should be measured relative to the animal. Evans et al. (2017) define affordances as a relational structure between technology and user that allows or constrains behavioral outcomes. Hashtags should not be seen as mere technological elements, as users and social contexts are essential for the emergence and performance of hashtags. As Rathnayake and Suthers (2018) note, hashtags can be considered as affordances, as platforms afford their creation and different acts emerge from their use. Highlighting the user-driven construction, Caliandro (2018, p. 18) claims that hashtags are “markers through which users develop a specific thread of conversation or self-categorize their own contents.” Similarly, Bruns and Burgess (2011) describe hashtags as a user-generated mechanism for topical tagging and collating online content. Zappavigna (2016) argues that hashtags operate as social metadata in the sense that they are a form of descriptive annotation produced by users, also indicating a shift toward coordinating activity and commentary rather than simply categorizing artifacts. These “user-generated” tags can relate to a practice termed as “folksonomy” (Wal, 2007) and function as “searchable signatures” (Schlesselman-Tarango, 2013). Instagram affordances promote visual and textual communication, and there is similarity with respect to the hashtag architecture (Highfield & Leaver, 2015) with Twitter. The use of hashtags on Instagram indicates less of a conversation. However, participation in a community, presentation of the self, and collective dimensions of engagement, such as supporting visibility, characterize the use of Instagram. For instance, examining the hashtag #hipster, Caliandro (2018, p. 569) notes that “Instagram functions as a public space through which internet users co-create a specific social imaginary related to the concept of hipsterism, and in doing so helping the research to better define this phenomenon.”

Although these studies viewed user-driven construction through hashtags as a basis for content classification, self-categorization may not necessarily result in well-defined categories. In other words, utterances in co-created social imaginaries (Caliandro, 2018) may not strictly adhere to the meanings associated with the hashtag. The act of using a hashtag with specific images indicates relevance from the perspective of the user. This subjective relevance is a central characteristic of social photography. From this perspective, objective meaning in social photography is at risk in collective settings and that demands constructs and concepts which can provide interpretation beyond what is afforded by meanings associated with objects included in such images. This, we argue, shows another less discussed character of hashtags, that is, inconsistency in meaning. We suggest that this internal incongruence—diversity of content within hashtags, sometimes exceeding direct meanings with which hashtags are associated—should be taken into account when examining hashtags. Figure 1 shows four pairs of Instagram images, each representing a hashtag. Images on the left in each pair can be directly connected to the meaning of the hashtag. Images on the right are not directly associated with the hashtag. Accordingly, even strictly defined Instagram hashtags, such as #Foodporn, #Trump, and #Brexit, which can drive content creation, can still include images that do not necessarily adhere to the meanings ascribed to the hashtag. General hashtags, such as #Instagood, include a range of images that indicate relevance from the perspective of users, rather than constructing a collective and objective meaning. This internal incongruence does not mean that images that do not directly express association with the hashtag do not belong in the hashtag. On the contrary, this inconsistency should be seen as a defining characteristic of user-driven construction. Internal incongruence in content is not similar to “undecidability” (Rubinstein & Sluis, 2013) as the latter is conceptualized based on the role played by metadata by releasing the image from its “stillness” and continuing reinvention. Instead, internal incongruence acknowledges the diversity that users bring into hashtags as they define relevance from their point of view in the process of content creation.

Figure 1.

Instagram hashtags and internal incongruence. (a) #Sushi. (b) #Trump. (c) #Brexit. (d) #Grafitti.

Although internal incongruence in social photography is unavoidable, current literature does not adequately deal with its causes and outcomes. This may have been caused by the socially oriented focus of Instagram studies. In other words, many Instagram studies focus on meaning-making within a networked social setting (i.e., hashtag) rather than how meaning is challenged in its construction. For instance, Tiggemann and Zaccardo (2016) focus on how body types are contained in Instagram images. Similarly, Rodriguez and Hernandez (2018) demonstrate how Instagram images reinforce hegemonic masculinity by fostering objectification of women. From the perspective of engagement, Filimonov and colleagues (2016) examine how the platform is used for specific purposes, such as political campaigns and mobilization. While these studies provide important insight, explaining how Instagram is used in different social contexts, they do not adequately explain internal diversity of content. We therefore argue that there is a need for a visually oriented line of inquiry focusing on Instagram content beyond meaning-making processes. Such work needs to consider effects of visual attributes, such as color combinations, shades, brightness, clutter, and complexity of online content.

Variability—differences among images that elicit similar content—is also crucial in approaching a meaning-independent reading of online images. Such variability can be caused by a range of factors including the use of filters, quality of cameras, lighting, and shapes and shades in the background. Figure 2 shows two images shared using the hashtag #Graffiti. These images are highly similar, as they include a female figure with a sword climbing a bridge located above a pipeline containing graffiti art. From the perspective of meaning, the objects in the images are similar. However, these images are considerably different from each other from the perspective of quality attributes, such as background details, angles, color variation, and brightness. Such differences may result in different reactions from users, since attributes such as color, as Kuzinas (2013) argues, can affect viewers independently of other elements. As Jue and Kwon (2013) demonstrate, color can be effective in estimating psychological states. Their work also shows that, while some colors, such as red and black, are perceived as aggressive and anxious, excessive use of black may darken the images and impressions of content. Moreover, people tend to associate brightness with positivity (Specker et al., 2018). According to Gong et al. (2017), backgrounds and hue influence the perception of color emotions to varying degrees. Examining visual attributes beyond explicit meaning is an interdisciplinary endeavor, and few studies take such an initiative. Hyun and Kim (2019), for instance, examine relationships between user characteristics and color features of images shared on Instagram. Their study shows associations between gender, agreeableness, neuroticism, openness, and neuroticism and visual attributes such as color diversity and harmony. Similarly, a study conducted by Kim and Hyun (2018) shows that pixel features, such as variance of RGB pixel values, hue, share of red color, and the share of warmth (sum of red, orange, and yellow) correlated with user personalities. While these studies provide evidence of correlation between visual properties and user attributes, such as personality, work that examines effects of such visual properties is underrepresented in social media research.

Figure 2.

Variability in Instagram images (#Grafitti).

Quantifying Social Photography: Image Segmentation as an Approach to Measuring Visual Affluence

In this section, we discuss how image segmentation can be used to develop a visually oriented basis for classifying images. Image segmentation allows the extraction of meaningful information from images by separating them into regions or objects and it is widely applied in a range of fields, including remote sensing, medical imaging, and pattern recognition (Rajinikanth & Couceiro, 2015). Previous work has demonstrated that image segmentation can be used to examine a variety of images, such as lung images (Skourt et al., 2018), catenary images (Wu et al., 2018), sonar images (Song et al., 2019), and breast ultrasound images (Xian et al., 2018; Xu et al., 2019) and help detect critical problems (e.g., breast cancer). While this is a well-established approach in fields such as medicine, its potential for explaining content in “everyday images” has not been examined. We segmented an image of a bird (Figure 3) to examine the possibility of using image segmentation to extract information from non-microscopic images. This image was segmented using the R EBImage package (Pau et al., 2010), which provides general-purpose functionality for image analysis. The “bwlabel” function included in the package detects connected sets in a binary image and assigns labels to each set. Setting threshold values for segmentation is crucial, as it can decide the granularity of segmentation. Thresholding produces image objects with binarized pixel values (Oleś et al., 2018) that can be used to isolate objects in an image. However, the number of “objects” identified by segmentation is not similar to the number of actual (physical) objects in images. Figure 3 shows three versions of the same image segmented with different threshold values. Colors in each segmented image represent each connected set of pixels. While a threshold value close to 1 or 0 returns a lower number of objects, a threshold value close to 0.4 returns a higher number of objects. This shows that a medium-level threshold can increase the granularity of segmentation, thereby providing a more nuanced assessment of the presence of different colors, objects, and shades in the image. This “richness” can be conceptualized as a “distant measure” of image content. As shown in Figure 1, the number of objects identified by the image segmentation function does not indicate the number of physical objects that a human reader or an object detection algorithm may recognize. Instead, the number of objects in images as identified by the “bwlabel” function of the EBImage package is an indicator of the visual richness of the image.

Figure 3.

Original versus segmented image. (a) Original image. (b) Segmented image (threshold: 0.90). No. of objects: 60. (c) Segmented image (threshold: 0.60). No. of objects: 394. (d) Segmented image (threshold: 0.40). No. of objects: 497. (e) Segmented image (threshold: 0.20). No. of objects: 304. (f) Segmented image (threshold: 0.10). No. of objects: 241.

As a metric, the number of objects identified by image segmentation is different from constructs such as visual complexity, clutter, and entropy that have been applied widely in vision research. Perceived visual complexity of images is caused by a range of factors, such as the quantity of objects, clutter, openness, symmetry, organization, and variety of colors (Oliva et al., 2004). Similarly, Pieters et al. (2010) argued that visual imagery, such as in advertisements, is complex if it has dense perceptual features and/or elaborate creative design. Although the number of objects identified through the image segmentation process indicates the extent of perceptual features and may correlate with the presence of objects and/or a range of colors in a given image, this measure cannot be considered as a metric of visual complexity. This is because the number of objects recognized by the segmentation function does not take into account the relationship between physical objects included in the image and the perceived complexity caused by their location or arrangement. Oliva et al. (2004) note that visual complexity relates to both object variety (i.e., quantity as well as the range of objects) and surface variety (i.e., complexity caused by the variety of materials and surface styles). Although image segmentation can detect the extent of visual stimuli in an image, it cannot differentiate between object variety and surface variety in visual content.

Visual clutter, another measure used to examine visual content, relates to a surplus of objects in a display, creating a “state in which excess items, or their representation or organization, lead to a degradation of performance at some task” (Rosenholtz et al., 2007, p. 3). As Moacdieh and Sarter (2007) noted, clutter relates not only to the number of objects in a display but also to their structure, organization, and order. Although defining clutter in terms of regions in an image rather than objects make a problem more traceable (Bravo & Farid, 2008), we do not treat the number of objects identified by the segmentation process as a metric of visual clutter. For instance, the image on the left (i.e., nature) in Figure 4 may be perceived as less “cluttery” and/or complex and more aesthetically pleasing than the image on the right (i.e., junk shop), although the former has a considerably higher number of objects (or regions) than the latter (see Figure 4c and d). Moreover, the number of segments in an image does not indicate visual entropy, a metric of system complexity which, according to Schieber and Gilland (2008), shows that a system is more complex if more information is needed for it to be specified. Entropy can be seen as a measure of diversity and it is maximized if items in a collection of things are different from each other (Stamps, 2003). Stamps claims that entropy is a predictor of impressions of visual diversity. Visual affluence is different from diversity or entropy, as images that include patterns consisting of the same object may include more segments than images in which the same object is not repeated, given that any other object is not present in such images.

Figure 4.

Segmentation of an image representing a scenery and object clutter. (a) Original image—Nature. (b) Original image—Junk Shop. (c) Segmented image—Nature. (d) Segmented image—Junk Shop.

Given that the proposed measure detects “regions” in images, we suggest that the number of objects identified through image segmentation can be considered as a measure of visual affluence of images. Affluence, as defined by the Oxford Dictionary (oxforddictionaries.com, n.d.), is “The state of having a great deal of money; wealth.” The notion of visual affluence is characterized by the “visual richness” as measured by the extent of regions of color present in a given image. For instance, Face B in Figure 5 (No. of objects: 91,110) can be considered as a more visually affluent photograph than Face A (No. of objects: Face A: 1170), although both images are close-up images of faces. Figure 6 provides a closer look at a selected region (i.e., the eye on the right) in Face B. As shown in Figure 6, detection of subtle details in the image, such as the human figure in the eye, may demand more effort from the viewer. We define visual affluence as the plenitude of visual stimuli, such as objects and surfaces containing a variety of color regions, present in visual imagery. From this perspective, complex, cluttery, and entropic images may be visually affluent if they contain a high number of color regions.

Figure 5.

Differences in visual affluence in facial photographs. (a) Original image—Face A. (b) Original image—Face A. (c) Original image—Face B. (d) Segmented image.

Figure 6.

A close-up of a selected area in a highly affluent image.

Differences in visual affluence between images within a hashtag may indicate internal incongruence as well as variability in image attributes. Figure 7 includes four segmented images representing two hashtags (#Trump and #Graffiti) that display internal incongruence as well as variability. As mentioned previously, the pair of images representing the hashtag #Trump shows internal incongruence (see Figure 1b), as the direct relevance of the image on the right (female figure) to the hashtag cannot be established without knowledge of the user’s perspective. In contrast, the image on the left is directly related to the hashtag, as it includes president Trump and information related to his performance. These images were segmented (Figure 7—top) to examine their affluence levels. The results showed that images that contain letters and dark, single-color backgrounds are less affluent than images that include subtle shades. Accordingly, the image that seems irrelevant to the hashtag (female figure) was more affluent (No. of objects: 757) compared to the more relevant image (No. of objects: 516). This shows that differences in visual affluence do not explain relevance. In other words, visual affluence does not depend on or relate to relevance.

Figure 7.

Internal incongruence and variability captured using image segmentation. (a) #Trump. (b) #Grafitti.

We also segmented the pair of images given in Figure 2 to examine differences in affluence between images that show variability. The segmented images are given in Figure 7 (bottom). The results showed that the larger image on the left that includes a complex background is more affluent than the other, although the images may convey similar meanings. From this perspective, visual affluence is capable of explaining variability more accurately than internal incongruence. To elaborate visual affluence further, we segmented three images representing the hashtag #food. These images were selected to focus on three different qualities: (1) details in the objects on the foreground (top), (2) patterns/details in the background (middle), and (3) blurriness (bottom). Figure 8 shows images before (left) and after segmentation (right). The image on the top that included detailed objects in the foreground had the highest number of objects (2,643). The segmentation function also identified the rough pattern in the background of the picture in the middle (No. of objects 780). However, the image with a less complicated background and a blurry region had the least number of objects (258). This shows that images containing objects with complex texture and background patterns are more affluent, while images that include blurry backgrounds are less affluent than others.

Figure 8.

Images with different levels of affluence (#Food). (a) Objects with texture, 2,643 objects. (b) Detailed/textured background, 780 objects. (c) Blurry background, 258 objects.

To apply the proposed measure across hashtags, we measured visual affluence in a sample representing five Instagram hashtags (#food, #nature, #graffiti, #minimalism, and #instagood). The sample used for analysis included 2,683 images (#Food: 500, #Nature: 518, #Minimalism: 468, #Graffiti: 500, #Instagood: 697) and was obtained using Netlytic (Gruzd, 2016) before Instagram decided to limit API access. Prior to examining differences among hashtags, we calculated the visual affluence of randomly selected images representing three hashtags (Figure 9: top: #Minimalism, middle: #Nature, and bottom: #Food). These images were segmented at three different thresholds (low: 0.25, middle: 0.50, and high: 0.75) to identify a global threshold level for analysis. In general, the number of objects was different between images at all three threshold levels. A threshold level of 0.5 captured a higher number of objects in three Instagram images representing three hashtags. Therefore, 0.5 was selected as the optimum global threshold level for segmentation. Table 1 shows minimum and maximum affluence levels, means and standard deviations, and skewness and kurtosis statistics for each hashtag. According to these statistics, two hashtags (#Nature and #Graffiti) had higher mean values than the other hashtags. Mean ranks and Mann–Whitney U test were used to examine differences between hashtag pairs, as the data for each hashtag were not normally distributed. Table 2 provides mean ranks and results of the Mann–Whitney U tests for ten pairs of hashtags in the dataset. The results showed that eight out of ten tests were significant. According to the results given in Table 2, #Food and #Instagood were not different from each other in terms of visual affluence. Similarly, #Nature and #Graffiti had similar affluence levels. In general, these results indicate that the measure can be used to detect differences among hashtags.

Figure 9.

Segmented Instagram images.

Table 1.

Descriptive Statistics—Visual Affluence in Instagram Hashtags.

Hashtag	N	Min	Max	M	SD	Skewness		Kurtosis
Hashtag	N	Min	Max	M	SD	Statistic	Std. error	Statistic	Std. error
#Minimalism	468	1	14,395	1,021.87	1,556.564	3.801	.113	21.214	.225
#Food	500	0	14,555	1,055.26	1,237.738	4.052	.109	31.285	.218
#Nature	518	1	17,394	1,860.85	2,317.030	2.833	.107	10.776	.214
#Instagood	697	1	13,661	1,119.98	1,570.803	3.567	.093	17.891	.185
#Graffiti	500	0	13,489	1,763.04	1,813.362	2.884	.109	12.561	.218

Table 2.

Mean Ranks and Mann–Whitney U Test Results.

Test	Mean rank	Sum of ranks	Mann–Whitney U	p
Test 1: #Food vs. #Nature	455.08	227,540.00	102,290.00	.000
Test 1: #Food vs. #Nature	562.03	291,131.00	102,290.00	.000
Test 2: #Food vs. #Minimalism	520.26	260,128.00	99,122.00	.000
Test 2: #Food vs. #Minimalism	446.30	208,868.00	99,122.00	.000
Test 3: #Food vs. #Graffiti	425.34	212,671.50	87,421.50	.000
Test 3: #Food vs. #Graffiti	575.66	287,828.50	87,421.50	.000
Test 4: #Food vs. #Instagood	620.10	310,050.00	163,700.00	.074
Test 4: #Food vs. #Instagood	583.86	406,953.00	163,700.00	.074
Test 5: #Nature vs. #Minimalism	567.01	293,712.50	83,132.50	.000
Test 5: #Nature vs. #Minimalism	412.13	192,878.50	83,132.50	.000
Test 6: #Nature vs. #Graffiti	494.85	256,332.50	121,911.50	.106
Test 6: #Nature vs. #Graffiti	524.68	262,338.50	121,911.50	.106
Test 7: #Nature vs. #Instagood	692.45	358,689.00	136,778.00	.000
Test 7: #Nature vs. #Instagood	545.24	380,031.00	136,778.00	.000
Test 8: #Minimalism vs. #Graffiti	388.87	181,993.00	72,247.00	.000
Test 8: #Minimalism vs. #Graffiti	574.01	287,003.00	72,247.00	.000
Test 9: #Minimalism vs. #Instagood	548.63	256,756.50	147,010.50	.004
Test 10: #Graffiti vs. #Instagood	606.08	422,438.50	117,903.50	.000
	711.67	355,836.50
	518.17	361,166.50

Conclusion

The visual perception and processes of meaning-making of the social media image are becoming urgent considering the variety and volume of images on social media platforms. Despite the apparent “visual turn” (Gibbs et al., 2015) of social media, research in this domain is still in its preliminary stages, when comparing it with the textual analysis of social media communication (Faulkner et al., 2018; Highfield & Leaver, 2016). The social media photograph and the interpretation of its image-based and intertextual content is more complex than that of a physical print. As discussed previously, the communicative purpose and immediate qualities of social media photography are intertwined with the dynamics of the platform and algorithmic processes (Rubinstein & Sluis, 2013). A recurring theme in the literature has been the question, challenge, and “ambiguity” of a single image interpretation. The huge amount of mobile social media images proliferates at considerable speed across networks, systems, and audiences, while rarely being looked at (Lister, 2013). In relation to the image economy of the web (Rubinstein & Sluis, 2013), the context for interpretation of individual images is difficult (Hand, 2012), challenging the traditional approach of visual qualitative research. Highlighting the issues of internal incongruence and variability, we encourage a new line of inquiry by framing visual affluence as a meaning-independent basis to capture the “richness” of online images. Image segmentation is an established measure, especially in life sciences, and we demonstrate that it can be used to quantify the richness of online social photography, including large samples. Visual affluence should be treated as a concept (or a visual property) rather than a “big data” analysis technique. Similarly, neither is it an alternative to techniques such as deep learning that are used for object identification in images. As visual affluence can be applied to a single image as well as any number of images, it is not subject to challenges related to deep learning, such as the need for large volumes of data for training algorithms, network overfitting, and brittleness (see Wason, 2018). Accordingly, this work should not be considered as an invention of a new technique to read images.

This discussion can be used to encourage what can be called a visually oriented turn in online social photography research, at least within the field of social media studies. Such a turn may benefit from hybrid methods that integrate automated data analysis with other methodologies, such as experimental designs and surveys. While our discussion of the literature focuses on highlighting issues such as internal incongruence, we have not adequately dealt with the rich body of academic work related to vision that can provide interdisciplinary insight into initiating such a turn. Previous work that shows associations between visual qualities and psychological state (e.g., Gong et al., 2017; Jue & Kwon, 2013; Kim & Hyun, 2018; Specker et al., 2018) can be extended with experiments examining correlations between visual affluence and such attributes. Moreover, survey-based research can examine correlations between user attributes and visual affluence. Such analysis may help explain the psychological effects of visual hashtags with varying degrees of affluence. This is particularly important as we have demonstrated that visual hashtags may contain different levels of affluence. Moreover, associations between user characteristics, such as personality aspects (see Hyun & Kim, 2019), content preferences, and visual affluence can be examined to understand how images of different levels of affluence appeal to certain personalities. Further work can also examine accumulation of and variances in affluence within hashtags from a more microscopic point of view. The proposed measure can be applied beyond online social photography. For instance, variances in visual affluence across frames in online videos can be used to examine user reactions to such content. Such analysis can extend work that focuses on platforms such as YouTube and Vimeo to a meaning-independent analysis of effects. Moreover, work on implications of visual affluence can benefit a range of disciplines beyond social media studies, such as advertising, branding, and political communication. For instance, previous work that examined effects of factors such as color and perceived complexity on behavioral outcomes (e.g., persuasion, comprehension, preferences) (e.g., Kareklas et al., 2014, 2019; van Mulken & Forceville, 2010) can be used as a basis to examine how visual affluence affects consumer reactions to marketing communication campaigns.

Empirical analysis that we have discussed has several limitations. First, our sample was limited and the distributions that represented each hashtag were not normal. Therefore, more analysis needs to be conducted using large samples. We collected our data before the platform limited accessibility for data collection purposes. Issues arising from global threshold levels should also be examined, as the effectiveness of segmentation depends on threshold levels.

Footnotes

The authors wish to thank the anonymous reviewers for their constructive comments and Dr Jaqueline Young at the University of Strathclyde for her feedback.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research,authorship,and/or publication of this article.

Funding

The author(s) received no financial support for the research,authorship,and/or publication of this article.

ORCID iD

Chamil Rathnayake

Author Biographies

Chamil Rathnayake (PhD,University of Hawaii) is a lecturer in the School of Humanities at the University of Strathclyde. His current research focuses on patterns of engagement in issue-response networks,with a special emphasis on novel forms of interaction allowed by new media affordances.

Irida Ntalla (PhD,City University of London) is a Lecturer in Media Management at Middlesex University in London. Her research interests include media and creative industries,media and art management,audience theory,digital cultures,and digital publishing.

References

Abidin

(2016). Visibility labour: Engaging with influencers’ fashion brands and # OOTD advertorial campaigns on Instagram. Media International Australia, 161(1), 86–100.

Ab. Jalil

Yunus

R. M.

Said

N. S.

(2012). Environmental colour impact upon human behaviour: A review. Procedia: Social and Behavioral Sciences, 35, 54–62.

Amadasun

King

(1989). Textural features corresponding to textual properties. IEEE Transactions on Systems, Man, and Cybernetics, 19(5), 1264–1274.

Batchen

(2002). Each wild idea: Writing photography, history. MIT Press.

Bravo

M. J.

Farid

(2008). A scale invariant measure of clutter. Journal of Vision, 8(1), 1–9.

Bruns

Burgess

J. E.

(2011). The use of Twitter hashtags in the formation of ad hoc publics. In Proceedings of the 6th European consortium for political research (ECPR) general conference (pp. 1–12). http://snurb.info/files/2011/The%20Use%20of%20Twitter%20Hashtags%20in%20the%20Formation%20of%20Ad%20Hoc%20Publics%20(final).pdf

Caliandro

(2018). Digital methods for ethnography: Analytical concepts for ethnographers exploring social media environments. Journal of Contemporary Ethnography, 47(5), 551–578.

Evans

S. K.

Pearce

K. E.

Vitak

Treem

J. W.

(2017). Explicating affordances: A conceptual framework for understanding affordances in communication research. Journal of Computer-Mediated Communication, 22(1), 35–52.

Fardouly

(2018). Instagram use and young women’s body image concerns and self-objectification: Testing mediational pathways. New Media & Society, 20(4), 1380–1395.

10.

Faulkner

Vis

D’Orazio

(2018). Analysing social media images. In Poell

Burgess

Marwick

(Eds.), SAGE handbook of social media (pp. 160–178). SAGE.

11.

Filimonov

Russmann

Svensson

(2016). Picturing the party: Instagram and party campaigning in the 2014 Swedish elections. Social Media + Society, 2(3). https://doi.org/10.1177/2056305116662179

12.

Gerlitz

Helmond

(2013). The like economy: Social buttons and the data-intensive web. New Media & Society, 15(8), 1348–1365.

13.

Gibbs

Meese

Arnold

Nansen

Carter

Gibbs

Carter

(2015). #funeral and Instagram: Death, social media, and platform vernacular. Information, Communication & Society, 18(3), 255–268. https://doi.org/10.1080/1369118X.2014.987152

14.

Gibson

J. J.

(1986). The theory of affordances: The ecological approach to visual perception. Lawrence Erlbaum.

15.

Ging

Garvey

(2018). “Written in these scars are the stories I can’t explain”: A content analysis of pro-ana and thinspiration image sharing on Instagram. New Media & Society, 20(3), 1181–1200.

16.

Gong

Wang

Hai

Shao

(2017). Investigation on factors to influence color emotion and color preference responses. Optik: International Journal for Light and Electron Optics, 136, 71–78.

17.

Gruzd

(2016). Netlytic: Software for automated text and social network analysis. https://netlytic.org/

18.

Guidry

J. P. D.

Austin

L. L.

Carlyle

K. E.

Freberg

Cacciatore

Jin

Messner

(2018). Welcome or not: Comparing #refugee posts on Instagram and Pinterest. American Behavioral Scientist, 62, 512–531.

19.

Hand

(2012). Ubiquitous photography: Digital media and society series. Polity Press.

20.

Heaps

Handel

(1999). Similarity and features of natural textures. Journal of Experimental Psychology, 25(2), 299–320.

21.

Highfield

Leaver

(2015, January). A methodology for mapping Instagram hashtags. First Monday. https://firstmonday.org/article/view/5563/4195

22.

Highfield

Leaver

(2016). Instagrammatics and digital methods: Studying visual social media, from selfies and GIFs to memes and emoji. Communication Research and Practice, 2(1), 47–62.

23.

Hyun

Kim

(2019). Instagram user characteristics and the color of their photos: Colorfulness, color diversity, and color harmony. Information Processing and Management, 56(4), 1494–1505.

24.

Jue

Kwon

(2013). Does colour say something about emotions? Laypersons’ assessments of colour drawings. The Arts in Psychotherapy, 40(1), 115–119.

25.

Kareklas

Brunel

F. F.

Coulter

R. A.

(2014). Judgment is not color blind: The impact of automatic color preference on product and advertising preferences. Journal of Consumer Psychology, 24(1), 87–95.

26.

Kareklas

Muehling

D. D.

King

(2019). The effect of color and self-view priming in persuasive communications. Journal of Business Research, 98, 33–49.

27.

Kember

(2013). Ambient intelligent photography. In Lister

(Ed.), The photographic image in digital culture (pp. 1–21). Routledge.

28.

Kim

Hyun

(2018). Using computer vision techniques on Instagram to link users’ personalities and genders to the features of their photos: An exploratory study. Information Processing and Management, 54(6), 1101–1114.

29.

Kuzinas

(2013). The power of colour on content: Associations, evoked by simple and complex pictures. Procedia: Social and Behavioral Sciences, 84, 1397–1402.

30.

Lister

(1995). Extracts from introduction to the photographic image in digital culture. In Wells

(Ed.), The photographic image in digital culture (pp. 218–227). Taylor & Francis.

31.

Lister

(2013). The photographic image in digital culture ( Lister

, Ed., 2nd ed.). Routledge.

32.

MacDowall

L. J.

de Souza

(2018). “I’d double tap That!!” Street art, graffiti, and Instagram research. Media, Culture & Society, 40(1), 3–22.

33.

Moacdieh

Sarter

(2007). Display clutter: A review of definitions and measurement techniques. Human Factors, 57(1), 61–100.

34.

Oleś

Pau

Sklyar

Huber

(2018). Introduction to EBImage. https://www.bioconductor.org/packages/devel/bioc/vignettes/EBImage/inst/doc/EBImage-introduction.html

35.

Oliva

Mack

M. L.

Shrestha

Peeper

(2004). Identifying the perceptual dimensions of visual complexity of scenes. Proceedings of the Annual Meeting of the Cognitive Science Society, 26, 1041–1046. https://escholarship.org/uc/item/17s4h6w8#author

36.

oxforddictionaries.com. (n.d.). Affluence. Retrieved February 12, 2019, from https://en.oxforddictionaries.com/definition/affluence

37.

Pau

Fuchs

Sklyar

Boutros

Huber

(2010). EBImage: An R package for image processing with applications to cellular phenotypes. Bioinformatics, 26(7), 979–981.

38.

Pieters

Wedel

Batra

(2010). The stopping power of advertising: Measures and effects of visual complexity. Journal of Marketing, 74(5), 48–60.

39.

Rajinikanth

Couceiro

M. S.

(2015). RGB histogram based color image segmentation using firefly algorithm. Procedia Computer Science, 46, 1449–1457.

40.

Rathnayake

Suthers

D. D.

(2018). Twitter issue response hashtags as affordances for momentary connectedness. Social Media + Society, 1–10. https://doi.org/10.1177/2056305118784780

41.

Rodriguez

N. S.

Hernandez

(2018). Dibs on that sexy piece of ass: Hegemonic masculinity on TFM Girls Instagram account. Social Media + Society, 4, 1–12.

42.

Rogers

(2018). Otherwise engaged: Social media from vanity metrics to critical analytics. International Journal of Communication, 12, 450–472.

43.

Rosenholtz

Nakano

(2007). Measuring visual clutter. Journal of Vision, 7(2), 1–22.

44.

Rubinstein

Sluis

(2008). A life more photographic. Photographies, 1(1), 9–28.

45.

Rubinstein

Sluis

(2013). Concerning the undecidability of the digital image. Photographies, 6(1), 151–158.

46.

Schieber

Gilland

(2008). Visual entropy metric reveals differences in drivers’ eye gaze complexity across variations in age and subsidiary task load. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 52, 1883–1887.

47.

Schlesselman-Tarango

(2013). Searchable signatures: Context and the struggle for recognition. Information Technology and Libraries, 32, 5–19.

48.

Skourt

B. A.

Hassani

A. E. L.

Majda

(2018). Lung CT image segmentation using deep neural networks. Procedia Computer Science, 127, 109–113.

49.

Song

Liu

Yan

(2019). Side scan sonar image segmentation and synthesis based on extreme learning machine. Applied Acoustics, 146, 56–65.

50.

Sontag

(1973). On photography. RosettaBooks, LLC.

51.

Specker

Leder

Rosenberg

Hegelmaier

L. M.

Brinkmann

Mikuni

Kawabata

(2018). The universal and automatic association between brightness and positivity. Acta Psychologica, 186, 47–53.

52.

Spence

(2018). Background colour & its impact on food perception & behaviour. Food Quality and Preference, 68, 156–166.

53.

Stamps

A. E.

(2003). Advances in visual diversity and entropy. Environment and Planning B: Planning and Design, 30, 449–463.

54.

Tiggemann

Zaccardo

(2018). “Strong is the new skinny”: A content analysis of #fitspiration images on Instagram. Journal of Health Psychology, 23, 1003–1011.

55.

van Esch

Heller

Northey

(2019). The effects of inner packaging color on the desirability of food. Journal of Retailing and Consumer Services, 50, 94–102.

56.

Van House

N. A.

(2011). Personal photography, digital technologies and the uses of the visual. Visual Studies, 26(2), 125–134.

57.

van Mulken

Forceville

(2010). The impact of perceived complexity, deviation and comprehension on the appreciation of visual metaphor in advertising across three European countries. Journal of Pragmatics, 42(12), 3418–3430.

58.

Wal

T. V.

(2007). Folksonomy. http://vanderwal.net/folksonomy.html

59.

Wason

(2018). Deep learning: Evolution and expansion. Cognitive Systems Research, 52, 701–708.

60.

Liu

Jiang

(2018). Catenary image segmentation using the simplified PCNN with adaptive parameters. Optik: International Journal for Light and Electron Optics, 157, 914–923.

61.

Xian

Zhang

Cheng

H. D.

Zhang

Ding

(2018). Automatic breast ultrasound image segmentation : A survey. Pattern Recognition, 79, 340–355.

62.

Wang

Yuan

Cheng

Wang

Carson

P. L.

(2019). Medical breast ultrasound image segmentation by machine learning. Ultrasonics, 91, 1–9.

63.

Zappavigna

(2016). Social media photography: Construing subjectivity in Instagram images. Visual Communication, 15(3), 271–292.

“Visual Affluence” in Social Photography: Applicability of Image Segmentation as a Visually Oriented Approach to Study Instagram Hashtags

Abstract

Keywords

Introduction

Social Photography

Visual Hashtags, Internal Incongruence, and Variability

Quantifying Social Photography: Image Segmentation as an Approach to Measuring Visual Affluence

Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

ORCID iD

Author Biographies

References