Sage Journals: Discover world-class research

Abstract

The scientific literature has presented evidence of the links between innovation and change and has published excellent methodologies to analyze them. Nowadays, international scientific programs like Horizon Europe prioritize social impact and co-creation; researchers need to develop methodologies to analyze the link of innovation with change and new knowledge and specially with social impact. This paper presents an innovative methodological approach to this endeavor using Social Media Analytics to investigate citizens' participation in paying attention to and differentiating between innovations with social impact and innovations without social impact. The method used to address this aim is Social Media Analytics, specifically through a Twitter sample on innovation and social impact composed of 16,794 tweets obtained during January–June 2021. The result obtained indicates that the definition of methodologies to capture citizens’ participation in paying attention to and differentiating between innovation and social impact is crucial for advancing this innovative methodological approach to analyze innovation with social impact.

Keywords

methodological approach innovation social media analytics social impact

Introduction

Methodologies for research innovation and social impact are a challenge nowadays. In this sense, one first step is to reflect on the meaning of innovation and how this concept could be linked to social impact and possible methodologies to address it. The term “innovation” has different meanings depending on its application, and usually there are some misunderstandings according to Kahn (2018). There are two definitions for innovation: the first one referred to an outcome and the second one to a process. Innovation can be applied in different domains in relation to outcomes, processes, mindset, strategies (Kahn, 2018). Moreover, innovation is also a current term linked with research and knowledge creation. For instance, one of the priorities for scientific programs like Horizon Europe is to promote research that includes co-creation, social impact, and innovation (European Commission, 2020). Although the EU has considered impact and social impact in previous research and innovation policies and framework programs, the current Horizon Europe incorporates a special emphasis on these strategies. The field of innovation is a transversal aspect in most research areas; business, education, health, experimental sciences, engineering, and agriculture are good examples of this transversality.

Understanding innovation as an outcome and process (Kahn, 2018), a challenge is to know which methodologies are available for analysing this topic. In fact, diverse disciplines have applied various methodologies to analyze innovation linked with change or the creation of new knowledge. In this sense, the European Commission has prioritized that research outcomes, changes and new knowledge should be linked to the societal impact during last years. The introduction of societal impact as a key indicator to evaluate the Framework Program for Research (Flecha, 2018) is evidence of this approach, and innovation should include this indicator as well. Social impact and societal impact both are terms used to explain the social improvements obtained through the transference of research results that represents effective steps towards the fulfilment of officially defined social goals (Pulido et al., 2018), such as UN Sustainable Development Goals or Missions of Horizon 2020, among the social goals defined by other institutions. Including a focus on social impact in the research work helps guarantee that the investment in it will contribute to improving people live conditions. This paper aims to review the different methodologies used to analyze innovation, with particular attention to those contributions that promote the integration of social impact. The innovative methodological approach is exemplified in this case with a specific method, Social Media Analytics, addressed to investigate citizens’ participation in paying attention to and differentiating between innovations with and without social impact.

The article presents an overview in which a review of diverse methodologies related to innovation is made and classified based on different main topics. The first topic explores the diversity of methodologies that have analyzed innovation from different fields. The second topic is based on methodologies focused on assessing national innovation systems. Finally, the third focuses on methodologies that have analyzed research proposals' innovation based mainly on the peer review process. The introduction ends with an explanation of the trend of the Framework Program to link innovation with societal impact aimed to present an innovative methodological approach with the method used in this study, Social Media Analytics.

Diverse Methodologies to Analyze Innovation

The literature review presents some findings extracted from the papers obtained through the combination of the searchable keywords “innovation” and “methodologies” of the last 5 years in the Web of Science and some relevant documents on this topic. The selection of the papers has centered on contributions focused on methods that include innovation as a crucial topic. There are quantitative and qualitative methods and mixed methods. Some of them focused on helping the guidance of innovation in different fields; others focused on analysing the innovation at macro and micro level, for instance, the innovation capacity of a country, an organization, or a research program. The following contributions selected exemplify some of the findings extracted from the literature review.

One of the topics is the diversity of methodologies carried out for analysing innovation from different fields. According to Druckman and Donohue (2020), we live in an era of innovation in social methods; for instance, linear models, mixed methods, systems frameworks, machine learning, and new approaches to fieldwork are available. These studies often benefit from interdisciplinary research teams. According to the authors, “On the qualitative side, we see the value of new technologies for more efficient data collections” (Druckman & Donohue, 2020; p. 16). Other methodologies used are based on the conception of social labs (Timmermans et al., 2020). Other methodologies used are based on the conception of social labs (Timmermans et al., 2020). The social lab is an inclusive methodology according to Timmermans et al. (2020) and consists in a space for “doing social experiments in a practical context where experts and stakeholders join together to initiate actions focused on tackling challenges without being constrained by predetermined project plans, (…) social labs provide precisely the ability to proactively experiment with circularity” (Timmermans, 2020:412). In relation to methodologies addressed to specific fields, there is an example of a methodology proficient in guiding successful investment in the IoT (Internet of Things) (Ammirato et al., 2019). According to the authors, the methodology combines two key sources: a) literature review and b) impressions collected from informal surveys and in-depth interviews. Other studies focused on methodologies related to innovation that help the replicability in a different context, promoting the anchoring for scaling (Seifu et al., 2020). Specifically in education, the field usually links innovation with change. For instance, in a study from Higher education, the methodology based on inquiry about “innovative action” embraces three compelling dimensions: a) the best alternative to achieve innovation, b) the best means (looking for efficiency and effectiveness) and c) the best results (evaluation of aims, means and results) (Penalva, 2021). In other fields, for instance, in business and health, innovation is linked with solutions or solving problems. Sudbury-Riley et al. (2020) reveal the Trajectory Touchpoint Technique (TTT), a service design methodology aimed at increasing innovation at the service of customer experiences. The design and development of the TTT uses design science research, a goal-oriented methodology that produces practical solutions to organizational problems (Sudbury-Riley et al., 2020). From the healthcare innovation, there is a study focused on methodologies based on problem-based approach. In this case, and according to the authors, the method needs to define the problem by applying techniques such as ethnographic research, market analysis and stakeholder exploration (Soliman et al., 2020). Still, there is a need to investigate how the different techniques interact with one another and how to develop the methodology to focus on how innovation should be oriented for the advancement of problem-driven or need-led innovation approaches (Soliman et al., 2020).

Another topic is focused on methodologies focused on the assessment of national innovation systems, as well as, for example, specific methodologies for studying innovative practices in policy. Regarding the assessment of the national innovation systems (Freeman, 1995; Lundvall, 1992; Nelson, 1993; OECD, 1997), there are contributions focused on methodologies based on science and technology statistics in order to evaluate the international position of those systems. As stated by Chaves et al. (2020), the methodology proposed enables intertemporal tracking of the trajectories of selected countries trajectories of selected countries “placed in cluster according to quantitative characteristics of their innovation systems” (Chaves et al., 2020; p. 44). Specifically, the methodology proposed helps form a link between the analysis of science and technology statistics and more structural assessments of the global capitalist dynamic. Further, it helps clustering countries depending on the results obtained through the dataset built (Chaves et al., 2020). Other studies, instead of quantitative methods, use qualitative ones. This can be seen, for instance, in the use of the Benchmarking method to rank the countries in relation to innovation (Gerlitz et al., 2020), or the use of the interviews in the example of the analysis of the innovative practices on equal participation of women in politics in Kerala linked to the Sustainable Development Goal (SDG) of the United Nations Organization (Babu & Jose, 2020). Another study found the sustainability of urban surfaces could be assessed with a method of SDG based on a top-down approach for identifying innovations’ potential contribution to the achievement of SDG before its introduction (Henzler et al., 2020). Likewise, it is important to refer to the Manuals the OECD (2015, 2018) has published in different versions for decades. The Frascati Manual is being published since 1963 and is devoted to the methodologies to collect and report data on research and experimental development. The Oslo Manual, on the other hand, deals, since 1992, with the methodologies related to data on innovation.

The last topic is focused on how research funding programs usually employ the peer review process to select which research proposals should be funded to guarantee innovation. Assessing this peer review process through a methodology for analysing the innovation of proposals is one of the concerns pointed out in these studies. Parreira et al. (2019) proposed a method that involves the use a multicriteria decision model under a multi-step decision-making process. However, other studies focus on the policy attitude within scientific innovation through qualitative inquiry to better understand how the policy attitude operates, considering that this attitude includes a collaborative ideal and influence of values and organizations (Friberg & Englander, 2019). This point is crucial for the authors because science is an integrated part of the policy and industry field, and policy attitude influences directly how the knowledge is perceived; qualitative inquiry is needed to “to explicate the human scientific meaning of a specific attitude driven by an interest in a sociocultural context” (Friberg & Englander, 2019; p. 1). Other studies are focused on how to identify innovation topics to assess the policymakers related to funding research programs inside this field. The study developed by Zhang et al. (2016) proposed an analytical method to cluster associated terms and phrases to constitute meaningful technological topics and their interactions and identify changing topical emphases (Zhang et al., 2016). In all, science and technology studies contribute with knowledge how to approach Responsible research and innovation (RRI) (Smith et al., 2021).

Innovation, Co-Creation and Social Impact in Framework Program for Research

International scientific programs like Horizon Europe prioritizes social impact as a key indicator and co-creation as a pathway to reach real improvements where innovation is included and addressed to this aim. According to Robinson et al. (2020) Horizon Europe integrated RRI as an overarching principle following the three O’s: “Open Innovation, Open Science, Open to the world ”(European Commission, 2016). The co-creation with citizens plays a key role in this program, and they have a triple role: being an end-user, being a relevant voice in terms of values and expectations and being an active player that can contribute with innovative ideas. Now, co-creation approaches are a consolidated trend aimed to collaborate in the design and create a solution based on multi-stakeholder collaboration (Robinson et al., 2020). This strategy also has the concept “Open” as a key word following the broad policies of the EU. Moreover, the co-creation intervention is also a trend in different fields, for instance, in Education (Ruiz-Eugenio et al., 2021)

There are different indicators and methodologies to evaluate the impact (scientific, societal and policy) of the research developed in the reference report on Monitoring the impact of EU framework programs (van des Besselaar et al., 2018). Institutional changes and MoRRI indicators represent a strong mechanism to asses engagement in Open Science (Robinson et al., 2020) and the societal impact (Flecha, 2018), contributing also to the evaluation of innovation. On the other hand, strategies identified to achieve social impact include a meaningful involvement of stakeholders and end-users through the project lifespan and public deliberation with a diverse public (Aiello et al., 2021). In fact, the evaluation of national innovation systems is simplified with a specific proposal considering that innovation should be addressed to sustainable growth, based on principles that take into account economic, social, and environmental (ecological) aspects, according to Bielinksa-Dusza and Hamerska (2021). In this context, it is also useful to consider the Triple Helix model developed by Etzkowitz & Leydersdorf (1995). The authors analyze the relationship between universities, business and government, and further development with the fourth helix incorporates society and impact. Moreover, the fifth helix adds the natural environment, that is, the socio-ecological transition (Carayannis & Campbell, 2010). These models allow analysing in more detail the national innovation systems.

There is a strong trajectory developing methodologies addressed to assess innovation, as seen in previous contributions reviewed in this article. The literature review identifies a trend to link innovation with social impact and results, particularity in the Framework Program for Research. However, there is still a need to develop methodologies to analyze this approach. To that end, the analysis of citizens’ perception of social impact should be included due to its relevance for open innovation. Hence, it is key to assess social impact through including citizens’ voices in research and innovation, as without them co-creation, necessary for research and innovation as indicated by European Commission, is not possible.

Method

This study has used Social Media Analytics with the specific method of Social Impact in Social Media (Pulido et al., 2020) in order to analyze citizens’ participation in paying attention to and differentiating between innovation with or without social impact. The social impact in social media (SISM) methodology aims to identify quantitative and qualitative evidence of the potential or real social impact of research shared on social media. In this case, the focus is innovation instead of research. The steps of this methodology include (a) criteria selection of the social media datasets, (b) search strategies for extracting social media data, (c) application of ethical criteria on internet and social media research, (c) elaboration of a codebook, and d) qualitative and quantitative analysis of the social media dataset.

Data Collection

In order to collect and analyze citizens’ voices around innovation and social impact, researchers established combinations of keywords that included (1) “Innovation” and (2) four keywords related to social impact: “results,” “improvement,” “impact,” or “social impact.” The data collection took place in Twitter and included a maximum of 10,000 tweets under each combination of keywords published from January 1st, 2021 to June 30th, 2021. If more than 10,000 tweets were found under a combination of keywords, the criterion was to select the 10,000 with more interactions among users, which was measured in terms of retweets.

In this study, researchers defined as a unit of analysis the whole tweet, including videos, images, link, and websites attached in the message. In addition, the number of retweets obtained by each message was also considered. In those cases where the extracted tweet was part of a Twitter thread or conversation, the whole thread was included in the analysis.

Sample

The sample for this study included a total of 16,794 tweets which were extracted using python connected with the API of Twitter. During the codification process, some messages were excluded from the analysis for the following reasons: (1) the tweets contained broken links, deleted messages or pages with restricted access, (2) the tweets did not include enough information to be codified and analyzed as evidence of innovation with or without social impact, and (3) the tweets were spam or contained disrespectful language such as insults. Table 1 presents the total sample and the final dataset across the combination of keywords:

Table 1.

Sample.

Combination of keywords	Extracted messages	Excluded due to lack of access	Excluded due to lack of information or spam messages	Final sample
Innovation + results	10,000	410	1001	8589
Innovation + improvement	5029	62	483	4484
Innovation + impact	701	14	115	572
Innovation + social impact	1064	31	218	815

Therefore, the final dataset included a total of 14,460 valid tweets that were analyzed as detailed in the results’ section.

Dialogic Codebook

The second step was the elaboration of the dialogic codebook for the Communicative Content Analysis, which was designed following a dialogic approach. The categories to be included in the codification process were discussed based on an egalitarian dialogue among researchers, and decisions were made based on validity claims instead of power claims. A first codebook was elaborated to include the categories needed to achieve the aims of the research. Then, a pilot was conducted to test the codebook. The final version of the codebook, which is presented below (Table 2), included both predefined categories and categories emerged from the pilot.

Table 2.

Codebook.

Innovation Social Impact
Code	Name	Definition
0	Not valid	There is a broken link or there is not enough information to include the message in the analysis dataset. In addition, those messages that are spam or include disrespectful language (i.e., insult)
1	No	There is no social impact as a result of an innovation
2	Potential	Although the innovation has not promoted social impact yet, it will potentially achieve social impact. It includes prototypes, designs and guidelines to promote social impact or future transferences of innovations, among others
3	Real	The innovation has achieved social impact and improved people’s lives
Evidence of social impact
Code	Name	Definition
0	Not applicable	The message does not reflect social impact as a result of an innovation
1	No	Users do not provide any evidence of how an innovation has achieved social impact
2	Supposed evidence of social impact	Although users claim that the innovation has somehow improved people’s lives, there is no evidence source of this improvement
3	Evidence of social impact	Users provide evidence of how innovations have improved people’s lives; links or source cited
Area of innovation
Code		Definition
-		Not applicable o not valid
E		Education
H		Health
B		Business
M		Marketing
C		Climate change
U		Urbanism
Other		Other not mentioned previously
Several		Two or more areas involved

Communicative Content Analysis

This study has followed the methodology in the field of Social Media Analytics, which is the Communicative Content Analysis. Drawing upon the principles of the Communicative Methodology, this novel approach places co-creation of knowledge in the core of the process (Gomez et al., 2019). Therefore, an egalitarian dialogue among plural and diverse voices is ensured throughout the process. In line with previous studies related to Communicate Content Analysis on social media and applying dialogic reliability (Pulido et al., 2020), in this study, researchers worked collaboratively and established a constant egalitarian dialogue among them during the whole process. All researchers discussed and had the codebook, which oriented the Communicative Content Analysis, and used it to decide whether to assign one code or another to each message. If researchers had any doubts during this process and the classification of the tweet was not clear, the case was discussed with other researchers using validity claims and reaching an agreement based on a consensus. This process ensures dialogic reliability, as it is based on the plurality of voices and a constant and egalitarian dialogue among researchers. We have done the qualitative analysis of each tweet of the final sample (1440 tweets). The results are shared quantitatively because it is more suitable with the aim of this paper to show whether there is or not a trend to link innovation with social impact.

Ethical Considerations

Regarding ethical considerations, the present research adheres to international ethical criteria related to social media data collection and corresponding analyses; in particular, we have followed the ethical guidelines for social media research recommended by European Commission (2018). We have perceived the risk of harm to and conserved the anonymity of users. Additionally, we have read the terms, conditions, and legalities of each of the social media channels, and we have used only public information without identifying any user. Likewise, the data were appropriately coded and anonymized to avoid the possibility of traceability. Sets of data have been secured, saved, and stored. The dataset analyzed and the calculations performed are available in the Supplementary Materials (dataset). We cannot share all raw data due to the current terms of the social media channels and the General Data Protection Regulation (GDPR).

Results

The first step is to identify whether citizens in social media are linking innovation with social impact or not. The category of innovation linked to social impact is crucial for defining the variables to apply in the sample related to this method. According to Table 3, there is a trend to link innovation with social impact in the sample analyzed.

Table 3.

Innovation and Social Impact.

Keywords	NO		YES		Potential SI		Real SI
Keywords	n	%	n	%	n	%	n	%
Innovation + results	7145	83.20	1443	16.80	943	10.98	500	5.82
Innovation + improvement	3775	84.19	702	15.65	432	9.63	270	6.02
Innovation + impact	312	54.55	257	44.93	150	26.22	107	18.71
Innovation + social impact	424	52.02	389	47.73	279	34.23	110	13.50

The global variables (YES or NO) help identify the percentage of the number of tweets that link innovation with social impact, being useful for a quick view. In addition, the division of potential and real social impact helps understand the gradient of social impact, whether it is potential or real (Pulido et al., 2018). The highest percentage of tweets with social impact (potential and real) is the combination keyword of innovation and social impact (47.73%). Although this result was expected due to the election of that specific combination of keywords, it is important to note that not all the tweets under this combination have social impact, and the combination of innovation and impact also has a high percentage of innovation and social impact (44.93%). Regarding the percentage of real social impact, the highest result is related to the combination of innovation and impact (18.71%), more than five points above the combination of innovation and social impact (13.50).

The method should help identify those tweets that include evidence of social impact or supposed evidence. The first one (social impact with evidence) helps citizens, policy makers and stakeholders go directly to the source that shares evidence of the improvements obtained, and the supposed evidence implies a contrasting task for trust in the statement checked. The following Table 4 classifies tweets without social impact evidence from those tweets that have evidence or supposed evidence of social impact. The results indicate that supposed evidence of social impact has a higher percentage of tweets than tweets with evidence of social impact in all four combinations of keywords.

Table 4.

Evidence of Social Impact.

Keywords	NO		Supposed		YES
Keywords	n	%	n	%	n	%
Innovation + results	556	38.53	599	41.51	277	19.20
Innovation + improvement	359	51.14	242	34.47	101	14.39
Innovation + impact	109	42.41	90	35.02	58	22.57
Innovation + social impact	234	60.15	106	27.25	49	12.60

There is another interesting result to be analyzed and defined as category (Area) with different possible variables defined in the codebook (see Table 5). This result is helpful in order to analyze which area has the highest percentage of innovation with social impact. In the case of innovation and improvement, health is the area with a highest percentage (31.95%), as well as in the case of innovation and results (57.20%). In innovation and impact the variable several areas (23.14%) is the one with the highest percentage together with other areas not included in the selected list (22.35%). Among the specific areas included, education has the highest percentage which represents the third position (16.08%). Regarding innovation and social impact, the first one is several areas (31.36%), followed by other areas (20.82%), and very close appears business (20.57%). In a lower level, although playing a role, health and education are areas where innovation and social impact are a consolidated trend.

Table 5.

Areas with Innovation Linked to Social Impact.

Keywords	Business		Education		Health		Marketing		Climate Change		Urbanism		Other		Several
Keywords	n	%	n	%	n	%	n	%	n	%	n	%	n	%	n	%
Innovation + results	127	18.12	85	12.13	401	57.20	29	4.14	199	28.39	70	9.99	298	42.51	231	32.95
Innovation + improvement	71	10.13	52	7.42	224	31.95	2	0.29	80	1141	18	2.57	181	25.82	73	10.41
Innovation + impact	37	14.51	41	16.08	30	11.76	4	1.57	21	8.24	6	2.35	57	22.35	59	23.14
Innovation + social impact	80	20.57	34	8.74	37	9.51	0	0.00	24	6.17	11	2.83	81	20.82	122	31.36

Last, Table 6 helps to view which type of tweets of innovation with or without social impact capture more attention from citizens. In this case, it is important to incorporate the average of the RT because it may clarify the results shown in Table 5. In all areas that are taken into consideration (business, education, health, climate change, urbanism, other areas, and several areas) except for marketing, the average RT is higher in innovation linked to social impact than the one without this link. Thus, considering the average RT, the tweets that link innovation with social impact obtain more attention from citizens than the ones that do not link innovation with social impact.

Table 6.

Tweets and Retweets Innovation Linked to Social Impact.

	Innovation linked to social impact			Innovation without link social impact
Areas	Tweets	Retweets	Average RT	Tweets	Retweets	Average RT
Business	315	439	1.39	1880	1709	0.91
Education	212	267	1.26	194	161	0.83
Health	692	1084	1.57	352	268	0.76
Marketing	39	29	0.74	770	581	0.75
Climate change	324	702	2.17	73	52	0.71
Urbanism	105	245	2.33	36	6	0.17
Other	613	1117	1.82	3556	4313	1.21
Several	486	1253	2.58	87	63	0.72

The quantitative results shared above help draw a general overview of the innovation and social impact. Analysing results from a qualitative side allows for a more in-depth analysis of the messages, delivering more qualitative information about the object of study. Some of the examples shared quantitative evidence. For instance, on the area of climate change, there is a tweet that reflects on how the inclusion of solar panels has achieved a 35% reduction of energy consumption due to the investment in leading technologies and being at the forefront of innovation. On the area of urbanism, there is a tweet on how innovation is linked to improve a better sanitation, the innovation achieved this aim, and the evidence is disseminated in a newspaper. Before introducing the innovation, the municipality used to generate 180 tons of garbage and now it has been reduced to 130 tons, and it is working to reduce it further. On the area of health, there is a tweet reflecting how the investment on innovation linked to improve the life conditions through better care given has achieved the improvement of 2500 patients.

Limitations

The method used to apply this innovative methodological approach to analyze the link between innovation and social impact poses some limitations. In particular, there are five limitations to highlight. The first is related to the keyword selection. There is a specific list proposed by researchers involved, but this list could be modified with other keywords to extract other datasets. The results are limited to this particular dataset. For getting a consolidated overview of it, it would be necessary to amply the list of these keywords. The second limitation is the period. We have extracted a dataset within a specific time; hence, the results are limited to this period of time. Therefore, it would be necessary to schedule a monitoring process during an extensive period, such as a year, for getting an in-depth overview. The third limitation is regarding privacy settings; according to the ethical considerations, we cannot reveal in detail any information of the users who tweet different messages. This limitation does not allow deepening on who publishes what regarding innovation and social impact. In addition, the fourth limitation is that researchers involved in his study have developed the dialogic codebook regarding the decisions of different codes defined. The codes that were more challenging to identify have been those regarding the difference of real or potential innovation with social impact; the analysis of these tweets needed a dialogic contrast of researchers involved to classify in the corresponding code. Last, the fifth limitation is that of spam or disrespectful messages, which is solved deleting them from the final dataset analyzed.

Discussion

Usually, innovation has been linked to change and the creation of new knowledge with diverse methodologies in order to analyze them (Chaves et al., 2020; Druckman & Donohue, 2020; Seifu et al., 2020; Soliman et al., 2020; Sudbury-Riley et al., 2020; Timmermans et al., 2020). The literature reviewed has identified a need to analyze research innovation linked to societal impact (Aiello et al., 2021; Bielińska-Dusza & Hamerska, 2021; Robinson et al., 2020). Societal impact is crucial for the Framework Program for Research, and for this reason, there is a need to develop methodologies addressed to assess innovation with social impact. The European Framework Program published documentation on how to evaluate the societal impact with a list of indicators and diverse methodologies (Flecha, 2018). Now is the time to delve deeper into the development of methodologies focused on innovation with social impact. This paper has presented an innovative methodological approach to this endeavor with an example of the application of Social Media Analytics under this approach to investigate citizens’ participation in Twitter related to paying attention to and differentiating between innovation with or without social impact. The selection of categories and variables and the presentation of the results quantitatively and qualitatively helps us to overview how citizens are paying attention and differentiating innovation with social impact. Furthermore, this methodological approach addressed the social impact from the design, data collection, and analysis to help identify and analyze this topic.

Including this innovative methodological approach could complement the assessment methods addressed to innovation and enrich them due to the inclusion of societal impact as a crucial element that the European Commission prioritizes. Finally, citizens are also important players paying attention to the core elements innovation and social impact.

Further Methodological Advances

Future applications of this innovative methodological approach will allow for relevant comparisons to be made to understand different applications and the evolution of data related to innovation and social impact. Moreover, this methodology could be adapted for different social media and to find out the differences that may exist in the use and relevance of the same keywords. In addition, the elements that may explain these differences could be investigated in relation to the specific characteristics and uses of the different social media. A further methodological aspect to deepen on in the analysis is how to delve into the reasons and justification for the opinions expressed through which the data is obtained.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research,authorship,and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research,authorship,and/or publication of this article: This work was supported by the INTURPYR: Tourism innovation in the Pyrenees for a unique tourist destination in the heart of the Pyrenees. EFA 026/15. Co-financed 65% by the European Regional Development Fund through the Interreg V A Spain/France/Andorra Programme - POCTEFA 2014–2020.

ORCID iDs

Carmen Elboj-Saso

Carme García Yeste

References

Aiello

Donovan

Duque

Fabrizio

Flecha

Holm

Molina

Oliver

Reale

(2021). Effective strategies that enhance the social impact of social sciences and humanities research. Evidence and Policy, 17(1), 131–146. https://doi.org/10.1332/174426420X15834126054137

Ammirato

Sofo

Felicetti

A. M.

Raso

(2019). A methodology to support the adoption of IoT innovation and its application to the Italian bank branch security context. European Journal of Innovation Management, 22(1), 146–174. https://doi.org/10.1108/EJIM-03-2018-0058

Babu

Jose

(2020). From the homestead to centerstage: A journey of empowerment. International and Multidisciplinary Journal of Social Sciences, 9(3), 230–259. https://doi.org/10.17583/rimcis.2020.4401

Bielińska-Dusza

Hamerska

(2021). Methodology for calculating the European innovation scoreboard—proposition for modification. Sustainability, 13(4), 2199. https://doi.org/10.3390/su13042199

Carayannis

E. G.

Campbell

D. F. J.

(2010). Triple helix, quadruple helix and quintuple helix and how do knowledge, innovation and the environment relate to each other? International Journal of Social Ecology and Sustainable Development (IJSESD), 1(1), 41–69. https://doi.org/DOI:10.4018/jsesd.2010010105

Chaves

C. V.

Ribeiro

L. C.

dos Santos

U. P.

Albuquerque

E. D. E.

(2020). Innovation systems and changes in the centre-periphery division: Notes on a methodology for determining country trajectories from science and technology statistics. CEPAL REVIEW, 2020(130), 45–64. https://doi.org/10.18356/256b68b2-es

European Comission (2020). Strategic plan 2020-2024 (pp. 1–59). https://ec.europa.eu/info/sites/default/files/rtd_sp_2020_2024_en.pdf

Druckman

Donohue

(2020). Innovations in social science methodologies: An overview. American Behavioral Scientist, 64(1), 3–18. https://doi.org/10.1177/0002764219859623

Etzkowitz

Leydesdorff

(1995). A laboratory for knowledge based economic development. EASST Review, 14(1), 14–19.

10.

European Commission (2016). Open innovation open science open to the world. Directorate-General for Research and Innovation. https://ec.europa.eu/digital-single-market/en/news/open-innovation-open-science-open-world

11.

European Commission (2018). Ethics in social science and humanities. https://ec.europa.eu/research/participants/data/ref/h2020/other/hi/h2020_ethics-soc-science-humanities_en.pdf Issue October

12.

Flecha

European Commission (2018). Societal impact Monitoring the iumpact of EU framework programmes.

13.

Freeman

(1995). The ‘national system of innovation’ in historical perspective. Cambridge Journal of Economics, 19(1), 5–24. https://doi.org/10.1093/oxfordjournals.cje.a035309

14.

Friberg

Englander

(2019). Taking policy for granted in the context of scientific innovation. International Journal of Qualitative Methods, 18, 1–7. https://doi.org/10.1177/1609406919831588.

15.

Gerlitz

Meyer

Prause

(2020). Methodology approach on benchmarking regional innovation on smart specialisation (RIS3): A joint macro-regional tool to regional performance evaluation and monitoring in central Europe. Entrepreneurship and Sustainability Issues, 8(2), 1359–1385. https://doi.org/10.9770/jesi.2020.8.2(80

16.

Gómez

Padrós

Ríos

Mara

L.C.

Pukepuke

(2019). Reaching social impact through communicative methodology researching with rather than on vulnerable populations: The Roma case. Frontiers in Education, 4(9). https://doi.org/10.3389/feduc.2019.00009.

17.

Henzler

Maier

S. D.

Jäger

Horn

(2020). SDG-based sustainability assessment methodology for innovations in the field of urban surfaces. Sustainability, 12(11), 4466. https://doi.org/10.3390/su12114466

18.

Kahn

K. B.

(2018). Understanding innovation. Business Horizons, 61(3), 453–460. https://doi.org/10.1016/j.bushor.2018.01.011

19.

Lundvall

B. Å.

(1992). In Lundvall

B. Å.

(Ed.), National systems of innovation: Towards a theory of innovation and interactive learning. Pinter Publishers.

20.

Nelson

(1993). National innovation systems: A comparative analysise. Oxford University Press.

21.

OECD (1997). National innovation systems. OECD Publications.

22.

OECD (2015). Frascati manual 2015: Guidelines for collecting and reporting data on research and experimental development, the measurement of scientific, technological and innovation activities. OECD Publishing. https://doi.org/https://doi.org/10.1787/9789264239012-en

23.

OECD (2018). Oslo manual 2018: Guidelines for collecting, reporting and using data on innovation the measurement of scientific, technological and innovation activities (4th ed.). OECD Publishing. https://doi.org/https://doi.org/10.1787/9789264304604-en

24.

Parreiras

R. O.

Kokshenev

Carvalho

M. O. M.

Willer

A. C. M.

Dellezzopolles

C. F.

Nacif

D. B.

Santana

J. A.

(2019). A flexible multicriteria decision-making methodology to support the strategic management of science, technology and innovation research funding programs. European Journal of Operational Research, 272(2), 725–739. https://doi.org/10.1016/j.ejor.2018.06.050

25.

Penalva

(2021). Innovation and leadership as design: A methodology to lead and exceed an ecological approach in higher education. Journal of the Knowledge Economy, 13, 430–446. https://doi.org/10.1007/s13132-021-00764-3.

26.

Pulido

C. M.

Redondo-Sama

Sordé-Martí

Flecha

(2018). Social impact in social media: A new method to evaluate the social impact of research. PLOS ONE, 13(8), Article e0203117. https://doi.org/10.1371/journal.pone.0203117

27.

Pulido

C. M.

Villarejo-Carballido

Redondo-Sama

Gómez

(2020). COVID-19 infodemic: More retweets for science-based information on coronavirus than for false information. International Sociology, 35(4), 377–392. https://doi.org/10.1177/0268580920914755

28.

Robinson

D. K. R.

Simone

Mazzonetto

(2020). RRI legacies: Co-creation for responsible, equitable and fair innovation in horizon Europe. Journal of Responsible Innovation, 8(2), 1–8. https://doi.org/10.1080/23299460.2020.1842633

29.

Ruiz-Eugenio

Toledo del Cerro

Gómez-Cuevas

Villarejo-Carballido

(2021). Qualitative study on dialogic literary gatherings as co-creation intervention and its impact on psychological and social well-being in women during the COVID-19 lockdown. Frontiers in Public Health, 9(March), 602964. https://doi.org/10.3389/fpubh.2021.602964

30.

Seifu

van Paassen

Klerkx

Leeuwis

(2020). Anchoring innovation methodologies to ‘go-to-scale’; a framework to guide agricultural research for development. Agricultural Systems, 182(2020), 102810. https://doi.org/10.1016/j.agsy.2020.102810

31.

Smith

R. D. J.

Kamwendo

Z. T.

Berndt

Parkin

(2021). Taking knowledge production seriously in responsible research and innovation. Journal of Responsible Innovation, 8(2), 1–10. https://doi.org/10.1080/23299460.2021.1935584

32.

Soliman

Mogefors

Bergmann

J. H. M.

(2020). Problem-driven innovation models for emerging technologies. Health and Technology, 10(5), 1195–1206. https://doi.org/10.1007/s12553-020-00450-5

33.

Sudbury-Riley

Hunter-Jones

Al-Abdin

Lewin

Naraine

M. V.

(2020). The trajectory touchpoint technique: A deep dive methodology for service innovation. Journal of Service Research, 23(2), 229–251. https://doi.org/10.1177/1094670519894642

34.

Timmermans

Blok

Braun

Wesselink

Nielsen

R. Ø.

(2020). Social labs as an inclusive methodology to implement and study social change: The case of responsible research and innovation. Journal of Responsible Innovation, 7(3), 410–426. https://doi.org/10.1080/23299460.2020.1787751

35.

van des Besselaar

Flecha

Radauer

(2018). Monitoring the impact of EU framework programmes.

36.

Zhang

Chen

Porter

A. L.

Zhu

(2016). Topic analysis and forecasting for science, technology and innovation: Methodology with a case study focusing on big data research. Technological Forecasting and Social Change, 105(1), 179–191. https://doi.org/10.1016/j.techfore.2016.01.015

Innovative Methodological Approach to Analyze Innovation and Social Impact

Abstract

Keywords

Introduction

Diverse Methodologies to Analyze Innovation

Innovation, Co-Creation and Social Impact in Framework Program for Research

Method

Data Collection

Sample

Dialogic Codebook

Communicative Content Analysis

Ethical Considerations

Results

Limitations

Discussion

Further Methodological Advances

Footnotes

Declaration of Conflicting Interests

Funding

ORCID iDs

References