Sage Journals: Discover world-class research

Abstract

The objectives of open government data initiatives range from enhancing transparency and accountability to increasing innovation and participation. However, there is a lack of knowledge of the extent to which the objectives of open government data initiatives are achieved. This article investigates the relationship between the objectives of open government data initiatives and the benefits delivered. A total of 168 survey responses concerning 156 open government data initiatives at different government levels worldwide suggest that operational and technical benefits are the benefits most often delivered, followed by economic benefits and, finally, societal benefits. Surprisingly, our study suggests that whether an open government data initiative delivers a benefit (e.g. increased openness, trust or innovation) is not significantly affected by having an objective related to the delivery of that benefit. The objectives of state- and national-level open government data initiatives are more often achieved than those of local- and regional-level open government data initiatives.

Points for practitioners

Key implications for professionals working in public management and administration are: attaining open government data objectives is easier said than done; the benefits delivered are most often operational and technical benefits, followed by economic benefits and, finally, societal benefits; the benefits are often in areas other than those of the open government data initiative’s objectives, suggesting a mismatch between the objectives and their attainment; state- and national-level open government data initiatives achieve their objectives more often than local- and regional-level open government data initiatives; and initiatives might mimic each other instead of aiming at delivering the benefits.

Keywords

attainment benefits delivery evaluation implementation objectives open data open government open government data open science

Introduction

Open government data initiatives (OGDIs) are often intended to achieve a variety of objectives, including enhanced transparency, participation and collaboration (Alexopoulos et al., 2013; Attard et al., 2016; Kassen, 2013), as well as the stimulation of innovation and economic value (Zeleti et al., 2016). To realise this, open government data (OGD) are published on the Internet by governments or publicly funded research organisations and can be reused by the public (Alexopoulos et al., 2013; Jetzek, 2015; Linders, 2013; Meijer et al., 2012). These efforts, however, are criticised for a number of reasons, including not taking the user’s point of view into account (Zuiderwijk, 2015) and a lack of convincing evidence regarding the impact and value created by OGDIs (Davies, 2013; Jetzek, 2015).

Considerable amounts of effort and money have been devoted to attaining the objectives of OGDIs (e.g. The Economist, 2013; The World Bank, 2013). Governments have high expectations regarding the achievement of such objectives (Kroes, 2011). Some OGDIs have successfully achieved their objectives (e.g. Attard et al., 2015; Evans and Campos, 2013), meaning that these initiatives have resulted in benefits that are related to the intended objectives. For example, when a particular OGDI aimed to increase governmental transparency, this OGDI actually managed to attain the benefit of increased governmental transparency. However, many OGDIs seem to have only partially achieved their objectives, or have not done so at all. An example of the latter is Kenya’s national OGDI, where the objective of data reuse was not achieved as the number of users remained stagnant and then decreased substantially (Brown, 2013). In addition, for many other OGDIs, it is not clear whether they have resulted in the delivery of the intended benefits.

Furthermore, while OGDIs are sometimes evaluated individually (e.g. Lee, 2014) or through larger projects (e.g. Kim et al., 2009), these evaluations usually evaluate all initiatives against the same criteria without looking at the intended objectives set by the OGDI (Susha et al., 2015) or without taking the particular context into account (Janowski, 2015). Moreover, evaluations of OGDIs are often carried out at a country or national level, whereas OGDIs may also be organised at the international or local level. It is thus unclear whether OGDIs at different government levels have achieved their objectives and delivered the benefits they intend to deliver.

We used a globally distributed survey to collect data on 156 OGDIs in 61 countries all over the world, and then investigated the relationship between the objectives of OGDIs at different government levels and the benefits delivered. As we could not directly measure the attainment of each objective, we compared the objective of each initiative to one or more related delivered benefits. For instance, when the objective was ‘Increased transparency within the government and/or its legislation’, the achievement of this objective was established by analysing the extent to which the benefit ‘Increased transparency’ had been delivered (see later). Documents like PDFs and initiatives that provide only processed rather than raw data (such as participation, petition and complaint initiatives) were outside the scope of this study.

Research background

The aim of our literature review was not to obtain a complete overview of all possible OGDI objectives as these might be considerably different. Instead, the aim was to gather background information concerning the main objectives mentioned in prominent and easily accessible OGD policies and in the literature. The following keywords were used in various combinations to find literature relevant to this research: ‘open data’, ‘open government data’, ‘public sector information’, ‘benefit’, ‘objective’, ‘aim’ and ‘goal’. We searched for papers in the Scopus, ACM Digital Library and Google Scholar databases. Scopus includes Elsevier (ScienceDirect), Springer, Taylor & Francis, Wiley Blackwell, Institute of Electrical and Electronics Engineers (IEEE), Sage, Emerald and many other sources. As suggested by Webster and Watson (2002), the citations in the identified articles were also examined to find additional relevant literature and to enrich the literature base. This search resulted in a rich collection of articles, which were categorised into clusters of OGDI objectives and OGDI benefits. Policy documents were searched for using Google and the websites of national governments.

Objectives of OGDIs

The US, the UK and many European countries have explicit and easily accessible policies concerning OGD. Together with the literature, these policies show that major objectives of OGDIs include increasing transparency and accountability, stimulating innovation, improving and supporting decision-making, stimulating data reuse, counteracting corruption, and providing new services and products (see Table 1). These objectives can be broken down further into their various components. For example, innovation can be related only to business developments or to public services (Jetzek et al., 2013; Schillemans et al., 2013). The objective of data reuse can be focused on various fields of government, such as the environmental sector, and can be in various forms, for example, for predictions or validations of policies (Cowan et al., 2015; Jocelyn et al., 2014). There are many objectives and some of them overlap and are interdependent, for instance, creating smarter cities may be done by stimulating innovation.

Table 1.

Objectives of open government data initiatives as mentioned in the literature and policy documents.

	Objectives of OGDIs	Source
Political and social	Increase accountability	Ubaldi (2013), European Commission (2013a), HM Government (2011)
	Increase transparency and/or openness	Jaeger and Bertot (2010), Huijboom and Van den Broek (2011), European Commission (2013a, 2013b), Obama (2009), Dawes and Helbig (2010), HM Government (2011)
	Counteract corruption	Bertot et al. (2010), Linders (2013)
	Promote social engagement	Ubaldi (2013), European Commission (2013a)
	Increase trust	Grimmelikhuijsen (2012), Sandoval-Almazán (2013), Ubaldi (2013), Lee and Kwak (2012)
Economic	Stimulate innovation	Jetzek et al. (2013), Schillemans et al. (2013), Tauberer (2012), Obama (2012)
Economic	Stimulate economic growth	Jetzek et al. (2013), Lakomaa and Kallberg (2013), European Commission (2013a), HM Government (2011)
Operational and technical	Stimulate research in data analytics	Cowan et al. (2015), Jocelyn et al. (2014), Chun et al. (2010), Ubaldi (2013)
	Improve the quality of collected information	Lee and Kwak (2012), European Commission (2013a), Bates (2012)
	Create smarter cities	Meijer and Bolívar (2016), Alawadhi et al. (2012), Veeckman and Van der Graaf (2015), Bates (2012)
	Improve services	Jetzek et al. (2013), Lakomaa and Kallberg (2013), Bakıcı et al. (2013), European Commission (2013a, 2013b), Obama (2012), HM Government (2011)
	Provide validation data for research	Kalampokis et al. (2011), Radl et al. (2013), Ubaldi (2013)
	Facilitate participation	Ubaldi (2013), Bates (2012), Evans and Campos (2013), Obama (2009)
	Consult citizens on the development of legislation (e-consultation)	Macintosh (2004), Whyte and Macintosh (2002), Abu-Shanab (2015)
	Share information between multiple governmental agencies	Harrison et al. (2012a), Dawes and Helbig (2010)
	Develop freedom of information legislation	Bertot et al. (2010), Worthy (2010)
	Increase collaboration	Ubaldi (2013), Lee and Kwak (2012), Lathrop and Ruma (2010), Obama (2009)
	Increase the efficiency of public authorities	Huijboom and Van den Broek (2011), Ubaldi (2013), Bates (2012)
	Provide access to government data	Jaeger and Bertot (2010), Gurstein (2011), Olalere and Lazar (2011)
	Reuse government data	Ubaldi (2013), Huijboom and Van den Broek (2011), Yu and Robinson (2012), European Commission (2013a)
	Obtain feedback from reusers and end users	Kassen (2013), Lee and Kwak (2012), European Commission (2013a)

Benefits delivered by OGDIs

OGD policies and the literature also describe the benefits that OGDIs can deliver. They include increased transparency, increased public engagement, increased collaboration, economic growth and easier discovery of data (see Table 2). Table 2 shows that many benefits may be delivered in different categories, including political and societal benefits, economic benefits, and technical and operational benefits.

Table 2.

Benefits delivered by open government data initiatives as mentioned in the literature and policy documents.

	Benefits delivered by OGDIs	Source
Political and societal	Increased transparency	Welle Donker et al. (2016), Zuiderwijk (2015), Kulk and Van Loenen (2012)
	Increased empowerment of the public	Gurstein (2011), Linders (2013)
	Improved policymaking processes	Chun et al. (2010), Harrison et al. (2012a)
	Increased public engagement	Lee and Kwak (2012), Chun et al. (2010)
	Increased scrutinisation of information and data released by the government	Harrison et al. (2012a), Yu and Robinson (2012)
	Stimulated knowledge development	Chun et al. (2010)
	Increased democratic accountability	Harrison et al. (2012a)
	Increased trust in government	Linders (2013)
	Increased participation	Evans and Campos (2013), Lathrop and Ruma (2010)
	New insights into the public sector	Lee and Kwak (2012), Ubaldi (2013)
	Equal access to information provided by government	Ubaldi (2013), Yu and Robinson (2012)
	Increased collaboration	Lee and Kwak (2012), Harrison et al. (2012a)
Economic	Stimulated competitiveness	Bakıcı et al. (2013)
	Contribution towards the improvement of products	Lee and Kwak (2012)
	Stimulated innovation	Ubaldi (2013), Lee and Kwak (2012)
	Reduced government spending	Bertot et al. (2010)
	Economic growth	Bertot et al. (2010), Arzberger et al. (2004)
	Contribution towards the improvement of services	Lee and Kwak (2012), Harrison et al. (2012a)
	Greater efficiency of government	Welle Donker et al. (2016), Kassen (2013), Moon (2002)
	Access to external problem-solving capacity and resources	Harrison et al. (2012b)
Technical and operational	Ability to reuse data	Ubaldi (2013), Yu and Robinson (2012)
	Contribution towards the improvement of administrative processes	Welle Donker et al. (2016), Harrison et al. (2012a), Coglianese (2009)
	Easier access to data	Ubaldi (2013), Gurstein (2011)
	Use of the wisdom of crowds: tapping into the intelligence of the collective	Lathrop and Ruma (2010)
	Fair decision-making by enabling comparison of different sources	Harrison et al. (2012a)
	Easier discovery of data	Villazón-Terrazas et al. (2011)

Research design

Questionnaire

A questionnaire was created containing the following sections: an introduction; a description of the open government initiative (including its objectives); the technologies and functionalities used in the initiative; the benefits delivered by the initiative; the initiative’s development and user barriers; and the demographics of the person completing the questionnaire (see Appendix 1, available online). The literature mentioned earlier was studied to obtain insight into the state of the art in open government research and to ensure that the questionnaire covered the topics mentioned in the literature. The questionnaire was tested by seven master’s students in domains related to open government. This resulted in changes to some questions in order to make them clearer and less ambiguous. Moreover, the labels of the five-point Likert scale were made clearer and some questions were made more concise. Then, a second round of testing took place involving eight people, of whom six were working or studying in the field of open government. No more changes were made since the testers did not have any further comments on the survey.

Data collection

Accessing data from all over the world is a challenging task. A free, massive open online course (MOOC) on ‘Open Government’ followed by participants from all over the world enabled the distribution of a worldwide survey. The five-week MOOC was taught by Delft University of Technology in the spring of 2016 and was provided through the EdX platform. The number of enrolments for the course varied between 3082 (at the start, on 14 March 2016) and 3607 (at the end, on 26 April 2016). Data on the participants’ backgrounds are presented later. In total, 153 countries were represented by the course participants.

Data were collected at the level of initiatives. One important assumption was that the data we collected about the OGDIs reflect the actual initiatives rather than the opinions of the people providing the data. In the questionnaire, each respondent provided information concerning the way he or she understood the OGDI. The majority of the respondents (57%) were involved in the OGDI that they assessed.

A stepwise filtration of the OGDIs was done based on the questionnaire data (see Figure 1). First, the complete data set consisted of 263 survey entries containing cases that qualified as OGDIs according to the respondents. Incomplete responses were removed from our sample, leaving 251 OGDIs. The initiatives that contained incorrect information or were related to non-existent initiatives were then removed, which resulted in 182 initiatives. Incorrect information concerned, for instance, PDF files about open data in general, links to initiatives with no relation to OGD or simple visualisation platforms or government web pages. The existence of the initiatives was checked by reviewing every website link of the given initiative. If this website link was a dead link or no link to an OGDI was provided, the data entry was rejected. Finally, from the 182 remaining initiatives, 168 responses concerning 156 OGDIs from 61 countries that fitted our definition of OGDIs were selected (for the overview, see Appendix 2, available online). This last step was done as follows. If the respondents indicated that the objective of the initiative was either the release of government data to the public as OGD or the use of OGD by the public (e.g. by citizens or journalists), the initiative was included in our selection. Furthermore, if the respondents referred to OGD later on in the survey and OGD appeared to be an important aspect of the initiative, we also included the initiative in our sample.

Figure 1.

The selection of OGDIs for this study.

Relating OGDI objectives to delivered benefits

The achievement of the objectives was measured by investigating the delivered benefits (as indicators) (see Figure 2).

Figure 2.

The way that objectives and benefits of OGDIs are related to each other in this study.

To measure whether the level of delivered benefits was significantly different for the OGDIs with and without a certain objective, a Mann–Whitney test was conducted (Mann and Whitney, 1947). The test was appropriate since the survey produced one independent categorical variable with two categories (whether a certain OGDI had a certain objective: yes or no) and one continuous dependent variable (the extent to which the benefit had been delivered). The Mann–Whitney test is the non-parametric equivalent of the independent t-test (Field, 2009: 540), and it had to be used since the sample did not meet the assumptions for parametric tests (i.e. the data were not normally distributed).

Data preparation

A reliability analysis was conducted to examine whether the independent variables consistently reflected the constructs that they were measuring (Field, 2009). For instance, we measured whether the objective ‘Create openness’ (the construct) had been attained by creating a scale of three benefits, namely, easier access to data, easier discovery of data and the ability to reuse data. The reliability analysis tested whether the three benefits consistently reflected the objective. Table 3 shows the reliability analysis results derived from Cronbach’s alpha test (Cronbach and Meehl, 1955).

Table 3.

Reliability analysis of benefits for specific objectives (Cronbach’s alpha test).

Objective	Benefits	Number of Benefits	Cronbach’s alpha
Create openness	Easier access to data	3	.870
	Easier discovery of data
	Ability to reuse data
The release of government data to the public as OGD	Easier access to data	3	.820
	Easier discovery of data
	Equal access to information provided by government
The use of OGD by the public (e.g. by citizens/journalists)	Ability to reuse data	2	.606
The use of OGD by the public (e.g. by citizens/journalists)	Increased scrutinisation of information and data released by the government	2	.606
Improve the functioning of the government	Improved policymaking process	4	.847
	Reduced government spending
	Greater efficiency of government
	Contribution towards the improvement of administrative processes
Development of services to participate in governmental processes	Contribution towards the improvement of services	4	.878
	Increased participation
	Increased public engagement
	Access to external problem-solving capacity and resources
Consultation of citizens in the development of legislation	Access to external problem-solving capacity and resources	2	.781
Consultation of citizens in the development of legislation	Use of the wisdom of crowds: tapping into the intelligence of the collective	2	.781
Stimulate innovation by companies	Stimulation of innovation	3	.886
	Stimulation of competitiveness
	Contribution towards the improvement of products

Murphy and Davidshofer (1988) state that alpha values below 0.6 are unacceptable, values of 0.7 are low, values between 0.8 and 0.9 are moderate to high, and values around 0.9 are high; however, according to others, a lower acceptance boundary can be adopted (Nunnally, 1967), namely, that alpha values of between 0.5 and 0.6 may still be acceptable. Except for one value, all values were moderate (.781) to high (.886). Cronbach’s alpha value for the construct ‘Use of OGD by the public’ was lower (.606), yet not unacceptable. Thus, the created scales of the objectives could be used for further analysis.

Respondents’ demographics and description of the OGDIs

Respondents’ demographics

Information concerning the respondents is presented in Table 4. Table 4 shows that most respondents work in the service sector (62.4%) or in the commerce sector (23.7%). Most have a master’s degree (42.6%) or a bachelor’s degree (28.4%). The majority (39.4%) have more than 10 years of experience in their field. Although most (57.4%) of the respondents are involved in OGDIs at least to a certain extent, a large proportion (42.6%) are not involved at all. The majority (94.6%) of all respondents trusted the government at least to a certain extent.

Figure 3.

The relation between the objectives and the delivered benefits of OGDIs.

Table 4.

Participants’ demographics.

Demographic	Descriptive	Percentage
Occupation sector	Services	62.4
	Agriculture	1.1
	Commerce	23.7
	Construction	1.1
	Finance	3.2
	Industry	8.6
Highest education level	Elementary school	4.1
	High school	14.2
	Associate’s degree	4.1
	Bachelor’s degree	28.4
	Master’s degree	42.6
	PhD degree	6.8
Level of expertise in their field	1–3 years	18.3
	3–5 years	16.4
	5–10 years	19.2
	>10 years	39.4
Involvement with OGDIs	No involvement at all	42.6
	Involved to a small extent	20.3
	Involved	18.9
	Involved to a large extent	7.4
	Completely involved	10.8
Trust in the government	No trust in government at all	5.4
	Trust the government to a small extent	20.4
	Trust the government	34.7
	Trust the government to a large extent	32.7
	Trust the government completely	6.8

Description of the OGDIs

The OGDIs that were assessed via the questionnaire were implemented in a total of 61 countries. The countries mentioned most often were the US, Brazil and the Netherlands (implementing 21%, 11% and 8% of the OGDIs, respectively). Table 5 presents the global distribution of the OGDIs. The table shows that most OGDIs (29%) are in the Anglo cluster, mainly including initiatives from the US; a large percentage (27%) are in the Latin American cluster, particularly consisting of initiatives from Brazil.

Table 5.

Global distribution of the OGDIs.

Cluster	OGDIs in this cluster
Africa	5% (9)
Anglo	29% (49)
Confucian	1% (2)
Eastern Europe	8% (13)
Germanic	10% (16)
Latin America	27% (45)
Latin Europe	9% (15)
Middle Eastern	2% (4)
Nordic	1% (2)
South-east Asia	8% (13)

Source: adopted from House et al. (2004).

The OGDIs in our sample are at national, international, local and regional levels. Table 6 shows the distribution of the levels at which the OGDIs are represented. In terms of percentages of the total number of cases, 35% of the initiatives are at the national level and 25% are at the local administrative level. Only 7% are international OGDIs.

Table 6.

OGDI characteristics (out of total number of 168 OGDI responses studied).

OGDI characteristics	Descriptive of the characteristics	Percentage
Administrative level of the initiative	Local	25%
	Regional	17%
	State	14%
	National	35%
	International	7%
Stage of public involvement	Start-up	18%
	Design	18%
	Implementation	27%
	Operation and maintenance	37%
Role of government	Operating	21%
	Implementing	20%
	Developing	18%
	Regulating	17%
	Funding	16%
	Advertising	7%
	Data supplier	1%
Government stakeholder involved in the initiative	Portal providers	10%
	Policymakers	10%
	Law enforcers	8%
	Defence department	7%
	Health department	9%
	Infrastructure department	8%
	Commercial	8%
Level of the government stakeholder involved in the initiative	Local government	11%
	Regional government	9%
	State government	9%
	National government	11%
	Other	7%

Public stakeholders are involved in the OGDIs of our sample in four different phases: the start-up phase, the design phase, the implementation phase and the operation and maintenance phase. As shown in Table 6, most are engaged in the operation and maintenance phase (37%), followed by some involvement in the implementation phase (27%) and equally low involvement in the start-up (18%) and design phases (18%). Like the public stakeholders, most of the government’s involvement is in the operation phase (21%) and the implementation phase (20%).

Findings and discussion on the attainment of OGDI objectives

Objectives of OGDIs

The second section of the article showed the diversity of OGDI objectives. The comprehensive list of OGDI objectives from the second section has been condensed into the objectives shown in Table 7. This table shows the number and percentage of OGDIs in our selection of 156 OGDIs that aimed at achieving a particular objective. The four major objectives were: create openness (62.8%), increase transparency (55.2%), engage government with citizens (50%) and use of OGD by the public (48.3%). Note that one OGDI can have multiple objectives and therefore the percentages do not add up to 100%.

Table 7.

Objectives of OGDIs.

Objectives of OGDIs	OGDIs with this objective (percentage of total OGDIs)
Create openness	108 (62.8%)
Increase transparency within the government and/or its legislation	95 (55.2%)
Engage government with citizens through social media channels	86 (50.0%)
Use of OGD by the public (e.g. by citizens or journalists)	83 (48.3%)
Release government data to the public as OGD	69 (40.1%)
Increase accountability of government	63 (36.6%)
Consult citizens on the development of legislation (e-consultation)	60 (34.9%)
Stimulate innovation by companies	59 (34.3%)
Share information between multiple governmental agencies	52 (30.2%)
Develop services to participate in governmental processes	44 (25.6%)
Improve the functioning of the government	41 (23.8%)
Increase trust in the government	37 (21.5%)
Develop freedom of information legislation	32 (18.6%)
Other	16 (9.3%)

Delivered benefits

After providing information concerning the objectives of the OGDI they selected, respondents were asked to answer questions about the benefits that the OGDI had delivered. The benefits were divided into political and societal benefits, economic benefits, and operational and technical benefits. Table 8 shows all the benefits that were presented to the respondents and the number and percentage of initiatives that delivered the specific benefit out of the total number of OGDIs given.

Table 8.

Benefits delivered by the OGDIs.

Benefits	Possible OGDI benefit	Delivery of this benefit by OGDIs
Benefits	Possible OGDI benefit	Average value	Median	Weighted number^a	Score^bPercentage
Political and societal	Increased transparency	2.95	4	496	59%
	Increased empowerment of the public	2.52	3	457	54%
	Improved policymaking processes	2.54	3	456	54%
	Increased public engagement	2.68	3	451	54%
	Increased scrutinisation of information and data released by the government	2.72	3	440	52%
	Stimulation of knowledge developments	2.62	3	427	51%
	Increased democratic accountability	2.26	3	426	51%
	Increased trust in government	2.46	3	425	51%
	Increased participation	2.53	3	423	50%
	New insights into the public sector	2.40	3	413	49%
	Equal access to information provided by government	2.54	3	403	48%
	Increased collaboration	2.71	3	379	45%
Economic	Stimulation of competitiveness	1.93	2	429	51%
	Contribution towards the improvement of products	1.90	2	389	46%
	Stimulation of innovation	2.22	3	373	44%
	Reduced government spending	1.90	2	346	41%
	Economic growth	2.06	2	327	39%
	Contribution towards the improvement of services	1.95	2	325	39%
	Greater efficiency of government	2.55	3	320	38%
	Access to external problem-solving capacity and resources	2.32	3	320	38%
Technical and operational	Ability to reuse data	2.43	3	462	55%
	Contribution towards the improvement of administrative processes	2.24	3	379	45%
	Easier access to data	2.75	3	396	47%
	Use of the wisdom of crowds: tapping into the intelligence of the collective	2.71	3	409	49%
	Fair decision-making by enabling comparison of different sources	2.26	3	377	45%
	Easier discovery of data	2.36	3	456	54%

Notes: ^aHere, the weighted number refers to the benefit extent (measured on an ordinal scale from 1 to 5) multiplied by the number of OGDIs in which it is delivered. ^bThis represents a score for the number of cases in which the given benefit was fully delivered. The maximum weighted number for each benefit is 840, namely, a score of 5 on the ordinal scale times 168 OGDI responses if the benefit was delivered fully in each of the OGDIs. The score is calculated by dividing the weighted number by the extent to which the benefit was fully delivered (the ordinal scale times the number of case responses; thus, 5 times 168).

The results of our survey show that the benefits delivered by the OGDIs are most often operational and technical benefits, followed by economic benefits and, lastly, societal benefits. The societal benefits might be harder to measure objectively and this might be why they were reported least. Moreover, the societal benefits are highly interconnected. Political and societal benefits like increasing transparency (58%) and the scrutinisation of information and data released by the government (51.5%) are almost equally delivered, followed by many others like participation (49.5%), trust (49.7%) and collaboration (44.3%). Economic benefits are mainly delivered in the form of contribution towards the improvement of services (45.5%) and greater efficiency of government due to public input (37.4%). The operational and technical benefits are delivered by easier access and discovery to data and the ability to reuse the data in most of the OGDIs. One should keep in mind that these results are subjective and depend on the viewpoint and opinion of the participants.

The benefits that were most often delivered were ‘Increased transparency’ (58% of the OGDIs yielded this benefit), ‘Ability to reuse data’ (54%), ‘Increased empowerment of the public’ (54%), ‘Improved policymaking processes’ (53%) and ‘Contribution towards the improvement of administrative processes’ (53%). None of these most-delivered benefits were economic benefits. Benefits that were delivered least by the OGDIs were ‘Greater efficiency of government’ (37%) and ‘Access to external problem-solving capacity and resources’ (37%), although a relatively high percentage of the OGDIs were still found to deliver these benefits and the percentage of the least-delivered benefits do not differ that much from the percentage of the most-delivered benefits.

Delivered benefits in relation to the OGDI objectives

The achievement of objectives was measured by investigating the delivered benefits (as indicators) (see earlier). Some objectives were related to multiple benefits, as can be seen in Table 9, but the relations of every objective with a benefit were checked individually. The median ranged from 1 to 5: 1 = ‘not attained at all’, 2 = ‘attained to a small extent’, 3 = ‘attained’, 4 = ‘attained to a large extent’ and 5 = ‘fully attained’.

Table 9.

Comparison of the level of benefits delivered by OGDIs with and without a certain related objective.

OGDI objective	Related delivered benefits of the OGDI	Median (mean): whether OGDI achieves objective (yes/no)		Mann–Whitney U
OGDI objective	Related delivered benefits of the OGDI	No	Yes	Mann–Whitney U
Create openness	Easier access to data	3 (3.38)	3 (3.37)	3211.50
	Easier discovery of data	3 (3.38)	3 (3.30)	3077.00
	Ability to reuse data	3 (3.03)	3 (3.02)	3184.50
Increase transparency within the government and/or its legislation	Increased transparency	2 (2.36)	2 (2.34)	3400.500
Release government data to the public as OGD	Easier access to data	3 (3.45)	3 (3.29)	3163.00
	Easier discovery of data	3 (3.44)	3 (3.22)	3069.50
	Equal access to information provided by government	3 (3.19)	3 (3.26)	3010.00
Use of OGD by the public (e.g. by citizens or journalists)	Ability to reuse data	3 (3.22)	3 (3.08)	3172.50
Use of OGD by the public (e.g. by citizens or journalists)	Increased scrutinisation of information and data released by the government	3 (2.96)	3 (3.31)	2909.00
Increase accountability of government	Increased democratic accountability	3 (3.33)	4 (3.35)	3075.00
Increase trust in government	Increased trust in government	3 (3.2)	3 (3.2)	3005.50
Improve functioning of the government	Improved policymaking process	3 (2.74)	3 (3.08)	2339.50
	Reduced government spending	2 (2.29)	2 (2.38)	2882.50
	Greater efficiency of government	2 (2.69)	3 (2.88)	2823.50
	Contribution towards the improvement of administrative processes	3 (2.78)	3 (2.88)	2948.00
Develop services to participate in governmental processes	Contribution towards the improvement of services	3 (2.98)	3 (3.02)	2883.50
	Increased participation	3 (3.37)	3 (3.17)	2347.00
	Increased public engagement	3.5 (3.4)	3 (3.10)	2389.50
	Access to external problem-solving capacity and resources	2 (2.41)	2 (2.46)	2841.00
Consult citizens on the development of legislation	Access to external problem-solving capacity and resources	2 (2.32)	2 (2.42)	2570.50
Consult citizens on the development of legislation	Use of the wisdom of crowds: tapping into the intelligence of the collective	2 (2.58)	3 (2.89)	2328.50
Engage government with citizens through social media	Increased public engagement	3 (3.38)	3 (3.11)	2084.50
Stimulate innovation by companies	Stimulation of innovation	3 (2.63)	3 (2.75)	2002.00
	Stimulation of competitiveness	2 (2.26)	3 (2.69)	1771.50
	Contribution towards the improvement of products	2 (2.17)	2 (2.47)	1808.00

Notes: *p <.05; **p <.001.

Table 9 provides the results of the Mann–Whitney test and the medians of the compared groups (i.e. whether the OGDI had a certain objective or not). Surprisingly, this test shows that the extent to which the benefits of OGDIs are delivered is not statistically different for OGDIs that have a related objective compared to those that do not have this objective. For example, the extent to which the benefit ‘Easier access to data’ was delivered was not significantly different for OGDIs aiming to create openness (Mdn = 3) compared to OGDIs that are not aiming to create openness (Mdn = 3), U = 3211.50, z = −0.114, p > .05¹. The differences between the means for the OGDIs with and without a certain objective are very similar for almost all of the benefits. Some medians for OGDIs with and without a certain objective differ slightly more than others. For instance, OGDIs with the objective to stimulate innovation by companies (Mdn = 3) were reported to have delivered slightly higher levels of the benefit ‘Stimulating competitiveness’ than OGDIs without this objective (Mdn = 2). Moreover, there were small differences in the extent to which the benefit ‘Use of OGD by the public’ was delivered, depending on whether or not the OGDI had the objective ‘Increased scrutinisation of information and data released by the government’. Furthermore, there were small differences in the extent to which the benefit ‘Improved policymaking processes’ was delivered, depending on whether or not the OGDI had the objective ‘Improve functioning of the government’. However, none of these benefits was significantly affected by whether the OGDIs had the related objective. Thus, the Mann–Whitney test results suggest that whether a benefit of OGDIs (e.g. creating openness, increase trust in the government or stimulating innovation by companies) is delivered is not significantly affected by having an objective related to that benefit (see Figure 3). Thus, the benefits are in areas other than ones the OGDIs aim for.

We examined whether we could find any patterns in the attainment of the objectives for different types of OGDIs. After we made a distinction between OGDIs at a state or national level (e.g. the USA’s² and Bulgaria’s³ open data initiatives) and OGDIs at a local or regional level (e.g. open data from Buenos Aires in Argentina⁴), we found a substantial difference in the results (see Table 10). The general trend was that the benefits delivered by state- and national-level OGDIs are affected more by a related objective compared to local- and regional-level OGDIs. Thus, the relation between the objectives and the benefits is stronger for state and national OGDIs (see Figure 4). For example, the benefit ‘Easier discovery of data’ appears to be significantly affected by the objective ‘Create openness’ for OGDIs at the national and state level (p = 0.003 (<0.05), Z = –.524 and U = 52.5). In contrast, for OGDIs at a local or regional level, this objective and benefit are not significantly related (p = 0.073 (> .05), Z = .937, U = 55.5). The same applies to the objective ‘Create openness’ in relation to the benefit ‘Ability to reuse data’.

Table 10.

Comparison of the level of benefits delivered by OGDIs at local or regional and national or state levels.

		Local or regional OGDIs			National or state OGDIs
		p	Z	U	p	Z	U
Create openness	Easier access to data	0.378	0.937	74.5	0.628	–0.524	52.5
	Easier discovery of data	0.073	–1.904	55.5	0.003	–3.01	16.5
	Ability to reuse data	0.185	–1.415	68	0.043	–2.138	29
Increase transparency within the government and/or its legislation	Increased transparency	0.553	–0.593	85.5	0.456	–0.828	48
Release government data to the public as OGD	Easier access to data	0.81	–0.241	93	0.771	–0.315	55
	Easier discovery of data	0.27	–1.103	69	0.111	–1.683	31
	Equal access to information provided by government	0.047	–2.126	53	0.01	–2.65	19
Use of OGD by the public (e.g. by citizens or journalists)	The ability to reuse data	0.201	–1.35	55	0.183	–1.435	28
Use of OGD by the public (e.g. by citizens or journalists)	Increased scrutinisation of information and data released by the government	1	0	90	0.63	–0.553	45
Increase accountability of government	Increased democratic accountability	0.408	–0.898	72	0.92	–0.108	54.5
Increase trust in government	Increased trust in government	0.977	–0.028	69.5	0.553	–0.632	30.5
Improve functioning of the government	Improvement of policymaking process	0.756	–0.372	67	0.367	–0.96	35.5
	Reduction of government spending	0.64	–0.526	64	0.858	–0.194	45.5
	Greater efficiency of government	0.208	–1.323	49.5	0.261	–1.186	32.5
	Contribution towards the improvement of administrative processes	0.259	–1.192	45.5	0.868	–0.215	53
Develop services to participate in governmental processes	Contribution towards the improvement of services	0.216	–1.331	41	0.831	–0.23	36.5
	Increased participation	0.753	–0.341	47.5	1	0	42
	Increased public engagement	0.46	–0.844	52	0.441	–0.847	44
	Access to external problem-solving capacity and resource	0.259	–1.268	45	0.945	–0.073	51.5
Consult citizens on the development of legislation	Access to external problem-solving capacity and resource	0.643	–0.525	57	0.63	–0.543	45
Consult citizens on the development of legislation	Use of the wisdom of crowds: tapping into the intelligence of the collective	0.88	–0.21	32	0.098	–1.72	0.085
Engage government with citizens through social media	Increased public engagement	0.063	–1.926	5	0.094	–1.675	11.5
Stimulate innovation by companies	Stimulation of innovation	0.296	–1.149	13.5	0.093	–1.682	11.5
Stimulate innovation by companies	Stimulation of competitiveness	0.698	–0.466	21	0.343	–0.949	19

Figure 4.

The relationship between the objectives and the benefits delivered by OGDIs at different levels.

However, in some cases, the opposite trend was found, that is, a higher value of p was found for OGDIs at a local or regional level compared to OGDIs at a national or state level. For instance, for the local OGDI called ‘Decide Madrid’, where the citizens of Madrid participate in budget debates, the objective of ‘Increasing transparency of the government and/or its legislation’ was attained through the benefit of ‘increased transparency’ benefits. The benefit ‘Access to external problem-solving capacity’ is delivered more often by OGDIs at a local or regional level that have the objective ‘Develop services to participate in governmental processes’ compared to OGDIs at a state or national level that have this objective.

We found that OGDIs from certain countries more often have benefits that match the objectives. For instance, there are more often matching benefits and objectives for Brazil (13.5%), the US (13.5%), Europe-wide OGDIs (6.13%), Greece, Spain, the Netherlands and India (5% each). However, this result is probably the effect of a higher number of participants from these countries, increasing the chance of having an OGDI where the objectives and benefits match, in relation to a lower number of OGDIs reported for the other countries. In addition, there is relatively more often a match between objectives and benefits for OGDIs where the public is involved in the implementation and operation phase than for OGDIs where the public is involved in the start-up or design phase, although this finding is not statistically significant. In more developed OGDIs, the stakeholders may have a more realistic view on which objectives can realistically be attained.

Discussion

Causes for the mismatch between objectives and benefits

Our study revealed that an OGDI having a certain objective did not significantly influence whether the benefits related to that objective were delivered. There is a mismatch between the benefits delivered and the objectives set. Examples of OGDIs in which there is a mismatch include the Greek e-Trikala OGDI (which aimed to create openness but instead used the wisdom of crowds) and the Diavgeia Transparency Programme (which aimed to increase democratic accountability but instead increased transparency). An explanation for this mismatch might be that the objectives are not focused on the situation at hand. Politicians and policymakers might not set objectives that take into account the context and societal problems that can be addressed by opening up the data. OGDI objectives are often generic, focusing on objectives like transparency, participation (Alexopoulos et al., 2013; Attard et al., 2016; Kassen, 2013) and economic value (Zeleti et al., 2016). This makes it difficult for practitioners to know exactly what should be done to achieve the objectives.

Furthermore, there are many assumptions and conditions underlying the benefits. For instance, to reuse data, a certain data user may need to acquire data-analysis skills, which requires specific training programmes (Zuiderwijk et al., 2015). When trying to achieve the OGDI objectives, the background of the data user is often not taken into account. In several cases, there was a narrow view on transparency, where transparency mainly referred to the visualisation or merely the opening of data. For instance, in one of the OGDIs, merely the ability to open up the emails of politicians was considered as transparency, whereas further content analysis of these emails was not considered.

Another possible explanation for the mismatch between the benefits delivered and the objectives set is that OGDIs might not be focused on achieving their objectives and may be based on copying other initiatives. We argue that the context and societal problems that OGDIs address should be better taken into account in OGDIs, and the assumptions and conditions for achieving OGDI objectives should be made explicit in open data policies. The objectives set in the start-up and design phases should be reviewed critically when OGDIs develop further. Objectives may need to be adjusted in the implementation and operation phases to ensure that there is a better match between set objectives and delivered benefits.

According to the literature, objectives should ideally be formulated according to ‘SMART’ principles (Doran, 1981). This means that they should be ‘Specific’ (What should the OGDI achieve?), ‘Measurable’ (How will we know that this has been achieved?), ‘Assignable’ (Who will do what to attain the objective?), ‘Realistic’ (What can realistically be achieved?) and ‘Time-related’ (When will this be achieved?) (Doran, 1981). Some of the examined OGDIs seem to be described more according to the SMART principles (e.g. the OGDI objectives of the Open Government Partnership of the US are specifically presented according to the SMART principles⁵) than others (e.g. for the OGDI of Punjab government public schools of India, the objectives are not specific⁶). Our research also suggests that OGDI objectives should be presented according to the SMART principles in order to obtain a more realistic understanding of the potential benefits that can be attained.

Study limitations

When interpreting the results of this study, one should take into account that we could not directly measure the fulfilment of each objective. We therefore compared the objective of each initiative to one or more related benefits delivered by the initiative. It is difficult to measure the attainment of some objectives since they consist of several sub-dimensions. It is therefore possible that the delivered benefits that we examined do not completely measure the objective of the OGDI.

Moreover, an important assumption in our study was that the data we collected reflect the actual initiatives rather than the opinions of the people providing the data. It is unclear to what extent the respondents have sufficient knowledge of the OGDIs to provide the correct information. However, we had reason to believe that at least most of the respondents were knowledgeable as they were participating in a MOOC on Open Government, the majority of the respondents (57%) were involved in OGDIs in general and 75% of them had at least three years of experience in the field (see earlier).

Furthermore, we do not claim that the 156 OGDIs in our sample are representative of the objectives of all OGDIs worldwide. One should keep in mind the context in which the objectives were collected. Most OGDIs were provided by respondents from the US, Brazil, the Netherlands and Spain. In addition, many of the studied objectives and benefits are influenced by other factors that we did not study. For example, increasing trust is not just influenced by OGDIs; it may also be affected by other developments and events, such as scandals and the corruption of politicians and the culture in a country. One should be aware that the studied factors are interrelated and complex.

Finally, we do not have information concerning when the OGDIs were launched. Some may have been launched only recently, and it may be too soon to establish whether their objectives have been achieved. For instance, it can take many years to increase trust in the government or increase government accountability. These limitations need to be taken into account in the interpretation of our results, and we recommend their further study in future research.

Conclusions

This study contributes to existing research by providing insight into the desired objectives of OGDIs and the extent to which these objectives are attained. Using data from 168 responses on 156 OGDIs worldwide, we found that certain objectives are more common than others. The most common objective is to ‘create openness’ (63% of the OGDIs), followed by the objective to ‘increase transparency within the government and/or its legislation’ (55%), to ‘engage with citizens through social media channels’ (50%), and the ‘use of OGD by the public (e.g. by citizens or journalists)’ (48%). The objectives in the categories ‘other’ (9%), ‘develop freedom of information legislation’ (19%) and ‘increase trust in the government’ (22%) were less prominent. Our survey showed that the OGDIs we studied mostly delivered operational and technical benefits, followed by economic benefits and, lastly, societal benefits. The main benefits delivered were increased transparency (58% of the OGDIs yielded this benefit) and the ability to reuse data (54%).

Our study shows that the benefits delivered are often in areas other than those in which the OGDIs’ objectives lie. This indicates that whether a benefit (e.g. increased openness, trust in the government or innovation by companies) is delivered is not significantly affected by having an objective related to that benefit, suggesting a mismatch between the two. This finding suggests that OGDIs might not be focused on achieving their objectives, but might be mimicking other initiatives. This has important implications as considerable amounts of money and effort are devoted to achieving these objectives, and this has not clearly resulted in the delivery of the related benefits. However, our study also shows that many OGDIs do deliver various benefits, although they do not seem to be closely related to the objectives of the OGDI. When we distinguished between OGDIs at different governmental levels (e.g. national and local), the analyses showed that the objectives of state- and national-level OGDIs are more often achieved compared to those of local- and regional-level OGDIs.

Further research should investigate why the objectives of state- and national-level OGDIs are more often achieved and whether the effects of local- and regional-level OGDIs can be improved by learning from other OGDIs. Practitioners should avoid copying each other’s initiatives and take into account the objectives, the context and the societal values that need to be delivered. OGDIs do not always deliver the expected benefits and the benefits may be in areas other than the intended ones. Future research should include an in-depth examination of how OGDIs can more effectively deliver the desired benefits.

Supplemental Material

RAS739115 Appendix 1 - Supplemental material for Investigating the attainment of open government data objectives: is there a mismatch between objectives and results?

Supplemental material, RAS739115 Appendix 1 for Investigating the attainment of open government data objectives: is there a mismatch between objectives and results? by Anneke Zuiderwijk, Rhythima Shinde and Marijn Janssen in International Review of Administrative Sciences

Supplemental Material

RAS739115 Appendix 2 - Supplemental material for Investigating the attainment of open government data objectives: is there a mismatch between objectives and results?

Supplemental material, RAS739115 Appendix 2 for Investigating the attainment of open government data objectives: is there a mismatch between objectives and results? by Anneke Zuiderwijk, Rhythima Shinde and Marijn Janssen in International Review of Administrative Sciences

Footnotes

Acknowledgement

This paper is related to the H2020 VRE4EIC project (see:

). The authors would like to thank their colleagues on this project for their input to this article,although the views expressed are the views of the authors and not necessarily of the project.

Notes

Dr Anneke Zuiderwijk is a researcher at Delft University of Technology,The Netherlands. She holds a PhD (with honour) in open data infrastructures and was ranked as one of the most prolific open data researchers (see Hossain et al.,2015). In 2016,Anneke won the Digital Governance Junior Scholar Award of the ASPA and DGS. See:

Rhythima Shinde is double master’s student in Data Science,in Faculty of Computer Science and in Engineering and Policy Analysis at Delft University of Technology,The Netherlands. She was the recipient of the Institute Award in her bachelor’s at IIT Bombay for three consecutive years. Her research interests are in leading technologies such as data sciences applied to societal systems like governance,energy or transport.

Professor Dr Marijn Janssen is a full Professor in ICT & Governance and chair of the Information and Communication Technology research group of the Technology,Policy and Management Faculty of Delft University of Technology,The Netherlands. He was ranked as one of the leading e-government researchers and has published over 380 refereed publications. For more information,see:

References

Abu-Shanab

(2015) Reengineering the open government concept: An empirical support for a proposed model. Government Information Quarterly 32: 453–463.

Alawadhi

Aldama-Nalda

Chourabi

et al. (2012) Building understanding of smart city initiatives. In: Scholl HJ, Janssen M, Wimmer MA, et al. (eds) International Conference on Electronic Government. Berlin: Springer, pp. 40–53.

Alexopoulos

Spiliotopoulou

Charalabidis

(2013) Open data movement in Greece: A case study on open government data sources. In: Panayiotis HK, Kostas M, Ioannis V, et al. (eds) 17th Panhellenic Conference on Informatics. Thessaloniki, Greece: ACM, pp. 279–286.

Arzberger

Schroeder

Beaulieu

et al. (2004) Promoting access to public research data for scientific, economic, and social development. Data Science Journal 3: 135–152.

Attard

Orlandi

Scerri

et al. (2015) A systematic review of open government data initiatives. Government Information Quarterly 32: 399–418.

Attard

Orlandi

Auer

(2016) Data Driven Governments: Creating Value Through Open Government Data. Berlin Heidelberg: Springer-Verlag.

Bakıcı

Almirall

Wareham

(2013) A smart city initiative: The case of Barcelona. Journal of the Knowledge Economy 4: 135–148.

Bates

(2012) ‘This is what modern deregulation looks like’: co-optation and contestation in the shaping of the UK’s Open Government Data Initiative. The Journal of Community Informatics 8(2).

Bertot

Jaeger

Grimes

(2010) Using ICTs to create a culture of transparency: E-government and social media as openness and anti-corruption tools for societies. Government Information Quarterly 27: 264–271.

10.

Brown

(2013) Why Kenya’s open data portal is failing – and why it can still succeed. Available at: https://sunlightfoundation.com/blog/2013/09/23/why-kenyas-open-data-portal-is-failing-and-why-it-can-still-succeed/

11.

Chun

Shulman

Sandoval

, et al. (2010) Government 2.0: Making connections between citizens, data and government. Information Polity 15: 1–9.

12.

Coglianese

(2009) The transparency president? The Obama administration and open government. Governance 22: 529–544.

13.

Cowan

Alencar

McGarry

et al. (2015) Adapting to climate change – an open data platform for cumulative environmental analysis and management. In: Gervasi O, Murgante B, Misra S, et al. (eds) International Conference on Computational Science and Its Applications. Cham: Springer, pp. 3–15.

14.

Cronbach

Meehl

(1955) Construct validity in psychological tests. Psychological Bulletin 52: 281.

15.

Davies

(2013) Open data barometer: 2013 global report. Available at: http://www.opendataresearch.org/dl/odb2013/Open-Data-Barometer-2013-Global-Report.pdf

16.

Dawes

Helbig

(2010) Information strategies for open government: Challenges and prospects for deriving public value from government transparency. In: Wimmer MA, Chappelet JL, Janssen M, et al. (eds) 9th International Conference on e-Government. Lausanne, Switzerland: Springer LNCS, pp. 50–60.

17.

Doran

(1981) There’s a S.M.A.R.T. way to write management’s goals and objectives. Management Review 70: 35–36.

18.

European Commission (2013a) Directive 2013/37/EU of the European Parliament and of the Council of 26 June 2013 amending Directive 2003/98/EC on the Re-use of Public Sector Information. Available at: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2013:175:0001:0008:EN:PDF

19.

European Commission (2013b) EU implementation of the G8 Open Data Charter. Available at: http://ec.europa.eu/newsroom/dae/document.cfm?doc_id=3489

20.

Evans

Campos

(2013) Open government initiatives: Challenges of citizen participation. Journal of Policy Analysis and Management 32: 172–185.

21.

Field

(2009) Discovering Statistics Using SPSS. London: SAGE.

22.

Grimmelikhuijsen

(2012) Transparency and Trust: An Experimental Study of Online Disclosure and Trust in Government. Utrecht: Utrecht University.

23.

Gurstein

(2011) Open data: Empowering the empowered or effective data use for everyone? First Monday 16.

24.

Harrison

Guerrero

Burke

, et al. (2012a) Open government and e-government: Democratic challenges from a public value perspective. Information Polity 17: 83–97.

25.

Harrison

Pardo

Cook

(2012b) Creating open government ecosystems: A research and development agenda. Future Internet 4: 900–928.

26.

HM Government (2011) Making open data real: A public consultation. Available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/78884/Open-Data-Consultation.pdf

27.

Hossain MA, Dwivedi YK and Rana NP (2016) State-of-the-art in open data research: Insights from existing literature and a research agenda. Journal of Organizational Computing and Electronic Commerce 26(1-2): 14–40.

28.

House

Hanges

Javidan

et al. (2004) Culture, Leadership, and Organizations: The GLOBE Study of 62 Societies. California, CA: Sage publications.

29.

Huijboom

Van den Broek

(2011) Open data: An international comparison of strategies. European Journal of ePractice 12: 4–16.

30.

Jaeger

Bertot

(2010) Transparency and technological change: Ensuring equal and sustained public access to government information. Government Information Quarterly 27: 371–376.

31.

Janowski

(2015) Digital government evolution: From transformation to contextualization. Government Information Quarterly 32: 221–236.

32.

Jetzek

(2015) The Sustainable Value of Open Government Data. Uncovering the Generative Mechanisms of Open Data Through a Mixed Methods Approach. Copenhagen: Copenhagen Business School.

33.

Jetzek

Avital

Bjorn-Andersen

(2013) The generative mechanisms of open government data. Paper presented at the 21st European Conference on Information Systems, Utrecht, the Netherlands.

34.

Jocelyn

Oliver

Gillian

(2014) Value in the mash: Exploring the benefits, barriers and enablers of open data apps. Paper presented at the 22nd European Conference on Information Systems.

35.

Kalampokis

Hausenblas

Tarabanis

(2011) Combining social and government open data for participatory decision-making. In: Tambouris E, Macintosh A and de Bruijn H (eds) International Conference on Electronic Participation. Delft, The Netherlands: Springer, pp. 36–47.

36.

Kassen

(2013) A promising phenomenon of open data: A case study of the Chicago Open Data Project. Government Information Quarterly 30: 508–513.

37.

Kim

Lee

(2009) An institutional analysis of an e-government system for anti-corruption: The case of OPEN. Government Information Quarterly 26: 42–50.

38.

Kroes

(2011) Opening remarks, press conference on open data strategy – SPEECH/11/872. Available at: http://europa.eu/rapid/press-release_SPEECH-11-872_en.htm

39.

Kulk

Van Loenen

(2012) Brave new open data world? International Journal of Spatial Data Infrastructures Research 7: 196–206.

40.

Lakomaa

Kallberg

(2013) Open data as a foundation for innovation: The enabling effect of free public sector information for entrepreneurs. IEEE 1: 558–563.

41.

Lathrop

Ruma

(2010) Open government: Collaboration, transparency, and participation in practice: ‘O’Reilly Media, Inc.’.

42.

Lee

(2014) Open Data Ireland: Evaluation framework. Available at: file:///H:/My%20Documents/Evaluation-Framework.pdf

43.

Lee

Kwak

(2012) An open government maturity model for social media-based public engagement. Government Information Quarterly 29: 492–503.

44.

Linders

(2013) From e-government to we-government: Defining a typology for citizen coproduction in the age of social media. Government Information Quarterly 29: 446–454.

45.

Macintosh

(2004) Characterizing e-participation in policy-making. In: System Sciences, 2004. Proceedings of the 37th Annual Hawaii International Conference on. IEEE, 10 pp.

46.

Mann

Whitney

(1947) On a test of whether one of two random variables is stochastically larger than the other. Annals of Mathematical Statistics 18: 50–60.

47.

Meijer

Bolívar

MPR

(2016) Governing the smart city: A review of the literature on smart urban governance. International Review of Administrative Sciences 82: 392–408.

48.

Meijer

Curtin

Hillebrandt

(2012) Open government: Connecting vision and voice. International Review of Administrative Sciences 78: 10–29.

49.

Moon

(2002) The evolution of e-government among municipalities: Rhetoric or reality? Public Administration Review 62: 424–433.

50.

Murphy

Davidshofer

(1988) Psychological Testing: Principles and Applications. Englewood Cliffs, NJ: Prentice-Hall.

51.

Nunnally

(1967) Psychometric Theory. New York, NY: McGraw-Hill.

52.

Obama

(2009) Memorandum for the heads of executive departments and agencies: Transparency and open government. Available at: https://www.whitehouse.gov/sites/whitehouse.gov/files/omb/memoranda/2009/m09-12.pdf

53.

Obama

(2012) Digital government. Building a 21st century platform to better serve the American people. Available at: https://obamawhitehouse.archives.gov/sites/default/files/omb/egov/digital-government/digital-government-strategy.pdf

54.

Olalere

Lazar

(2011) Accessibility of US federal government home pages: Section 508 compliance and site accessibility statements. Government Information Quarterly 28: 303–309.

55.

Radl

Skopek

Komendera

et al. (2013) And data for all: On the validity and usefulness of open government data. In: Proceedings of the 13th International Conference on Knowledge Management and Knowledge Technologies. ACM, p. 29.

56.

Sandoval-Almazán R (2015) Open Government Success Factors in Government Websites: The Mexican Experience. In: I. Management Association (ed.) Open Source Technology: Concepts, Methodologies, Tools, and Applications. Hershey, PA: IGI Global, pp. 1619–1635.

57.

Schillemans

Twist Mv Vanhommerig

(2013) Innovations in accountability: Learning through interactive, dynamic, and citizen-initiated forms of accountability. Public Performance & Management Review 3: 407–435.

58.

Susha

Zuiderwijk

Janssen

, et al. (2015) Benchmarks for evaluating the progress of open data adoption: Usage, limitations, and lessons learned. Social Science Computer Review 33(5): 613–630.

59.

Tauberer

(2012) Open government data. Available at: http://opengovdata.io/

60.

The Economist (2013) Show me the money. When transparency is the handmaiden to innovation. Available at: http://www.economist.com/blogs/graphicdetail/2013/11/open-data-0

61.

The World Bank (2013) New partnership seeks to bring benefits of open data to developing countries. Available at: http://www.worldbank.org/en/news/press-release/2013/09/18/new-partnership-seeks-bring-benefits-open-data-developing-countries

62.

Ubaldi

(2013) Open government data: Towards empirical analysis of open government data initiatives. In: OECD Working Papers on Public Governance. Paris: OECD.

63.

Veeckman

Van der Graaf

(2015) The city as living laboratory: Empowering citizens with the citadel toolkit. Technology Innovation Management Review 5(3): 6–17. http://timreview.ca/article/877.

64.

Villazón-Terrazas

Vilches-Blázquez

Corcho

et al. (2011) Methodological guidelines for publishing government linked data. In: Wood D (ed.) Linking Government Data. New York: Springer, pp. 27–49.

65.

Webster

Watson

(2002) Analyzing the past to prepare for the future: Writing a literature review. MIS Quarterly 26: xiii–xxiii.

66.

Welle Donker

Van Loenen

Bregt

(2016) Open data and beyond. International Journal of Geo-Information 5: 48.

67.

Whyte

Macintosh

(2002) Analysis and evaluation of e-consultations. e-Service Journal 2: 9–34.

68.

Worthy

(2010) More open but not more trusted? The effect of the Freedom of Information Act 2000 on the United Kingdom central government. Governance 23: 561–582.

69.

Robinson

(2012) The new ambiguity of ‘open government’. UCLA Law Review Discourse 59: 178–208.

70.

Zeleti

Ojo

Curry

(2016) Exploring the economic value of open government data. Government Information Quarterly 33: 535–551.

71.

Zuiderwijk

(2015) Open Data Infrastructures: The Design of an Infrastructure to Enhance the Coordination of Open Data Use. ‘s-Hertogenbosch: Uitgeverij BOXPress.

72.

Zuiderwijk

Janssen

Dwivedi

(2015) Acceptance and use predictors of open data technologies: Drawing upon the unified theory of acceptance and use of technology. Government Information Quarterly 32: 429–440.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

1.12 MB

0.58 MB

0.00 MB