Sage Journals: Discover world-class research

Abstract

Evaluating design ideas is an integral part of designing built environments. It involves multiple stakeholders with diverse backgrounds reviewing design solutions by studying their form and performance data. Although there are computational systems for supporting evaluation tasks, they are either highly specialised for designers or configured for a particular workflow with limited functions. We developed a Design Analytics method aiming at a collaborative and data-driven evaluation of alternatives in the design-evaluate-feedback cycle. Adopting this approach, we introduce D-ART as a prototype system composed of customisable Web interfaces for presenting design alternatives, enabling stakeholders to participate in data-informed discourse on alternatives and providing feedback to the design team. Its system design considers requirements gathered through literature review, critical analysis of the existing systems and collaboration with our industry partners. Finally, we assessed D-ART’s design through an expert review evaluation, which generally reported positive results on the system’s goals.

Keywords

data-driven design participatory design design analytics design alternatives visual analytics design evaluation

Introduction

The information complexity of architectural designs compounded with the stakeholders’ diversity can significantly hinder the collaborative evaluation of alternatives and decision-making.^1–3 Collaboration can suffer from ambiguity in design information, incomplete or missing details or terms, unavailability, unfitness of means for information sharing, sequential discourse, etc. Some of these can be addressed through tools encouraging the stakeholders’ involvement in, for example, design review meetings,⁴ online design modelling,⁵ planning.⁶ However, these existing tools are either highly specialised for designers or configured for specific purposes overlooking most stakeholder needs. Therefore, there is a need to understand better and support design collaboration between design stakeholders and designers. We pose and seek answers to the following three main research questions in this study: (a) how can we enable collaborative design decision-making on alternative solutions starting as early as possible in the design process? (b) What are the critical system features to support data-informed evaluation of design alternatives by design stakeholders? And (c) how do the interactive design-collaboration interfaces look and can be an integral part of design-share-feedback workflow ecosystems?

These are important questions for several pragmatic and theoretical reasons. Using an asynchronous and online system can allow stakeholders to investigate designs and help start an equal dialogue. Each can share their opinion without having the discussion dominated by few stakeholders, as usually observed in meetings.⁴ One of the bottlenecks in successful collaboration between designers and design stakeholders is the lack of collaboration support tools that can seamlessly integrate with various design workflows and, at the same time, can engage the stakeholders in decision-making. This bottleneck may increase the cost of collaboration and decrease the motivation for implementing new practices. Hence, a collaborative tool should communicate with various other systems without any unnecessary layers of tasks to motivate more feedback cycles, increase stakeholders’ engagement in design and lead to higher levels of satisfaction in the design outcome.⁷

In this paper, we present the Design Alternatives Reporting Tool (D-ART). As part of the Design Analytics research program, D-ART explores the potentials of interactive data visualisations in collaboratively evaluating design alternatives using built environments' design form and performance data [40]. Design Analytics combines visual analytics and computational design methods to reveal design data and inform decision-making.^8–10 We developed D-ART based on a set of high-level system requirements we compiled through a design-study method.¹¹ D-ART is a platform-independent and online system that can be integrated with any design workflow.

Developing such a system can face significant challenges. The diversity of the stakeholder interests and their expertise levels demand different abstractions of complex design data. Second, such systems should accommodate interactive techniques for enabling the expressiveness of design feedback. And third, compiling, structuring and presenting stakeholders’ rich and diverse input on design ideas will require computational methods that can detect, filter and link conversations to design ideas. D-ART prototypes are mainly means to explore the potential of visual analytics interfaces for data presentation, feedback input and feedback presentation. We conducted a formative evaluation of the D-ART’s minimally viable version to validate and improve our understanding of the data-informed collaborative decision-making.

Background

This research builds on three fields of knowledge: Design, Computer Supported Collaborative Work (CSCW), and Visual Analytics (VA). In the intersection of these knowledge fields, we narrowed the focus on Design Analytics, Collaborative Visualisation and Participatory Design.

Collaborative visualisation

Collaborative Visualisation is an emerging field at the intersection of Computer-Supported Collaborative Work (CSCW) and Visual Analytics. Isenberg et al.¹² define collaborative visualisation as: ‘The shared use of computer-supported, (interactive) visual representations of data by more than one person with the common goal of contribution to joint information processing activities’. This ‘shared-use’ can be synchronous, for example, accessed simultaneously, or asynchronous, for example, accessed at different times by different users in the space or distributed locations.^13,14 Asynchronous distributed collaborative analysis can result in higher quality outcomes, broader discussions and more detailed reports than face-to-face collaboration.¹⁵ However, most existing research in Collaborative Visualisation focuses on the co-located synchronous collaborative analysis.^16–18 Both modes demand a system-support for exploration and collaboration.¹⁹ The main challenge in developing an asynchronous distributed collaboration system is determining the design decisions leading to successful collaboration using visualisation media. Heer and Agrawala²⁰ examined a set of distributed asynchronous web-based collaboration systems and listed design considerations, such as modularity, view sharing and bookmarking, cost of integration. Among these designs, the practical ones translated to system features helped us develop D- ART prototypes.

Participatory design

D-ART aims to increase the involvement of stakeholders in design. To understand the possibilities and methods for increasing involvement, we surveyed some of the works in the Participatory Design field. Participatory focuses on engaging design stakeholders such as clients, users, consultants, construction teams, etc., in the design process. Participatory design approaches, particularly in the early stages, may help better-understanding client needs and lead to fewer disagreements and design changes.²¹

Identifying the tool needs to support stakeholders’ involvement in design decision-making depends on understanding the stakeholders’ participation and engagement levels. Arnstein²² defines three categories of participation: ‘non-Participation’, ‘Degree of Tokenism’ and ‘Degree of Citizen Power’. At Non-Participation, the objective is not to enable stakeholders to participate but to enable powerholders to influence stakeholders through manipulation. At the Degree of Tokenism, stakeholders can voice their opinion and inform the powerholders of their input. However, compared to Degree of Citizen Power, this category of participation lacks the tools and practices for ensuring that the participants’ voices and opinions are heard and followed. Building on Arnstein’s work, Wilcox²³ defined five levels of participation. The Degree of Tokenism has two levels of involvement: ‘informing’ and ‘consulting’. Furthermore, the Degree of Citizen Power has three levels of participation: ‘deciding together’, ‘acting together’ and ‘supporting’.

The tool types for the digital collaboration systems can be studied considering the stakeholders’ expected interaction with the content. For example, Jutraz and Zupancic²⁴ categorise such tools considering the higher-level tasks performed at different levels of participation: tools for informing, tools for communication, tools for decision-making, tools for collaboration process and tools for empowering process and engagement. In comparison, the tools give control to the stakeholders for empowering them in decision-making, the tools for informing the public limit the feedback to be received from the citizens.

Design analytics

The use of data leads to better decisions in fields of science, engineering and design. While the designer’s experience and tacit knowledge are valuable in identifying aspects of design problems and solutions, they may remain limited when working with multi-criteria problems with complex and often conflicting objectives.²⁵ Today’s computational tools allow for incorporating performance analysis in the early phases of design.²⁶ Designers use parametric or non-parametric design tools with analysis tools, such as EnergyPlus,²⁷ Radiance^28,29 and OpenStudio.³⁰ This combination allows generating design performance data in the early design phases. Design data presented in a visual format enhances design cognition.³¹ The visual forms, combined with interactivity, can enable designers to explore more and switch their attention to different objectives.

Collaborative design systems

We compared three systems considering if they can support asynchronous and distributed collaboration with design stakeholders. There are few systems as the adoption of collaboration systems in architecture—especially in the AEC industry—is still in its early stages.

Modelo³² is an online commercial Web application for design assets management. It supports sharing CAD files from, such as Rhino,³³ Revit,³⁴ SketchUp,³⁵ 3DsMax.³⁶ Its presentation features include images, interactive 3D views, and VR walkthroughs. It can also enable feedback, for example, as 3D annotations and text comments. Although Modelo has a robust 3D presentation feature, it lacks data visualisation for presenting performance metrics and feedback given by the stakeholders. In addition, a comparison of multiple alternatives is not supported in Modelo, which we believe is necessary as every alternative reveals different aspects of the problem at hand. DesignLink is a decision-support environment that allows professionals from diverse backgrounds to share their data, make sense of shared data from different collaborators, and decide ‘trade-offs’ based on performance.³⁷ It combines data across other design systems into customisable interfaces for presenting, for example, cost, physics, spatial composition information. The users can evaluate design alternatives through 2D visualisations in tabular form and images. However, DesignLink does not offer feedback functionality for commenting or discussions.

Konieva et al.⁵ developed an online urban design system to facilitate communication between design stakeholders using an interface connected to a CAD environment. The interface controls can modify parameters in the script, allowing stakeholders to generate different combinations and view the results in 3D. In addition, users can hide and show different design layers to change the level of fidelity. The system also displays analytical data on the 3D view and the side panel. It lacks feedback controls as well as comparison controls.

Our examination of these collaboration systems reveals three main high-level system features. Each system we investigated supports only a subset of these three main features, while D-ART aims to provide all three features (Table 1).

Table 1.

A comparison of the asynchronous collaboration systems (Y: System offers the feature).

System features	Content	Modelo	DesignLink	Konieva et al.
Presentation	Form	Y	Y	Y
	Performance		Y	Y
	Feedback	Y
Comparison	Form		Y
	Performance		Y
	Feedback
Feedback input	Form	Y
	Performance
	Feedback	Y

Design data presentation

The presentation of design alternatives’ form and performance data and stakeholders’ associated feedback.

Feedback input

The stakeholders' input as feedback on the design components and performance. The input should support continuous discussions through exchanges of ideas, replies and answers.

Alternatives comparison

The comparison of design alternatives’ forms and how they perform; and the stakeholders’ feedback and reaction to the design alternatives.

Methodology

We surveyed the current research on interactive systems for data-informed, asynchronous and sustained collaboration between designers and design stakeholders. Parallel to this, we worked with our industry partners to learn from their workflows and identify needs for sharing design alternatives with the stakeholders. We applied the lessons learnt from these activities to develop prototypes through use-case-driven system development and combined interactive visualisations of design form and performance data in the interfaces for ‘sense-making’.⁸ These activities are based on an extensive problem-driven research method called design study.³⁸

The design study method’s fundamental principle is iterative development and evaluation of ‘designs’. It involves ‘analysis of a specific real-world problem faced by domain experts, designing a [interactive] visualisation system that supports solving this problem, validating the design, and reflecting on lessons learnt to refine visualisation design guidelines’.³⁸ For our research, its practical benefit is identifying solution features in iterations. The method is divided into nine stages contained in three main phases. The first is the Precondition phase with three stages: learn, winnow and cast (e.g. selecting collaborators). The Core phase is composed of four stages: discover, design, implement and deploy. The final phase, Analysis, involves the reflections stage. Each phase has its evaluation strategy, and every stage is connected to the other eight stages.

We developed five high-level system requirements parallel to developing D-ART prototypes guided by these requirements. We also used these requirements to develop system features and their formative evaluations in increments.

DRQ1. Present each design alternative with customisable visualisation of form and performance data and the structured stakeholders’ feedback.

DRQ2. Enable feedback as comments and questions on design form and performance data and responding to the other feedback.

DRQ3. Allow visual comparison of form, performance and feedback on design alternatives.

DRQ4. Accessible online, decoupled from the design environments, workflows and systems.

DRQ5. Provide adaptable interfaces to accommodate a variety of stakeholders with different interests and backgrounds.

System description

D-ART adapts client-server Web application architecture to enable communication between the design data server and the D-ART’s interfaces, visualise data on browsers and collect feedback from the stakeholders (Figure 1). The high-level requirements were refined in use-cases and data models during the system development. D-ART receives three different data types: 1) design form, 2) performance metrics as categorical or numerical data and 3) design objectives. This generality enables reusing D-ART in multiple collaboration scenarios (DRQ4). The stakeholders’ input is associated with the alternatives and persistently stored for retrieval and report generation.

Figure 1.

System overview: Design modelling is independent of D-ART and can be of any modelling or computing environment. D-ART can be linked through a custom adaptor or directly by pushing data in the data repository, for example, FlowUI¹⁰ initiates storing data in the backend.

D-ART interface composition

D-ART interfaces consist of five main interfaces (Figure 2) (c.f. 40 for details). Projects Browser shows projects with a set of design alternatives that designers curate and upload through D-ART. Their summary data is accessible on-demand on an expandable panel (Figure 3). Grid View displays alternatives on a juxtaposed layout with controls to access the other views (DRQ1) (Figure 4). In this view, the stakeholders can choose the detail level, compare alternatives side-by-side (DRQ3), and access Alternative Overview (Figure 5) or Building Components Overview (Figure 6). The stakeholders can review and respond to others’ feedback in threads. Comments are captured with additional metadata that shows the creator of the input and date. The stakeholders’ input in the discourse is used in summary generation.

Figure 2.

D-Art interfaces and transition between them.

Figure 3.

Project browser allows for browsing projects and displaying project information.

Figure 4.

Grid view presents alternatives and their performance summaries on expanding and collapsing panels.

Figure 5.

Alternative view enables inspection of an alternative’s performance and feedbacks.

Figure 6.

A block-by-block analysis of performance in building components view.

Alternative View supports inspection of design data and stakeholders’ feedback on a particular alternative (DRQ1) (Figure 5). The data on a customisable tabular view shows the select metrics and their targets set in the requirements (DRQ5). The designers choose the images and 3D models at the time of curating the alternatives. The stakeholders can create data visualisations like a bar chart to study the relationship between different metrics (DRQ5). They can select any combination of metrics to display on graphs. Providing feedback is a dominant feature in D-ART to motivate discussion among the stakeholders.^39,40 The design stakeholders can express their opinions, ask questions or request changes on the feedback interface (DRQ2, DC6).

Building Components View supports inspecting the parts of an alternative, which can be any cluster of geometry with a particular meaning, for example, residential or commercial blocks, as in a massing model (Figure 6) (DRQ2). Alternatives Comparison View enables comparing multiple alternatives considering their form and performance data and review feedback given to selected alternatives (DRQ3) (Figure 7).

Figure 7.

Comparison of alternatives and providing feedback on the compare view.

Evaluation

To validate the design of D-ART, we performed an expert review as a formative evaluation. We questioned how domain experts might receive our approach to the collaborative assessment of design alternatives. The domain experts were asked to comment on the system’s objectives and potential in practice using D-ART’s minimally viable version.

We invited 7 experts (5 female and 2 male) with a minimum of 5-year experience in the AEC industry. They played different roles as architects, project managers, construction managers, interior designers, BIM managers, computational designers or researchers. Four participants never used data visualisation systems in their projects, and the rest had used tools such as Tableau and PowerBI. All experts held in-person meetings to communicate design information with stakeholders. One expert noted sharing annotated PDF reports for receiving feedback.

We used the first session as a pilot to refine the review process factors, for example, the task details and their completion time, interview questions and the setup for hosting D-ART. The evaluation took place in online meetings where each participant individually joined the evaluation session. Each session took between 1.5 to 2 h. Following a brief introduction of D-ART, the experts were asked to review alternatives, compose charts and respond to the other stakeholders’ feedback. This was followed by semi-structured interviews where we asked the experts their opinion on our approach for engaging the stakeholders in providing feedback considering design data on alternatives.

Results

We structured the results by the high-level themes that emerged during the coding of the findings.

Data presentation

Overall, the experts described the data presentation features as ‘very good’ and ‘smooth’ and hinted at improvements. Four experts asked about the performance metrics. For example, Expert-5 asked: ‘…what does it [commercial count] mean? Is it the number of floors?’ and Expert-1 asked: ‘What is View [metric]?’ These questions highlighted the need to have clear names for the metrics and to show brief descriptions on-demand. However, this is essentially a recommendation for the projects, not D-ART, as it is data-type agnostic in its architecture. Further, Expert-1 and -2 were confused when comparing metrics in the bar chart, possibly because of the unit mismatch or significant differences in magnitudes. Finally, Expert-4 suggested using category labels that can identify target ranges. The expert exemplified such labels with the energy efficiency cards used to explain the energy performance of home appliances. Consumers do not need to know what constitutes a good measure; the card indicates how appliances comply with standards. Expert-4 noted that data presentation could help to assess designs by focussing on a few performance metrics. The expert said, ‘there might be ten criteria and the stakeholder only…focuses on two of them leading to choose an alternative…the tool...might shorten this gap’.

Feedback input and presentation

The experts agreed on improvements for the feedback features, for example, supporting annotations on images, 3D models and charts. For example, Expert-6 recommended support for image or file attachments to comments. While text comments would cover most clients’ concerns, adding annotations and image sketches will support other edge cases. Expert-3 suggested using voice recording for commenting to make the stakeholders feel closer to the designers and motivate them to express their opinions independent from typing restrictions. Expert-5 suggested using reactions such as ‘Like’ and ‘Dislike’ to encourage quick feedback. However, they indicated that these abstract reactions might be too vague to support decision-making.

Expert 1 noted that ‘This [feedback presentation] is not an easy question because we are not used to seeing such tools’. Experts-2, 5 and 6 suggested enhancements to the comments display, such as filtering by comment type, tracking To-Do tasks progress and hiding resolved comments. These features are implemented in the following version.

Alternatives comparison

The experts described the comparison features as ‘pretty good’, ‘handy’, ‘gives clear information’. Expert-4 started evaluating designs using the Compare View first and then focused on one alternative in the Alternative View. The expert suggested that the users should be able to move between Compare View and Alternative View without losing thoughts or should be able to ‘mute’ an alternative in the compare view. Expert-3 was confused about the positive colour association of green and red’s negative colour association on tables. Expert-1 did not remember using the Compare View because the Compare button was not activated until a selection was made in the Grid View. The frequent shifts in the locus of attention when comparing and commenting has been reported as an issue by Expert-3: ‘I want to see the charts while I am giving feedback’. This shows the importance of localised feedback or annotations on visualisations. Expert-3 and -6 shared a similar concern on the lack of visualisation of the stakeholders’ progress or choices to the designers and other stakeholders.

Workflows integration

Expert-1 believed that D-ART’s real-world value would increase as it integrates with more design workflows. Expert-6 found value in the D-ART’s current support for one typical workflow and commented, ‘You do not have to make this tool work for every design software. if it were limited to Rhino, it would also be good’. Further, the experts had some suggestions to enhance the workflow integration. Expert-5 suggested having an intermediate interface between the design system and D-ART to modify and select the design details for a different audience. Expert-1 proposed a dynamic link between the design files and D-ART’s repository to achieve a real-time environment and keep stakeholders constantly in the loop. Expert-1 disagrees with Expert-5, who believes that the designers should control the frequency of feedback cycles. Expert-6 highlighted a technical concern when transferring complex models in the cloud.

D-ART in other domains

The experts agreed that D-ART could help evaluate ‘alternatives’ in other domains such as Web design, engineering design, product design. Expert-1 questioned, ‘why it cannot be used for designing an engine or ...’ Expert-3 noted that D-ART could be used where there is a comparison between choices. This suggestion is significant and deserves to be studied further.

Discussion

Overall, D-ART received a positive response. The experts agreed to support collaboration through design data presentation, feedback input and comparing alternatives. We summarise the issues we identified in Table 2 and guide the discussion about the results.

Table 2.

A summary of the issues in categories identified through the expert review.

Issue	Description	Category	Experts
IS1	Stakeholders not understanding the meaning of some of the performance metrics	Data presentation	1,4,5,6
IS2	Limitation of bar charts when working with multiple units or significant gaps between values	Data presentation	1, 2
IS3	Limitation of the single target value in communicating achievement levels	Data presentation	4
IS4	Annotation system support for localised feedback input	Feedback input	1,2,3,4,5,6
IS5	Image attachment support for feedback input	Feedback input	6
IS6	Quick reactions as feedback such as ‘like’ and ‘dislike’	Feedback input	5
IS7	Comments presentation features, for example, filtering by type and hiding resolved comments	Feedback presentation	2,5,6
IS8	Colour association bias when using ‘green’ and ‘red’ colours	Alternatives comparison	3
IS9	Intermediate design data uploading interface	Workflows integration	5
IS10	Supporting the review of large CAD models	D-ART in other domains; workflow integration	6

The confusion about the meanings of performance metrics (IS1) can be due to unfamiliarity or unclear names. The complexity of design data would benefit from having permanent descriptors to create a common ground for discussions. It is also crucial for the design team to onboard the stakeholders to a project by providing descriptions of the terms expected in any collaborative decision-making. D-ART can present on-demand definitions of the metrics.

IS2 is about the limits of having one type of chart. While bar charts are simple and frequently used, they are limited in working with multiple data units. The significant differences between the values make the comparison difficult and ‘meaningless’, as Expert-1 noted. Also, it remains necessary to consider different metrics simultaneously. D-ART can implement an option for filtering only comparable metrics in a specific type of charts (e.g. bar charts). Adding other charts, such as radar charts, can enhance the D-ART’s comparison capabilities (Figure 8 Right). However, adding new types of charts does not come without any cost: the stakeholders should be familiar with their use.

Figure 8.

(Left) The updated comment design allows users to expand, hide and filter comments. (Right) Added option to visualise performance data on radar charts.

Expert-4 mentioned that using one target can be insufficient to evaluate a metric (IS3) and recommended using some performance classifications for comparing alternatives. However, the creation of classifications is not trivial, for example, the values cannot be universal. For example, the same metric may have different efficiency value depending on the climate.

The experts agreed on annotations as localised feedback (IS4). However, designing a unified annotation feature for multiple types of visualisations has technical challenges. One solution would be to have data-agnostic screenshot-based annotations attached to a comment. While this could work with any presentation, it lacks the benefits of capturing data-custom annotations. The stakeholders can provide feedback on, for example, a 3D component by directly attaching a comment. As another solution, annotations can be done through code references in the discussions. A data-agnostic annotation would integrate with any data type that D-ART might grow to support. However, data-custom annotations would refine input and support the generation of more thorough reports for reviewing feedback. The latter will also enable the recreation of feedback visualisations. IS5 and IS6 focus on enhancements for the feedback features adapting to the stakeholders’ level of interest and experience. Advanced stakeholders might be interested in attaching reference images or sketches. Enabling quick feedback such as ‘Like’ and ‘Dislike’ ratings can help less experienced stakeholders and have the negative obscuring-information effect of why and how they like and dislike an alternative.

The comments on Feedback Presentation were minimal, possibly due to the novelty of the problem. IS7 sheds some light on its importance. The original feedback input area was taking space from the display. We reimplemented it to expand or collapse (Figure 8-Left). We also added filtering features to show only a subset of comment types and resolved or unresolved comments. Expert-4 recommended having upper and lower target limits (IS3). While domain experts may know about, for example, good ‘energy use’, stakeholders may not. Hence, Expert-4’s suggestion on using performance labels can help compare alternatives without interpreting their implications. However, the classification of such labels is not trivial. The first intuition is that the designers could create them, which assumes that the labels can be universal, which is not usually the case. For example, the same metric may have different efficiency criteria depending on the climate or location. Nevertheless, we find the utility of these labels high for decision-making.

IS9 highlights a bottleneck in the current integration of D-ART with CAD workflows. The required modifications to the CAD models are increasing the integration friction. We see this as a technical challenge but not necessarily directly questioning the design analytics approach. There could be various solutions for this problem, for example, the models can be saved in lower resolution and decomposed into a set of smaller models that are accessed on-demand. Another potential solution can be an intermediate interface to control the data density and access frequency. The interface could mediate sifting the models before shared. Developing such an interface for all integration workflows would be challenging as each CAD tool would have its specifications. However, such an interface would remain necessary for reducing integration friction.

Conclusion and future work

Collaboration is a core activity in design and can happen between diverse stakeholders. Successful collaboration requires a system of tools that integrate with the existing design ecosystem. D-ART aims to support collaboration at different levels of participation by empowering design stakeholders with a degree of citizen power and help them to reach an agreement. D-ART integrates tools for design data presentation, communication as feedback input and comparing alternatives. The main challenge was the integration of these functions in a tool to work together and not as independent features. D-ART can be seen as an instance for such integration. The expert review validates our approach and provides insights for the visualisation and interface design improvements. Although we have implemented some of the feedback we received on D-ART’s current version, such as filtering capabilities and radar charts, the following features are planned for future work.

Annotation

The experts highlighted a need for design data annotation to localise feedbacks in the relevant context, that is, visualisations. The feature addressing this need should enable annotating diverse data visualisations such as images, 2D charts, 3D geometry, tables, etc. The added complexity with this feature may create usability challenges and cognitive overload for non-expert users. In addition, managing the scalability of annotations can be another challenge.

Feedback presentation and report exportation

While D-ART supports basic filtering, there is still room for improvement. Complex feedback queries across alternatives and the generation of interactive online reports are examples of future enhancement.

D-ART in other workflows

D-ART works independently from design environments. However, we validated its integration with one design workflow to test its generic design. More work is needed to test its integration with other design workflows.

D-ART in other domains

The experts agreed that D-ART design is reusable in different fields of design. As an experiment, we uploaded real estate alternatives to D-ART in a scenario for shopping for a house in Vancouver. The result encouraged testing D-ART in evaluating alternatives other than designs, for example, consumer products, cars or even insurance policies.

As a formative evaluation, the expert review helped us assess D-ART’s plausible use in design data sharing and collaboration of diverse stakeholders. Following this evaluation, a critical question is how D-ART, and our approach, would be received in the real world where the stakeholders work on real projects. Expert-5’s comment demonstrates concerns around the cost-benefit trade-off using D-ART. The expert noted that ‘…the primary shortcoming, and if it exists, is the increased [design] time as there will be more feedback cycles and more discussions back and forth’. We believe that this trade-off can be measured if and when we have the opportunity to test D-ART in practice. Beyond its technical integration in a design workflow, social and professional implications will be critical to measure. For this reason, we believe that D-ART provides a platform ready to use in answering such questions. Its implementation-level challenges have been relatively solved. Our industry partners demonstrated a high interest in using D-ART. These encourage us to run a summative study for testing its social and practical acceptability.

We envision the summative design study to be conducted by real users in our industry partner’s firm. We can observe the potential of D-ART, the users’ subjective and objective responses through observations and interviews. Outside of the lab, we can also answer if and how our approach can change the practice. A summative evaluation will require a systematic preparation and execution that we plan as future work. As one of the experts said, ‘…professional practice always falls behind the ongoing development in the field of design communication due to the fear of change’. We should aim to strengthen the cooperation between research and practice in the AEC industry to eliminate such fears.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research,authorship,and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research,authorship,and/or publication of this article: This research was funded by MITACS Canada and Stantec Consulting (Ref. No: IAT19453).

ORCID iD

Halil Erhan

References

Klein

Sayama

Faratin

, et al. The dynamics of collaborative design: Insights from complex systems and negotiation research. Concurrent Eng 2003; 11(3): 201–209.

Sanders

EBN

Stappers

. Co-creation and the new landscapes of design. CoDesign 2008; 4(1): 5–18.

Bilal

Oyedele

Qadir

, et al. Big data in the construction industry: A review of present status, opportunities, and future trends. Adv Engineering Informatics 2016; 30(3): 500–521.

Fernando

Bassanino

, et al. Designing a novel virtual collaborative environment to support collaboration in design review meetings. J Inf Tech Construction 2013; 18: 372–396.

Konieva

Joos

Herthogs

, et al. Facilitating communication in a design process using a web interface for real-time interaction with grasshopper scripts. In Blucher Des. Proc Porto, Portugal: Editora Blucher, pp. 731–738.

Holzer

Downing

Optioneering: A new basis for engagement between architects and their collaborators. Architectural Des 2010; 80(4): 60–63.

Al-Kodmany

. Using visualization techniques for enhancing public participation in planning and design: process, implementation, and evaluation. Landscape Urban Plann 1999; 45(1): 37–45.

Erhan

Salmasi

Woodbury

. Visa: A parametric design modeling method to enhance visual sensitivity control and analysis. Int J Architectural Comput 2010; 8(4): 461–483.

Erhan

Abuzuraiq

Zarei

, et al. What do design data say about your model? A case study on reliability and validity, 2020.

10.

Erhan

Abuzuraiq

Zarei

, et al. FlowUI. Combining directly interactive design modeling with design analytics. 25th International Conference on Computer-Aided Architectural Design Research in Asia: Design in the Age of Humans. CAADRIA, 2020.

11.

Design alternatives reporting tool. Available at: https://d-art-79714.web.app/# (Accessed on Feb. 28, 2022).

12.

Isenberg

Elmqvist

Scholtz

, et al. Collaborative visualization: Definition, challenges, and research agenda. Inf Visualization 2011; 10(4): 310–326.

13.

Beck

Beedle

Van Bennekum

, et al. Manifesto for agile software development. (Accessed on Feb. 28, 2022). Online 2001, https://agilemanifesto.org.

14.

Dix

Finlay

, et al. Human-Computer interaction. Pearson Education 2004.

15.

Benbunan-Fich

Hiltz

Turoff

. A comparative content analysis of face-to-face vs. asynchronous group decision making. Decis Support Syst 2003; 34(4): 457–469.

16.

Agrawala

Beers

McDowall

, et al. The two-user responsive workbench: support for collaboration through individual views of a shared space. In: Proceedings of the 24th annual conference on Computer graphics and interactive techniques, New York, NY, USA, August 1997, pp. 327–332.

17.

Anupam

Bajaj

Schikore

, et al. Distributed and collaborative visualization. Computer 1994; 27(7): 37–43.

18.

Brodlie

Duce

Gallop

, et al. Distributed and collaborative visualization. In: Computer Graphics Forum, Volume 23. Hoboken, NJ: Wiley Online Library, pp. 223–251.

19.

Shneiderman

. Creativity support tools: Accelerating discovery and innovation. Com ACM 2007; 50(12): 20–32.

20.

Heer

Agrawala

. Design Considerations for Collaborative Visual Analytics. Inf Visualization 2008; 7(1): 49–62.

21.

Caixeta

Bross

Fabricio

, et al. Value generation through user involvement in healthcare design. In: Proceedings of 21st Annual Conference International Group for Lean Construction (IGLC), Brazil, Fortaleza, 29 July–2 August 2013.

22.

Arnstein

. A ladder of citizen participation. J Am Inst Planners 1969; 35(4): 216–224.

23.

Wilcox

. The guide to effective participation. (Accessed on Feb. 28, 2022). Partnership Brighton 1994, http://www.partnerships.org.uk/guide/index.htm.

24.

Jutraz

Zupancic

. Criteria for cooperative urban design through digital system of design tools. In: 16th International Conference on Information Visualisation, Montpellier, France, 11–13 July 2012, IEEE, 2012, pp. 416–421.

25.

Bernal

Marshall

Okhoya

et al. Parametric analysis versus intuition-assessment of the effectiveness of design expertise. In: 37th eCAADe and 23rd SIGraDi Conference, Porto, Portugal, 16 September 2019, 2019.

26.

Anton

Tanase

. Informed geometries. Parametric modelling and energy analysis in early stages of design. Energ Proced 2016; 85: 9–16.

27.

Crawley

Lawrie

Winkelmann

, et al. Energyplus: creating a new generation building energy simulation program. Energy and Buildings 2001; 33(4): 319–331.

28.

Radiance — radsite. Available at: https://www.radiance-online.org/ (Accessed on June 22, 2021).

29.

Daysim 4.0 download (free). Available at: https://daysim.software.informer.com/4.0/ (Accessed on June 22, 2021).

30.

Guglielmetti

Macumber

Long

. Openstudio: an open-source integrated analysis platform. In: 12th Conference of International Building Performance Simulation Association, CO, USAGolden: Tech Report, Natl Renew Energ Lab. (Nrel), 2011.

31.

Munzner

. Visualization Analysis and Design. Boca Raton, FL: CRC Press, 2014.

32.

Modelo. Available at: https://modelo.io/ (Accessed on Feb. 28, 2022).

33.

McNeel

Rhinoceros 3d (version 6.0), 1998. Available at: https://www.rhino3d.com/ (Accessed on Feb. 28, 2022).

34.

Autodesk Revit. Available at: https://www.autodesk.com/products/revit/overview (Accessed on May 29, 2021).

35.

Trimble Sketchup. Available at: https://www.sketchup.com/ (Accessed on May 29, 2021).

36.

Autodesk . 3dsmax. Available at: https://www.autodesk.com/products/3ds-max/overview (Accessed on May 29, 2022).

37.

Holzer

. Sense-Making Across Collaborating Disciplines in the Early Stages of Architectural Design. PhD Thesis (Embedded Practice), RMIT University, Melbourne, Australia, 2009.

38.

Sedlmair

Meyer

Munzner

. Design study methodology: Reflections from the trenches and the stacks. IEEE Transactions Visualization Computer Graphics 2012; 18(12): 2431–2440.

39.

Willett

Heer

Hellerstein

, et al. Comment space: structured support for collaborative visual analysis. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 2011; 3131–3140.

40.

Alsalman

. Data-Informed Collaboration in Evaluating Design Alternatives. MSc Thesis, Simon Fraser University, Burnaby, 2021.