International Journal on Interactive Design and Manufacturing manuscript No. (will be inserted by the editor)
Data exchange and multi-layered architecture for a collaborative design process in Virtual Environments Monica Bordegoni · Francesco Ferrise · Marco Ambrogio · Francesco Caruso · Fabio Bruno
Received: date / Accepted: date
Abstract The design review process of new products is time consuming, requires the collaboration and synchronization of activities performed by various experts having different competences and roles, and is performed using different tools and different product representations. In order to improve the performances of the overall product design process, it would be beneficial the availability of Computer aided tools supporting conceptual design and analysis activities within an integrated environment based on a multi-disciplinary model paradigm. The paper presents an environment named PUODARSI that allows designers to modify the shape of a product through haptic interaction and to test in real-time the structural and fluid-dynamics impact of these changes. The research work demonstrates that the integration of modeling tools based on haptic interfaces, fluid-dynamics analysis tools, and Virtual Reality visualization systems is feasible through proper data model exchange and handling while preserving real-time computation. Keywords Virtual Design · Interactive Simulation · Haptic Interaction · Multi-disciplinary Model · Design Review
1 Introduction The process of product design and engineering comprises various activities that start from market analysis and concludes with product distribution and maintenance. Progresses in M. Bordegoni · F. Ferrise · M. Ambrogio · F. Caruso Dipartimento di Meccanica Politecnico di Milano Via La Masa 1, 20156, Milano (Italy) Tel.: +39 02 2399 8232 Fax: +39 02 2399 8282 E-mail:
[email protected] F. Bruno Dipartimento di Meccanica Universita’ della Calabria Tel.: +39 0984 494623 Fax: +39 0984 484673 E-mail:
[email protected]
2
the product development process have been pushed by factors such as design costs, performances required and reduced time-to-market. We have seen progresses concerning methodologies, tools, and technologies for supporting the various phases of the product development in the last decades (Ulrich and Eppinger (2003)). Activities like product definition and conceptualization, aesthetic design, detailed design, testing and simulation, and manufacturing are today performed in a digital context. Recently, we are facing a trend where traditional computer aided tools (CAS/CAD/CAE/CAM) are being included into a more comprehensive environment named Virtual Prototyping (Rix et al (1995)). Traditionally, CAS/CAD tools have been used for the design of a new product, including its aesthetic shape and components details and assembly. CAE tools allow the simulation of various aspects and performances of a product and are used in order to adapt the design to satisfy specific requirements. In this evolving situation, some emerging aspects have arisen, which need to be considered in future developments: – The users of design and engineering tools are more demanding, and ask for tools that are conforming to their habits and skills rather than the opposite, where they are obliged to adapt themselves to the various technologies. – Companies often operate in a context of Virtual Enterprise where the activities are actually distributed in the various offices, or are sub-contracted to other companies in a dynamic way. – Concurrent engineering paradigms are being implemented by companies and mainly consist of restructuring the organizations of design and engineering activities through a parallel and close collaboration among the various deputy offices and human resources. – Product testing is performed in digital environments and aims at checking whether product requirements are met by designs, and at studying design alternatives in case targets are not met. The developments of future methods and technologies for product development have to consider these users needs and requirements. From the lists of emerging aspects to consider, two major points are particularly important: – user-centered tools; – full product testing. The concept of user-centered has to do with the fact that tools need to be designed with users in mind, considering their skills, know-how, working conditions and activities. Full product validation consists of methods and tools allowing all users involved in the various and differing activities related to product testing and design-review to work together, at the same time on the same project in a coherent and consistent way, and also from places that may be geographically dislocated. From these considerations, we have identified some topics that require being further studied and researched: – new interaction modalities of design and engineering tools that better meet users expectation. Examples are haptic interfaces that allow physical interaction with the virtual products (Bordegoni and Cugini (2007)), and mixed and augmented reality systems that allow mixing virtual and physical objects within the users working space (Haller et al (2006)). – multi-disciplinary model paradigms supporting effective collaborative design-review (van der Auweraer (2008)). This requires the study of tool-set that can be used by several users possibly distributed
3
– to test various product aspects, operating on different views and representations of the same product model. In addition, there should be a close loop between the re-design and testing activities, that should be possible to perform interactively and in a unique environment. This paper intends to analyze issues concerning interaction modalities and problems related to the integration of tools and data models in the context of virtual design, and to describe and discuss an example of a possible solution implemented through a case study. The research has been carried out in the context of the PUODARSI (Product User-Oriented Development based on Augmented Reality and interactive Simulation) project, funded by the Italian Ministry of University and Research (http://www.kaemart.it/puodarsi). The paper is organized as follows. In Section 2 the authors reason about the relation between product design problems, tools and data models and a case study related to design review of aesthetic products. Section 3 presents the conception of the system we have developed aiming at integrating shape modification performed through haptic interfaces and fluid-dynamics interactive simulation. Section 4 presents an overview of the state of the art of hardware and software technologies at the basis of the system development, and Section 5 describes the operational principles and the implementation of the system. Finally, Section 6 discusses the research results described in the paper and draws some conclusions.
2 Problems, tools and data models The technological development in the context of product design and engineering has followed an approach where each problem or point of view has been supported by means of a specific and dedicated tool. For example, there are tools for designing products or more precisely to define in mathematical and geometrical terms the shape (CAS/CAD), tools for analyzing various aspects of products in their lifecycle environments and situations (CFD, structural FEM, multi-body, etc.), there are tools for realistic visualization of products, etc. Typically, each tool is based on a proper data model. Actually, a data model may be consisting of several data models. For example, lets consider a CAD system: it uses a model for representing the geometry (B-reps, NURBS), a model (tessellated) for rendering the geometry, and a model for representing the product features. This approach can be described as a multi-layered modeling architecture including layers concerning product design problems, related tools and related data models (Figure 1). The use of different tools each one based on its own data model implies the necessity to learn and use different user interfaces, and causes difficulties, errors and loss of time for exchanging data and for guaranteeing their consistency during data model conversion and modification. In order to be fully effective, the emerging paradigm of virtual prototyping should be based on the integration of tools supporting product development, and of related models. A better integration would allow users to use a unique user interface, and would support a coherent correlation among data and procedures managing those data. This paper addresses the evolution of virtual prototyping techniques on the basis of a case study consisting of design review and structural and fluid-dynamics analysis of aesthetic products.
4
Fig. 1 Multi-layered architecture relating product design problems, tools and data models.
2.1 Case study: design review of aesthetic products The conceptual design of the shape of aesthetic products is usually performed by the designer, here intended as the design team, who is the responsible of the project (Figure 2). The design team includes stylists and CAS/CAD engineers. Their design solution(s) is passed to analysts, who are expert engineers on analysis and simulation methodologies and tools. In the case study that we are addressing, the mathematical model representing the design solution is passed to the CFD (Computational Fluid Dynamic) analyst for the analysis of fluid-dynamics aspects, and to the structural FEM (Finite Element Method) analyst for the study of structural aspects. According to the analyses results, it is very common that the shape of the object has to be modified. The structural FEM or CFD experts may suggest how to modify the shape and solve the problem. The types of modification to apply could be described in terms of features (intended as parameters about geometry and functionality/semantics). Thus, for example, the designer can describe the required changes in terms of aesthetic features [CC1], the CFD analyst could provide fluid-dynamics feature expressing that a vortex has formed in a certain part of the product model, and that it could be fixed by modifying the shape in a specific way (for example, increase the curvature in a point, create a more flat surface in a certain area). Since the modification impacts the aesthetic value of the product, the design team usually try to find a compromise that takes into account both aesthetic and technical aspects as suggested by the engineers. The design and analysis activities are performed by the various users by means of different tools (oriented to design, analysis and information visualization), each using specific data models.
5
Perform structural FEM analysis
FEM expert
FEM analysis Tool Visualization Tool
designer Modify shape
Design Tool Visualization Tool
Perform CFD analysis
CFD expert
CFD analysis Tool Visualization Tool
Fig. 2 Users roles and tools during product design review and testing.
Referring to the example that we are considering, the problems and tasks, tools and data models are the following (Table 1). Design tasks: CAS/CAD tools are used to create geometry of shapes; they use CSG, B-rep, NURBS representations, feature, tessellated representations. Visualization task: several tools are available and can be used for (stereo) realistic rendering; they use tessellated representations. Analysis task: CFD and structural FEM tools are used for performing analysis and simulation using mesh-based and voxelbased representations.
Table 1 Problems, tools and model representation Problem/Task Product Design Realistic Visualization Analysis
Tool CAS/CAD tools 3D rendering tools(RTT, ...) Analysis/simulation tools (structural FEM, CFD, ...)
Model Representation CGS, Feature, Tessellated Tessellated Mesh, Voxel
This case study, although simple, highlights anyway complex problems related to the conversion of data models and to the maintenance of their consistency during these mappings. In fact, models require to be converted when passed from design to analysis and visualization tools; this activity is error prone, tedious and time consuming.
2.2 Towards a multi-disciplinary model paradigm The model of the product being designed and tested can be considered from three different points of view:
6
– style surface (created and modified by the designers) – external boundary surface plus its surrounding space (used for fluid-dynamics analysis) – inner boundary surface (used for structural studies). Currently, if the style surface of object is modified, the other two models do not change accordingly and automatically. In fact, there are al least three different models that are kept and handled separately. Ideally, the three views should be consistent so that modifications propagate from one model to the other. In this way, the various users may work on the different views of the model without the necessity to convert models and to check their versions. The PUODARSI system described in this paper proposes a unique environment that can be used in a collaborative way by designers and engineers, where the object shape can be visualized in a realistic modality and at the same time by the two users (Figure 3). The environment allows the designer to modify the object shape easily and intuitively, through some haptic devices, and the engineer to run the analysis and simulation on the new shape. This is reiterated up to reaching a consensus about the aesthetic and technical aspects. Besides, the various solutions can be performed in comparative way.
designer
Modify shape
PUODARSI system
Perform analysis
engineer
Design tool Visualization tool Analysis tool
Fig. 3 Users roles and tools using the PUODARSI system.
3 System conception The PUODARSI environment is an interactive system that allows users to visualize in 3D or in an Augmented Reality (AR) environment the geometrical model of a product, to perform fluid-dynamics tests in real-time and to modify the geometrical shape of the object. The system is intended to be used for design review and analysis of a product whose geometrical representation is available. The geometrical representation of the product can be reconstructed from a real object by means of reverse engineering techniques, or it can be created using a CAD tool. The object model is imported within the PUODARSI environment and shown to users in stereoscopy or through the use of AR technologies. Two are the users of the system: designer and analyst. The designer can evaluate the aesthetic shape of a product; the analyst is in charge of performing fluid-dynamics analysis (CFD) of the product model. The result of the CFD analysis is shown through the visualization of the velocity field of air around the object. The model and the analysis data may be visualized in a stereoscopic modality; alternatively, analysis data may be visualized onto the physical prototype of the product by using an AR technology (Figure 4).
7
image of the PUODARSI augmented reality environment where CFD analysis results can Fbeig.seen4 Conceptual super-imposed onto a physical prototype of the product.
The evaluation of fluid-dynamics characteristics of the model may reveal some problems (such as vortexes) that require a modification of the shape of the object in order to be fixed. Therefore, the analyst may decide to ask the designer to modify the shape providing specific information about the motivation and the area where to intervene. The designer can modify the shape of the product quickly, easily and intuitively using a haptic interface (Figure 5). The modification is performed on the basis of the requests made by the analyst, and also taking into account stylistic preferences. The users can also insert annotations that may be related to the object geometry or to the analysis data. For example, the analyst can attach an annotation such as vortex generates in this area, surface should be made more flat to an area of the object surface. The PUODARSI environment can be though of as a simulation of a wind tunnel where one can simulate the interaction of air with an object in a virtual environment, and where the object shape can be modified directly and interactively as if one were inside the wind tunnel, but without actually perturbing the phenomena.
3.1 Case study The PUODARSI environment is an interactive system that allows users both designers and analysts- to collaborate on the review of the design of a product. A case study has been identified, which consists of the design review of a motorbike windscreen.
8
Fig. 5 Conceptual image of the PUODARSI haptic design environment. In a first scenario (Scenario 1) the geometry of the product has been developed using a CAD tool. In a second scenario (Scenario 2), the product is physically available and its geometrical representation has been created using a reverse engineering process (Figure ??). The geometrical representation of the object can be visualized within the environment. The designer can evaluate the shape of the windscreen; the analyst may decide to run the fluid-dynamics analysis of the shape. The system graphically visualizes the velocity field of air around the object. The visualization can be stereoscopic, or in case a physical prototype of the windscreen is available (Scenario 2), the velocity field of air is visualized on top of the physical prototype through the use of an augmented reality system. The evaluation of fluid-dynamics characteristics of the model may reveal some problems (such as vortexes) that require a modification of the shape of the object in order to be fixed. Therefore, the analyst may decide to ask the designer to modify the shape providing specific information about the motivation and the area where to intervene. The designer operates on the shape easily and intuitively using a haptic interface. The modification is performed also taking into account style preferences. The PUODARSI environment consists of several hardware and software components supporting the activities and operational modalities described above. The environment includes a stereoscopic visualization system allowing realistic rendering of the product model, a haptic interface for applying modifications to the model, and a real-time software application supporting functionalities for performing fluid-dynamics analysis on that.
9
Real Object
!
Reverse Engineering
!
CAD model
Fig. 6 Physical windscreen and its geometrical model reconstructed through a Reverse Engineering process.
3.2 Theoretical background The architecture of the environment is demanding especially for what concerns the performances required (for example, interactive haptic rendering, real-time simulation), and the integrability of hardware and software components. The main issues addressed are the following: – model of object should be suitable for visualization, haptic rendering and also simulation. This means that the geometrical model should be accompanied by some other parameters concerning the physical properties of the object; – haptic rendering requires real-time computation allowing interactive modification of geometry of the object; – the fluid dynamic analysis should be run almost in real-time in order to provide immediate feedback to the users. For what concerns the haptic rendering, we have focused on the so called haptic modeling that addresses the modeling of virtual shapes using haptic technologies. The novelty of these technologies is that they allow users to touch, feel, manipulate and model objects in a 3D virtual environment that is similar to a real natural setting. Several applications based on haptic modelling have been developed. Some applications have been developed with the aim of providing haptic interaction with volume dataset, without actually providing realistic force feedback (Avila and Sobierajski (1996); Iwata and Noma (1993)). Solutions based on geometric and volumetric
10
modeling for enabling local deformation are presented in (Celniker and Gossard (1991)). More general models support global deformations, and mimic the global and local effects of plasticity, mass preservation and surface tension that are typical of clay material (Dewaele and Cani (2004)). Some other applications are more related to physics-based shape modeling. An overview of physics-based deformable models is provided in (Nealen et al (2006)). Some sculpting systems have been developed based on haptic force associated with dynamic subdivision of solids, which give users the illusion of manipulating semi-elastic virtual clay (Celniker and Welch (1992); Dachille et al (2001); Jagnow and Dorsey (2002)). In order for the haptic device to exert appropriate forces in response to users actions the virtual object and its properties are simulated by means of a physics-based model. Various physics-based modeling techniques have been developed. Just to quote examples of such applications, Terzopoulous and Fleisher provide a basis for physics-based design supporting simple interactive sculpting using viscoelastic and plastic models (Terzopoulos and Fleischer (1988)), Celniker and Welch investigated deformable B-splines with linear constraints (Celniker and Welch (1992)). Most of the above-mentioned systems are based on the use of the point-based Phantom stylus by SensAble Technologies Inc. for interacting with the virtual clay. The only physically based shape modeling system commercially available is FreeForm by SensAble Technologies Inc., which is based on the Phantom haptic device http://www. sensable.com. Users work directly with the digital clay using the Phantom stylus as a modeling tool. Hardness and surface smoothness of the clay can be varied, and different modeling tools can be selected. The material can be removed using some carving operators, but the user can also work from inside out pulling and deforming the shape. Finally, our research group has developed a system based on haptic technology for the generation and evaluation of digital shapes for aesthetic purposes. The system has been developed in the context of the TnD project - a project partially funded by the European Union (http://www.touch-and-design.eu) and has been described in several papers (Bordegoni and Cugini (2007)). A following project is ongoing for developing a system based on a new concept of haptic interface for shape evaluation and supporting a stereoscopic ergonomic setup. This new system is being developed within the framework of the European project SATIN Sound And Tangible Interfaces for Novel product design (Bordegoni et al (2008)). Regarding Computational Fluid Dynamic (CFD) applications, they allow users to investigate the behavior of the fluids in a fixed volume by numerical solving the differential equations of Navier-Stokes, through a discretization process that uses the finite volume method (standard approach), the finite elements or the finite differences (Ferziger and Peric (1999); Anderson (1995)). A CFD simulation is divided into three distinguished phases: pre-processing, simulation and post-processing. The first step consists of the definition of the control volume, the generation of the mesh, and the physical parameters of the fluid. All these data are passed to a solver that computes the solution. Scientific visualization algorithms are used to visualize the results of the simulation that generally consist of datasets. The visualization application aims at presenting to users the simulation data in intuitive and direct way. Some research works have tried to create real-time simulations of the fluid behaviors in virtual dynamic fields (Chen et al (1997)).
4 Overview of hardware and software components The hardware and software components of the system have been decided upon a thorough analysis of the state of the art in the reference domains: 3D visualization, haptic interfaces
11
and interactive simulation. Some choices concerning hardware components have been done considering the various technologies already available at our labs; software components have been selected among available open source libraries. For what concerns visualization systems, several VR technologies have been considered (Dai (1998)), such as: – the full immersion stereoscopic HMD, 5DT 3D HMD 800: (http://www.5dt.com); the augmented see-through NOMAD HMD: (http://www.nomad.com); the Nvisor ST see-through HMD (http://www.nvisinc.com). – projection-based systems, such as a system developed within the context of the European project SATIN (Bordegoni et al (2008)), and the SenseGraphics system (http://www. sensegraphcs.com) – the wall display system CyViz: http://www.cyviz.com. For what concerns open source software libraries supporting stereoscopic visualization, we have considered VTK (http://www.vtk.org), openSceneGraph (http://www. openscenegraph.org/) and OpenSG (http://opensg.vrsource.org/trac). These libraries have been considered mainly because of the possibility of integrating external libraries and of importing dataset generated by simulation environments. Particularly interesting for our aims is VTK, which is a scientific visualization library that manages CFD structured and unstructured grids. VTK library has been successfully integrated with the Augmented Reality library ARToolKit (http://www.hitl.washington.edu/artoolkit/). This AR library is named VTK4AR. This library has been used for the visualization of CFD data in an AR environment as described in (Bruno et al (2006)). VTK is also available in a version supporting a graphical interface that is named Paraview (http://www.paraview.org) that allows us to manage VTK data without the need of using a programming language. This is very useful to use in place of VTK during the initial system prototyping phase. For what regards the haptic system for haptic modelling, we have considered the various technologies at our disposal in our labs and evaluated the benefits and disadvantages in using one technology compared to other ones. Specifically, we have considered the following haptic devices: the Phantom device by SensAble (http://www.sensable.com), the HapticMaster by MOOG-FCS (http://www.moog-fcs.com/robotics) and the Virtuose system by Haption (http://www.haption.com). The following open source software libraries supporting real-time haptic rendering have been considered: CHAI3D (http://www.chai3d.org), H3D (http://www.h3d.org), HaptikLibrary (http://www.haptik-library.org), OpenHaptic (http://www.sensable.com), and osgHaptics (http://www.vrlab.umu.se/research/ osgHaptics). Some tests have been performed using these libraries, mainly with the aim of evaluating the possibility of integrating them with the other software components of the system. On the basis of the results of these benchmarking activities it has been decided to use the Phantom device mainly because all the libraries for haptic rendering support these devices since are very common and diffused. For what concerns the software library, it has been decided to use a generic haptic library, such as the H3D library, instead of using a proprietary library like the Sensable OpenHaptic, because it allows us to develop the application for the Phantom device, but also to run it with other haptic devices also supported by the library. According to the requirements of the PUODARSI framework some issues have to be considered for what concerns the selection of the interactive simulation environment. One issue is related to the possibility of creating pre-processing steps that can be run automatically; a second issue concerns the compatibility of simulation results with an external scientific visualization environment.
12
It has been demonstrated that the STL file format is a good compromise for exchanging tessellated geometrical data among different software. The mesh generation algorithm of our system is expected to receive an STL geometric file and to generate a mesh that is compatible with the CFD solver. The solver should compute the solution in real-time, and also should give output results that are easy to manage by using the selected scientific visualization environment (VTK environment in our case). The software libraries that satisfy these requirements are the following: Deal II, OpenFlower, Comsol Multiphysics and OpenFOAM. Deal II (http://www.dealii.org) is an open source library written in C++ that is fast enough for our aims, but is too limited since it does not support unstructured grids and works only with QUAD (2D) and HEX (3D) elements. OpenFlower (http: //openflower.sourceforge.net) is also written in C++, supports non structured grids and requires GMSH (http://www.geuz.org/gmsh) as mesh generation algorithm. Comsol Multiphysics (http://www.comsol.com/) allows users to run multi-physics simulation; it is not only a solver but it also supports the pre and post processing steps. Some tests have been carried out on these libraries to verify the feasibility of performing real-time simulations (Ambrogio et al (2008)). Finally, OpenFOAM (http://www.opencfd.co.uk/openfoam) is written in C++ and runs exclusively on Linux/UNIX operating systems. It receives in input a mesh in the GMSH format thanks to the gmshToFoam function, and generates an output file that is compatible with VTK/Paraview thanks to the foamToVTK function.
5 PUODARSI environment The definition of the PUODARSI architecture has been made upon the selection of hardware and software components made on the basis of the state of the art analysis. The PUODARSI environment is an interactive system that allows users to visualize realistically the geometry model of a product, visualize the velocity field of a fluid around it, modify the geometry in an intuitive way (by using haptic interfaces) and see how the modification influences fluid-dynamics characteristics of the model. Therefore, the PUODARSI environment consists of the following main three modules: 1. module for haptic modification of product shape, 2. module for 3D rendering of object shape, annotations and of scientific data, and 3. module for interactive CFD analysis. For what concerns the haptic interface used for shape modification, it has been decided to use the SenseGraphics 3D-IW immersive workbench that consists of a Phantom haptic device integrated with a stereo visualization system including a CRT monitor, a semitransparent mirror and some Stereographics shutter glasses. This configuration allows a good collocation of the haptic and the stereoscopic visualization space, and is more ergonomic compared to HMDs that users are not keen on wearing for long periods. Regarding the software architecture, the major aspect that has been considered for the selection of the software libraries is the support for data exchange from the haptic application to the CFD simulation environment, and from CFD module to the data visualization in a Virtual/Augmented Reality environment. The selection of the libraries also considers that the initial file received in input is a CAD geometric file in STL format. For what concerns the selection of the haptic library, the environment should include a library that allows us to modify a tessellated CAD model. Actually, for the purpose of the project, that is mainly performing a feasibility study, a simple single contact point deformation algorithm based on a three-dimensional Gaussian curve is enough.
13
The following sections present the system architecture, its modules and their implementation and integration.
5.1 System architecture On the basis of the hardware selection and of the system requirements and software benchmarks, the following software libraries have been selected: – – – –
H3D haptic library for deforming tessellated models; OpenFOAM library for performing real-time CFD analysis; VTK visualization library for scientific data visualization. ARToolKit as pattern-based tracking system for the visualization module.
H3D library has been considered the most appropriate solution for haptic rendering since it already includes some useful algorithms for implementing tessellated surfaces deformation. OpenFOAM library allows us to manage input files from CAD modeling software through the GMSH module, and to export the results to VTK. ARToolKit is suitable for supporting the implementation of an augmented reality application integrated with the VTK library. In order to satisfy the system requirements, we have first studied separately the four modules and defined stand-alone components, and then we have implemented the connections among the components. The system architecture is based on the use of different computers for the visualization/haptic interaction and the simulation system; this choice has been made mainly because the OpenFOAM library works only on Linux operating systems, and also because it is useful to use an independent and more powerful hardware for numerical analysis, that certainly needs more computational resources than the other system components. The Visualization System has been defined as the main component that exchanges data with all the other components and serves as rendering engine all the system components, as shown in Figure 7. The integration of software components is mainly based on data exchange. All data pass through the VTK library that exchanges geometrical models in STL format with the haptic system and with the CFD solver. Moreover, it receives results generated from the CFD analysis (exported in VTK format), processes these data and visualizes them.
5.2 System implementation This section presents the PUODARSI environment, describing in details the implementation issues of its components and their connection. 5.2.1 CFD analysis module The module for CFD analysis is implemented using the OpenFOAM library. The OpenFOAM library can be described as a collection of several different solvers for continuum mechanics problems, including computational fluid dynamic problems. All solvers and functions are open source and written in C++. The type of representation of the geometry of the object that one intends to analyze with the CFD module has been studied considering the possibility of easily exchanging this
14
CFD Solver
Annotation
OpenFOAM
VTK Visualization System
VTK Haptic Interaction
Augmented Reality
ARToolKit
H3D
Fig. 7 Software components of the PUODARSI environment.
geometry with the haptic module used for applying modifications. Since haptic libraries usually allow us to work easily especially with tessellated surfaces, it has been necessary to find a way to import tessellated geometries, create a control volume around these geometries to simulate the fluid volume and then generate a 3D mesh on the volume obtained by the subtraction of the volume of the model from the control volume. For this purpose, the GMSH software library has been used for importing the geometry and generating the mesh. GMSH library works with .GEO files that are written in ASCII code using a very simple syntax; it can import STL files but does not work correctly when defining additional geometries (for example, the control volume). Therefore, a C++ routine for converting .STL files into .GEO file format has been implemented. The code works in such a way that, once the conversion is done, a control volume is created around the model. After completing the definition of the geometry it is possible to generate the mesh (made of tetra elements) using GMSH library in batch mode, simply executing the following command line: gmsh -3 filename.geo where filename.geo is the file obtained from the conversion from .STL. The output is a mesh file: filename.msh. Afterwards, the mesh can be converted in OpenFOAM format using the gmshToFOAM function included in the OpenFOAM package. OpenFOAM requires that a folder related to a specific performed analysis is defined, including ASCII files containing information like initial conditions, initial and final time, viscosity of the flow, etc. Those files are independent from the geometry to analyze, and so they have to be defined only at the beginning of the process. Once all information is defined, the analysis can be launched using the solver routine icoFoam, a transient solver for incompressible, laminar flow of Newtonian fluids. Once the analysis is complete, the results are sent to the Visualization System that renders these data. The results of the analysis are transformed into a format that is compatible with VTK, using the function foamToVTK. This function creates a VTK file containing the
15
definition of a VTK dataset and information about pressure and velocity for each vertex of the grid. The Visualization Toolkit VTK is an open source library for Scientific Visualization written in C++. This library allows developers to easily visualize data obtained from scientific simulations. In our project VTK has been used to visualize geometry in STL format, and also the velocity field around the geometry that is obtained from the CFD analysis. By importing the dataset as a VTK unstructured grid it is possible to choose different modalities for representing a fluid flow field, such as stream tubes, stream ribbons, glyphs and so on, according to the preferences of the analyst that is performing the fluid-dynamics analysis. In the example reported in the following (Figure 8), we have decided to use streamribbons to represent the velocity field. Streamlines are curves that are tangent to the velocity vector field of the flow, while the color represents the scalar field. The class vtkStreamRibbon automatically generates streamribbons from a dataset containing velocity components on the vertices of the elements that compose the dataset.
Fig. 8 Fluid-dynamics analysis results performed on the initial and then on the modified geometry of a windscreen..
Figure 8 shows the results of the CFD analysis performed on an initial geometry of a windscreen and then shows the resulting analysis performed on the same object where the geometry has been modified using the haptic module. The VTK module has been already integrated as a standalone module with the ARToolKit environment that supports Virtual/Augmented Reality applications in a framework named VTK4AR (Bruno et al (2006)). This application allows users to see the real object and the analysis results in a unique augmented environment. So the future release of PUODARSI system will also support Scenario 2 described in section 3.1, allowing users to see the analysis results (velocity fields) superimposed on a physical model of the windscreen through the use of the augmented reality environment. The position and orientation of the physical object will be detected through the pattern-based tracking system supported by the ARToolKit. 5.2.2 Haptic shape modification module The environment allows the designers to modify the object shape using a haptic module. The hardware component of module is based on the SenseGraphics system consisting of an aug-
16
mented reality visualization system, a haptic Phantom device and a 3D Connexion SpaceNavigator (http://3dconnexion.com). The user sees a 3D representation of the virtual object, interacts haptically with it through the Phantom device and uses the 3D mouse for easily positioning and rotating the model of the object in space. The user handles the Phantom stylus: he moves it over the surface for evaluating the curvature; the surface modification modality is activated when the button located on the stylus is pushed. The haptic shape modification module has been implemented using the H3D library. H3D library is an open source haptic library written in C++ that works with geometries in the X3D format; since the model is initially available in STL format, it is necessary to create a converter from STL to X3D and from X3D to STL. In this way, the module reads STL models, applies modification to the geometry according to users haptic interaction, and saves the modified geometries in STL format. The STL model is imported and transformed in X3D format, and then handles as a deformable shape (named DeformableShape). We have implemented a function that handles shape modification and manages plasticity values that are initially defined at a fixed default value, but that can be changed dynamically by the designer during the shape modification task. The haptic system deforms the geometry following a 3D Gaussian curve; this is done through the H3D function GaussianFunction, which uses a fixed width at the beginning. A new deformation tool has been added to the Gaussian deformation that is the ChainMail deformer and the module allows users to change the deformation modality in real-time. The system also provides users the possibility to change the width and the amplitude of the Gaussian function in order to define the area of influence of the deformation. The SenseGraphics system does not include a tracking system allowing detecting the users point of view. Consequently, it may happen that the point of contact between the haptic device and the virtual surface does not correspond precisely. The misalignments problem has been solved by implementing a snap function acting as a sort of attractive magnetic surface that attracts the haptic stylus used by the user towards the surface. This functionality has been implemented through the H3D function MagneticSurface. When the modification phase is finished, a new STL file representing the modified shape is generated; this model can be passed to the CFD module for running the analysis. It is also possible to save the deformed model in an X3D file named verN.x3d that the system automatically converts into an STL file named verN.stl, where N stands for the number of times the file has saved. This feature is particularly useful during the design review phase of a product. In fact, the system allows the user to go back to the previous step of the shape modification and forward to the current one. This is implemented by saving a version of the model in the RAM memory every time the Phantom stylus is in contact with the virtual surface of the object and the button located on the stylus is pushed for activating the modification modality. For practical reasons, this function can be used to store up to eight versions of the model. The model is visualized through the Visualization System. The system is based on the visual rendering library used by the haptic module is VTK instead of H3D. The system includes a VTKrenderer function that has been specifically implemented for rendering 3D VTK objects (such as the output of the CFD module and annotations), X3D shapes and DeformableShapes. The visualization output is shown through the active stereoscopic window of the SenseGraphics system. Figure 9 shows a designer that is modifying the shape of a windscreen model using the haptic module. The user is looking at a stereoscopic view of the model and is interacting haptically with it by using the stylus of the Phantom haptic device. As yet the haptic modification module does not take into account issues concerning the
17
continuity of the curve or the quality of the curvature typically reached in surface modeling during the modification operations.
Fig. 9 Use of haptic module based on the SenseGraphics system for the modification of the model of the windscreen.
5.2.3 Modules integration Once the modules have been defined and implemented separately, it has been created a connection between them. The CFD application, using OpenFOAM library has been implemented on a machine running Linux operating system; the haptic module and the Visualization System, using H3D and VTK libraries, have been implemented on a machine running Windows operating system. The haptic module and the Visualization System, and the CFD solver communicate through file exchange mode that is based on a Server/Client architecture using a TCP/IP connection (class socket). Figure 10 shows how the connection between the two systems has been managed. The common data format used for exchanging the geometric representation of the object among the various modules is STL, which is appropriately converted within each module (Figure 11).
18
Fig. 10 Connection of CFD analysis module, and the haptic module and the Visualization System. 5.3 Use of the PUODARSI system Two users operate on the PUODARSI system: designers use the haptic module and analysts use the CFD module. The two users use the PUODARSI system in a collaborative way (Figure 12). The users can work in the same environment or can be geographically dislocated. The analyst may start the CFD simulation: the system automatically generates a mesh from the .STL geometry of the object (using the GSMH module) and the related volume, and computes the fluid-dynamics analysis. The results are converted into a VTK compatible format that is sent to the Visualization System. At the moment the resulting data are visualized in a 3D environment. The next release of the system will allow the user to see the CFD analysis data displayed on top of the physical prototype of the object in an augmented reality environment. The designer can then use the haptic module for modifying the shape of the object. The modified shape is converted into an STL format and can be used for performing further analysis.
6 Discussion and Conclusions The paper has presented the preliminary results if the PUODARSI project aiming at developing an integrated environment for design reviews and real-time CFD analysis of aesthetic products. Some problems have been encountered during the selection of the components for the development of the environment, mainly from the software point of view, due to the exchange of information among the various components. The paper has described the available
19
H3D
CAD tool
STL model
VTK
OpenFOAM
Fig. 11 Data exchange among the PUODARSI modules.
hardware and software components to use for the system development and the ones selected including motivations. Most of the components have been selected considering issue concerning integration among the various libraries rather than specific performances. The PUODARSI system supports two different setups: one is for the designer that uses the SenseGraphics workbench for performing modifications on the object geometry, and the other one is for the analyst that runs CFD analyses and visualizes the results in a 3D visualization environment. In a future release of the system the users will be able to add notes to the geometry and to the analysis results through the annotation module; in addition the Visualization System will support and augmented reality modality allowing engineers to visualize velocity vector of the flow on top of the physical prototype of the object analyzed. The same object will be rendered in two different windows while the annotation will be related to the model and shared by users. The annotation module is at the moment under development, and is based on the VTK library for managing notes; it will be necessary to create two or more visualization windows for the same environment, or create a communication protocol between the two applications that share the same models. The PUODARSI system demonstrates the feasibility of the integration of various tools and data models used in the context of virtual design. Specifically, it has been demonstrated that the STL file format is a good compromise for exchanging tessellated geometrical data among different software application.
20
Fig. 12 PUODARSI collaborative environment. The system takes as input STL files, and the analysis and post-processing steps are completely independent from the modeling one. This will allow us in the future to develop the framework as an add-on for other modeling tools, such as the ones we are developing within the context of other two research projects, TnD and SATIN projects, but also for the FreeForm system. Acknowledgements Part of the research described in the paper has been developed in the context of the PUODARSI (Product User Oriented Development based on Augmented Reality and interactive SImulation) project (http://www.kaemart.it/puodarsi), partially funded by the Italian Ministry of University and Research.
References Ambrogio M, Bruno F, Caruso F, Muzzupappa M, Bordegoni M (2008) Interactive cfd simulation in virtual reality. In: Proocendings of XIX ADM Ingegraf Anderson JD (1995) Computational Fluid Dynamics. McGraw-Hill van der Auweraer H (2008) Frontloading design engineering through virtual prototyping and virtual reality: industrial applications. In: Proocedings of the TMCE Edited by I. Horvath and Z. Rusak Avila RS, Sobierajski LM (1996) A haptic interaction method for volume visualization. In: VIS ’96: Proceedings of the 7th conference on Visualization ’96, IEEE Computer Society Press, Los Alamitos, CA, USA, pp 197–ff. Bordegoni M, Cugini U (2007) Haptic interface for real-time evaluation and modification of shape design. In: Proceedings of the ASME 2007 Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC/CIE 2007, Las Vegas, Nevada, USA
21 Bordegoni M, Cugini U, Covarrubias M (2008) Design of a visualization system integrated with haptic interfaces. In: Proocedings of the TMCE Edited by I. Horvath and Z. Rusak, Izmir, Turkey Bruno F, Caruso F, Ferrise F, Muzzupappa M (2006) Vtk4ar: An object oriented framework for scientific visualization of cae data in augmented reality. In: Gallo G, Battiato S, Stanco F (eds) Eurographics Italian Chapter Conference, Eurographics, pp 76–81 Celniker G, Gossard D (1991) Deformable curve and surface finite-elements for free-form shape design. SIGGRAPH Comput Graph 25(4):257–266, DOI http://doi.acm.org/10.1145/127719.122746 Celniker G, Welch W (1992) Linear constraints for deformable non-uniform b-spline surfaces. In: SI3D ’92: Proceedings of the 1992 symposium on Interactive 3D graphics, ACM, New York, NY, USA, pp 165– 170, DOI http://doi.acm.org/10.1145/147156.147191 Chen JX, da Vitoria Lobo N, Hughes CE, Moshell JM (1997) Real-time fluid simulation in a dynamic virtual environment. IEEE Computer Graphics and Applications 17(3):52–61, DOI http://doi. ieeecomputersociety.org/10.1109/38.586018 Dachille F, Qin H, Kaufman A (2001) A novel haptics-based interface and sculpting system for physics-based geometric design. Computer Aided Design 33:403–420 Dai F (1998) Virtual Reality for Industrial Applications. Springer Dewaele G, Cani MP (2004) Virtual clay for direct hand manipulation. In: Eurographics (short papers), Ferziger JH, Peric M (1999) Computational Methods for Fluid Dynamics. Springer Haller M, Billinghurst M, Thomas BH (2006) Emerging Technologies of Augmented Reality. IGI Publishing, Hershey, PA, USA Iwata H, Noma H (1993) Volume haptization. In: Proocedings of IEEE 1993 Symposium on Research Frontiers in Virtual Reality, pp 16–23 Jagnow RC, Dorsey J (2002) Virtual sculpting with haptic displacement maps. In: Proceedings of Graphics Interface, pp 125–132 Nealen A, Mueller M, Keiser R, Boxerman E, Carlson M (2006) Physically based deformable models in computer graphics. Computer Graphics Forum 25(4):809–836, DOI 10.1111/j.1467-8659.2006.01000.x Rix J, Haas S, Teixeira J (eds) (1995) Virtual Prototyping: Virtual Environments and the Product Design Process. Springer Terzopoulos D, Fleischer K (1988) Modeling inelastic deformation: viscoelasticity, plasticity, fracture. Computer Graphics pp 269–278 Ulrich KT, Eppinger SD (2003) Product Design and Development, third edition edn. McGraw-Hill
