What have the Earths' Mantle & an Egyptian Mummy ...

6 downloads 0 Views 741KB Size Report
This paper will start with an overview of the Manchester Visualization Centre .... The Manchester Museum have a large collection of Egyptian mummies and as ...
Manchester Visualization Centre

What have the Earths' Mantle & an Egyptian Mummy in Common? W T Hewitt, J Leng, J Brooke, & P Lever CSAR & Manchester Visualization Centre University of Manchester Manchester M13 9PL United Kingdom [email protected]

Abstract This paper will start with an overview of the Manchester Visualization Centre (MVC), what we do, why we do it and who we do it to. The main elements of the paper will be to present four examples of our work: Visualization of Egyptian Mummy 1766, one of the Egyptian mummies belonging to the University Museum. We will explain what we did with the CT scan of this mummy, the special techniques that had to be employed to produce a variety of views of the mummy. The Terra project at the University of Liverpool run computational simulations of the Earths' mantle on the RAY T3E-200E at Manchester. The data from the simulations is distributed over a unique and complex grid structure. We will present the results of our work for this group in providing ways in which they can visualize their results. The VIPAR provides a portable framework for the development of parallel visualization modules, including: an automatic parallel module generator, and a dataflow network editor for building parallel visualization. We present details of the Multi-Pipe Rendering project, a joint research effort by AVS, SGI, KGT, and the MVC, in conjunction with the International AVS Centre. We have developed a new view rendering environment for any AVS/Express visualization application, taking full advantage of parallel and large scale scene rendering facilities of SGI Multi-Pipe systems.

1 Manchester Visualization Centre Manchester Computing (MC) provides local information and computing services to the University of Manchester and is also a leading national and international centre for advanced information retrieval services, high performance computing and data visualization. It hosts three major national services: •

It is one of the four national Network Operation Centres providing the network infarstrucutre (JANET and SuperJANET) to all 192 higher education insititutions in the UK.



MIMAS provides on-line access to many large and complex datasets such as the Census of Population statistics and recurrent government survey data. Spatial information is also available using the Bartholomew Digital Map Data, Census of Population Digital Boundary Data and images from the Landsat and SPOT satellites. For scientific researchers, MIMAS also supports the Cambridge Structural Database System including the Brookhaven Protein Data Bank, the Mössbauer Effect Reference Database and the Beilstein CrossFireplus Reactions Service. For research of a more general nature the Department maintains COPAC, a nationally accessible on-line system providing unified access to the consolidated catalogues of some of the largest university research libraries in the UK and Ireland.



CSAR: The flagship supercomputer service for UK academia and commerce. The main facility is a 816 processor Cray T3E-1200E. It is supplemented by a 128 processor Origin2000, an 8 processor Fujitsu VPP300, and has a pre-release 8 processor SGI/Intel IA-64 based system. The near future will see the addtion of a 16 processor Compaq alpha system, a 128 processor Origin3000 and a very large IA-64 based system.

MC can trace its history back to the invention of the world's first stored-program computer here in the University in June 1948. Within a year or two of that momentous development, computing services were being offered to fledgling users who were eager to harness the power of the new machines. Manchester Visualization Centre (MVC) is the group within Manchester Computing which provides graphics, visualization, image processing, and multimedia services to both local and national users. In addition to the installation, and maintenance of software across campus, it provides a public 24 hour Visualisation Immersive Projection (VIP) Laboratory (Figure 1) that permits high quality virtual reality displays, with full interactive control. The VIP Lab consists of a large semicylindrial screen, three high quality projector systems, active stereoscopy support, and surround sound. It is front-ended by a 40 processor SGI Onyx2 with six graphics piples, making it the most powerful visual Figure 1 The VIP Laboratory in MVC supercomputer in the UK, and probably in Europe too. MVC offers a high level of consultancy, education and training in these areas, and provides specialist support to UK academia in visualization, cluster computing and multimedia. It offers extensive facilities for scanning pictures, and 35mm slides, digitising large diagrams or maps, optical mark reading, real-time digitising of video tapes. Similar extensive facilities are available for producing 35mm slides, high quality colour output from A4 to A0, and creating and editing both analogue and digital videos. It has a very active research and development group funded by grants from EPSRC, JISC, the Commission of the European Union, and a number of national and international companies, covering visualization, virtual reality, computer animation, multimedia, distributed computing, high performance computing and collaborative working.

1.1 The International AVS Centre MVC hosts the International AVS Centre (IAC) which as a catalyst for expanding the user base of AVS/Express and AVS5, the primary products of advanced Visual systems Inc. The main objective is to increase the functionality of these products by fostering discipline-specific module development and new uses for the software. The core activity of the IAC is to distribute "added value" technology for AVS beyond what is available in the core products, in an open source web site. The main activities carried out at the IAC are to: •

Maintain and enhance the AVS/Express and AVS5 Module Repositories.



Manage the IAC Web Site



Carry out Selected Research Projects on behalf of AVS Inc.



Support User Group Activities



Participate in AVS Product and Solution Testing and Quality Assurance.

2

The AVS Consortium is made up of a number of AVS Sponsors and Affiliates who are funding and providing direction for the International AVS Centre. They are Advanced Visual Systems, KGT Inc., and SGI Inc.

2 Egyptian Mummy 1766 The Manchester Museum have a large collection of Egyptian mummies and as part of their research they have conducted many non-invasive tests on these objects. In fact they pioneered noninvasive testing for the study of Egyptian mummies. This work included the mummy known as 1766 being given a CT scan. We took this scan data, did a special segmentation of the scan and then used the result to produce volume visualizations and MPEG computer animations. These were used to aid the Manchester Museum's research into Egyptology and were on display at the museum, for three months, as part of an exhibition on the future of computing. •

Date: Roman Period (1st/2nd century A.D.)



Provence: Fayoum



Length: 166cm



Date of acquisition: 1895-6

A body was enclosed in bandages coated in resin, painted with three horizontal registers of funerary scenes. There is a cartonnage cover of three separate pieces for the head, breast and feet are gilded and the sole of the foot cover is decorated with painted figures. Imitations of serpent-bracelets, rings, necklaces inlaid with glass to represent semi-precious stones, and a bunch of flowers held in the womans's hand decorate the cartonnage breast cover.

With permission from [1].

Figure 2 Mummy 1766 The picture on the right shows museum researchers performing an endoscopy on mummy 1766. These noninvasive studies have enabled them to take small tissue samples from the mummy with as little harm as possible. The tissue samples were used for biochemical investigations. The CT scan produced a volume of images from Xray radiation. This type of scan images hard materials well so the skull and mask like cartonnage cover are imaged particually well. The cartonnage cover has a complex shape which obscures the front of the skull and face making it difficult to use standard volume visualization techniques. A computer program was written to find the contour at the back of the mask cover. This contour was used to separate them and produce two data sets one of the head and one of the cover. The head and cover data were mathematically altered and recombined to produce a third set of data in which the skull and the mask can be viewed without the cover obscuring the head. These three data sets were rendered to give 3D images using AVS/Express, that was also used to produce computer generated animation of the data sets. The volume visualization technique

3

Figure 3 The Mummy undergoing an endoscopy

used is volume rendering which allows some data to be made transparent. Using this technique the cover could be removed from the head, then the bandages from the face, then the soft tissues from the skull.

Figure 4 The Cartonnage Cover

Figure 5 The Skull

The images were produced using volume visualization to reveal details that can not be seen by researchers without unwrapping and so damaging the dummy. Both images (Figure 4 and Figure 5) are taken from the same angle and show how the head is positioned relative to each other. For example MPEG movies are provided on the conference CD. This work was done in collaboration with the Manchester Museum in particular Professor R A David who is in charge of the Egyptian project. The Department of Diagnostic Radiology produced the original CT scan of the mummy.

3 Seismic Tomography & Convection Modeling of The Earth's Mantle As part of our national support of supercomputing we have been collaborating with the UK base of the Terra Consortium at the University of Liverpool to develop some better visualization techniques for their data. They run computational simulations of the Earth mantle's (the layer between the crust and core), which run on 512 processors of the CSAR Cray T3E-1200E for up to 12 hours. The study of mantle currents are important for understanding both continental drift as well as volcanic and seismic activity. The mantle is a relatively thick layer which extends nearly halfway to the centre of the Earth/sphere. The spherical shell being modeled is much thicker than the shells used for ocean or atmospheric modeling, it is 2900 km compared to the 100 km of the atmosphere (the radius of the Earth is 6370 km). The mantle is modelled as a thick viscous liquid, usually by finite element analysis, the ocean and atmosphere on the other hand are usually modelled by finite difference methods. The mantle is crucially different from the ocean and atmosphere in that it is a thick rather than thin shell. Thus the effects of curvature are fundamental to the visualization. The shape and structure of this spherical shell makes it closer to that of astronomy simulations of stars or planets rather than ocean or atmosphere modelling. Seismograms from just over 3000 seismic stations resulting from around 10,000 earthquakes have been collected. The travel times of these seismic observations are inverted by velocity tomography so the group can compare the lateral variations in seismic velocity produced with the lateral temperature perturbations of the mantle modeling. In this way they can verify the simulation, by comparing theory with observation. The problems can be boiled down to two main areas, modeling or inverting this spherical geometry and visualizing it in 3D. We are also faced with the challenge of comparing observational data with the result of simulations. The two main problems are modelling or inverting this sphereical geometry and visualizing it in 3D.

4

3.1 Modeling This Spherical Shell The solution to the inversion of seismic data dates back to the 1930's. Since the variations are dominantly radial the model is effectively reduced to 1D, depth is the only factor. Given this history it has been quite common to analyse the data by 2D projections (Figure 6).

Figure 6 2D projections for two shells of given depth.

3.2 3D Visualization We have concentrated on visualizing the tomography data initially. The group converts the simulation data to the tomography format, and hence can visualize both types of data from this format. We will look to visualize the unconverted simulation data in future work. The data mesh for the tomography is made of cells. The cells are not defined by array data as they are for atmospheric modeling but by a data structure. The data cells are defined in a spherical coordinate system so implicitly the cells have curved surfaces. Visualization systems use straight lines for all surfaces and these are defined in a Cartesian coordinate system. To transform the simulation cells into renderable objects they must be resampled in Cartesian space to fit the original curved shape enough to tessellate. The complexity of the data and of visualization systems mean that this reader is best produced by a graphics expert rather than a geophysics researcher. The number of cells required to visualize the data mean the results will be slow to manipulate unless specialist graphics hardware is used. Special Challenges of Visualizing Thick Spherical Shells There are several reasons why this type of data is difficult to visualize. •

Graphics renderers only handle straight lines, spheres have few straight lines, are defined by spherical coordinates and to get a well formed tessellated cell set extra data cells must be introduced.



The number of cells needed to visualize this data means special data reduction/handling techniques are required.



Placement of reference information is important since this allows the researcher to relate features inside the sphere to surfical features. The user needs "volcanic" hot spots (e.g. Hawaii, Iceland), tectonic plate boundaries and coastlines to be present in each image.



The shell is thick and placed close to the centre of the sphere, the angular component of the data is more important than it would be for atmospheric data.



Perceptual problems with understanding a spherical shell: o

A sphere looks the same from every angle so it is difficult to orientate.

o

Perspective has little effect.

o

Colour, transparency and shading sometimes work to make the pseudo-colour difficult to interpret. There is stronger darker band of colour around the outer circumference of the sphere.

5



Problems with depth: o

A 2D projection of one layer gives no depth information, is distorted and does not link around on itself.

o

The original visualization only related to depth slices. When seen in 3D many depths are seen at the same time, the depth values need a new normalization function to make 3D images valid.

o

A cut plane through the sphere shows how the data varies with depth but a plane is 2D and loses 3D information i.e. data with an angular component is lost.

o

Internal data with an angular component may be self-obscuring, difficult to position because of problems with perspective and there is ambiguity in its relation to the reference data.

Visualization of The Earth's Mantle As A Spherical Shell A C data structure defines the cells used by the simulation. The organization of data cells is shown in Figure 7, which shows the cells are placed in a ring around a latitude band. The bands at this resolution are 5o and the number of cells in each band increase from the pole to the equator; the volume of each cell is equal. There are higher resolution tomography data sets with 1o latitude bands, which has 25 as many data cells as the 5o data. The convection simulations give finer resolution, approaching the equivalent of 0.4o bands. The coordinates for each cell are defined in a spherical coordinate system (latitude, longitude and radius) and so have implicitly curved edges. There are three cells at the pole and these are in the shape of a curved triangular prisms while the rest of the cells are curved hexahedrons (rectangular prisms).

Figure 7 A schematic diagram to show how one shell of tomography cells are arranged. AVS/Express was used for this application. A number of modules were developed in C and C++ with the Developers Edition of the software. User interfaces were designed for these modules and application networks built so they could be used at the Terra group's site in Liverpool using the Visualization Edition of AVS/Express. Cells used by a visualization system are defined as being of a particular type, e.g., prism or hexahedron. A coordinate is given for each apex of the cell, then the connections between each coordinate are specified and finally a data value is given for each cell (cell data) or for each apex (node data). Initially the data was mapped to just the cells explicitly defined by the C structure (Figure 7). The coordinates were calculated for the apex of each cell in turn even though neighboring cells would have common apex locations. The coordinate of each apex would appear at least twice in the coordinate array, once for each cells with

6

a shared apex. Not surprising the resulting cells did not tessellate, gaps appeared between most latitude bands. More interestingly there was a banding effect across the surface caused by light shading. The shading had two causes. Firstly between latitude bands not only were there gaps but there were cell overlaps which stopped the surface being smooth. Secondly there was a shading effect between cells of the same band, this was caused by two adjacent cells with shared apices referencing different coordinates on the coordinates array although they should be the same point in 3D space. The reason for this seems to be the inaccurate nature of floats and coordinates are stored as floats. The result was very unsatisfactory; the solution was to increase the number of cells but to keep the number of coordinates as low as possible.

Figure 8 All coordinates appear once in the coordinate array so there is no across latitude band shading but the right bottom hemisphere is resampled while the left top is not. Resampling The Cell Sets Removing Duplicate 3D Locations From the Coordinates Array The resampling method removed all duplicate 3D Locations from the coordinates array. Coordinates were calculated for the cells of a whole latitude band so neighbouring cells with common apexes would never reference difference points on the coordinate array for the same point in 3D space. This removed most of the shared apexes from the coordinate array but some incidental ones remained. The first cell on each band begins at the same longitude and will share an apex coordinates with the bands above and below. The bands that lie on each side of the equator share all their adjoining apexes. It is a default throughout that when there are two possible coordinates that the one from the upper band will be used. Neighbouring Bands With A Harmonic Number of Cells Occasionally adjacent bands would both have a number of component cells that would be divisible by the same small integer, e.g., in the low resolution data the second band has 3 cells and the third band has 12 cells, both are divisible by 3. This effect causes an incidentally shared apex with its known shading problems. In these cases the coordinate from the upper band is selected.

Figure 9 Shows the first four latitude bands, viewed from the South Pole, the dots show where there are duplicate 3D locations in the coordinates array.

7

Resampling The Poles In all resolutions of the data the latitude band at the poles consists of 3 triangular prisms and the next band consists of 9 hexahedron. For each pole the cells were resampled to give 9 triangular prisms, 3 for each of the original prisms. The coordinates were used from the neighbouring band whether upper or lower. Resampling The Hexahedrons In All Other Latitude Bands Each cell from the other latitude bands should consist of one curved hexahedron. This does not tessellate because it is impossible to give it curved edges they must be straight. Each shell has the same number of cells, they tile up on each other so extra cells only need to be added within each shell. The solution must be scalable so it can be extended to higher resolution data. The resampling solution was to map one hexahedron for all of the original simulated data cells but not to map it to the full extents of the cell and to add triangular prisms to fill in the gaps. There were gaps being caused by either cells in the band immediately above or below not having apices exactly in line with the cell being looked at. The number of cells increases in each band from the South Pole to the equator and then decreases up to the North Pole. A solution was found that was symmetrical about the equator so now we can just consider the geometry below the equator. Each data cell consists of one hexahedron and a number of triangular prisms. The hexahedron is cut so it changes from its initial rectangular prism shape to that of a trapezoid prism and triangular prisms are added to give the cell a curvature that tessellates with the cells in the latitude bands above and below. All hexahedrons will keep four of their original apices but some may keep six. Imagine you are looking at the surface of a hexahedron so it looks like a rectangle. Now imagine the cells in the band above, they are smaller and there may be one or two which lie across the top of our cell. Our hexahedron must curve at the apex of each of the cells above if it is to tessellate. To do this we cut our cell from the apex of the cell above to the bottom left apex of our cell giving us one triangular prism and a trapezoid prism. If there are two apexes above we cut twice and get two triangular prisms and one trapezoid prism. The cells in the band below are larger and also do not align with our cell. We must make our trapezoid prism curve to tessellate with the cells below. This time we cut the cell from the last apex above to the first apex below, we keep a trapezoid prism but adding another triangular prism.

Figure 10 A diagram showing the resampling scheme. It is the surface of a group of cells in 3 latitude bands projected in 2D. The cells appear rectangular; they are below the equator so each band has progressively more cells. We are looking at the cell with the thickened border; the thin line shows how it is cut into two triangular prisms and a trapezoid prism to make it curve with the bands above and below it. Reading Cell Sets For Better Display And Data Management The cell sets produced by the method given above gave a smooth evenly shaded sphere. Unfortunately the whole data had been resampled so it contained about 3 times the number of cells. It stopped being interactive and problems with visualization methods become clearer.

8

One of the most common techniques used in visualization is the bounding box, its shows the extents of the data. For a spherical shell the bounds are not easy to detect, the extents of the data are clear when it is seen as the sphere's surface but one of the extents are completely embedded within the other, i.e. the inner most shells is entirely within the outermost one. All the shells are needed for visualization but removing cells as early in the pipeline as possible increases interactivity e.g. when the sphere is cut the data values across the plane must be calculated but after cutting any data cells that are completely internal can be removed. Separating the outermost shell and innermost shell from the rest is helpful as they can then be used to mark the data's bounds. To achieve both of these objectives the innermost shell, the outermost shell and the rest of the shells were placed in separate groups of cell sets. Each of these three groups could then be separated or kept together and passed down different visualization pipelines to produce the most appropriate images.

Figure 11 These figures use the three groups of cell sets one for the inner most shell, the outermost one and one for all the other shells. The picture on the left shows how these can be used to create a visualization while the picture on the right shows that the internal data was removed early in the visualization pipeline to make it more interactive

Figure 12 Two further visualizations. The picture on the left is of an isosurface, the outer most shell is semitransparent while the inner most shell is opaque and shows the bounds of the data. The picture on the right shows how the inner most shell is used to display reference data that would otherwise be cut away. Reference Data, Pseudo-Colour and Data Normalization The tomographic data of the Earth's mantle is strongly related to other information, volcanic "hot spots", tectonic plate boundaries and coastlines. It is vital that this reference data is always visible and unambiguously displayed. For tectonic plate boundaries and coastlines a simple polyline could be used to join the appropriate coordinates into a line. The "volcanic" hot spots are traditionally viewed as a ring centering on the location of the hot spot on the Earth's surface. To achieve this an equation was used to calculate 12 points around the hot spot. All of the reference data is stored in spherical coordinates and then converted to Cartesian space. The equations used to ring the hot spot works in the spherical domain, there are two equations to do this. Unfortunately each equation has

9

discontinuities at certain spherical coordinates, the problem areas, either around the pole or at the equator. A patch is needed between equations to make the rings approximately circular at all locations on the Earth's surface. The colour used for all the data, the simulated data and the reference data and the background adheres to the standard colour representations of the field of mantle studies. The tomography data is given as percentage velocity perturbation, which means the data is tightly clustered around the zero value. Many cells have a zero; i.e. have a zero perturbation or are at the average value. The scientists interest is in the data close to zero either just faster or slower than average so a saturated colour scale is used at the extents of the data values that rapidly changes around zero. A new normalization function was added to give the root mean square of data values for each layer. The colour map has been designed to highlight data values close to but not at zero. The lower layers have data values more closely clustered around zero than the outer layers. The normalization function makes the variation in the inner most layers as prominent as the outer most one while still using the same colour map for all layers.

Figure 13 The image on the left uses un-normalized data while the one on the right uses normalized data.

4 Visualization in Parallel (VIPAR) 4.1 Introduction The main aim of the VIPAR project was to provide a portable software environment to support users who are developing parallel modules for use within visualizations systems. The VIPAR framework has been designed to be portable across both different modular visualization environments (MVE) and parallel platforms. The project has achieved a number of major goals: portability across both MVEs and DM-MIMD architectures, tools to aid the generation and management of parallel modules, and performance gains shown in the results obtained so far. 4.2 System Architecture The system architecture is shown below (Figure 14) and as well as the modules “underneath AVS” has developed two specific tools described later: data composition tool to generate templates for parallel modules (DDTool), and a management tool for parallel visualization applications (PiMMS). DD Tool: A tool to automatically generate a parallel module. DDT allows the user to describe the data composition strategy and other criteria when constructing a parallel module and produces a template module structure. PiMMS: The Parallel Module Management System provides facilities to control the placement and characteristics of parallel modules when they are being used in a visualization system. VPRvsi: The Visualization System library provides an interface to the other support libraries by handling the data in the form the native MVE uses and mapping it onto the structures used by the independent and general support

10

libraries (VPRidd and VPRdd). A specific version of the VPRvsi libraries must be written for each visualization system that VIPAR supports.

Figure 14 System Archtitecture VPRprf: The Performance library contains routines to provide performance feedback from parallel modules to the VIPAR environment. These range from simple clock timings to defining states and indicating changes from one state to another. VPRidd: routines to calculate data distribution patterns and other useful utilities. These functions are characterised as being independent of the underlying parallel architecture. VPRdd: routines to distribute, manage and composite data. They are used to implement the mechanisms made available in the VPRvsi. This library is dependant upon the underlying parallel support libraries being used to distribute data and form communications. 4.3 Prototype implementation The prototype of DDTool was developed using Tcl/TK and versions of the support libraries have been developed for AVS/Express and Iris Explorer. Both versions sit on top of MPI and have been tested with the mpich and LAM implementations of MPI. An initial prototype of PiMMS has been developed for AVS/Express. 4.4 DDTool The DDTool prototype (Figure 15) produces a parallel module template, which supports intra module parallelism. Using the tool it is possible to create and use multiple parallel modules in a single AVS/Express network. The Data Decomposition Tool generates parallel modules template based on the user inputs: •

Input data distribution schemes;



Neighbourhood and boundary data processing;



Parameters to the module;



Output data composition;



Parallel platform specific information.

4.5 Parallel Module Components The Structure of a parallel module, generated by DDTool, has four main components (Figure 16): •

Distributor: handles the field input to a parallel module;



Compositor: handles the output from a parallel module;



Harness: control process for the parallel module:



Worker: user inserts application code to processes data portions in parallel.

11

Figure 15 DDTool Prototype

Figure 16 Structure of a Parallel Module 4.6 PiMMS The prototype of PiMMS has been designed as a module in AVS/Express. The tool provides the following functionality: •



alter the parameters for a particular parallel module which include: -

number of processes:

-

nodes the processes execute on;

-

distribution, neighbourhood, boundary schemes.

plots of timings from each of the parallel modules control components.

This tool allows the user to experiment with different numbers of processes and distribution patterns without the need to regenerate any application code.

12

4.7 Case Study: Parallel Isosurface A series of tests have been performed using a parallel isosurface module. The module structure was described using DDTool and then generated for both the AVS/Express and Iris Explorer visualization systems. The same extract of application code is used in both versions of the module illustrating the extent of the portability achieved by VIPAR. The module accepts a 3D uniform dataset and a parameter which sets the value of the surface to extract. Each worker is sent a subblock of the complete 3D dataset and the value of the surface to extract. Individual workers extract a surface portion at the level chosen and represent this as a series of triangles. These surface portions are eventually sent back to the harness and composed together to form the complete surface.

The parallel module (PM_ISO) is shown in Figure 17 has been opened to show the internal components. The trirender module is used to convert the list of triangles into a format that can be rendered by the display module (Uviewer3D).

Figure 17AVS/Express Parallel Isosurface Module

4.8 Timings Tests have been performed using a number of different datasets. The tests were performed on a cluster of eight Silicon Graphics Indy’s (R4400, 64 Mbytes) inter-connected via both ATM (155 Mbits/s) and ethernet (10 Mbits/s). The software used to implement the prototype was AVS/Express 3.0 and the MPI implementation from Argonne National lab and Mississippi State University, MPICH 1.0.13.

Figure 18 Some Sample Isosurfaces

13

The tests were conducted using three different sized datasets, and for each dataset different timings were taken when generating isosurfaces (Figure 18): •

Processing time when all input data has to be distributed to each worker (labelled “data+ value” in the graphs);



Processing time when just isosurface value is redistributed and not the input dataset (labelled “value” in the graphs);

All the tests were duplicated using both the ethernet and ATM interconnections. Finally a serial version of the above parallel code (labelled VPR Serial in the graphs) was also tested by embedding it within an AVS/Express module. The tests were carried out using up 7 of the Indy’s in the cluster; the eighth always hosts the MVE and is where the harness process executes. 4.9 Analysis of results It can be seen in all three cases, using ATM, the parallel module performs better than the serial code as we reach three worker processes. This is the initial goal of the VIPAR framework. Further optimisations could be made to both the parallel and serial code to improve the performance. It is important to note that when comparing the ethernet and ATM results that while the peak bandwidth of the ATM connection are 155Mbits/s, the Indy’s I/O rate restricts this to an effective bandwidth of around 34 Mbits/s. The timings taken (an example is shown in Figure 19) when just distributing the isosurface value is a valuable measurement as it is usual practice to load a dataset and then iteratively adjust the isosurface value. Finally the anomaly when we reach three worker processes is due to the brute- force block distribution of the small head dataset. The middle worker process ends up with the bulk of the work when distributed across three workers .

Figure 19 Example Timings 4.10 Achievements The results so far have illustrated the portability of the VIPAR system across both MVEs and DM-MIMD architectures. This is emphasised by the fact that the application code for the parallel isosurface executes across different MVEs without any changes. DDTool handles the generation of the visualization system specific code.

14

The PiMMS tool also allows the application to experiment with different numbers of processes and distribution patterns without the need of the application programmer to generate code. 4.11 Future work Modifications will made to the VIPAR framework to handle inter module parallelism. To analyse the effect of the changes to the VIPAR framework a parallel module implementing triangle decimation will be developed. This will be coupled with the existing parallel isosurface to measure improvements when the composition/redistribution phases are removed. The PiMMS tool will be enhanced to provide useful performance feedback to aid the users decisions on altering and placing the parallel modules. This will involve interfacing to existing performance tools rather than reengineering these within PiMMS

5 Multi-pipe Rendering Project 5.1 Introduction The goal of any visualization tool is to provide insight into data and this may be enhanced by placing the investigators in a large-screen and/or virtual environment. AVS/Express already provides an extensive set of visualization tools but currently lacks any standard Virtual Environment (VE) support. Existing solutions have extended the functionality via the usual method of adding user modules and with projects such as VIVRE have shown this to be a valid approach. However such solutions have the drawback of requiring changes to be made to the application, adding modules to an existing application network. This impacts on application portability, a strength of AVS/Express with its multiple platform support. Having the advantage of access to the AVS/ Express source code afforded us the opportunity to integrate support more tightly, to minimize the impact on the interface, retain portability across application plat-forms, introduce portability between VE platforms and make support as transparent as possible. For VE support a number of solutions are available but the MPU library from SGI provides a wide-range of output configurations including support for desktop systems, Immersadesks, CAVES and RealityCenters. MPU not only makes it possible to use large projection output but also allows multiple rendering pipes to be combined for any display thereby improving rendering performance. 5.2 Approach Support for MPU rendering has been added directly as a new renderer layer inside AVS/Express. Existing support for a number of platforms means that AVS/Ex-press is already well structured. Figure 20 shows how the application and Object Manager works through an Independent Renderer Layer which transparently pass-es instructions to the selected renderer. Using MPU means that a number of renderer processes are started which can take advantage of both multi-processor and multi-pipe systems. Each MPU process now provides distributed OpenGL rendering capabili-ties. To maintain consistency with other AVS/Express renderers, the existing OpenGL layer was extended to take advantage of MPU. As the independent layer traverses the scene graph to render a frame, calls are made to the MPU layer. These calls are tokenized and stored in shared memory (which is used to pass data to MPU). The tokens are later processed by the MPU ren-derers and the original OpenGL calls recreated. Tokenizing all the OpenGL calls is not efficient, as for example a complex isosurface will include many nodes, each of which will typically include a call to glVertex3f(), glColor3f(), glNormal3f() and so on. Storing all these tokens will impact heavily on time and memory resources. AVS/Express provides its ren-dering capabilities through the support of common high-level primitives (which closely map to those of OpenGL) and in these instances all the data for a prim-itive (vertex, colour and normal arrays) can be pack-aged together in one pass with only one token to identify it. Later, at rendering time, a high-level ren-dering function will process that package and only then will all the individual OpenGL calls be made. Figure 21 shows a sample of a sequence of tokens representing a frame, including tokenized OpenGL calls and high-level primitives.

15

AVS/Express also uses a cache system and this functionality was included in the MPU layer to reduce the amount of data transfer to shared memory and improve performance. A two level cache system was employed: firstly keeping a copy of an object in shared memory as more than one MPU renderer process may require ac-cess to it at different times. Secondly, each renderer process makes a GL List that may be repeatedly called in preference to traversing the same OpenGL tokens or high-level primitive packages. The MPU library blocks while a frame is being ren-dered which stops the application from performing visualization tasks. This problem is further exacerbated when considering realtime updates in an immersive environment, for which AVS has not been designed. An intermediate Frame Management Process (FMP) was introduced to alleviate this problem by separating the handling of the rendering from the application process. When a frame is ready (fully tokenized) the application notifies the FMP which in turn notifies the MPU processes and is, itself, blocked. The application continues with its tasks and produces the next frame.

Figure 20 AVS/Express layers, showing independence from the new MPU renderer.

Figure 21 Sample sequence of tokens representing single OpenGL calls (data not shown) or high-level primitives. Note that the application process is eventually blocked, so that there is no build-up of frames should the production of frames be faster than the consumption rate of the renderers. Figure 22 shows how the various processes are connected and where notifications and requests for data are made. It also shows that event information is passed back to the application process via a socket so that the AVS/Express event handling system can be invoked. Additionally the head-tracking (using trackd software) process passes on information to the FMP. For head-tracking support use of the FMP means that the camera can now be decoupled from the application, the FMP piloting the centre of the view with camera offsets without interruption of the application. If a new frame is ready then the FMP will immediately use it, otherwise it will re-render the previous frame accord-ing to the new camera position. Figure 22 shows that the head-tracking data feeds directly into the FMP which will trigger the renderers to redraw with the updated camera, without interrupting the main AVS/Express application process

16

Figure 22 Connectivity of processes in the MPE system, use of shared memory and use of notify 5.3 Conclusions The AVS/Express Multi-Pipe Edition is now available as a product and has been tested extensively at a number of key sites. Figure 23 shows AVS/Express MPE in use at Tokyo Institute of Technology, a KGT customer site on a Powerwall display. Currently support is included for rendering the majority of geometric and volumetric primitives that AVS/Express provides, within most configurations supported by MPU. The next phase of work includes the addition of further support for head-tracking, wand interaction and VE user interface tools. Problems have arisen during the implementation, in particular picking of objects has proved difficult to solve, as the existing picking mechanism in AVS/Ex-press is tightly coupled with the standard renderers and assumes a single window is in use rather than multiple windows/channels as found in some MPU configura-tions. In most applications the MPU renderer layer remains transparent and very little effort has been needed to get existing applications working with MPE. The future addition of interactors suitable for use in a VE will re-quire changes at the application level but such interac-tors will still work when using the standard renderers and, as such, the transparency of the MPE will be re-tained. The approach taken here can be seen to be useful as a model for similar projects requiring the porting of an existing application to multi-pipe rendering hardware. In some instances it is neither easy or possible to completely re-write the entire application so that it can manage multiple processes/threads and shared memo-ry. Additionally, if frame decomposition mode is used with multi-pipe hardware, the application must also provide support for the management of multiple frames corresponding to a sequence of time steps. The approach presented here provides a mechanism for re-cording the scene data from within the application and then manipulating that data independently in external processes.

6 Acknowledgements We wish to record our thanks to many people who have contributed to this paper: •

Collaborators who have supplied the problems, The Manchester Musuem, Huw Davies at the Unviersity of Liverpool



Staff at MVC and CSAR who have worked on these problems including George Leaver, Andrew Dodd, Mary McDerby, and Yien Kwok.



The UK Research Councils who funded the CSAR service, and particularly the EPSRC who funded the VIPAR research project (grant GR/K40390)



Now previous employees: Andrew Grant and Steve Larkin



Our industrial partners who have supplied both advice and resources: SGI, KGT, AVS Inc, and CSC.

17

Figure 23 Multi-pipe Powerwall display using MPE. (Computer Center, Tokyo Insitute of Technology, Tokyo, Japan)

7 Notes About the Authors W T Hewitt is Director of Manchester Visualization Centre, Director of the International AVS Centre and also CSAR Users Services Managers. J Leng was in Manchester Visualization Centre but is now visualization support officer for CSAR. P Lever is part of MVC and the International AVS Centre J Brooke is head of special projects for CSAR. The staff of CSAR and MVC share an open plan office area.

8 References [1] David, A R Editor, The Manchester Museum Mummy Project, 1979 [2] Davies, J H and Bunge, H P, Seismically `fast' geodynamic mantle models, submitted to Geophys. Res. Lett. [3] Bunge H P and Davies, J H, Tomographic images of a mantle circulation model, submitted to Geophys. Res. Lett. [4] Rhodes, M and Davies, J H, Global tomographic detection of mantle plumes in the uppermost lower mantle, submitted to Geophys. J. Int. [5] http://mantle.esc.liv.ac.uk/davies/AGU99/ [6] Advanced Visual Systems Inc., AVS/Express Developer's Reference, Release 3.0 [7] Wilson, P R, Solar and Stellar Activity Cycles, Cambridge University Press, 1994 [8] http://www.csar.cfs.ac.uk/

18