Aerial Videography in Built Heritage Documentation

Aerial Videography in Built Heritage Documentation: The Case of Post-Independence Architecture of Malaysia Delas Santano Centre for Crea tive Content and Digital Innovation University of Malaya Kua la Lumpur, Ma laysia [email protected]

Abstract— The Unmanned Aerial Ve hic le (UAV) has become increasingly affordable in the market and has been reduced to smaller sizes, providing a n entry level UAV for a ny beginner. For this paper we have utilized the GoP ro action camera as the chosen camera, since DJI has an established gimba l for it and also the fact that the GoPro setup is considered the entry level, which is available to anyone and also produces good quality. Recent research collaboration has identif ied built heritage in Malaysia with a scope of being built after the nation’s independence between 1957 and 1970. Three buildings from this category were shot with the UAV, the National Mosque, Tun Abdul Razak Apartments or famously known as the Pekeliling Flats, and the Rifle Range Flats in Penang. This research will present the utilization of UAV in documenting a built heritage, its challenges in operating the machine, and documenting the sites. This study also briefly desc ribes how aeria l videography can be beneficia l for visualization artists. In addit ion to aerial capturing, one aspec t of built heritage documentation deals with e-rec onstruction including dig ital drawings, 3D modeling, virtual realit y, etc. This can be achieved by using exist ing visual materials as a reference to generate a virtualized version of the building. In some points visualization artist needs full access to the building, as existing re ference images are insufficient. But as mentioned above, sometimes acce ss is lim ited to terms and condit ions mostly related to safety. In this situation one of the best alternatives is to use UAV drones. Aerial v ideography can be beneficial for visualization artists, first as the reference, and secondly to be combined with virtua lized materials for augmented reality purposes as the land where flats were located on, will be covered by newly built s kyscrapers in the near future. Keywords— UAV, Aerial Videography, Built Heritage Preservation, e-Reconstruction, Virtual Reality, Augmented Reality

I.

INTRODUCTION

Ae rial videography has become popular by introduc ing the affor dable, ready-to-fly drone s. These dr ones ca n be see n in different for ms from winged type to mu lti rotors, up to eight. Drones, better known as UAV today, standing for Unmanned Aerial Vehicle [1] , have be come increasin gly affor dable in the market and have reduced to smaller sizes providin g an entry le vel UA V for almost all beginners. Ma nufacturers such as DJI

Human Esmaeili Knowle dge Ma nagement Centre, i-Knowmedia Faculty of Creative Multimedia, Multimedia University, Cyberjaya, Malaysia [email protected]

have opened the gateway from entry level to pro, for everyone from hobbyists to pr ofessional videographers, utilizin g these new capabilities. The co mmonly availa ble dr ones to the pub lic are the mu lti rotors, rangin g from 4-rotors to 8-rotors. The 4ro tors known as qua d copter, are used for this research projec t. Baxter [2] state s that aerial view fro m aerial photography has an inte nse effec t on the way public s picture their world. It will create an e mot ional impac t in the viewer towards the sub ject in the c ontext of archivin g what and how the past looked like for the case of Pekelilin g Flats. Co mbinin g the rule s a nd principle s of on ground cine matogra phy, the documenter controls the UAV to get the required shots to do cument the buildin gs. Utilizing the GP S in the UAV provided a challenge in terms of flyin g and shooting the buildings. Enough distance be tween the drone and the building is required in or der to frame the target area . This paper will pr esent the challenges of utiliz ing and ope rating the UAV, in documentation of built heritage . Co mbined with a high -re solution camera, the UAV will be an excellent tool for data acquisition with great coverage area of a site [3]. II. STUDY AREA The Post-Inde pende nce Archite cture Atlas (PIAA), is a colla boration betwee n the Centre for Creative Content and Digital Innova tion, University of Mala ya, Kuala Lumpur, Ma laysia, and the Knowledge Mana gement Centre, iKnowmedia, Multimedia University, Cyberja ya, Malaysia. The project is a bout documenting the buildin gs whic h were built after Malaysian Independence. These targe ts were se lec ted based on their architectura l importance. A period of 20 years from 1957 to 1977 was covered by this stu dy. Two buildings fro m this c ate gory were shot with the UAV, which are the National Mosque and the Tun Abdul Razak Apartments or fa mously known a s the Pekeliling Flats. Documenting built herita ge through digital data will allo w for future use of that data or even re interpre tation of objects or site s tha t will be inaccessible or have deteriorated [4] .

This research was funded by a Grant from University of Malaya (UMRG #CG001-2013).

The National Mosque does not provide a ccess to the roof or the mina rets due to the secur ity reason of the mosque itself and safety issues. The Peke lilin g Flats ha ve bee n on schedule to be de molished by the City Council and to be replac ed by new housin g projects. The building ha s been closed for more than 6 years after the last fa mily moved out in 2008 due to ne gotiation issue s. The de molition works of the Peke liling Flats has started in early 2014 block by block. During the time the flats we re abandoned it wa s famously known as black area of drug crimes. Co ntractors that were assign ed to de molish the build ing evacuated the whole area of the flats a nd cordoned off the territory to preve nt any tre spassers rega rdin g to the safety issues. Therefore it wa s dangerous to enter the complex for the archivist to document the build ing. The Rifle Range flats in Penang are still occupied by tena nts and sc hools and public a menities hea vily populate the area . III. THE AERIAL RIG In doc umenting the 3 buildings for the aerial videography, the re searcher used the DJI Phanto m 2 quad copter. The syste m is an intermediate level in aerial drone videogra phy and more budget-friendly for resea rcher without ha ving to c hoose a bigge r a nd more expensive syste m. Another considera tion in flying the qua d copter is to make sure tha t the locations are clear of physical obstructions such as power lines. The manufac turer also recommends a voiding any powe r tra nsformer or anythin g that could create an electromagn etic fie ld, which c ould disr upt the GPS and c ompass of the quad copter and could result in loss of control.

Spec ifica tions of the syste m as in Fig. 1 a nd Fig. 2: 1. DJI Phantom 2 Quad-copter. 2. 2-axis gimbal. 3. GoPro He ro 3. 4. First-Person-Vie w (FP V) Syste m.

Fig.2: FPV Setup

The 2-axis gimbal allows the ca mera to be stabilized to ha ve smooth fo ota ge. The FP V system is utilized for the pilot and the filmmaker, who is also the c amera man, to allow better flying through the live view of wha t the qua d c opter sees [5]. The FP V syste m used the 5.8GHz cha nnel to communicate be tween the transmitter and the LCD receiver. Havin g the FP V asse mbly will also assist in composing the shots taken. Just like any other production consideration, the key fa ctor of choosing the ca mera is ca pabilities and th e budget. Go Pro Hero 3 at the time was the late st action ca mera that was equipped with advanced feature s a nd capabilit ies. The ca mera offers capa bilities of a me dium-pro ca mera but is designed for any ac tion a ctivities or situation. Price wise , it is also much cheaper than mediu m-pro ca meras and eve n cheaper than the ba sic c onsumer ca meras. With today’ s adva nce ment in, combin ing UA V with digital cameras, high-resolution photos and vide os can be ca ptured with lower budgets [6]. As the target of the ca pture is for archivin g, a higher resolution of the da ta is chosen. Ha ving that in mind, the Go Pro cameras offers 1920x 1080 (FullHD) resolution that is usable in audiovisual data acquisition with reasonable file size. This will ensure sustainable quality for repr esentation on large scree ns. An action ca mera can be attached to the UAV with the assistance of a gimbal, whic h helps to sta bilize the ca mera to avoid shaky footages. The size of the ca mera will affect the gimbal size and also the UAV’s size. Bigger cameras will require bigger and powerful gimba l and a lso more powerful UA V. For this paper we have utilized the GoPro action ca mera as the chosen ca mera since DJI has an esta blishe d gimbal for it and also th e fact that the GoPro setup is considered the e ntry level that is available to anyone and a lso pr oduce s broadcast industry quality.

Fig.1: Drone setup photo

Setting the Go Pro Hero 3 for audiovisual capture c overs the resolution, FPS and FOV as per Fig. 3. The Go Pro Hero 3 ha s three options for the Fie ld of View (FOV), Narrow, Mediu m, and Wide. Narrow was chosen as the settin g to avoid or reduce lens distor tion in th e footage. With having a Narrow

FOV, the quad copter has to be place d further a wa y fro m the subject to cover the whole location, unlike a Wide view FOV.

Only when it reached the top floor s of the flats, it managed to register a ll 7 sate llite signals to fly smoothly. This issue resulte d in capturing less a erial shots for the interior.

Fig. 3: GoPro Hero 3 Setup photo

IV. DATA ACQUISITION AND ANALYSIS Aerial audiovisua l capture was done in the morning, as weathe r a nd the sun could a ffect the AV ca pture result. The conte nt initially identified to captur e are e xterior shots, focusing on the architec ture of the buildin gs such as the minare ts, mosque roof , flats’ height and flats’ rooftop. Shots take n by the ca mera will be dolly, pan and tilt. The quad copter utilizes the GP S to sta bilize itself and also to allo w better control for the operator. The gu sting wind affected the stability of the quad copter in fly ing a nd sta ying at one position. In Fig. 4, the qua d copter had fe w bouts of sha ke while ca pturin g the site when it reac hed 150 mete rs.

Fig. 5: Quad copter at Interior/Hallway of Pekeliling

The data were then importe d with the GoPro Stu dio App that allows the foota ges to be transcoded (for better handling in the Post Processing sof tware. Due to the nature of the FOV fro m the GoPro Stock (initially installe d) Lens, even with a “Narrow” FOV settin g, when nearing the subject ma tter in the shooting, le ns distortion will happe n (ha ppens for c lose targets). Using th e Adobe Pre miere CC2014, the latest preset in lens distortion fix, lens distortion wa s fixed in the footage with minor adjustment to the prese t para meters. To fur ther sta bilize the foota ge, warp sta bilizer was applied and succe ssfully smoothed the dolly, tilt and pan motion of the ca mera. In regards to weather and sun, the researcher notic ed of shadows from the propellers be ing re corded in the footage. After a few times observing, this happened due to the height and position of the sun towards the quad copter/ca mera is a t 10 o’clock or 2 o’ clock, similar to a 45-degr ee a ngle. This issue unf ortunately is not fixable in post pr ocessin g. The aerial video captured from the interior parts of the Pekeliling Flats, was gra iny due to insufficient light, pe netrating from the roof . However it wa s fixed by using the Denoiser plugin of fered by the RED GIANT.

Fig. 4: National Mosque at 150m above ground level.

Whilst go ing forward it wa s a lso not stea dy as the wind blew it side wa ys and the GPS was adjusting the coordinate s to stabilize the dr one. In the location shoot for the Pe keliling Fla ts, the quad copter wa s placed inside the flats at the ha llway as in Fig. 5, where the se flats have a hollo w structure . With the flats bein g 17 levels high, poor GP S signal was obser ved in this situation. It requires 7 GP S satellite signal to be re gistered for it to be stable with GP S flyin g mode , but at the time of shooting with its position it only read 3 satellite signals. P oor sate llite rece ption resulted in instability, whic h required more manua l control and relatively more ba ttery usage.

For the Rifle Range Flats location, flyin g the quad copter was restricte d to only long shots (Fig. 6), as the area is hea vily popula ted by re sidents. Public safety and their properties must always be considere d first should there be any disturbances that could happen to the quad copter whilst fly ing. This also resulte d in a roof view of the buildings (Fig. 7) as flyin g lower poses more risks. To accommodate the line of view between the opera tor a nd the quad copter, it had to be flown very high reachin g to 300 meters above the ground level. This is known since th e quad copter utilizes the GP S c oordinates together with its internal compa ss in relation to the re mote con trol.

Fig. 6: Long shots of Rifle Range Flats

Fig. 8. The Last Staircase

Fig. 7: Rifle Range Flats Roof View Fig. 9. The Elevator Room

V. VISUALIZATION BASED ON THE E XISTING REFERENCES Existing reference materials such as photos, maps, plans, drawin gs, etc . are essentia l in visualiza tion of the built heritage [7]. This be comes more tangib le when a visualization artist has limited or no access to the buildin g due to the safety issues and in some ca ses in absence of the building mainly because of demolition. VI. THE METHODOLOGY : HOW A DRONE WAS USED TO HELP THE VISUALIZATION ARTIST ? For the case of the Pe keliling Fla ts, limite d access to the roof le d to using the ca ptured aerial videos as the reference material to c omplete the visualization. In a ddition to the roof there were other sections that could not be modele d without using a drone, as it was not safe to walk into those areas for reference photography. The la st stairc ase , whic h is the connec tion between the last floor and the roof, also the elevator roo m located on the roof are two exa mples of those area s. Fig. 8, Fig. 9, and Fig. 10, sho w the last staircase, the elevator room and the roof , respec tively.

Fig. 10. The Roof

The pattern for the staircase on a ll floor s wa s the sa me but it was assumed tha t it might be different when it reached to the roof. The c aptured referenc es by the dr one helped to eliminate this uncertainty. Fig. 11 and Fig. 12, show the 3D mode l of the last staircase and the elevator roo m re spectively. The 3D modelin g of these flats was main ly focused on the concrete structure, as they were the first affor dable housing project in Ma la ysia, using Industrialize d Build in g Syste m (IBS) a nd prefa brication methods [8].

de taile d areas. Degr adation of the concrete structure wa s ma inly caused by see page and water le akin g issues [9], which can be seen in Fig. 13.

Fig. 11. The 3D model of the Interior, Showing the Last Staircase

Fig. 13. Marked areas After the First Phase of the Aerial Videography

Fig. 12 . The 3D Model of the Roof

Aerial vide ography provided an overvie w of the building. A c omple te aerial videography helped the visualiza tion artist to study the entire building from angles, which ca nnot be captured without using UAV. This is more useful when no technical drawin gs are available, but somehow limite d to the exterior. The selected target for this study has a simple pattern repea ted on all floors. Therefore the main measurements were done in the ground floor, as the overall shape is a box. However to make sur e tha t the pa ttern from down to top is the sa me. A quick scan was done with the dr one main ly to capture interior parts. It is important to mention that the building’ s hollo w struc ture e nabled our capturin g team to perfor m the aerial videogr aphy of the interior , which was a n advanta ge compare d to our other targe ts. The Visua lized 3D mode l completely follows the original proportion of the rea l fla ts. Because of the simplicity of the structure, at the first stage the UA V was only used for the limited access a rea s, as mentioned earlie r. After e-reconstruction of the Pekelilin g Flats to their or igina l state , a decision was made to visualize the a bandoned version of the flats. The main idea was to create a virtua l reality e nvironment where users ca n experience a tra nsition be twee n or igina l and abandoned states. For this purpose more reference materials were nee ded to study the aging pattern of the flats. Again limited acc ess to the higher floors led to use the UA V to capture the refe rence materia ls. Unlike the first time, this time more small de tails needed to be documented. Therefore initially capture d aerial videos were studie d to mark the areas with high level of details as the targets for the sec ond phase of the capturing. Fig. 13, shows some of the highly

After markin g the areas with high level of details, the second phase of aeria l videography was started by foc using on small details. Considering time, budge t, and workf orce, an optimized data sa mpling was required to enable our visua lization artist to visualize the abandoned version of the flats by follo wing the a ging pattern. Fig. 14, shows a closer look at one of the marked areas, capture d in the second phase of the aeria l videography.

Fig. 14. The Second Phase of the Aerial Videography

The captured data were used in the e -reconstruction of the abandone d version, which was actually modifyin g the in itia lly e-reconstructe d version. The original concrete structure wa s left almost intact. Therefore the major modification wa s around te xturing and adding the aging e lements, mostly the plants. However by stu dying the dama ge d area it wa s reve aled that these da mages were appeared afte r the demolition wa s started. ‘Da mage’ is referred to th e broke n concrete parts not the effect of the weather. Therefore in visualiza tion of the abandone d ve rsion, these br oken con cre te parts were ignored as the main story of the Pe keliling Flats ends right before the first day of the de molition. The story of the Pekeliling Fla ts is a virtual time travelin g betwe en the aba ndoned and the or iginal versions, from 2014 to 196 7, vice versa . This interac tive stor y will be simulated in the Unity 3D ga me engine . Going through the tec hnical details is out of the area

of th is study as the main foc us is on the aerial videography and collecting the reference materials. VII. USING AERIAL VIDEOGRAPHY FOR AUGMENTED REALITY PURPOSES The land, where the Pekeliling Flats stood, for a long period of time, will be soon covered by newly built sky scrapers. A complete aerial vide ogra phy was perfor med over the area. The se videos can be used for augmented reality pur poses. A fly-through video with a gradual transitio n between the a bandoned and the origina l state s (vice versa) can be generated using the obje ct trac king technology. This will be use d only for the exterior. The interior will be c ompletely repla ced by the 3D model. VIII. CONCLUSION AND RECOMMENDATION Operating the DJI Phanto m in GP S mode is the safest wa y to fly, especially for beginners to intermediate leve l operators. With the comp licacy of re gisterin g for GP S signals, it is not suitable for indoor/covered flying espe cia lly for beginner pilots. The a dvanced mode in the qua d copter is ava ilable, but it requires sufficie nt flyin g time to be able to ope rate the quad copter in said mode as it does not use GP S sa tellite signals to stabilize itself. The latest qua d copter of DJI “ Inspire 1” can better cover indoor/covered loca tions. At the time of the data collec tion, the “ Go Pro Hero 3” was the la te st model offered by the GoPro. The next re lea se after it was the “ Go Pro Hero 3+” which was re leased couple of months after it. I t has one major upgrade that makes it superior than the GoPro Hero 3, e ven though the hardware specifications are the sa me. With the I SO Limit contr ol in the GoPro Hero 3+ fir mware setting, it a llo ws the usage of the camera in dark/low light locations without ha ving to worry about noise, unlike in the GoPro Hero 3 with the Auto-ISO. For better post-processing of the data in overcomin g stabilization and lens distortion, shooting in 2.7k-resolution (2 704x1524 pixels) is better tha n the 1080p-resolution. It offers more room for refra ming the shot in post-pr oduction and compensating for the stabilization and lens distortion, to finish the data in 1080 re solution. Although 2.7k-resolution will increa se the file size and de mand more storage space for the ca mera files and Post-Production Process. An up grade of the stock lens to a zoom lens like the 16m m from Rage Ca m can provides closer or tighter shots of objects without having to fly near to the subje ct or area. This study a lso reviewed a real case of th e data collection for e-rec onstruc tion of architectura l heritage by focusin g on limited access area s. These days dr ones are bec omin g an

essentia l tool rather than optiona l. Considerin g the safety a nd limited access, recently a dr one was used by the Duba i’s polic e force in the pr ocess of saving a window cleaner, where the techn ica l tea m could analyze the faulty device and the situation, before making any dec ision [10]. In the case of the Pekeliling Flats, virtual reconstruction could be a diffic ult pr ocess if not c omp letely impossible, in a bsence of the drone. The video capture d by the drone can a lso be used for augmented reality purposes by replac ing the real mode ls with the e-reconstructed one s, using object-trackin g technolo gy. ACKNOWLEDGMENT This research project was funde d by a Grant from the University of Ma laya (UMRG #CG001-2013). The author s would like to thank the Centre for Creative Content a nd Digital In novation (CCCDI), University of Malaya, Kuala Lumpur (3cdi.um.edu.my). Many thanks to the Knowledge Manage ment Centre & University Researc h Centre, iknowmedia, Faculty of Crea tive Multimedia, Multimedia University, Cy berjaya , Malaysia, for projec t collabor ation. Spe cia l tha nks are extended to HW-ARTS (www.hwarts.c om) for the 3D modelin g a nd visualization imagery.

REFERENCES [1] The UAV. (2014, November 19). The Future Of The Sky [website]. Available: http://www.theuav.com/ [2] Baxter,K. (2014). Grounding the Aerial: The observer’s view in Digital Visualisation for Built Heritage. Electronic Visualisation and the Arts. 2014:163-170. [3] Stal, C,. Lonneville, B., Nuttens, T., De Maeyer, P., De Wulf, A., 2014. “Highly Detailed 3D Modelling of Mayan Cultural Heritage Using and UAV”. 25 FIG Congress 2014, 1-14. [4] Richards, J.D., Niven, K., Jeffrey, S., 2014. “Preserving Our Digital Heritage: Information Systems for Data Management and Preservation.” 2013(29):311-326. [5] Lisa M. King (2014) “Will drones revolutionise ecotourism?, Journal of Ecotourism”, 13:1, 85-92, DOI: 10.1080/14724049.2014.948448 [6] Opfer, N. D., & PE, D. R. S. “Unmanned Aerial Vehicle Applications and Issues for Construction.” 121 st ASEE Annual Conference and Exposition. 2014. Retrieved from http://www.asee.org/file_server/papers/attachment/file/0004/4608/Unma nned_Aerial_Vehicles_In_Construction_2-16-2014.pdf [7] P. Pauwels, R. Verstraeten, R. De Meyer, and J. Van Campenhout, “Architectural Information Modelling for Virtual Heritage Application.” International Conference on Virtual Systems and Multimedia (VSMM), pp. 18 -23, Archaeolingua, 2008. [8] W. A. Thanoon, L. W. Peng, M. R. A. Kadir, M. S. Jaafar, and M. S. Salit, “The Experiences of Malaysia and other countries in industrialised building system,” In Proceeding of International Conference on Industrialised Building Systems, pp. 10-11, September 2003. [9] N. B. Kiong, and Z. B. Akasah, “Analysis Building Maintenance Factors For IBS Precast Concrete System: A Review. Analysis, vol. 2, no. 6, pp. 878-883, 2012. [10] Yahoo Tech, Alyssa Bereznak (2014, November 7). Drone Saves Window Washer [Online]. Available: https://www.yahoo.com/tech/drone-saves-window-washer101935642539.html