Download File - ORION Cybernetics

PREPARING PEOPLE FOR THE FUTURE… …THROUGH THE INTEGRATION OF TECHNOLOGY AND FACILITIES℠

TIA-942 based Data Centre Design Checklist

Extracts from ORION Cybernetics™’ Requirements Traceability Framework July 2012

R

D

Governance & Client Advocacy for Intelligent Infrastructures Data Centre Consultancy, Design, Optimization, Consolidation Data Centre Management Strategies, Compliance & Auditing Data Centre Infrastructure Management (DCIM) Enterprise Program & Project Management IM/IT Relocation Specialists

T F A

TIA942-based Data Centre Design Checklist – RTM v120716

1

Document Details

TIA942-based Data Centre Design Checklist Extracts from ORION Cybernetics™’ Requirements Traceability Framework

1.1

Document Change Control

Revision No. 1.0

Date of Issue

Author(s)

Description of Change

31 May 2012

D. Kevin Orr

Document creation - draft

Date of this revision: 1.2

31 May 2012

Distribution

Name 1.0

Title Internal distribution

Date of Issue 31 May 2012

Distributed By Document creation

1.3 Acknowledgement(s) This document and its content has been developed in concert with ORION’s Requirement Traceability Matrix Framework (RTM); attributes of the TIA942 Data Center Standard; and elements of the Certified Data Centre Management Professional curriculum. Per: D. Kevin Orr Managing Director & President ORION Cybernetics™ …integrating People, Technology, Facilities℠ © 2012 ORION Cybernetics™ . All rights reserved.

Page 2


ORION Cybernetics™ 1.4

Notice of Confidentiality

The information disclosed herein is copyrighted and is proprietary to ORION Cybernetics™ and is not to be used or disclosed to unauthorized persons without the written consent of ORION Cybernetics™. The recipient of this document shall respect the security status of the information. This document is intended only for the use of the individual(s) or entity to which it is addressed and contains information that is privileged and confidential. If the reader is not the intended recipient, or the employee or agent responsible for the deliver of the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this document in error, please notify us immediately by telephone at +1 (888) 358-4562, or on email at [email protected].

ORION Cybernetics™ …integrating People, Technology, Facilities℠ © 2012 ORION Cybernetics™ . All rights reserved.

Page 3


2

About ORION Cybernetics™

ORION Cybernetics™ is a knowledge-based firm specializes in the field of hand's-on planning, program management and the delivery of intelligent infrastructures and turnkey information management services to the Owner, Architect, Information Technology and/or the Facilities Manager. Client Advocacy & Governance for Intelligent Infrastructures Data Centre Consultancy, Design, Optimization, Consolidation Data Centre Management Strategies, Compliance, Auditing Certified Data Centre Management Professional Enterprise Program & Project Management Data Centre Infrastructure Management IM/IT Relocation Specialists Specializing in the integration of planning and programming of facilities and information services, ORION Cybernetics™ balances the Customer's facilities and property management teams with information technology. Focused on the evolution of the data centre environment, ORION Cybernetics™ provides for increased project oriented flexibility, while reducing costs for energy management and operations. Our ongoing commitment is to provide increased responsiveness to the end Customer and simplified operation for the information technology/facility/data centre manager. Our objective is to create environments tailored to the specific corporate cultures and needs of our clients. PREPARING PEOPLE FOR THE FUTURE, THROUGH THE INTEGRATION OF TECHNOLOGY AND FACILITIES℠

ORION Cybernetics™ Ottawa, ON Canada K4A 1S5 [email protected] www.orioncybernetics.com Tel: +1 (613) 590-1400 Fax: +1 (613) 590-1445 NA: +1 (888) 358-4562


Page 4


3

Table of Contents

1

Document Details .................................................................................................................. 2 1.1 Document Change Control ........................................................................................... 2 1.2 Distribution ........................................................................................................................ 2 1.3 Acknowledgement(s) ..................................................................................................... 2 1.4 Notice of Confidentiality ................................................................................................. 3

2

About ORION Cybernetics™ ................................................................................................ 4

4

Introduction............................................................................................................................ 6

5

Design Criteria ....................................................................................................................... 7

6

Site Selection and Tier Levels ............................................................................................... 8

7

Building Considerations ........................................................................................................ 9

8

Equipment Configuration ................................................................................................... 10

9

Rack Layout ......................................................................................................................... 11

10

Raised Floors ..................................................................................................................... 12

11

Cable Distribution ............................................................................................................. 13

12

Copper Cabling ................................................................................................................ 14

13

Copper Cabling System Design ...................................................................................... 15

14

Fibre Cabling ..................................................................................................................... 16

15

Fibre Cabling System Design ........................................................................................... 17

16

Earthing and Bonding ....................................................................................................... 18

17

Power Supplies .................................................................................................................. 19

18

Close Control Air Conditioning ........................................................................................ 20

19

Fire Protection ................................................................................................................... 21

20

BMS & Security................................................................................................................... 22

21

Data Center Infrastructure Management (DCIM) ......................................................... 23

22

ORION's Data Centre Practice Offerings - Professional Services ................................. 25


Page 5


4

Introduction

The comprehensive evaluation, analysis, planning, design and operation of a data centre crosses many disciplines with breadth and depth. This ‘design checklist’ serves as an analytical tool to audit data centre compliance against best practices and organizational requirements. Much of the criteria documented herein is based on the following TIA Standard and Addendums: ANSI/TIA-942-2005: Telecommunications Infrastructure Standard for Data Centers ANSI/TIA-942-1 (March 2008): Data Center Coaxial Cabling Specifications and Application Distances TIA-942-2 (March 2010): Telecommunications Infrastructure Standard for Data Centers Addendum 2-Additional Guidelines for Data Centers, specifies revised requirements for temperature and humidity in data centers.


Page 6


5

Design Criteria 1. What are the business/organization drivers for the Data Centre a. Current b. Future, when and what 2. Who are the stakeholders a. Owner b. Operator c. Contractor d. Consultant 3. Project start and end date 4. What is the critical level of the data centre 5. What is the budget 6. What type of Data Centre: Enterprise, Hosted, Co-location 7. Any particular standards to be considered 8. What currently exists, infrastructure and resilience 9. What disaster recovery plans/requirements 10. Has any Uptime “Tiering” been specified 11. Where will the location ne/planned to be 12. What size has been allocated 13. What electrical power is available and when 14. Is there any fire detection/suppression system in place 15. What HVAC is in place 16. What communications currently exist, LAN, WAN and/or MAN 17. IS there any containment in place/planned, intra and inter 18. What racks are currently being used, planned to be used and why 19. What IT Equipment exists or is planned 20. What security plans exist or have been stated, e.g. CCTV, Access 21. Any ongoing maintenance arrangements in place or planned


Page 7


6

Site Selection and Tier Levels 1. 2. 3. 4. 5. 6. 7.

8.

9. 10. 11.

12. 13. 14.

Does the structure already exist What geographic/seismic problems are there Any subsidence problems known about Are there any logistical access problems What distance from any known flight path, e.g. 0.4km What is the distance from the nearest highway or railway, e.g. 0.8km What is the distance from the following, e.g. 1.6km a. Nuclear Facility b. Munitions plant c. Defence Plant What is the distance from the nearest, e.g. 0.4km a. Airport b. Research Lab c. Chemical plant d. River e. Coastline f. Dam What is the distance from the nearest military Base, e.g. 0.8km Any known security/criminal problems in the area What availability of services, and note Service Provider details: a. Gas b. Water c. Electricity d. Telecoms e. Sewage What development plans exist for the surrounding area What EMI risks in the area, e.g. GSM masts, radar, microwave What Tiering levels are required if any


Page 8


7

Building Considerations 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26.

What is the minimum ceiling height to finished floor, e.g. 2.6m Is the ceiling false or under-base Is the min distance from ceiling to sprinkler heads greater than 460mm Confirm the minimum floor loading to be greater than 7.2 kPa, ideally 12kPa Confirm the ceiling/floor has a minimum hanging capacity of 1.2kPa, niom 2.4kPa Confirm lighting spec has a minimum of 500 Lux (horiz) and 200Lux (vert) at 1m above the finished floor in the middle of the aisles Confirm the doors to be a minimum of 1m wide and 2.13m high without doorsills Confirm that doors open outward or side/side or are removable Doors to have no centre posts Doors to be fitted with either industrial grade, or access control locks What are the local rules re Exit requirements Confirm walls, floor and ceilings to be/are sealed/painted to minimize dust Confirm sealing finish colour to be of light colour to enhance lighting Confirm floors to have anti-static properties to IEC 61000-4-2 What will the size of the telecom room be, e.g. min of 6.6, 8.4, 10.2 sq/m Confirm size of entry room/lobby is not less than 9 sq/m Confirm all exit doors, stairs, ramps and pathways to have Emergency lighting Confirm furthest distance to a fire exit is no more than 18m if only one exit Confirm escape routes to have a minimum width of 750mm and 1-lux lighting Have all fire extinguisher locations been planned What smoke extraction system is in place Fire doors to survive a min of 30-mins Are the fire door hinges fire rated Confirm fire door hinges/frames to be fitted with intumescent strips Confirm walls are 30-min fire rated, sealed and are built slab-to-slab Confirm one wall to be suitable for mounting heavy equipment


Page 9


8

Equipment Configuration 1. List the flowing information to be installed in the data centre a. Equipment b. Quantities c. Dimensions d. Weight e. Size f. Colour g. U size h. Power consumption i. Interconnection to/from i. Switches, core and edge ii. Servers iii. Patch panels iv. KVM v. ODF/MDF vi. Routers vii. Telco gateways viii. Storage devices ix. Power devices, e.g. UPS x. Cooling devices, e.g. CRAC’s xi. Security equipment xii. Fire protection/alarms xiii. Any other powered device 2. Consider equipment in other areas, e.g. Command and Control Centre 3. What cabling/switch strategy to use to interconnect cabinets 4. What data set/cluster strategy to use based on equipment fit


Page 10


9

Rack Layout 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21.

Confirm Hot Aisle – Col Aisle layout to be used Confirm 7-tile pitch configuration to be used Confirm either no-doors or perforated doors to be used Confirm a minimum of 63% perforation on the doors Will doors need to be locked, if so what security, e.g. key, card or remote Is the floor strong enough for racks and contents Confirm the floor to be level throughout Confirm the racks are level, x and y Confirm layout allows for heavier equipment in the bottom of the rack Will the racks be bolted to the floor base Confirm all racks to be bayed Confirm all racks to be earthed to the data centre earthing system Confirm that the rack colour to be a light shade to maximize lighting Confirm that there is a min clearance of 3.8cm between equipment and front/rear doors Confirm that at least 1m of clearance at the front door, preferably 1.2m Does the rack spec ensure that all nominated equipment will fit horizontally Confirm that vertical patch cord management shall be fitted in all racks Confirm that horizontal patch cord management shall be fitted in all racks If cable is to enter rack from the bottom, confirm cable entry will be sealed Confirm racks to have anti-tilt legs fitted Are there any open frames required for the Data Centre


Page 11


10 Raised Floors 1. 2. 3. 4.

Confirm that the min ceiling height from finished floor is to be a minimum of 2.6m Confirm ceiling finish to be a light colour to enhance lighting Confirm floors to have anti-static properties to IEC 61000-4-2 The loading of the floor to be as follows:

Grade

Concentrated load over 25mm sq

Concentrated load over 300mm sq

Uniformly distributed load

Light

1.5kN

2.7kN

6.7kN/m2

Medium

3.0kN

4.4kN

8.0kN/m2

Heavy

4.5kN

N/A

12.0kN/m2

Extra Heavy

4.5kN

N/A

12.0kN/m2

In addition this grade is required to sustain a total load of 11kN applied equally on 4 points, each point 25mm2 on a 200mm x 200mm2 configuration anywhere on the system.

5. Confirm the minimum floor loading to be greater than 7.2kPa, ideally 12kPa 6. Raised floor heights to be as follows: Floor area

Height of raised floor, VDI 2054

200-‐500m2

Approx. 400mm

500-‐1000m2

Approx. 700mm

1000-‐2000m2

Approx. 800mm

>2000m2

>800mm

Under 600mm Use normal support pedestals

7. 8. 9. 10. 11. 12.

Over 600mm Will require HDG lateral support/stringers

If data cables to be under floor, ensure they are under cabinet or hot aisle Power cables to be fitted in the cold aisles, virtually at base level Ensure ALL floor spaces are sealed, especially at the sides Ensure all ramps, steps and balustrades are designed in, disables/access Ensure the SRG is designed, then fitted before False Floor Confirm that every 6th metallic pedestal will be bonded to the SRG


Page 12


11 Cable Distribution 1. 2. 3. 4.

Confirm that the cable trays, overhead or under-floor has been mapped. Agree to patch panel I.D. plan Confirm all cables will be labelled on both ends Re the distances below, EN 50174 allows no separation in the final 15m Separation Distance Without a divider or with a non-‐metallic divider

Aluminum divider

Steel divider

Unscreened power cable & UTP IT cable

200 mm

100 mm

50 mm

Unscreened power cable & FTP/SFTP IT cable

50 mm

20 mm

5 mm

Screened power cable & unscreened IT cable

30 mm

10 mm

2 mm

Screened power cable & screened IT cable

0 mm

0 mm

0 mm

5. 6. 7. 8. 9.

Ensure any cables of different families cross at right angles Confirm that all cable trays are connected to Earth network every 15=20, Confirm cable tray sizing based on maximum cable planned fill Plan for 45% fill factor on internal tray containment If adjacent overhead trays to be used, confirm a 300mm distance between the top of the lower and bottom of the higher tray 10. Confirm all trays/cables route passing though walls will be fir stopped 11. Ensure all relevant cable bend radius is maintained 12. Label all pathways with to/from points


Page 13


12 Copper Cabling 1. 2. 3. 4.

Confirm which class of copper cabling to be used, e.g. Cat6A Confirm which sheathing type to be used, e.g. F/FTP Is fibre channel to be used on the copper cabling Confirm the installation team qualified to install, terminate, test and certify the copper cable 5. Confirm that the alien cross talk measures of separating the 1st 5-20mm of cable has been deployed 6. If screened cable is being deployed confirm that all foils/braids have been correctly earthed to the panel and the panels have been earthed to the cabinet bus-bar


Page 14


13 Copper Cabling System Design 1. 2. 3. 4. 5. 6. 7.

Confirm number of interconnects are allowed with the relevant standard Confirm that any copper backbone will support the transmission protocols Confirm that all patch cords will be used intra-cabinet Will the manufacturer guarantee the design Has ‘tiering’ /resilience for inter-cabinet cabling been considered Has sufficient growth capacity per cable route been considered Confirm the installation team has/will test correctly to the relevant standard


Page 15


14 Fibre Cabling 1. 2. 3. 4. 5. 6. 7. 8. 9.

What inter-building links are required What intra-building links are required What intra-Data Centre links are required Confirm with type of MM fibre cabling to be used, e.g. OM3 Confirm which SM fibre class to be used, e.g. G625D Confirm number of fibres per cable Confirm cable construction, e.g. TB, Universal What connector type to be used in the patch panels Has a pre-terminated solution been considered


Page 16


15 Fibre Cabling System Design 1. What IEEE standards with the fibre be required to support, Day1/long term 2. Fibre selection table, which fibre type Ethernet, Mbps

100m

300m

500m

2000m+

10

OM1

OM1

OM1

OM1

100

OM1

OM1

OM1

OM2

1,000

OM1

OM2

OM2

OM1

10,000

OM3

OM1

OM3

OM1

40,000

OM3

OM1

OM1

OM1

100,000

OM3

OM1

OM1

OM1

3. Confirm the bandwidth & fibre interfaces to be used on the equipment 4. Confirm the light budget calculations are within manufacturer limits 5. Has the ODF been sized for Day1 and future growth


Page 17


16 Earthing and Bonding 1. Confirm that the following (where relevant) shall connect to the main earthing terminal: a. Water services pipes b. Gas service pipes c. Other ducting/pipes d. HVAC e. Exposed metal parts of the building f. The lighting protective system g. Metallic sheath of the telecommunications cables 2. Confirm that the main earth bond conductor shall be no less than 6AWG or 4.1mm diameter 3. Confirm that all protective conductors shall be green/yellow sheathed 4. Confirm that all cabinets are fitted with earthing bus-bars and all equipment are earthed to it 5. Confirm that every 6th metallic pedestal with be bonded to the SRG 6. Confirm that a Transient Suppression Plate has been fitted to the SRG


Page 18


17 Power Supplies 1. Confirm the day1 and expansion loads 2. Confirm power requirements for the data centre a. Nameplate values (derate by 40%) b. Assume UPS efficiency of 88% c. Lighting assume 21.5w/sq m d. Cooling to take 100% of it cooling capacity e. % of future expansion capacity f. Agree to peak load factor, e.g. 50% g. Agree to a overrating capacity, e.g. 25% h. Add 3% for additional generator capacity 3. Confirm the requirement and specification for a Transfer switch, e.g. generator deployment 4. Confirm the tiering arrangements for power provision, e.g. N, N+1, 2N 5. Confirm the metering arrangements, e.g. utility input, UPS output, generator output, PDUs, output of the transfer switch 6. Confirm the earthing system to be used in the data centre, e.g. TN-S 7. Based on the tiering requirement confirm the specific redundancy arrangements/specifications of the UPS and generator sets, e.g. power/KVA provision and quantities 8. Confirm UPS type, e.g. VFI 9. Confirm UPS/battery support time of IT critical loads, e.g. 15-30 mins 10. Confirm siting and wiring arrangements for the Emergency Stop facility 11. Specific safety signage for UPS, Batteries and Generators 12. Confirm power to the rack arrangements, e.g. 3 or single phase, 16 or 32A connection, plug type. Ensure PDU plug matched to socket 13. Confirm where the Power sockets will be placed, e.g. directly under the rack or overhead tray 14. Specify PDU, e.g. sequential start-up, power conditioning, web management, load monitoring, environmental monitoring


Page 19


18 Close Control Air Conditioning 1. Confirm specification and cooling capacity of CRAC units, including N+1 and future capacity planning 2. Confirm siting arrangements for the CRAC and External Heat Exchangers 3. Ensure water supply is provided to the CRAC locations for humidity control 4. Ensure drainage route is planned and fitted 5. Plan pipe work routes from CRAC to External heat Exchangers 6. Ensure External Heat Exchangers have a stable and secure concrete plinth 7. Confirm appropriate power arrangements for supply to all CRACs and External Heat Exchangers 8. Confirm initial operating parameters for CRAs, e.g. temperature and humidity 9. Specify if an air and water side economizer is required or appropriate. If appropriate ensure that data centre, power and cooling adjustments are made. 10. Confirm ceiling/air plenum arrangements: a. Open ceiling b. False ceiling c. False ceiling with hot/cold aisle to air distribution dividers to ceiling space. Re-arrange fire options d. Cold air supply from overhead air containment (no false floor) 11. Confirm cold-aisle perforated tiles ensure maximum air flow 12. Ensure cold-aisle perforated tiles are no less than 2.4m from the CRAs 13. Confirm that all open spaces in the racks will be fitted with blanking panels 14. Confirm any under floor cable entries are blocked with brush panels 15. Confirm no shelf mounted equipment 16. Testing points to be 1.5m above the floor and 500mmm in front of all racks


Page 20


19 Fire Protection 1. Confirm development of the Fire Safety Plan 2. Emergency exits, lighting and signage 3. Fire detection a. Type, e.g. ASD b. Define areas to be covered, including under floor c. Confirm Fire Detection category, e.g. L5 d. Confirm detectors will be placed a minimum of 25mm from the ceiling or 600mm maximum. Note the detectors must be a minimum of 460mm from top of racks. Ceiling voids must also be covered e. Confirm maximum detectors spacing from walls, i.e. 7.5m and no more than 10.6m apart f. Confirm detectors at least 1m from a CRAC and 500mm clearance 4. Fire Alarms, including manual alarms 5. Multi-level fire suppression system a. Including pre-suppression as required b. Ceiling level c. Under floor d. Operations Centre e. Confirm suppressant type is appropriate f. Fire extinguishers g. Staff training drills, procedures and management structure h. Maintenance plan i. Fire certification from local authorities 6. Pressure testing of the room 7. Confirm what links and information into the building BMS and DCIM platform 8. Ensure the Fire Designer Certificate is complete before final payment 9. Ensure space has been allocated for the Fire Suppression bottles 10. Ensure under floor cables are low-smoke and old/non used cables are removed


Page 21


20 BMS & Security 1. Ensure Security Plan is multi-layered a. Site b. Room c. Rack d. The plan to contain i. Overall site security ii. CCTV iii. Access Control iv. BMS v. Fire Detection Integration of all electronic systems 2. Confirm Data Centre Access Control will be either Swipe Card, Biometric or Proximity 3. Confirm all locks will be industrial grade at the room doors 4. Confirm the CCTV system will monitor all Access controlled doors 5. Ensure any rack based security uses a hierarchical model for reporting to the NMS


Page 22


21 Data Center Infrastructure Management (DCIM) In order to better manage the critical information gap between the physical and virtual layers in the data centre, consider six key questions data centre managers should ask when implementing a data centre infrastructure management (DCIM) solution. Data centre managers are challenged to maintain or increase IT availability, server utilization and efficiency in the face of rising costs and demands. One strategy to help address these challenges is virtualization, which increases server utilization and enables greater energy efficiency. But virtualization also makes it more difficult to predict the demand on physical systems a t any given time. As a result of the uncertainty in physical system demand, the data centre manager is compelled to reserve a resource buffer against overload at peak capacity. A holistic DCIM approach provides new insights into the relationships between facilities and IT infrastructure components, enabling data centre managers to better optimize their data centre resource consumption. 1) What are the critical infrastructure assets in my data centre and what are their respective mission interdependencies? 2) If asked by my management, could I quickly retrieve a comprehensive monitoring and analysis report of all the assets in my data centre, and would this report identify bottlenecks? 3) Can my data centre manage day-to-day workloads with flexibility while servicing on-demand needs as they arrive for my company? 4) Does this DCIM solution provide the trend analysis and intelligence needed for my team to deliver a comprehensive capacity management plan? 5) What intuitive web-based executive summary and planning tools are provided in the solution? 6) Will the DCIM solution align with IT and business services?


Page 23


Forward-thinking data centers are using DCIM solutions today. These smart data center managers realize that they have to stop guessing & start planning. The days of over provisioning or using Visio and Excel for data center management are -- by necessity -over. DISCOVER… VISUALIZE… MODEL… CONTROL… REPORT… PREDICT…


Page 24


22 ORION’s Data Centre Practice Offerings - Professional Services Client Advocacy & Governance for Intelligent Infrastructures I. Data Centre Consultancy, Design, Optimization, Consolidation II. Data Centre Management Strategies, Compliance, Auditing III. Enterprise Program & Project Management IV. Data Centre Infrastructure Management V. IM/IT Relocation Specialists


Page 25


“Plans may be useless but the planning process is indispensable." Dwight D. Eisenhower


Page 26