Cloud Computing

UNIVERSITY OF PATRAS SCHOOL OF ENGINEERING DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING

Doctoral Dissertation

« MONITORING AND CONTROL OF

DISTRIBUTED WEB SERVICES ON CLOUD COMPUTING INFRASTRUCTURE»

DECHOUNIOTIS DIMITRIOS

DIPLOMA IN ELECTRICAL AND COMPUTER ENGINEERING

DISSERTATION No: 318 PATRAS - 2014

2

ΠΑΝΕΠΙΣΤΗΜΙΟ ΠΑΤΡΩΝ ΠΟΛΥΤΕΧΝΙΚΗ ΣΧΟΛΗ ΤΜΗΜΑ ΗΛΕΚΤΡΟΛΟΓΩΝ & ΤΕΧΝΟΛΟΓΙΑΣ Η/Υ

Διδακτορική Διατριβή «ΠΑΡΑΚΟΛΟΥΘΗΣΗ ΚΑΙ ΕΛΕΓΧΟΣ ΚΑΤΑΝΕΜΗΜΕΝΩΝ ΔΙΚΤΥΑΚΩΝ ΥΠΗΡΕΣΙΩΝ ΣΕ ΥΠΟΛΟΓΙΣΤΙΚΗ ΑΡΧΙΤΕΚΤΟΝΙΚΗ ΝΕΦΟΥΣ»

ΔΕΧΟΥΝΙΩΤΗΣ ΔΗΜΗΤΡΙΟΣ

ΔΙΠΛΩΜΑΤΟΥΧΟΣ ΗΛΕΚΤΡΟΛΟΓΟΣ ΜΗΧΑΝΙΚΟΣ ΚΑΙ ΤΕΧΝΟΛΟΓΙΑΣ Η/Υ

ΑΡΙΘΜΟΣ ΔΙΑΤΡΙΒΗΣ: 318 ΠΑΤΡΑ - 2014

3

4

Acknowledgements Foremost I would like to thank my advisor professor Spyros Denazis for his continuous support, motivation and guidance during the research and writing of this thesis. Besides my advisor, I would like to thank the rest of my thesis committee: Prof. Antonios Tzes and Prof. Evaggelos Dermatas for their encouragement, insightful comments, and their help whenever I needed. I would like to express my sincere gratitude to Prof. George Bitsoris for his contribution of time, immense knowledge and being an excellent example of teacher. My sincere thanks also goes to Dr. Andreas Kind and Dr. Xenofontas Dimitropoulos for offering the internship opportunity in the IBM Research Laboratory of Zurich and leading me working on exciting research topics. I feel grateful that most time of my research I work together with my close friend Nikolaos Leontiou, who is also PhD student in the same lab. Our collaboration helps me to be more productive and improve the quality of my research. I thank my close friend Nikolaos Athanasopoulos for his patience, advices and ideas which help me a lot to complete this thesis. I thank my close friends and fellow labmates Nikolaos Efthymiopoulos, Athanasios Christakidis and Maria Efthymiopoulou for their support, advices, productive discussions and fun we have had all these years. I thank my friends Andreas Lambropoulos, Rania Tsioulou, Elli Andrikogiannopoulou, Anastasia Zisimatou and Spilios Kastanis for their friendship and support. Last but not least I thank and dedicate this dissertation to my parents Panayiotis and Nikoletta and my brother Giorgos, who always support me in all my pursuits.

Dechouniotis Dimitrios Patras February 2014

5

6

Εκτεταμένη Περίληψη στα Ελληνικά Greek Extended Abstract 1. Περιγραφή της Διατριβής. Η Υπολογιστική Αρχιτεκτονική Νέφους (ΥΑΝ – cloud computing) είναι ένας πρόσφατος τρόπος ανάπτυξης και παροχής δικτυακών υπηρεσιών που αναπτύχθηκε μέσα στην τελευταία δεκαετία. Με την γρήγορη εξέλιξη των υπολογιστικών και αποθηκευτικών τεχνολογιών και την διάδοση του διαδικτύου, οι υπολογιστικοί πόροι έχουν γίνει πιο ισχυροί και προσιτοί για όλους από κάθε άλλη εποχή. Αυτή η νέα τεχνολογική τάση έχει σαν αποτέλεσμα τη δημιουργία ενός νέου μοντέλου υπολογιστικής αρχιτεκτονικής, της Υπολογιστική Αρχιτεκτονική Νέφους (ΥΑΝ), στο οποίο οι υπολογιστικοί (CPU, μνήμη και αποθηκευτικά μέσα) και δικτυακοί πόροι είναι οργανωμένοι σε υπολογιστικά κέντρα (data centers) που βρίσκονται διασκορπισμένα σε όλο τον κόσμο. Οι πόροι αυτοί παρέχονται σαν προϊόν το οποίο μπορεί να μισθώνεται κατά απαίτηση από τους χρήστες του διαδικτύου. Η ΥΑΝ προβάλλει πειστικά επιχειρήματα στους πελάτες έτσι ώστε να μεταφέρουν τις δικτυακές εφαρμογές από ιδιόκτητες υποδομές και εγκαταστάσεις σε πλατφόρμες ΥΑΝ. Τα πιο σημαντικά κίνητρα είναι το μοντέλο πληρωμής ανάλογα με τη χρήση, η δυναμική κατανομή πόρων, η εύκολη επεκτασιμότητα, η μείωση των εξόδων λειτουργίας και συντήρησης καθώς και η εύκολη πρόσβαση από ετερογενής συσκευές. Με αυτό τον τρόπο, οι πελάτες δεν είναι υποχρεωμένοι να επιβαρύνονται με την αγορά του απαραίτητου εξοπλισμού ενός ιδιωτικού υπολογιστικού κέντρου και τα έξοδα λειτουργίας και συντήρησής του, αλλά έχουν τη δυνατότητα να μισθώνουν ανάλογα με τις ανάγκες μόνο τους απαραίτητους υπολογιστικούς και δικτυακούς πόρους για την εγκατάσταση των δικτυακών τους εφαρμογών και να πληρώνουν μόνο για τη χρήση τους. Αυτό συνεπάγεται επίσης μείωση της κατανάλωσης ηλεκτρικής ενέργειας, των βλαβών του υλικού (hardware) και του αναγκαίου εξειδικευμένου τεχνικού προσωπικού. Οι πάροχοι υπηρεσιών ΥΑΝ χωρίζονται σε τρεις κατηγορίες. Όπως φαίνεται και στο Σχήμα 1, οι πάροχοι είναι διασυνδεμένοι μεταξύ τους σε ένα ιεραρχικό πολυεπίπεδο μοντέλο, όπου ο πάροχος του ενός επιπέδου είναι χρήστης του παρόχου του προηγούμενου στρώματος. Στο κατώτερο επίπεδο βρίσκονται οι πάροχοι της Υποδομής ως Υπηρεσίας (ΥωΥ – Infrastructure as a Service) που προσφέρουν όλο τον αναγκαίο υπολογιστικό και δικτυακό εξοπλισμό, όπως εξυπηρετητές, αποθηκευτικές συσκευές και δρομολογητές, με τη μορφή εικονικών μηχανών (ΕΜ – Virtual Machine). Στο δεύτερο επίπεδο οι πάροχοι της Πλατφόρμας ως Υπηρεσίας (Platform as a Service) παρέχουν την υποστήριξη των λειτουργικών συστημάτων και τα απαραίτητα περιβάλλοντα ανάπτυξης λογισμικού. Αυτά τα δύο πρώτα στρώμα ονομάζονται μαζί ως επίπεδο πόρων (utility computing). Στο ανώτερο επίπεδο οι

7

πάροχοι Λογισμικού ως Υπηρεσίας (Software as a Service) δημιουργούν οποιαδήποτε δικτυακή εφαρμογή.

Χρήστης Δικτυακή Διεπαφή

ΛωΥ Επίπεδο πόρων

ΠωΥ

ΥωΥ

Σχήμα 1 - Ιεραρχικό Μοντέλο Παρόχων ΥΑΝ

Οι δικτυακές υπηρεσίες, οι οποίες αναπτύσσονται σε πλατφόρμες ΥΑΝ, έχουν διαφορετικές απαιτήσεις μεταξύ τους από άποψη λειτουργικού κόστους, αξιοπιστίας και ασφάλειας. Για το λόγο αυτό τα είδη των ΥΑΝ έχουν κατηγοριοποιηθεί ως εξής: 







Δημόσιες ΥΑΝ προσφέρουν τους πόρους τους σαν υπηρεσίες στο ευρύ κοινό, παρέχοντας όλα τα απαραίτητα υπολογιστικά και δικτυακά μέσα αλλά έχοντας χωρίς έλεγχο πάνω στο δίκτυο, τα δεδομένα και τις ρυθμίσεις ασφαλείας. Αυτό μπορεί να αποτελεί εμπόδιο για εταιρικές υπηρεσίες με αυστηρές προδιαγραφές. Ιδιωτικές ΥΑΝ έχουν σχεδιαστεί για αποκλειστική χρήση από ένα και μόνο οργανισμό και προσφέρουν υψηλά επίπεδα ελέγχου, αξιοπιστίας και ασφάλειας. Υβριδικές ΥΑΝ είναι ένας συνδυασμός δημόσιων και ιδιωτικών ΥΑΝ, τα οποία είναι ευπροσάρμοστα και εκμεταλλεύονται τα πλεονεκτήματα των δυο προηγούμενων ΥΑΝ με σκοπό να εκπληρώσουν τις προδιαγραφές όλων των υπηρεσιών. Εικονικές ιδιωτικές ΥΑΝ εκμεταλλεύονται την τεχνολογία των εικονικών ιδιωτικών δικτύων (Virtual Private Networks) και επιτρέπουν στους παρόχους 8

υπηρεσιών να σχεδιάσουν τη δική τους τοπολογία και ρυθμίσεις ασφαλείας πάνω σε ένα σύνολο δημοσίων ΥΑΝ. Το παραπάνω μοντέλο των παρόχων και των υποκείμενων υποδομών προσδίδει τα ακόλουθα σημαντικά γνωρίσματα στις πλατφόρμες ΥΑΝ:  

   

Γεωγραφικά κατανεμημένες υπηρεσίες και καθολική δικτυακή πρόσβαση από ετερογενείς συσκευές. Η λειτουργία των εφαρμογών καθορίζεται από ένα Συμφωνητικό Επιπέδου Υπηρεσίας, ΣΕΥ, (Service Level Agreement - SLA) μεταξύ του παρόχου και του πελάτη. Δυναμική παροχή πόρων ανάλογα με την ζήτηση της υπηρεσίας. Κοινή δεξαμενή πόρων μπορεί να κατανέμεται δυναμικά σε πολλαπλούς χρηστών πόρων. Μείωση του κόστους λειτουργίας και συντήρησης όπως η κατανάλωση ηλεκτρικής ενέργειας και αστοχιών του εξοπλισμού. Πληρωμή ανάλογα με τη χρήση των πόρων.

Παρόλη την γρήγορη ανάπτυξη, η ΥΑΝ είναι ακόμα στα αρχικά της βήματα και υπάρχουν πολλά θέματα τα οποία πρέπει να επιλυθούν. Επιπλέον νέα θέματα προκύπτουν συνεχώς από διάφορες βιομηχανικές εφαρμογές. Τα πιο σημαντικά προβλήματα διαχείρισης των ΥΑΝ περιγράφονται σύντομα παρακάτω: 







Η αυτοματοποιημένη παροχή πόρων (automated resource provisioning) λύνει το πρόβλημα του διαμοιρασμού των υπολογιστικών πόρων προς τις εφαρμογές με τέτοιο τρόπο έτσι ώστε να ικανοποιούνται όλες οι απαιτήσεις ανεξαρτήτως των διακυμάνσεων του φορτίου όσο και απότομες κι αν είναι. Η μετακίνηση των εικονικών μηχανών (virtual machine migration) είναι μια τεχνική για την αποφυγή εξάντλησης των πόρων ενός συνόλου εξυπηρετητών. Η συγκέντρωση των υπηρεσιών (service consolidation) είναι ένα σχετικό πρόβλημα που ασχολείται με τη μεγιστοποίηση της χρήσης των πόρων του συστήματος ελαχιστοποιώντας παράλληλα την ενεργειακή κατανάλωση σε περιβάλλον ΥΑΝ. Η εγκατάσταση μερών διαφορετικών εφαρμογών σε ένα συγκρότημα εξυπηρετητών επιτρέπει τη χρησιμοποίηση μόνο του αναγκαίου αριθμού εξυπηρετητών και την ενεργοποίηση ή απενεργοποίηση αυτών ανάλογα με τη ζήτηση. Η διαχείριση ενέργειας (energy management) αντιμετωπίζει το πρόβλημα της κατανάλωσης ηλεκτρικής ενέργειας το οποίο δεν είναι αμελητέο πια ως κόστος. Πολλοί πάροχοι θέτουν την εξοικονόμηση ενέργειας ως έναν από τους πρωταρχικούς τους στόχους κατά τον σχεδιασμό του καταμερισμού των πόρων του συστήματος.

9











Η παρακολούθηση και ανάλυση της δικτυακής κίνησης (traffic monitoring and analysis) είναι ένα ενδιαφέρον ερευνητικό θέμα εξαιτίας του μεγάλου όγκου των δεδομένων προς επεξεργασία. Είναι σημαντικό να υπάρχει μια άμεση εικόνα του τι συμβαίνει σε πραγματικό χρόνο στο υπολογιστικό κέντρο για την αποφυγή αστοχιών του υλικού, διατήρηση υψηλής αξιοπιστίας και ασφάλειας και για τον απαραίτητο σχεδιασμό επέκτασης ή τροποποίησης των υπαρχόντων υπολογιστικών και δικτυακών πόρων. Η ασφάλεια δεδομένων (data security) είναι άλλο ένα σημαντικό ερευνητικό θέμα των ΥΑΝ. Ο πάροχος πρέπει να παρέχει εμπιστευτικότητα για ασφαλή πρόσβαση και μεταφορά δεδομένων και μηχανισμούς ελέγχου (auditability) για επιβεβαίωση αν οι ρυθμίσεις ασφαλείας εφαρμόζονται ή όχι. Λογισμικά περιβάλλοντα (software frameworks) πρέπει να αναπτυχθούν για το προγραμματισμό των διάφορων διεργασιών σε ένα ΥΑΝ περιβάλλον και για μια εύκολα επεκτάσιμη και ανεκτική σε σφάλματα (fault-tolerant) επεξεργασία δεδομένων. Τεχνολογίες αποθήκευσης και διαχείρισης δεδομένων (storage technologies and data management) για κατανεμημένη επεξεργασία εφαρμογών που παρουσιάζουν μεγάλες απαιτήσεις σε αυτό το τομέα. Πρωτότυπες αρχιτεκτονικές που συνδυάζουν διαφορετικές τεχνολογίες. Μέχρι τώρα οι ΥΑΝ χρησιμοποιούν κυρίως κεντρικοποιημένες δομές. Μπορούν όμως να συνδυαστούν όμως και με κατανεμημένες δικτυακές αρχιτεκτονικές, όπως τα διομότιμα δίκτυα (peer-to-peer networks) για το διαμερισμό βίντεο ή άλλων πραγματικού χρόνου εφαρμογών πολύ μεγάλης κλίμακας.

2. Αντικείμενο της Διατριβής. Στη παρούσα διδακτορική διατριβή δύο από τα προαναφερθέντα ερευνητικά θέματα επιλύονται. Αρχικά αναπτύσσεται μια τεχνική παρακολούθηση της δικτυακής κίνησης με σκοπό την εύρεση λειτουργικών σχέσεων μεταξύ των διάφορων μερών μιας δικτυακής εφαρμογής. Στο δεύτερο μέρος επιλύεται το πρόβλημα της αυτοματοποιημένη διανομής των πόρων σε δικτυακές εφαρμογές που μοιράζονται ένα κοινό περιβάλλον ΥΑΝ.

2.1 Παρακολούθηση δικτυακών εφαρμογών σε περιβάλλον ΥΑΝ Οι σύγχρονες δικτυακές εφαρμογές είναι πολυεπίπεδες, αφού απαρτίζονται από διαφορετικά μέρη και είναι εγκατεστημένες σε υπολογιστικά κέντρα δημόσιων ή ιδιωτικών ΥΑΝ. Επειδή η σωστή λειτουργία των δικτυακών υπηρεσιών περιγράφεται από το ΣΕΥ και η παραβίασή του έχει οικονομικό αντίκτυπο, υπάρχει ανάγκη για ολοκληρωμένα εργαλεία διαχείρισης τα οποία παρακολουθούν την λειτουργία και την απόδοση των εγκατεστημένων εφαρμογών με σκοπό να προλαμβάνουν κάθε πιθανό πρόβλημα. 10

Ένα τυπικό υπολογιστικό κέντρο περιέχει τα συστατικά μέρη χιλιάδων υπηρεσιών. Αυτά τα μέρη αλληλεπιδρούν με ένα ακαθόριστο και μη τυποποιημένο τρόπο με σκοπό να ολοκληρώσουν διαφορετικού τύπου συναλλαγές (transactions) μιας εφαρμογής. Οι διαχειριστές του δικτύου συνήθως δεν έχουν μια πλήρη και ξεκάθαρη εικόνα των συνθηκών λειτουργίας των εφαρμογών. Τα σημερινά συστήματα δικτυακής παρακολούθησης παρέχουν πληροφορίες και στατιστικά βασιζόμενα απλά στις δικτυακές πόρτες (network ports) χωρίς περαιτέρω ερμηνεία του πως συνεργάζονται τα λειτουργικά μέρη μιας υπηρεσίας. Στόχος αυτού του κεφαλαίου της διατριβής σε σχέση με την υπάρχουσα βιβλιογραφία είναι η δημιουργία ενός εργαλείου ανάλυσης της δικτυακής κίνησης έτσι ώστε να γίνονται κατανοητές οι λειτουργικές σχέσεις μεταξύ μερών των κατανεμημένων δικτυακών υπηρεσιών. Πιο συγκεκριμένα: 

    

Η δημιουργία ενός γράφου υπηρεσιών (service overlay), που θα απεικονίζει τις δικτυακές και λειτουργικές σχέσεις μεταξύ των διαφόρων μερών των εγκατεστημένων εφαρμογών σε ένα υπολογιστικό κέντρο. Αξιόπιστη αναγνώριση των σχέσεων χωρίς καμία πρότερη γνώση για αυτές. Μέτρηση του σθένους μια σχέσης. Εύκολη και εκτός γραμμής (offline) διαδικασία εκπαίδευσης του συστήματος. Γρήγορη και πραγματικού χρόνου (online) επεξεργασία των δεδομένων. Μικρός αριθμός παραγόμενων λανθασμένων σχέσεων.

Στο Σχήμα 2 απεικονίζεται η δομή των υπηρεσιών και οι διασυνδέσεις μεταξύ των χρηστών και των εξυπηρετητών ή των βάσεων δεδομένων που είναι συνδεδεμένοι με μια συγκεκριμένη εφαρμογή. Αυτός ο γράφος είναι πρωτεύον εργαλείο για πολλές εργασίες ενός διαχειριστή που εντάσσονται στο πεδίο της ανάλυσης της απόδοσης (performance analysis) και της ανάλυσης των αρχικών αιτίων (root cause analysis). Για παράδειγμα η ανίχνευση λανθασμένων εγκαταστάσεων (misconfiguration) ή διαδικτυακών επιθέσεων και ο σχεδιασμός για την επέκταση η μετατροπή των ΥΑΝ υποδομών. Η συλλογή των ιχνών των δεδομένων (traces) γίνεται μέσω του Netflow, μιας τεχνολογίας που μπορεί να συλλέξει σε πραγματικό χρόνο σε ένα δρομολογητή διάφορες πληροφορίες για δικτυακές ροές (network flows). Στη περίπτωσή μας συλλέγουμε χρονοσφραγίδες (timestamps) και τυπικές πληροφορίες ροών, όπως ΙΡ διευθύνσεις, δικτυακές πόρτες (ports) και πρωτόκολλα. Αρχικά γίνεται μια προεπεξεργασία των δεδομένων έτσι ώστε να βρεθούν όλα τα πιθανά ζεύγη δικτυακών ροών που μπορούν να αντιστοιχούν σε κάποια σχέση μεταξύ λειτουργικών μερών μιας υπηρεσίας.

11

Clients

116:7002

cs relationship ss relationship ms relationship

50:7004

Clients

Clients 56:53 Clients

Clients

Clients 116:7004 50:7002 16:88 2:111

2:755

1016:80 Clients

2:5353

56:659 91:728

1009:22 Clients

443:53 91:53

Clients 243:53

222:25 Clients

Clients

Clients 114:53

116:514 Clients

157:53

Clients

3538:80

24:1352

623:53

427:53

2:5353 218:53

Σχήμα 2 - Γράφος Υπηρεσιών

Βασιζόμενοι σε συγκεκριμένα βασικά και ευρέως αποδεκτά πρότυπα κατηγοριοποιούμε τα πιθανά ζεύγη σε περιπτώσεις μεταξύ πελάτη-εξυπηρετητή (client-server), εξυπηρετητή-εξυπηρετητή (server-server)και πολλών εξυπηρετητών σειριακά (multi server), όπως φαίνεται και στο Σχήμα 3. Στη συνέχεια ένας ασαφής μηχανισμός συμπερασμού (ΑΜΣ) αναπτύχθηκε με στόχο να αποφαίνεται αν ένα ζεύγος ροών αντιστοιχεί σε κάποια πραγματική σχέση και να μετράει το σθένος της. Η ασαφή λογική είναι κατάλληλη όχι μόνο για αυτή την ταξινόμηση, καθώς μπορεί να αποφαίνεται σωστά για την ύπαρξη ή μη κάποιας σχέσης, αλλά και κατά πόσο συχνή ή χαλαρή είναι η επικοινωνία μεταξύ των διαφόρων μερών. Ο ΑΜΣ αποτελείται από ένα μικρό σύνολο κανόνων ασαφούς λογικής (π.χ. 30 κανόνες), οι οποίοι παράγονται από έναν υβριδικό γενετικό αλγόριθμο με υψηλό ποσοστό ταξινόμησης (μεγαλύτερου του 90%).

12

Clients

DNSServ

WebServ

Clients

WebServ

DB

(b)

(a) WebServ

AppServ

DB

(c) Σχήμα 3 – Ζεύγη ροών: (a) πελάτη- εξυπηρετητή, (b) εξυπηρετητή-εξυπηρετητή, c) πολλών εξυπηρετητών σειριακά

Οι κύριες συνεισφορές αυτού του εργαλείου ανίχνευσης δικτυακών σχέσεων είναι οι ακόλουθες:   





Ανακαλύπτονται οι δικτυακές σχέσεις που αφορούν το λειτουργική δομή των εφαρμογών χωρίς να υπάρχει πρωτύτερη γνώση για αυτές. Η μέθοδος με ακρίβεια τις υφιστάμενες διασυνδέσεις και απεικονίζει σωστά το σθένος τους. Οι παραγόμενες λανθασμένες σχέσεις είναι λίγες. Ο υβριδικός γενετικός αλγόριθμος που χρησιμοποιείται για την εκπαίδευση έχει υψηλό ποσοστό ταξινόμησης και χρειάζεται μόνο ένα μικρό όγκο δεδομένων εκπαίδευσης για να παράγει τους κανόνες. Αυτή η διαδικασία μπορεί να γίνεται ξεχωριστά από την αξιολόγηση και εκτός γραμμής (offline). Επίσης μπορεί να εφαρμοστεί σε οποιοδήποτε είδος ΥΑΝ (ιδιωτικό ή δημόσιο) και για κάθε είδους εφαρμογές. Το σύνολο των κανόνων (ΑΜΣ) που χρειάζονται για την ταξινόμηση των σχέσεων και τη μέτρηση του σθένους τους είναι σχετικά μικρό. Συνήθως τριάντα κανόνες είναι αρκετοί για να ταξινομήσουν σωστά κάθε πιθανή δικτυακή σχέση. Επιπλέον λόγω του ότι η διαδικασία της αξιολόγησης δεν χρειάζεται πολύπλοκους υπολογισμούς μπορεί να εφαρμοστεί και σε πραγματικό χρόνο. Ο σχηματιζόμενος γράφος υπηρεσιών αποτελεί για τους διαχειριστές ενός υπολογιστικού κέντρου ένα χρήσιμο εργαλείο για περαιτέρω ανάλυση της κατάστασης του δικτύου ,των σφαλμάτων που συμβαίνουν και των επιπτώσεων τους, καθώς και για μελλοντικό σχεδιασμό απαραίτητων επεκτάσεων ή αλλαγών του συστήματος.

13

2.2 Αυτοματοποιημένη κατανομή πόρων σε περιβάλλον ΥΑΝ Το δεύτερο μέρος της παρούσας διατριβής (κεφάλαιο 3) ασχολείται με το θέμα της αυτοματοποιημένης κατανομής των υπολογιστικών πόρων ενός υπολογιστικού κέντρου ΥΑΝ σε ένα σύνολο εγκατεστημένων δικτυακών εφαρμογών. Η σύγχρονη τεχνολογία της εικονικοποίησης (virtualization technology) είναι ο κύριος παράγοντας για την «συστέγαση» (consolidation) πολλών κατανεμημένων υπηρεσιών σε υπολογιστικά κέντρα ΥΑΝ. Σε αυτά τα κέντρα οι πάροχοι προσφέρουν τις κατάλληλες υποδομές, από άποψη υλικού και εργαλείων ανάπτυξης λογισμικού, τις οποίες οι εφαρμογές χρησιμοποιούν δυναμικά και ανάλογα με τις απαιτήσεις τους. Οι δικτυακές υπηρεσίες θεωρούνται κατανεμημένες για δύο λόγους. Ο πρώτος είναι ότι αποτελούνται από αρκετά συστατικά μέρη, όπως δικτυακοί εξυπηρετητές (web servers) και βάσεις δεδομένων (databases), τα οποία εγκαθίστανται σε ξεχωριστά ΕΜ. Ο δεύτερος λόγος είναι ότι πανομοιότυπα αντίγραφα των υπηρεσιών είναι εγκατεστημένα σε ένα ή περισσότερα υπολογιστικά κέντρα ΥΑΝ. Η λειτουργία μιας υπηρεσίας περιγράφεται από ένα ΣΕΥ μεταξύ του παρόχου και του πελάτη, στο οποίο καθορίζονται συγκεκριμένες ονομαστικές τιμές - στόχοι για διάφορα δικτυακά μεγέθη που αντιστοιχούν στα επιθυμητά επίπεδα ποιότητας υπηρεσίας, ΠΥ (Quality of Service – QoS). Τα πιο συνηθισμένα δικτυακά μεγέθη που χρησιμοποιούνται είναι ο χρόνος απόκρισης των αιτημάτων (request response time) ο αριθμός των εξυπηρετημένων αιτήσεων (throughput) μέσα σε ένα σταθερό χρονικό διάστημα. Από τη μεριά του πελάτη, οι στόχοι είναι η επίτευξη προκαθορισμένων τιμών των δικτυακών μεγεθών και η εξασφάλιση ενός εγγυημένου επιπέδου ΠΥ με το ελάχιστο οικονομικό κόστος. Αυτοί οι στόχοι μπορεί να είναι αντίθετοι με τους στόχους του παρόχου, ο οποίος επιθυμεί το σύστημα διαχείρισης να κατανέμει βέλτιστα τους πόρους σε κάθε εγκατεστημένη υπηρεσία με τέτοιο τρόπο έτσι ώστε να επιτυγχάνονται οι στόχοι των δικτυακών μεγεθών, να διασφαλίζεται η διαθεσιμότητα των πόρων ανεξαρτήτως του εισερχόμενου φορτίου αιτήσεων και να ελαχιστοποιείται το λειτουργικό κόστος του κέντρο ΥΑΝ ( πχ. η κατανάλωση ηλεκτρικής ενέργειας). Πρέπει να αναφερθεί ότι το φορτίο αιτήσεων των δικτυακών εφαρμογών γενικά είναι απρόβλεπτο και να μεταβάλλεται αρκετές τάξης μεγέθους. Επίσης οι υφιστάμενοι υπολογιστικοί πόροι είναι πεπερασμένοι και υπόκεινται σε περιορισμούς. Όλοι οι προαναφερθέντες παράγοντες καθιστούν τη διαχείριση των υποδομών ΥΑΝ ένα ανοιχτό και ενδιαφέρον ερευνητικό θέμα. Όπως φαίνεται στο σχήμα 4 υπάρχουν δυο επίπεδα ελέγχου των δικτυακών εφαρμογών σε υποδομές ΥΑΝ. Πιο συγκεκριμένα το γενικό επίπεδο (global level) και το τοπικό επίπεδο (local level). Στο γενικό επίπεδο, οι ελεγκτές προσπαθούν είτε να μοιράσουν το φορτίο κάθε εφαρμογής μεταξύ των αντίγραφών της ή να εξισώσουν την απόδοση των υποκείμενων υπολογιστικών κέντρων ΥΑΝ σύμφωνα με ένα δείκτη απόδοσης (π.χ. χρησιμοποίηση επεξεργαστικής ισχύς). Οι ελεγκτές τοπικού επιπέδου ασχολούνται κυρίως με θέματα που αφορούν τη λειτουργία των εφαρμογών και των αντίστοιχων εξυπηρετητών στους οποίους είναι εγκατεστημένες. 14

Local Level

Local Level AC+RA

AC+RA

Site 1

Site 2  12 ,  22 ,  ,  2n

11 , 12 ,  , 1n

1 ,  2 ,,  n Global level

 13 ,  32 ,  ,  3n

 14 ,  42 ,  ,  4n

Local Level

Local Level

AC+RA

AC+RA

Site 4

Site 3 Σχήμα 4 – Αρχιτεκτονική Ελέγχου Υποδομών ΥΑΝ

Για παράδειγμα ο έλεγχος αποδοχής, ΕΑ, (admission control) ο οποίος αποδέχεται ή απορρίπτει αιτήσεις ανάλογα με το εισερχόμενο φορτίο και η κατανομή πόρων, ΚΠ, (resource allocation) που καθορίζει τους πόρους του κάθε ΕΜ (επεξεργαστική ισχύ, μνήμη και εύρος δικτύου) είτε αλλάζοντας το χρονικό ποσοστό που ανατίθεται σε μια εφαρμογή ο επεξεργαστής ή χρησιμοποιώντας το μηχανισμό δυναμικής διαβάθμισης της τάσης και της συχνότητας (Dynamic Voltage and Frequency Scaling) του επεξεργαστή έτσι ώστε να αλλάζει ο ρυθμός εξυπηρέτησης των αιτήσεων. Οι προτεινόμενες λύσεις στη βιβλιογραφία για τους ελεγκτές τοπικού επιπέδου χρησιμοποιούν θεωρία ουρών ή γραμμικά καταστατικά μοντέλα από τη θεωρία ελέγχου για τη μοντελοποίηση της δυναμικής λειτουργίας των εφαρμογών. Αυτές οι τεχνικές μοντελοποίησης συνδυάζονται με κλασσικές τεχνικές ελέγχου, όπως αναλογικοί – ολοκληρωτικοί – διαφορικοί ελεγκτές (Proportional – Integral – Derivative, PID), ή με μοντέρνες θεωρίες ελέγχου, όπως ο έλεγχος πρόβλεψης μοντέλου (model predictive control) ή ο προσαρμοστικός έλεγχος (adaptive control) ή 15

με θεωρία βελτιστοποίησης για να επιτύχουν τους στόχους του ΣΕΥ. Ωστόσο όλες οι παραπάνω τεχνικές έχουν κάποια μειονεκτήματα. Συνήθως οι περισσότερες από αυτές ασχολούνται είτε με το πρόβλημα του ΕΑ ή με το ΚΠ χωριστά. Αυτό μπορεί να οδηγήσει είτε σε υπερτροφοδότηση πόρων (overprovisioning) αν ο σχεδιασμός του καταμερισμού των πόρων έχει γίνει για το μέγιστο φορτίο ή ένα μεγάλο μέρος των εισερχόμενων αιτήσεων να απορρίπτεται επειδή η στατική ανάθεση υπολογιστικών πόρων ωα αδυνατεί να τις εξυπηρετήσει. Επίσης δεν εξετάζουν αν το επιλεγμένο σημείο λειτουργίας του συστήματος είναι εφικτό και δεν ικανοποιεί τους υφιστάμενους περιορισμούς του συστήματος. Τελικά οι περισσότερες μελέτες δεν παρέχουν καμία ανάλυση της ευστάθειας της προτεινόμενης λύσης. Στη παρούσα διδακτορική διατριβή παρουσιάζονται δυο διαφορετικοί ελεγκτές τοπικού επιπέδου, οι οποίοι επιλύουν τα παραπάνω μειονεκτήματα και παρέχουν εγγύηση της απόδοσης των δικτυακών εφαρμογών ανεξαρτήτως των μεταβολών του εισερχόμενου φορτίου και των περιορισμών του συστήματος. Πιο συγκεκριμένα:        

Ικανοποίηση των στόχων που καθορίζονται από το ΣΕΥ. Βέλτιστη χρήση των πόρων του συστήματος. Σχεδιασμός και υλοποίηση ελεγκτών που επιλύουν τα προβλήματα των ΕΑ και ΚΠ ταυτόχρονα . Ακριβής μοντελοποίηση της λειτουργίας των εφαρμογών. Ικανοποίηση των υπαρχόντων περιορισμών του συστήματος. Εύρεση εφικτών σημείων λειτουργίας του συστήματος βάσει του ΣΕΥ. Εγγύηση και ανάλυση της ευστάθειας του συστήματος. Εφαρμογή καινοτόμων μεθόδων από τη θεωρία ελέγχου.

Το ΕΑΚΠ (έλεγχος αποδοχής και κατανομή πόρων) είναι ένα αυτόνομο πλαίσιο (framework) μοντελοποίησης και ελέγχου, το οποίο παρέχει ακριβή μοντέλα και λύνει ενοποιημένα τα προβλήματα ΕΑ και ΚΠ των δικτυακών εφαρμογών που είναι συγκεντρωμένες σε υπολογιστικά κέντρα ΥΑΝ. Στόχος του ΕΑΚΠ είναι να μεγιστοποιεί την είσοδο των αιτήσεων των χρηστών στη παρεχόμενη υπηρεσία εκπληρώνοντας παράλληλα και τις προδιαγεγραμμένες απαιτήσεις ΠΥ. Μια ομάδα γραμμικών καταστατικών μοντέλων με επιπρόσθετες αβεβαιότητες έχει χρησιμοποιηθεί με στόχο να καλύψει τις μεταβολές του εισερχόμενου φορτίου και την ποσοτικοποίηση των μη γραμμικών όρων του συστήματος. Για το σχεδιασμό του δυικού ελεγκτή, χρησιμοποιήθηκε ένας ελεγκτής από τη θεωρία συνόλων (settheoretic controller) που μας δίνει τη δυνατότητα να διασφαλίζεται η ευστάθεια του συστήματος και η ικανοποίηση όλων των υπαρχόντων περιορισμών. Επίσης το ΕΑΚΠ έχει τη δυνατότητα επιλογής του σημείου λειτουργίας του συστήματος ανάμεσα από διαφορετικά εφικτά σημεία ισορροπίας (equilibrium points), τα οποία

16

System Constraints Feedback u Controller

LTI Models Equillibrium Point

Σχήμα 5 – Δομή του ΕΑΚΠ πλαισίου μοντελοποίησης και ελέγχου.

έχουν υπολογιστεί μέσω της επίλυσης ενός προβλήματος προγραμματισμού. Η δομή του ΕΑΚΠ φαίνεται στο σχήμα 5.

γραμμικού

Ο δεύτερος τοπικός ελεγκτής που παρουσιάζεται σε αυτή τη διατριβή είναι ένα αυτόνομο πλαίσιο (framework) μοντελοποίησης και ελέγχου κατανεμημένων δικτυακών εφαρμογών σε περιβάλλον ΥΑΝ, το οποίο λύνει συγχρόνως τα προβλήματα ΕΑ και ΚΠ με ενιαίο τρόπο. Πιο συγκεκριμένα, ένα γραμμικό παραμετρικά μεταβαλλόμενο (Linear Parameter Varying) καταστατικό μοντέλο χρησιμοποιείται για να περιγράψει τη δυναμική συμπεριφορά του συστήματος. Αυτό το είδος των γραμμικών καταστατικών μοντέλων είναι κατάλληλο για μοντελοποίηση της λειτουργίας των δικτυακών εφαρμογών επειδή αυτή εξαρτάται από διάφορες παραμέτρους όπως ο ρυθμός εξυπηρέτησης και ο ρυθμός των εισερχόμενων αιτήσεων. Αυτές οι παράμετροι μπορούν εύκολα να ενσωματωθούν στη συγκεκριμένη κατηγορία μοντέλων. Στη συγκεκριμένη περίπτωση σαν παράμετρο του μοντέλου χρησιμοποιούμε την πρόβλεψη του ρυθμού άφιξης των εισερχόμενων αιτήσεων που προκύπτει από ένα προβλεπτή Holt. Οι συνθήκες λειτουργίας των υπηρεσιών που μοιράζονται ένα σύνολο εξυπηρετητών αποφασίζονται σύμφωνα με διάφορα κριτήρια βελτιστοποίησης. Ένα σύνολο εφικτών ονομαστικών σημείων λειτουργίας υπολογίζεται με τέτοιο τρόπο έτσι ώστε να ικανοποιούνται οι προϋποθέσεις ΠΥ. Στο σχεδιασμό της στρατηγικής ελέγχου υπολογίζεται ένας καταστατικός ελεγκτής ανάδρασης (state feedback controller) και μια περιοχή ελκτικότητας (domain of attraction), τέτοια ώστε να εγγυάται η τοπική ασυμπτωτική ευστάθεια και η ικανοποίηση των υφιστάμενων περιορισμών του συστήματος για ολόκληρο το σύνολο των επιθυμητών εφικτών σημείων ισορροπίας. Η υλοποίηση του συγκεκριμένου ελεγκτή είναι εύκολη, καθώς η υπολογιστική πολυπλοκότητά του είναι μικρή και σε κάθε χρονικό διάστημα μόνο ένα πρόβλημα γραμμικού προγραμματισμού και επιλογής του σημείου λειτουργίας πρέπει να επιλύονται. Η συνολική δομή του αυτόνομου πλαισίου μοντελοποίησης και ελέγχου απεικονίζεται στο σχήμα 6.

17

System Constraints ~



 Predictor

Feedback u Controller

LPV Model Equillibrium Point

Σχήμα 6 – Δομή του αυτόνομου πλαισίου μοντελοποίησης και ελέγχου.

Οι κύριες ερευνητικές συνεισφορές των δυο προαναφερθέντων ελεγκτών τοπικού επιπέδου σε υποδομές ΥΑΝ στην παρούσα διδακτορική διατριβή συνοψίζονται στα παρακάτω σημεία:    



Ακριβής μοντελοποίηση της δυναμικής λειτουργίας δικτυακών υπηρεσιών που μοιράζονται τους πόρους ενός συνόλου εξυπηρετητών. Υψηλή απόδοση και ικανοποίηση των προκαθορισμένων στόχων ενός ΣΕΥ. Ικανοποίηση των φυσικών περιορισμών του συστήματος. Καθορισμός ενός συνόλου εφικτών ονομαστικών σημείων λειτουργίας του συστήματος, τα οποία επιλέγονται ανάλογα με τις παρούσες συνθήκες λειτουργίας. Εγγύηση της ευστάθειας του συστήματος.

3. Διάρθρωση της Διατριβής. Η διατριβή είναι οργανωμένη σε τέσσερα κεφάλαια. Στο πρώτο κεφάλαιο παρουσιάζεται μια γενική περιγραφή του τι είναι ΥΑΝ, δίνοντας κάποιους βασικούς ορισμούς και κάποια χαρακτηριστικά γνωρίσματα. Περιγράφεται ποιος είναι ο ρόλος των παρόχων, η αρχιτεκτονική των επιχειρηματικών και τεχνικών μοντέλων καθώς και η κατηγοριοποίηση των υποδομών ΥΑΝ ανάλογα με τη χρήση τους. Επίσης παρουσιάζονται συνοπτικά τα πιο σημαντικά τρέχοντα ερευνητικά προβλήματα σχετικά με τις ΥΑΝ. Το δεύτερο κεφάλαιο ασχολείται με την παρακολούθηση των κατανεμημένων δικτυακών υπηρεσιών. Αρχικά δίνονται όλες οι απαραίτητες θεμελιώδεις πληροφορίες και έννοιες της ασαφούς λογικής. Έπειτα δίνονται οι ορισμοί των ροών (flows), των γεγονότων (events), των ζευγών των ροών (flow pairs) και των ζευγών των γεγονότων (event pairs) που είναι απαραίτητοι για την ανάλυση των Netflow δικτυακών ιχνών και την προεπεξεργασία για την μετέπειτα αξιολόγησή τους από τον 18

ΑΜΣ. Μετά από αυτά τα προπαρασκευαστικά βήματα περιγράφεται αναλυτικά πως παράγεται ένα σύνολο ασαφών κανόνων από έναν υβριδικό γενετικό αλγόριθμο. Τέλος η απόδοση του ασαφούς μηχανισμού ταξινόμησης δοκιμάζεται σε πραγματικά ίχνη από ένα δίκτυο εταιρείας και μιας πανεπιστημιούπολης και τα αποτελέσματα δείχνουν ότι ο μηχανισμός ταξινόμησης είναι ακριβής και μπορεί να αναγνωρίσει τις υφιστάμενες λειτουργικές σχέσεις χωρίς να έχει καμία πρωτύτερη του δικτύου. Το τρίτο κεφάλαιο παρουσιάζει τους δύο ελεγκτές τοπικού επιπέδου που επιλύουν το πρόβλημα του καταμερισμού των πόρων της ΥΑΝ. Πρωτίστως δίνονται οι βασικοί ορισμοί μοντελοποίησης και θεωρίας ελέγχου και έπειτα μια αναλυτική περιγραφή των υπαρχουσών μεθόδων μαζί με τα πλεονεκτήματα και τα μειονεκτήματα τους. Ο πρώτος ελεγκτής που παρουσιάζεται είναι το ΕΑΚΠ. Εξηγείται πως προκύπτει και αναγνωρίζεται η ομάδα των γραμμικών καταστατικών μοντέλων και διατυπώνονται μαθηματικά οι περιορισμοί του συστήματος. Στη συνέχεια παρουσιάζονται τα κριτήρια και τα βήματα για τον υπολογισμό ενός εφικτού ονομαστικού σημείου λειτουργίας που εγγυάται την ικανοποίηση των περιορισμών. Μετά παρουσιάζεται ο σχεδιασμός του δυικού ελεγκτή και οι ειδικές ιδιότητες που προσδίδονται στο σύστημα κλειστού βρόχου. Τέλος παρουσιάζεται η απόδοση του πλαισίου ΕΑΚΠ σε ένα πραγματικό σύστημα και η σύγκρισή του με δυο άλλες γνωστές τεχνικές ελέγχου. Στο δεύτερο μέρος του κεφαλαίου παρουσιάζεται ο δεύτερος ελεγκτής τοπικού επιπέδου. Αρχικά περιγράφονται αναλυτικά το γραμμικό παραμετρικά μεταβαλλόμενο μοντέλο, οι υφιστάμενοι περιορισμοί του συστήματος και ο προβλεπτής της εισερχόμενης κίνησης. Στη συνέχεια παρουσιάζονται η διαδικασία υπολογισμού ενός συνόλου ονομαστικών σημείων λειτουργίας του συστήματος και ο σχεδιασμός της στρατηγικής ελέγχου με σκοπό την εγγύηση της ευστάθειας και της ικανοποίησης των περιορισμών του συστήματος σε κάθε περίπτωση. Στο τελευταίο μέρος του κεφαλαίου περιέχεται η υλοποίηση του ελεγκτή σε ένα πραγματικό περιβάλλον ΥΑΝ και η σύγκρισή του με έναν ελεγκτή πρόβλεψης μοντέλου. Στο τέταρτο κεφάλαιο υπογραμμίζονται όλα τα συμπεράσματα της διατριβής γύρω από την παρακολούθηση και τον έλεγχο των δικτυακών εφαρμογών σε πλατφόρμες ΥΑΝ. Επίσης παρουσιάζονται κάποιες απόψεις για τη συνέχιση της έρευνας μετά τη διατριβή. Τέλος στο παράρτημα παρουσιάζεται οι δημοσιεύσεις σε διεθνή συνέδρια και περιοδικά που προέκυψαν κατά τη διάρκεια της εκπόνησης της διδακτορικής διατριβής και η σχετική βιβλιογραφία.

19

Abstract 1. Description and Objectives of the Dissertation Cloud computing has recently emerged as a new paradigm for hosting and delivering services over the Internet. With the rapid development of processing and storage technologies and the success of the Internet, computing resources have become cheaper, more powerful and more available than ever before. These available resources (CPU, storage and network) are organized in data centers that are spread around the glove. This technological trend has enabled the realization of a new computing model called cloud computing, in which resources are provided as a product that can be leased by users through the Internet in an on-demand fashion. Cloud computing offers to customers many attractive arguments to shift their business and web applications from proprietary infrastructure to a cloud computing platforms. The most important incentives are the pay-as-you-go pricing model, the dynamic resource allocation, the high scalability, the reduction of the operating and maintenance expenses and the easy access from many different devices. Nowadays, a customer is not obliged to buy all the necessary hardware equipment of a private data center and pay for its operation and maintenance, but he can lease on-demand only the necessary computing and network resources to develop his applications and pay only for the usage. This means that he eliminates the cost of the electrical consumption, the hardware failures and the number of expertise in IT department. Cloud computing providers are separated in three categories. The providers are connected on a hierarchical layered model, where the provider of one layer is customer of the previous level provider. At the first layer of this model, Infrastructure as a Service (IaaS) providers offer all the necessary computing and network equipment, e.g. servers, storage devices and routers, switches, in terms of Virtual Machines (VMs). On the next level, Platform as a Service (PaaS) provider offer operating system support and software development frameworks. These two layers are usually called the utility computing layer. On the top level, Software as a Service (SaaS) providers develop on demand any application over the Internet. Web applications, which are deployed on cloud computing platforms, have different requirements in terms of operating cost, reliability and security. Thus some certain types of cloud have been emerged. Public clouds offer their resources as services to the general public. They provide all the essential computing and network resources but they usually lack control over data, network and security settings, which is an obstacle to deploy business services with strict requirements. Thus private or internal clouds have been designed for exclusive use by a single organization and it offers high level of control, reliability and security. Hybrid cloud is a mixture of public and private cloud models. It offers more flexibility than both public and private clouds and leverages the benefits both of them in order to fulfill the requirements of all services. 20

Finally Virtual Private cloud exploits the virtual private network (VPN) technology that allows service providers to design their own topology and security settings such as firewall rules on the top of public clouds. The above layered model of the providers and the underlying infrastructure provides the following important features to the cloud computing platforms:      

Geo-distribution of applications and ubiquitous network access from many different devices. Service oriented applications which operation is described by a Service Level Agreement (SLA) between the provider and the customer. Dynamic resource provisioning according to the current service demand. Shared pool of resources can be adaptively assigned to multiple resource consumers. Reduction of operating cost and risks such as electrical consumption and hardware failures. Pay-per-use pricing model.

Although the rapid development, cloud computing is currently at its early steps and there are many issues still to be addressed. Furthermore, new challenges continuously emerge from many industrial applications. The most important research problems of cloud computing are the following: 









Automated resource provisioning addresses the problem of allotting the computing resources to all applications in such a manner that their SLA requirements are satisfied and high agility is achieved towards rapid demand fluctuations such as in flash crowd effect. Virtual Machine migration is a technique, which moves the VM of an application in order to avoid hotspots and bottleneck of resources in a specific group of servers. Service consolidation is relative to the above problem and it concerns the maximization of the resource utilization while minimizing energy consumption in a cloud computing environment. Hosting many components of different applications on a server cluster allows using only the necessary number of servers and activating or deactivating them according to the service demand. Energy management addresses the problem of electrical consumption of data centers, which is not a negligible cost any more. Many providers set energy saving as one of their primary goal in the design of their capacity planning. Traffic monitoring and analysis is a challenging problem because of the huge volume of information data that need to be processed. Having an online view of what is going on in a data center is important in order to

21



  

avoid hardware failures, preserve high reliability and security and planning necessary expansions and changes of computing and network resources. Data security is another important research topic in cloud computing. The infrastructure provider must achieve confidentiality for secure data access and transfer and auditability for attesting whether security setting of applications has been tampered or not. Software frameworks should be developed for scheduling tasks and for scalable and fault-tolerant data processing. Storage technologies and data management for distributed processing of data-intensive tasks. Novel architectureσ which try to combine different technologies, such as cloud computing and peer-to-peer networks in order to achieve media delivery in a distributed fashion.

This thesis addresses two of the aforementioned research topics. Firstly a useful traffic monitoring technique is developed to unveil the underlying relationships between the various components of web services. Secondly automated resource provisioning problem is addressed using two different control theoretic solutions. Nowadays web applications are usually multi-tier, since they consist of many different components, and are mainly installed in data centers of private or public clouds. Due to the financial impact of SLAs, integrated management tools, which automatically monitor the operation and performance of consolidated services, are necessary in order to proactively handle any arising problem. A typical data center usually consolidates the components of thousand applications. These components interact in an unpredictable way in order to complete different types of transactions. Network administrators usually do not have a clear view of the services operation conditions. The current monitoring systems provide data and statistics of traffic conditions based on ports without further interpretation about how the service components collaborate. Relatively to the disadvantages of the studies in the literature, the thesis focuses on the creation of a tool for analysis of network traffic, which mines relationships between network and service components and measures their strength without having any prior knowledge of existing applications. The main contributions are the following: 

 

The creation of a service graph – overlay, which depicts the operational dependencies between the components of the distributed web services in a data center. Reliable identification of the underlying relationships without any a priori knowledge about them. Quantification of the relationship’s strength. 22

  

Easy and offline training procedure of the inference mechanism. Fast and online process of the traffic data. Small number of erroneously produced data.

This service overlay illustrates the structure of services and interconnections between network clients and servers or databases that are affiliated with a specific service. This overlay is vital for several administrative tasks, such as detecting misconfigurations or DoS attacks, monitoring service performance and design or expansion of the cloud computing infrastructure. NetFlow traces, collected from an enterprise network and a campus network of a university, include time information and typical flow attributes, i.e. IP addresses, ports and protocols. Initially a simple algorithm, based on very simple and realistic traffic patterns, is applied that reveals all possible dependencies. Then a flexible fuzzy inference mechanism (FIM) is developed, which determines whether the candidate relationship is true or not and measures its strength. The second part of our study is the design and training of FIM. FIM consists of a small set of fuzzy rules (i.e. 30 rules) produced by a hybrid genetic algorithm, which is a training process with high classification rate. The main contributions are the following:   



We discover relationships that correspond to the application structure, such as web and file servers, without any prior knowledge. The generated erroneous relationships are few and our approach measures accurately the strength of every dependency. The hybrid genetic algorithm has a high classification rate and needs only a small fraction of training data (10%). Also it is an offline process, applicable to different networks. The set of fuzzy rules is not necessarily large: 30 rules suffice to classify correctly all candidate relationships. Furthermore the classification by the fuzzy rules set can be done online.

The other research topic, which is addressed in this thesis, is automated resource provisioning of consolidated web services on cloud computing platforms. Modern virtualization technology is the key factor for the consolidation of many distributed web services into large data centers. Providers offer an execution platform with all the essential means (i.e hardware infrastructure and software tools) that applications can use on demand. Web services usually consist of many components such as web servers and databases, which are deployed on separate VMs. These VMs can be hosted within one machine or span across many machines. The performance of a web service is described in a SLA between the service provider and the customer; the Service Level Objectives (SLOs) are the network metrics, which prescribe exactly the desired Quality of Service (QoS) levels. The most common SLOs are the request response time and the served request rate during a fixed time interval. From the customer’s view, the goals are the satisfaction of some predefined nominal values of 23

the SLOs and a guaranteed level of QoS with the minimum financial cost. These objectives are contradictive to the goals of the provider, who aims for a management system that allocates the resources to each service optimally in a manner that achieves SLOs, ensures availability of resources under any workload circumstances and minimizes the operational cost (e.g. the consumed power energy). In addition, the workload of these web services is generally unpredictable and highly variant, while the available computing resources are finite and subject to constraints. All these factors make the management of the cloud computing infrastructure an open and challenging research field. There are two layers of control of web services, specifically, the local level and the global level. On the global level, controllers try to share the volume of requests among the replicas of an application or equalize the performance at each of the underlying data centers according to a performance indicator. Local level controllers mainly concern application specific or server specific problems, i.d. admission control (AC) that rejects requests under peak workload conditions and resource allocation (RA), which determines for each VM resources (CPU, memory, network bandwidth) either by changing their capacity or using the Dynamic Voltage and Frequency Scaling (DVFS) mechanism of servers to change their service rate. The existing solutions about local level controllers in literature use queuing theory or linear state space models from control theory to model the dynamic operation of the services and modern control techniques, such as PI or model predictive control (MPC), or optimization theory to achieve the SLA objectives. However they have some certain drawbacks. They usually address the problem of admission control and capacity planning separately. This can be lead to overprovisioning whether the capacity planning is design only for peak demand and a large fraction of incoming requests can be rejected because static resource allocation cannot serve them. They do not examine if the selected operating points are feasible and satisfy the underlying system constraints. Finally most of them do not provide any stability analysis of the proposed solutions. In the thesis two local level controllers are presented, which solve the above limitations and guarantee the performance of the application under the system constraints and any variation of their incoming workload. The main contributions of the thesis are:       

Satisfaction of the defined targets by the SLA. Optimal usage of the computational and network resources. Design and implementation of dual controllers that address admission control and resource allocation simultaneously. Accurate modeling of the web service operation. Satisfaction of the underlying system constraints. Computation of feasible operation points according to SLA. Guarantee and analysis of the system stability.

24



Application of novel methods of control theory to recent network research problems.

ACRA (Admission Control and Resource Allocation) is an autonomic modeling and control framework that provides accurate modeling, joint admission control and capacity allocation among different consolidated services on cloud computing data centers. The objective is to maximize the provider’s revenue (by maximizing admittance of customers to the provided service), while fulfilling QoS requirements. A group of linear state-space models with additive uncertainties is used in order to cover the variation of workload and quantify the system’s nonlinearities. For the design of the dual controller, a set-theoretic controller is used which provides stability and satisfaction of the underlying system constraints. Finally ACRA has the ability to adapt between several operating points, because it always assures that the system will be driven in the neighborhood of a feasible equilibrium point. The second local level controller of the thesis is an autonomous modeling and control framework that solves admission control and resource allocation simultaneously in a unified way. In specific, a Linear Parameter Varying (LPV) state space model is adopted to capture the dynamic behavior of the underlying infrastructure. The operating conditions are determined according to an optimization criterion. A feasible operating point, which satisfies the desired QoS nominal values, is computed. The resulting stabilizing state feedback control law is an affine state-dependent control law that can be easily implemented. The computational complexity of the controller implementation is small, since at every time instant only a linear program and a point location problem are solved. Finally, convergence to the feasible operating point and satisfaction of the system’s constraints are guaranteed, for a number of desired operating points of interest.

2. Structure of the Dissertation This dissertation is structured in four chapters. The first chapter is a general tutorial of what cloud computing is, giving some basic definitions and key features. The role of providers, the architecture of the business model and the categorization of cloud computing infrastructure are demonstrated. Also the most important open research problems are analyzed briefly in this chapter. The second chapter is dedicated to the monitoring of the consolidated wed applications. Initially the fundamental definitions and ideas of fuzzy logic are given. Then the definitions of flows, events, flow pairs and event pairs are given in order to process the data from NetFlow traces and prepare them for classification by FIM. After this preparation step, it is described in full details how the set if fuzzy rules are produced by a hybrid genetic algorithm. Finally the performance of this fuzzy logic classifier is tested on real traces from a private business network and the network of a university campus and the results illustrates that it is accurate and can clearly identify the underlying relationships without any previous knowledge of the network. 25

Third chapter presents the two local level controllers that address the problem of capacity provisioning. Firstly, all the necessary definitions of modeling and control theory are given. Then there is an analytical description of the existing methods and their pros and cons. The first presented controller is ACRA. It is described how the group of linear state space model is inferred and identified. Then the sets of system constraint are formulated and it is explained what are the criteria and the steps to compute a feasible operating point that guarantees the underlying constraints. After this step, it is illustrated the design and the special properties of the dual set-theoretic controller are illustrated. The evaluation section shows the performance of ACRA and a comparison with two other well-known controllers. Finally the components of the second local level controller are presented. Firstly, the LPV modeling, the underlying constraints and the request rate predictor are analytically described. Then the computation of a set of feasible operation point and the procedure of computing a control strategy, which guarantees the stability of the system and the satisfaction of the constraints, is explained. The last section of this chapter includes the evaluation of the controller on a real testbed and the comparison with a corresponding MPC method. The fourth chapter highlights all the conclusion of the dissertation about monitoring and control of web applications on cloud computing platforms. Also some aspects of possible future work are given. Appendix presents the bibliography and the dissertation publications in international journal and conferences.

26

Contents CHAPTER 1 – Cloud Computing. ....................................................................................... 31 1.1 Overview of cloud computing .................................................................................... 31 1.1.1 Definitions .......................................................................................................... 31 1.1.2 Characteristics .................................................................................................... 32 1.2 Providers and Business Model ................................................................................... 32 1.3 Types of Cloud and Cloud computing technologies .................................................... 34 1.3.1 Types of Cloud ................................................................................................... 34 1.3.2 Architectural design of data centers ..................................................................... 35 1.3.3 Distributed File System ....................................................................................... 37 1.3.4 Distributed Application Framework .................................................................... 37 1.4 Key Features ............................................................................................................. 38 1.5 Research Challenges .................................................................................................. 40 1.5.1 Automated Resource Provisioning ...................................................................... 40 1.5.2 Virtual Machine Migration .................................................................................. 41 1.5.3 Service Consolidation ......................................................................................... 41 1.5.4 Energy Management ........................................................................................... 42 1.5.5 Traffic Management and Analysis ....................................................................... 42 1.5.6 Data Security ...................................................................................................... 43 1.5.6 Software Frameworks ......................................................................................... 43 1.5.7 Storage technologies and Data Management ........................................................ 44 1.5.8 Novel Architectures ............................................................................................ 44 CHAPTER 2 – Monitoring of Distributed Cloud Services. .................................................. 46 2.1 Fuzzy Logic............................................................................................................... 48 2.1.1 Linguistic Variables ............................................................................................ 49 2.1.2 Membership Functions ........................................................................................ 49 2.1.3 Set of Fuzzy Rules and Inference Mechanism ..................................................... 50 2.1.4 Defuzzification ................................................................................................... 51 2.2 Mining Network Relationships .................................................................................. 51 2.2.1 Pre-processing Data ............................................................................................ 52 2.2.2 Flows and Events ................................................................................................ 52 2.2.3 Flow and Event Pairs .......................................................................................... 53 2.2.4 Confidence Variables .......................................................................................... 55 2.2.5 Relationship Discovery ....................................................................................... 57

27

2.3 Fuzzy Inference Mechanism ...................................................................................... 58 2.4 Evaluation ................................................................................................................. 61 2.4.1 Simulations ......................................................................................................... 61 2.4.2 Real Data ............................................................................................................ 63 2.5 Conclusions ............................................................................................................... 67 CHAPTER 3 – Control of Distributed Web Services on Cloud Computing Infrastructure .... 70 3.1 Basic Definitions of Modeling and Control Theory .................................................... 76 3.1.1 Linear Time Invariant State Space Models .......................................................... 76 3.1.2 Linear Parameter Varying State Space Models .................................................... 77 3.1.3 Stability .............................................................................................................. 78 3.2 ACRA: A Unified Admission Control and Resource Allocation Framework for Virtualized Environments ................................................................................................ 81 3.2.1 Modeling and System Identification .................................................................... 82 3.2.2 State and Input Constraints.................................................................................. 84 3.2.3 Determination of the Equilibrium Point ............................................................... 85 3.2.4 ACRA Controller Design .................................................................................... 85 3.2.5 Evaluation .......................................................................................................... 88 3.3 An Autonomous Admission Control and Resource Allocation Framework for Consolidated Web Applications ....................................................................................... 96 3.3.1 Modeling and Identification ................................................................................ 97 3.3.2 Request Rate Predictor ...................................................................................... 100 3.3.3 State and Input Constraints................................................................................ 101 3.3.4 Determination of the Operating Point ................................................................ 101 3.3.5 Controller Design.............................................................................................. 102 3.3.6 Evaluation ........................................................................................................ 109 CHAPTER 4 – Conclusions and Future Work. .................................................................. 118 4.1 Conclusions ............................................................................................................. 118 4.2 Future Work ............................................................................................................ 119 Publications ...................................................................................................................... 122 Bibliography ..................................................................................................................... 124

28

Lists of figures Figure 1.1 – Business Model of Cloud Computing................................................... 33 Figure 1.2 Layered Architecture of Data Center....................................................... 36 Figure 2.1 – Structure of a Fuzzy Logic System....................................................... 48 Figure 2.2 – Memebership Functions: (a) singleton, (b) triangular, (c) trapezoidal, (d) Gaussian shape ........................................................................................................ 49 Figure 2.3 – Membership Functions of a Linguistic Variable ................................... 50 Figure 2.4 – Defuzzification of a Fuzzy Logic System ............................................. 51 Figure 2.5 – Relationship Discovery System............................................................ 52 Figure 2.6 – Flow Pairs: (a) cs flow pair, (b) ss flow pair, (c) ms flow pair .............. 55 Figure 2.7 – Training: Average Classification Rate of Simulations .......................... 62 Figure 2.8 – Evaluation: Average Classification Rate of Simulations ....................... 63 Figure 2.9 – Real Data: Average Classification Rate of Training Process................. 65 Figure 2.10 - Real Data: Evaluation of the produced FIMs ...................................... 65 Figure 2.11 – Graph of Relationships ...................................................................... 68 Figure 3.1 – Control Architecture of Cloud Computing Infrastructure ..................... 71 Figure 12 – Structural Diagram of ACRA Framework ............................................. 82 Figure 3.3 – Group of LTI Models ........................................................................... 89 Figure 3.4 –State and Input Constraints set Sz, Sv(z), Positevely Invariant Set Δ, Target Set R........................................................................................................................ 91 Figure 3.5 - Average Response Time of ACRA, PI, MPC........................................ 92 Figure 3.6 – Reference and Admitted Request Rate ................................................. 93 Figure 3.7 – Performance of MPC, PI controllers using a single LTI Model ............ 94 Figure 3.8 – Average Response Time and VMs Capacity Allocation of ACRA ....... 95 Figure 3.9 – System Architecture............................................................................. 96 Figure 3.10 – Structural Diagram of Autonomous Framework. ................................ 97 Figure 3.11 - DoAs Ri of each operating point xref,I , i=1,…,N.................................106 Figure 3.12 - Tested Diagram – VMi,j of ith application on jth server. .......................109 Figure 3.13 - Overall Performance of Consolidated Applications. ...........................112 Figure 3.14 - DoA of operating points (red lines and circles), DoA of the desired operating point (blue line and square) and System Trajectory (black solid line). .....113 Figure 3.15 – Alternative solution of point location problem. .................................113 Figure 3.16 - Comparison with MPC Controller .....................................................115

29

Glossary of Acronyms IT IaaS PaaS SaaS VM VPC VPN MDC GFS HDFS SLA CRM SLO QoS ISP TPM FIM MIMO MISO AppList GA HGA AC RA DVFS LTI LPV MPC DoA LF ACRA RLS PI BFR

Information Technology Infrastructure as a Service Platform as a Service Software as a Service Virtual Machine Virtual Private Cloud Virtual Private Network Modular Data Center Google File System Hadoop Distributed File System Service Level Agreement Customer relationship Management Service Level Objective Quality of Service Internet Service Provider Trusted Platform Module Fuzzy Inference Mechanism Multi-Input Multi-Output Multi-Input Single-Output Application List Genetic Algorithm Hybrid Genetic Algorithm Admission Control Resource Allocation Dynamic Voltage and Frequency Scaling Linear Time Invariant Linear Parameter Varying Model Predictive Control Domain of Attraction Lyapunov Function Admission Control and Resource Allocation Recursive Least Square Proportional Integral Best Fit Rate

30

CHAPTER 1 – Cloud Computing. Cloud Computing is associated with a new paradigm for provisioning services over the Internet. Cloud computing becomes attractive since it eliminates the requirements for users to plan ahead for provisioning, and allows enterprises to start from the minimum and increase resources only when there is a rise on service demand. Available resources (e.g., CPU, storage and network) are provided as general utilities that can be leased and released by users through the Internet in an on-demand fashion. In a cloud computing environment, the provider is separated in the following two roles. The infrastructure providers manage cloud platforms and lease resources according to a usage-based pricing model to service providers, who rent resources from one or many infrastructure providers to serve the end users. The emergence of cloud computing has made a tremendous impact on the Information Technology (IT) industry over the past few years, where large companies such as Google, Amazon and Microsoft compete to provide more powerful, reliable and cost-efficient cloud platforms, and business enterprises seek to reshape their business models to gain benefit from this new paradigm. This rest of this chapter presents a summary of the most important characteristics, models and research challenges of the existing cloud computing platforms. The following section demonstrates an overview of cloud computing, including its definition and the most dominant features. Section 1.2 describes the business and the operational models of cloud computing. Section 1.3 includes the existing types of clouds and the architectural design of data centers. Section 1.4 summarizes the most important key features of the cloud computing infrastructures. Finally the last section includes the research challenges that have not been fully addressed yet.

1.1 Overview of cloud computing 1.1.1 Definitions There are many definitions of cloud computing, but they all seem to focus on just certain aspects of the technology. We adopt the definition of cloud computing provided by the National Institute of Standards and Technology (NIST) [1] as it covers opinion all the essential aspects of cloud computing: Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. The NIST definition lists five essential characteristics of cloud computing: on-demand self-service, broad network access, resource pooling, rapid elasticity or expansion, 31

and measured service. It also lists three "service models" (software, platform and infrastructure), and four "deployment models" (private, public, hybrid and virtual private) that together categorize ways to deliver cloud services. The definition is a mean for broad comparisons of cloud services and deployment strategies, and provides a baseline for discussion from what is cloud computing to how to optimally use cloud computing. In [2], authors compared over twenty different definitions from a variety of sources to confirm a standard definition. 1.1.2 Characteristics Cloud computing provides several different characteristics that attract many customers to shift from traditional service computing to this new type of infrastructure. The most important are summarized below: 







No fixed investment: Cloud computing uses a pay-as-you-go pricing model. A service provider does not need to invest in the infrastructure to start gaining benefit from cloud computing. He simply rents resources from the infrastructure provider according to his own needs and pays only for the usage. Easy access: Services hosted in the cloud are generally web-based. Therefore, they are easily accessible through any device with Internet connections, such as desktop and laptop computers or cell phones and PDAs. Reducing operating cost: Resources in a cloud environment can be rapidly allocated and de-allocated on demand. Hence, a service provider no longer needs to allocate resources according to the peak load. This provides huge savings since resources can be released to save on operating costs when service demand is low. Highly scalable: Infrastructure providers offer large amount of resources from data centers and make them easily accessible. A service provider can easily expand its service to large scales in order to handle rapid fluctuations of service demands (e.g., flash-crowd effect).

1.2 Providers and Business Model Cloud computing adopts a layered business model [3]. The services of each layer are offered by a different provider and every layer is considered as a customer of the previous layer. In practice, cloud computing services can be grouped into three categories: Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and Software as a Service (SaaS), 

Infrastructure as a Service: IaaS refers to on-demand provisioning of infrastructural resources, usually in terms of Virtual Machines (VMs). The cloud owner who provides IaaS is called an IaaS provider. An example of IaaS provider is Amazon EC2 [4]. 32





Platform as a Service: PaaS refers to providing platform layer resources, including operating system support and software development frameworks. Examples of PaaS providers are Google App Engine [5], Microsoft Windows Azure [6] and Force.com [7]. Software as a Service: SaaS refers to providing on demand applications over the Internet. Examples of SaaS providers are Salesforce.com [7], Rackspace [8].

Figure 1.1 depicts the architecture of the layered business model. The PaaS provider runs its cloud on top of an IaaS provider’s cloud. However, usually IaaS and PaaS providers are often parts of the same organization. This is the reason that PaaS and IaaS providers are often called the infrastructure or cloud providers. In more details, the IaaS providers are responsible for managing the physical resources of the cloud, including physical servers, routers, switches, power and cooling system that are typically located in data centers. A data center usually contains thousands of servers that are organized in racks and interconnected through switches, routers or other fabrics. Usually provides exploit the benefits of virtualization at this level. Virtualization technology, such as Xen [9] and VMware [10] abstracts away the details of physical hardware and provides dynamic virtualized

End User Web Interface

SaaS Utility Computing

PaaS

IaaS

Figure 1.1 – Business Model of Cloud Computing

33

resources for high-level applications. A virtualized server is commonly called a virtual machine (VM). Virtualization is a fundamental element of cloud computing, as it provides the capability of pooling computing resources from clusters of servers and dynamically assigning or reassigning virtual resources to applications on-demand. Typical issues at this layer include hardware configuration, fault tolerance, traffic management, power and cooling resource management. The platform layer is built on top of the infrastructure layer and it consists of operating systems and application frameworks. The purpose of this layer is to minimize the burden of deploying applications directly into VM containers. Finally at the highest level of the hierarchy, the SaaS providers offer the actual cloud applications. Different from traditional applications, cloud applications can leverage the automatic-scaling feature to achieve better performance, availability and lower operating cost.

1.3 Types of Cloud and Cloud computing technologies 1.3.1 Types of Cloud There are many issues to consider when moving an enterprise application to the cloud environment. Some service providers are mostly interested in lowering operation cost, while others may prefer high reliability and security. Accordingly, there are different types of clouds, each with its own pros and cons: 





Public clouds: A cloud in which service providers offer their resources as services to the general public. They offer several key benefits to service providers, including no initial capital investment on infrastructure and shifting of risks to infrastructure providers. However, public clouds lack fine-grained control over data, network and security settings, which hampers their effectiveness in many business scenarios. Private (internal) clouds: are designed for exclusive use by a single organization. A private cloud may be built and managed by the organization or by external providers. A private cloud offers the highest degree of control over performance, reliability and security. Hybrid clouds: A hybrid cloud is a combination of public and private cloud models that tries to address the limitations of each approach. In a hybrid cloud, part of the service infrastructure runs in private clouds while the remaining part runs in public clouds. Hybrid clouds offer more flexibility than both public and private clouds. Specifically, they provide tighter control and security over application data compared to public clouds, while still facilitating on-demand service expansion and contraction. On the contrary, designing a hybrid cloud requires carefully determining the best split between public and private components.

34



Virtual Private Cloud: An alternative solution to addressing the limitations of both public and private clouds is called Virtual Private Cloud (VPC). A VPC is a platform running on top of public clouds. The main difference is that a VPC leverages virtual private network (VPN) technology that allows service providers to design their own topology and security settings such as firewall rules. VPC is essentially a more holistic design since it not only virtualizes servers and applications, but also the underlying communication network as well. Additionally, for most companies, VPC provides seamless transition from a proprietary service infrastructure to a cloud-based infrastructure, owing to the virtualized network layer.

Most service providers select the right cloud model according to the business scenario. For example, computation-intensive scientific applications are best deployed on public clouds for cost-effectiveness. Arguably, certain types of clouds will be more popular than others. In particular, it was predicted that hybrid clouds will be the dominant type for most organizations. However, virtual private clouds have started to gain more popularity. 1.3.2 Architectural design of data centers A data center contains thousands of devices like servers, switches and routers. Proper planning of this network architecture is critical, as it will heavily influence applications performance and throughput in such a distributed computing environment. Furthermore, scalability and resiliency features need to be carefully considered. Currently, a layered approach is the basic foundation of the network architecture design, which has been tested in some of the largest deployed data centers. The basic layers of a data center consist of the core, aggregation, and access layers, as shown in Figure 1.2. The access layer is where the servers in racks physically connect to the network. There are typically 20 to 40 servers per rack, each connected to an access switch with a 1 Gbps link. Access switches usually connect to two aggregation switches for redundancy with 10 Gbps links. The aggregation layer usually provides important functions, such as domain service, location service, server load balancing, and more. The core layer provides connectivity to multiple aggregation switches and provides a resilient routed fabric with no single point of failure. The core routers manage traffic into and out of the data center. A popular practice is to leverage commodity Ethernet switches and routers to build the network infrastructure. In different business solutions, the layered network infrastructure can be elaborated to meet specific business challenges. Basically, the design of the data center network architecture should meet the following objectives [11], [12], [13]:

35

Internet

Core

…..

…..

Aggregation

Access

Figure 1.2 Layered Architecture of Data Center







Uniform high capacity: The maximum rate of a server to server traffic flow should be limited only by the available capacity on the network-interface cards of the sending and receiving servers, and assigning servers to a service should be independent of the network topology. It should be possible for an arbitrary host in the data center to communicate with any other host in the network at the full bandwidth of its local network interface. Free VM migration: Virtualization allows the entire VM state to be transmitted across the network and migrate a VM from one physical machine to another. A cloud computing hosting service may migrate VMs for statistical multiplexing or dynamically changing communication patterns to achieve high bandwidth for tightly coupled hosts or to achieve variable heat distribution and power availability in the data center. The communication topology should be designed so as to support rapid virtual machine migration. Resiliency: Failures will be common at scale. The network infrastructure must be fault-tolerant against various types of server failures, link outages, or server-rack failures. Existing unicast and multicast communications should not be affected to the extent allowed by the underlying physical connectivity. 36

 

Scalability: The network infrastructure must be able to scale to a large number of servers and allow for incremental expansion. Backward compatibility: The network infrastructure should be backward compatible with switches and routers running Ethernet and IP. Because existing data centers have commonly leveraged commodity Ethernet and IP based devices, they should also be used in the new architecture without major modifications.

Another area of rapid innovation in the industry is the design and deployment of shipping-container based, modular data center (MDC). In an MDC, normally up to a few thousands of servers, are interconnected via switches to form the network infrastructure. Highly interactive applications, which are sensitive to response time, are suitable for geo-diverse MDC placed close to major population areas. The MDC also helps with redundancy because not all areas are likely to lose power, experience an earthquake at the same time. 1.3.3 Distributed File System Google File System (GFS) [14] is a proprietary distributed file system developed by Google and specially designed to provide efficient, reliable access to data using large clusters of commodity servers. Files are divided into chunks of 64 megabytes, and are usually appended to or read and only extremely rarely overwritten or shrunk. Compared with traditional file systems, GFS is designed and optimized to run on data centers to provide extremely high data throughputs, low latency and survive individual server failures. Inspired by GFS, the open source Hadoop Distributed File System (HDFS) [15] stores large files across multiple machines. It achieves reliability by replicating the data across multiple servers. Similarly to GFS, data is stored on multiple geo-diverse nodes. The file system is built from a cluster of data nodes, each of which serves blocks of data over the network using a block protocol specific to HDFS. Data is also provided over HTTP, allowing access to all content from a web browser or other types of clients. Data nodes can talk to each other to rebalance data distribution, to move copies around, and to keep the replication of data high. 1.3.4 Distributed Application Framework HTTP-based applications usually conform to some web application framework such as Java EE. In modern data center environments, clusters of servers are also used for computation and data-intensive jobs such as financial trend analysis, or film animation. MapReduce [16] is a software framework introduced by Google to support distributed computing on large data sets on clusters of computers. MapReduce consists of one Master, to which client applications submit MapReduce jobs. The Master pushes 37

work out to available task nodes in the data center, striving to keep the tasks as close to the data as possible. The Master knows which node contains the data, and which other hosts are nearby. If the task cannot be hosted on the node where the data is stored, priority is given to nodes in the same rack. In this way, network traffic on the main backbone is reduced, which also helps to improve throughput, as the backbone is usually the bottleneck. If a task fails or times out, it is rescheduled. If the Master fails, all ongoing tasks are lost. The Master records what it is up to in the file system. When it starts up, it looks for any such data, so that it can restart work from where it left off. The open source Hadoop MapReduce project [17] is inspired by Google’s work. Currently, many organizations are using Hadoop MapReduce to run large dataintensive computations. Finally Table 1.1 demonstrates the main features of the most representative commercial products. Cloud Product

Microsoft Windows Azure

Amazon Elastic Compute Cloud (EC2)

Google Application Engine

Salesforce.com

Class of Provider

PaaS

IaaS

PaaS

PaaS

Applications

Compute & Storage

Compute & Storage

Web Applications

Storage & CRM

Virtualization

OS level

OS level

Application Container

CRM solution

User Access

Web –based Administration & SDK Tools

Amazon EC2 Command-line Tools

Web –based Administration Console

Web Interface, App

Supported Frameworks

.NET Services, SQL, Dynamic CRM, Sharepoint Services

Linux based Amazon Machine Image (AMI)

Python

Apex, Scontrols

Table 1.1 – A survey of the main features of commercial cloud products

1.4 Key Features Cloud computing provides different features from the traditional service computing. The most important features of the infrastructure and service providers are summarized below,

38















Geo-distribution and ubiquitous network access: Clouds are generally accessible through the Internet and use it as a service delivery network. Any device with Internet connectivity is able to access cloud services. Furthermore, in order to achieve high network performance and localization, clouds consist of data centers located at many locations around the world. Thus a service provider can easily leverage geodiversity to achieve maximum service utility. Service oriented: In a cloud, each IaaS, PaaS and SaaS provider offers his service according to the Service Level Agreement (SLA) negotiated with his customers. SLA assurance is therefore a critical objective of every provider. Multi-tenancy: In a cloud environment, services owned by multiple providers are co-located in a single data center. The performance and management issues of these services are shared among service providers and the infrastructure provider. The owner of each layer only needs to focus on the specific objectives associated with this layer. However, multitenancy also introduces difficulties in understanding and managing the interactions among various stakeholders. Shared resource pooling: The infrastructure provider offers a pool of computing resources that can be dynamically allocated to multiple consumers. Such dynamic resource assignment capability provides much flexibility to infrastructure providers for managing their own resource usage and operating costs. For instance, an IaaS provider can leverage VM migration technology to succeed in a high degree of server consolidation, hence maximizing resource utilization while minimizing cost such as power consumption and cooling. Dynamic resource provisioning: One of the key features of cloud computing is that computing resources can be obtained and released in an on demand fashion. Compared to the traditional model that adopt static resource provisioning according to peak demand, dynamic resource allocation allows service providers to acquire resources based on the current demand, which can significately lower the operating cost. Self-organizing: Since resources can be allocated on demand, service providers are responsible to manage their resource consumption according to their own needs. Furthermore, the automated resource management feature yields high agility that enables service providers to respond quickly to rapid variations of service demand such as the flash crowd effect. Reducing business risks and maintenance expenses: By outsourcing the service infrastructure to the clouds, a service provider shifts its business risks, i.e. hardware failures, to infrastructure providers, who often have better expertise and are better equipped for managing these risks. In addition, a service provider can cut down the hardware maintenance and the staff training costs. 39



Utility-based pricing: Cloud computing employs a pay-per-use pricing model. The exact pricing scheme may vary from service to service. For example, a SaaS provider may lease a VM from an IaaS provider on a perhour basis. On the contrary, a SaaS provider that offers on demand customer relationship management (CRM) may charge its customers based on the number of clients it serves. Utility-based pricing lowers service operating cost as it charges customers on a per-use basis. However, it also introduces complexities in controlling the operating cost.

1.5 Research Challenges Although cloud computing is very promising and well-established in industry, there are many challenging issues for the research community which have not been fully addressed yet. Furthermore, new challenges continuously emerge from many industrial applications. This section includes the most interesting research topics of cloud computing field. 1.5.1 Automated Resource Provisioning One of the key features of cloud computing is the capability of allotting and releasing resources on demand. The objective of a service provider is to allocate resources from the cloud to satisfy its service level objectives (SLOs), while minimizing its operational cost. In particular, it is not easy to determine online how to map SLOs such as Quality of Service (QoS) requirements to low-level resource requirement such as CPU and memory requirements and concurrently achieve high agility and response to rapid demand fluctuations such as in flash crowd effect. Automated service provisioning is not a new problem. Dynamic resource provisioning for Internet applications has been studied extensively in the past [18], [19]. These approaches typically involve, 1. Constructing an application performance model that predicts the number of application instances required to handle demand at each particular level, in order to satisfy QoS requirements. 2. Periodically predicting future demand and determining resource requirements using the performance model. 3. Automatically allocating resources using the predicted resource requirements. Application performance model adopts various techniques, such as Queuing theory [18], Control theory [34] and Statistical Machine Learning [35]. Additionally, there is a distinction between proactive and reactive resource control. The proactive approach uses predicted demand to periodically allocate resources before they are needed. On the other hand, the reactive approach addresses immediate demand fluctuations before periodic demand prediction is available. Both approaches

40

are important and necessary for effective resource control in dynamic operating environments. 1.5.2 Virtual Machine Migration Virtualization can provide important benefits in cloud computing by enabling virtual machine migration to balance load across the data center. In addition, VM migration enables robust and highly responsive provisioning in data centers. Virtual machine migration has evolved from process migration techniques [20]. More recently, Xen and VMWare have implemented “live” migration of VMs that involves extremely short downtimes ranging from tens of milliseconds to a second. It is [21] pointed out that migrating an entire OS and all of its applications as one unit allows to avoid many of the difficulties faced by process level migration approaches, and analyzed the benefits of live migration of VMs. The major benefits of VM migration are to avoid hotspots. However, this is not straightforward. Currently, detecting workload hotspots and initiating a migration lacks the ability to respond to sudden workload changes. Moreover, the in memory state should be transferred consistently and efficiently, with integrated consideration of resources for applications and physical servers. 1.5.3 Service Consolidation Service consolidation is an effective technique to maximize resource utilization while minimizing energy consumption in a cloud computing environment. Live VM migration technology is often used to consolidate VMs hosted by multiple underutilized servers onto a single server, so that the remaining servers can be set to an energy-saving state. The problem of optimally consolidating servers in a data center is often formulated as a variant of the vector bin-packing problem [22], which is an NPhard optimization problem. Various heuristics have been proposed for this problem [23], [24]. Additionally, dependencies among VMs, such as communication requirements, have also been considered recently [25]. However, service consolidation activities should not deteriorate application performance. It is known that the resource usage (also known as the footprint [26]) of individual VMs may vary over time [27]. For server resources that are shared among VMs, such as CPU, bandwidth, memory cache and disk I/O, maximally consolidating a server may result in resource congestion when a VM changes its footprint on the server [28]. Hence, it is sometimes important to observe the fluctuations of VM footprints and use this information for effective service consolidation. Finally, the system must quickly react to resource congestions when they occur [27].

41

1.5.4 Energy Management Improving energy efficiency is another major issue in cloud computing. It has been estimated that the cost of powering and cooling accounts for 53% of the total operational expenditure of data centers [29]. In 2006, data centers in the US consumed more than 1.5% of the total energy generated in that year, and the percentage is projected to grow 18% annually [23]. Hence infrastructure providers are under enormous pressure to reduce energy consumption. The goal is not only to reduce energy cost in data centers, but also to meet government regulations and environmental standards. Designing energy-efficient data centers has recently received considerable attention (Green Computing). This problem can be approached from several directions. For example, energy efficient hardware architecture that enables slowing down CPU speeds and turning off partial hardware components [30] has become common place. Energy-aware job scheduling [31] and service consolidation [24] are two other ways to reduce power consumption by turning off unused machines. Recent research has also begun to study energy-efficient network protocols and infrastructures [32]. A key challenge in all the above methods is to achieve a good trade-off between energy savings and application performance. In this respect, few researchers have recently started to investigate coordinated solutions for performance and power management in a dynamic cloud environment [33]. 1.5.5 Traffic Management and Analysis Analysis of data traffic is important for today’s data centers. For example, many web applications rely on analysis of traffic data to optimize customer experiences. Network operators also need to know how traffic flows through the network in order to take many of the management and planning decisions. There are several challenges for existing traffic measurement and analysis methods in Internet Service Providers (ISPs) networks and enterprise to extend to data centers. Firstly, the density of links is much higher than that in ISPs or enterprise networks, which deteriorates the scenario for existing methods. Secondly, most existing methods can compute traffic matrices between a few hundred end hosts, but even a modular data center can have several thousand servers. Finally, existing methods usually assume some flow patterns that are reasonable in Internet and enterprises networks, but the applications deployed on data centers, such as MapReduce jobs, significantly change the traffic pattern. Further, there is tighter coupling in application’s use of network, computing, and storage resources, than what is seen in other settings. Currently, there is not much work on measurement and analysis of data center traffic. Greenberg et al. [12] report data center traffic characteristics on flow sizes and concurrent flows, and use these to guide network infrastructure design. Benson et al. [36] perform a complementary study of traffic at the edges of a data center by examining SNMP traces from routers. 42

1.5.6 Data Security Data security is another important research topic in cloud computing. Since service providers typically do not have access to the physical security system of data centers, they must rely on the infrastructure provider to achieve full data security. Even for a virtual private cloud, the service provider can only specify the security setting remotely, without knowing whether it is fully implemented. Thus the infrastructure provider must achieve the following objectives: 1. Confidentiality for secure data access and transfer. 2. Auditability for attesting whether security setting of applications has been tampered or not. Confidentiality is usually achieved using cryptographic protocols, whereas auditability can be achieved using remote attestation techniques. Remote attestation typically requires a trusted platform module (TPM) to generate non-forgeable system summary (i.e. system state encrypted using TPM’s private key) as the proof of system security. However, in a virtualized environment like the clouds, VMs can dynamically migrate from one location to another; thus directly using remote attestation is not sufficient. In this case, it is critical to build trust mechanisms at every architectural layer of the cloud. Firstly, the hardware layer must be trusted using hardware TPM. Secondly, the virtualization platform must be trusted using secure virtual machine monitors [37]. VM migration should only be allowed if both source and destination servers are trusted. Recent work has been devoted to designing efficient protocols for trust establishment and management [38], [37]. 1.5.6 Software Frameworks Cloud computing provides a platform for hosting large-scale data-intensive applications. Typically, these applications leverage MapReduce frameworks such as Hadoop for scalable and fault-tolerant data processing. Recent work has shown that the performance and resource consumption of a MapReduce job is highly dependent on the type of the application [39], [40], [41]. For instance, Hadoop tasks such as sort are I/O intensive, whereas “grep” requires significant CPU resources. Furthermore, the VM allocated to each Hadoop node may have heterogeneous characteristics. For example, the bandwidth available to a VM is dependent on other VMs collocated on the same server. Hence, it is possible to optimize the performance and cost of a MapReduce application by carefully selecting its configuration parameter values [39] and designing more efficient scheduling algorithms [40], [41]. By mitigating the bottleneck resources, execution time of applications can be significantly improved. The key challenges include performance modeling of Hadoop jobs (either online or offline), and adaptive scheduling in dynamic conditions. Another related approach argues for making MapReduce frameworks energy-aware [41]. The essential idea of this approach is to turn Hadoop node into sleep mode when 43

it has finished its job while waiting for new assignments. To do so, both Hadoop and HDFS must be made energy-aware. Furthermore, there is often a trade-off between performance and energy-awareness. Depending on the objective, finding a desirable trade-off point is still an unexplored research topic. 1.5.7 Storage technologies and Data Management Software frameworks such as MapReduce and its various implementations such as Hadoop and Dryad are designed for distributed processing of data-intensive tasks. As mentioned previously, these frameworks typically operate on Internet-scale file systems such as GFS and HDFS. These file systems are different from traditional distributed file systems in their storage structure, access pattern and application programming interface. In particular, they do not implement the standard POSIX interface, and therefore introduce compatibility issues with legacy file systems and applications. Several research efforts have studied this problem [42], [43]. For instance, the work in [42] proposed a method for supporting the MapReduce framework using cluster file systems such as IBM’s GPFS. Patil et al. [43] proposed new API primitives for scalable and concurrent data access. 1.5.8 Novel Architectures Currently, most of the commercial clouds are implemented in large data centers and operated in a centralized fashion. Although this design achieves economy-of-scale and high manageability, it also comes with its limitations such high energy expense and high initial investment for constructing data centers. Recent work [44], [45] suggests that small size data centers can be more advantageous than big data centers in many cases: a small data center does not consume so much power, hence it does not require a powerful and yet expensive cooling system; small data centers are cheaper to build and better geographically distributed than large data centers. Geodiversity is often desirable for response time-critical services such as content delivery and interactive gaming. For example, Valancius et al. [45] studied the feasibility of hosting video-streaming services using application gateways (a.k.a. nano-data centers). Another related research trend is on using voluntary resources (i.e. resources donated by end-users) for hosting cloud applications [46]. Clouds built using voluntary resources, or a mixture of voluntary and dedicated resources are much cheaper to operate and more suitable for non-profit applications such as scientific computing. However, this architecture also imposes challenges such managing heterogeneous resources and frequent churn events. Also, devising incentive schemes for such architectures is an open research problem.

44

45

CHAPTER 2 – Monitoring of Distributed Cloud Services. Today’s networks of large organizations, such as universities or enterprises, rely on many distributed and non-standardized applications. These applications are usually multi-tier, since they consist of many different components, and are mainly deployed on data centers of private or public clouds. Availability and performance of these services are important for revenue-generating business processes, so enterprises enter service level agreement (SLA) with Internet service providers (ISPs). Due to the financial impact of SLAs, there is great research interest in service management and integrated management tools that automatically monitor the performance of multi-tier applications and that can also autonomously handle arising problems. Monitoring and management of such systems have become a critical and complex issue. The purpose of a management system is to monitor vital attributes of network in an automatic manner and to take action whenever needed. After the set-up of a network, network components interact in an unpredictable way, obstructing administrators from having a clear view of the network operation. Even a simple task, such as fetching a web page from a web server, involves many network assets (DNS server, web server, databases) that have an operational relationship. Network administrators use mainly tools like MRTG [47] that illustrate characteristics and statistics of traffic volume based on ports. The major drawback of these monitoring tools is that they do not provide any further information about how network assets collaborate in order to complete a specific task. Unveiling these dependencies can help administrators to have an integral view of the structure of network applications. A functional and structural model of a service or application is a powerful tool for the maintenance, expansion and performance analysis of the service. It helps administrators to detect which component of a service is responsible for a possible failure and which other business processes will be affected. These two problems are known as root cause analysis and business impact analysis, respectively. The first step in building an operational model of a service is to completely understand the interaction of the network components that comprise an integrated application. Because of the complexity and the heterogeneity of enterprise networks, there are many types of relationships and dependencies between the parts of multi-tier applications. In [48], Keller et al. present a good definition and classification of the different types of dependencies among service components. This classification is based on many characteristics, such as locality, domain, component type and activity as well as dependency detection method and strength. There are many studies on the area of monitoring issues of distributed and virtualized data centers which are implemented on cloud computing infrastructure. The following papers are the most representative and closer to our point of view. In [49], a passive technique is suggested using flow attributes and a fuzzy inference engine to infer dynamic dependencies from multi-tier applications. The disadvantage of this work is 46

that the inference engine was based on human experience and it was not flexible. Furthermore, it focused only on multi-tier applications without examining other kinds of dependencies. Agarwal et al. [50] demonstrated a data-mining method to unveil dynamic dependencies in e-business systems. This approach focuses only on intrasystem dependencies and does not seem to scale for large number of customers. In [51], authors used state machines to describe the structure of applications. They illustrated the trade-off between simplicity and analysis depth, but their study is restricted to particular applications (i.e. HTTP or FTP). Aiello et al. [52] modeled network hosts’ behavior and created communities of interest. Although they demonstrate interesting parameters for relationship discovery, their main scope is to discover clusters of interacting hosts without examining whether they are interested in a particular application. Sherlock [53] is a system that uses a centralized inference engine and distributed agents to form a service-level dependency graph for fault localization. Although Sherlock infers dependencies in depth, it cannot extract many relationships simultaneously but extracts dependencies from one server or one service at a time. Kandula et al. presented eXpose [54], a novel technique to discover communication rules between network components. They extract communication rules over a network using an information theoretic approach. Although eXpose discovers the majority of the underlying dependencies, it does not extract the patterns which govern these underlying communication relationships or their strength. Relatively to the disadvantages of the studies in the literature, the thesis focuses on the creation of a tool for analysis of network traffic, which mines relationships between network and service components and measures their strength without having any prior knowledge of existing applications. The main contributions are the following: 

    

The creation of a service graph – overlay, which depicts the operational dependencies between the components of the distributed web services in a data center. Reliable identification of the hidden relationships without any a priori knowledge about them. Quantification of the relationship’s strength. Easy and offline training procedure of the inference mechanism. Fast and online process of the traffic data. Small number of erroneously produced data.

This chapter introduces a new inference mechanism that extracts relationships between network components and measures their strength without having any prior knowledge of existing applications [59]. We discover expected relationships, such as DNS queries before connection to a server or database, and hidden relationships between servers of a distributed networked file system (ports 7000–7010) or between web (port 80) and authentication servers (i.e. Kerberos port 88). After detecting these dependencies we build a service overlay that illustrates the structure of services and 47

the structure of communication among network hosts and servers or databases that are affiliated with a specific service. This overlay can benefit several administrative tasks, such as detecting misconfigurations or DoS attacks, monitoring service performance and design or expansion network infrastructure. Our work is based on NetFlow traces collected from an enterprise network and a campus network of a university. We do not collect data from log files of main servers of these networks because this work is tedious and would lose a lot of relationships, which do not include these computers. From these traces time information and typical flow attributes are used, like IP addresses, ports and protocols. Our work is twofolded. First we apply a simple algorithm that reveals all possible dependencies. Then we develop a flexible fuzzy inference mechanism (FIM) that determines whether the candidate relationship is true or not and measures its strength. The second part of our study is the design and training of FIM. FIM consists of a small set of fuzzy rules (i.e. 30 rules) produced by a hybrid genetic algorithm, which is a training process with high classification rate. Our approach discovers a wide variety of network relationships and measures their strength. Since our training process is an offline procedure and produces a small set of rules in order to extract the hidden dependencies, an online implementation seems feasible. The remainder of the chapter is structured as follows. Initially, all the essentials principles of fuzzy logic are described. Sections 2.2 and 2.3 contain an analytical description of our dependency discovery algorithm and the training procedure, respectively. Section 2.4 demonstrates the implementation and evaluation of the approach. Finally Section 2.5 summarizes the conclusions of the proposed study.

2.1 Fuzzy Logic

Inference Mechanism

Figure 2.1 – Structure of a Fuzzy Logic System

48

Outputs

Rules Set

Defuzzification

Fuzzification

Inputs

A fuzzy logic system can be defined as the nonlinear mapping of an input data set to a scalar output data [55]. It consists of four main parts: fuzzification, rules, inference engine, and defuzzification. These components and the general architecture of a fuzzy logic system are shown in Figure 2.1.

2.1.1 Linguistic Variables Linguistic variables are the input or output variables of the system whose values are words or sentences from a natural language, instead of numerical values. A linguistic variable is generally decomposed into a set of linguistic terms. For example, let temperature (t) be the linguistic variable which represents the temperature of a room. To qualify the temperature, terms such as “hot” and “cold” are used in real life. These are the linguistic values of the temperature. Then, T(t) = {“cold”, “cool”, “nominal”, “warm”, “hot”} can be the set of decompositions for the linguistic variable temperature. Each member of this decomposition set is called a linguistic term and can cover a portion of the overall values of the temperature. 2.1.2 Membership Functions Membership functions are used in the fuzzification and defuzzification steps of a fuzzy logic system, to map the non-fuzzy input values to fuzzy linguistic terms and vice versa. A membership function is used to quantify a linguistic term. There are different forms of membership functions such as triangular, trapezoidal, piecewise linear, Gaussian, or singleton (Figure 2.2). The most common types of membership functions are triangular, trapezoidal, and Gaussian shapes. The type of the membership function can be context dependent and it is generally chosen arbitrarily according to the user experience [56]. Note that, an important characteristic of fuzzy logic is that a numerical value does not have to be fuzzified using only one membership function. In other words, a value can belong to multiple sets at the same time, as it is shown in Figure 2.3.

Figure 2.2 – Memebership Functions: (a) singleton, (b) triangular, (c) trapezoidal, (d) Gaussian shape

49

Figure 2.3 – Membership Functions of a Linguistic Variable

2.1.3 Set of Fuzzy Rules and Inference Mechanism The mapping of the inputs to the outputs for a fuzzy system is in part characterized by a set of condition - action rules in If-Then form, If premise Then consequent Usually, the inputs of the fuzzy system are associated with the premise, and the outputs are associated with the consequent. Generally, these If-Then rules can be represented in multi-input multi-output (MIMO). Although they can be multi-input single-output (MISO) if there is only one out associated with the consequent. The set of these rules contains the knowledge of the expert about the how the system works. The evaluations of the fuzzy rules and the combination of the results of the individual rules are performed using fuzzy set operations. The operations on fuzzy sets are different than the operations on non-fuzzy sets. Let μΑ and μΒ are the membership functions for fuzzy sets A and B. Table 2.1 contains possible fuzzy operations for OR and AND operators on these sets, comparatively. The mostly used operations for OR and AND operators are max and min, respectively. For complement (NOT) operation, the following equation is used for fuzzy sets.

  1  A _

A

OR (union)

AND(interception)

MAX

max{  A ( x),  B ( x)}

MIN

min{ A ( x),  B ( x)}

ASUM

 A ( x )   B ( x)   A ( x )  B ( x )

PROD

 A ( x)  B ( x )

BSUM

min{1,  A ( x)   B ( x)}

BDIF

max{ 0,  A ( x)   B ( x)  1}

Table 2.1 – Fuzzy Set Operation 50

After evaluating the result of each rule, these results should be combined to obtain the final result. This process is called inference. The results of individual rules can be combined in different ways. The maximum algorithm is generally used for accumulation max{  A ( x),  B ( x)} . 2.1.4 Defuzzification After the inference step, the overall result is a fuzzy value. This result should be defuzzified to obtain a final numerical output. This is the purpose of the defuzzifier component of a fuzzy logic system. Defuzzification is performed according to the membership function of the output variable. For instance, assume that we have the result in Figure 2.4 at the end of the inference. In this figure, the shaded areas all belong to the fuzzy result. The purpose is to obtain a numerical value, represented with a dot in the figure, from this fuzzy result.

Figure 2.4 – Defuzzification of a Fuzzy Logic System

There are many different algorithms for defuzzification. The mostly used is Center of Gravity which determines the final numerical value of output according to the following equation, max

U

 u (u )du

min max

  (u )du

min

where U is the numerical result of defuzzification, u is the output variable and μ(u) is the membership function after accumulation.

2.2 Mining Network Relationships In this section, the problem of detecting relationships between IT elements by examining attributes of flow records is considered. Collection of the necessary information is performed by Aurora [57], a flow-based network profiling system 51

based on NetFlow/IPFIX [58]. Flow-based traffic information, such as source or destination IP address and ports, timestamps, protocol and packet or octet volumes, is collected. More specifically, by examining the timestamps of flows we would like to deduce whether the coexistence of particular flow pairs in a time window can be interpreted as a relationship between two different components that execute an individual task or a part of an application. By mining all these relationships between network components of a service, we can obtain a clear overview in terms of which elements of network infrastructure are responsible for the provisioning of a particular service. Figure 2.5 illustrates the structure of our approach. In the following paragraphs, we explain the different components of our system.

AppList

Flows

HGA

Extracted Event Pairs

FIM

Relationships

Figure 2.5 – Relationship Discovery System

2.2.1 Pre-processing Data Before we formalize flow dependency, it is essential to introduce some dominant parameters of relationship discovery. In an enterprise or a university campus network the traffic volume is very high, so our algorithm has to examine a large amount of flows, which are time and resource consuming. Furthermore, a large portion of these flows does not contain any meaningful information for mining dependencies. For the aforementioned reasons, we make a distinction of hosts in a network. By examing the destination (IP address and port) of a flow and the TCP flags we can understand whether a network asset is an active part of an application, because it receives many more requests on a specific port (or ports in the case of co-location) than the other ports. We mention that we do not rely on well-known ports, because we intend to include also non-standardized applications and p2p relationships. We create a list of these tuples (IP address and port) and we focus on flows which include these tuples either as source or as destination, in order to extract candidate relationships. We call this list application list (AppList). 2.2.2 Flows and Events We define a flow f as a 3-tuple of the following basic traffic attributes:

f  (srcIP, dstIP, dstPort )  F 52

(2.1)

where srcIP and dstIP are the IP address of source and destination host, respectively, and dstPort is the TCP/UDP service port of the destination host. The set of all flows is denoted by F. We do not include the service port of the source host because it is usually random. Additionally we define the destination of a flow f as the tuple:

dst( f )  (dstIP, dstPort )

(2.2)

An event - typically a NetFlow/IPFIX record - is defined as,

e  ( f , t s , t e , octs , pkts)

(2.3)

where f is the flow as defined and ts and te are the start and end timestamps of a flow event, respectively, and octs and pkts are the number of bytes and packets of a flow, respectively. The set of all events is denoted as E. Finally we denote the set of all events of a given flow f as,

E( f )  e  E | f e  f 

(2.4)

2.2.3 Flow and Event Pairs Nowadays, most network applications are distributed and depend on many different assets. For this reason an event cannot reveal any association between IT infrastructure. The main idea of our approach is to mine flow pairs that can show any functional relationship between hosts in the application list. We define three generic flow pairs that cover every possible relationship. In the first case, we examine how network clients interact with the hosts of AppList. Considering any two flows in F, we define the client–server flow pair (cs flow pair) if the following conditions are satisfied:  

The first and the second flow have an identical source IP address and do not belong to the AppList. The destination of the first flow is different from the destination of the second flow and both destinations belong to AppList.

An example of such a flow pair is a pair between a DNS server and a web server. When a user sends an HTTP request to a web server, a flow between the client and the DNS server always precedes the flow between the client and the web server. The next pair between any two flows in F examines whether a request from a client to a host of the application list triggers a flow between the host of the application list and another host of AppList. For instance, this flow pair extracts relationships between a web server and a back-end database when a client sends a request to a web server and consequently it communicates with a database that contains the contents all the necessary information. Formally, we define a server–server flow pair (ss flow pair) if the following prerequisites are satisfied: 53





The source IP address of the first flow is not included in the AppList and the destination of the first flow belongs to the AppList. The destination of the second flow belongs to the AppList. The destination IP address of the first flow is identical to the source IP address of the second flow.

Finally we define a flow pair that detects whether a request from a host to another host of the application list triggers a second flow from the second host to a third one of the application list. Relationships of multi-tier services and updates between servers can be revealed by this flow pair. More specifically, considering any pair of flows in F we define a multi-server flow pair (ms flow pair) if the following conditions are satisfied:  

Source IP addresses of both flows are included in the AppList and the destination of both flows belongs to the AppList. The destination IP address of the first flow is identical to the source IP address of the second flow.

Assuming any two different flows fi and fj in F, we encode the previous considerations in the following expressions of three flow pairs functions:

1 if scrIPi  scrIPj and (dst(fi ), dst(f j ))  AppList fp cs ( f i , f j )   otherwise 0

(2.5)

1 if dstIPi  scrIPj and (dst(fi ), dst(f j ))  AppList and srcIPi  AppList fp ss ( f i , f j )   (2.6) otherwise 0

1 if dstIPi  scrIPj and (srcIPi , dst(fi ), dst(f j ))  AppList fp ms ( f i , f j )   otherwise 0

(2.7)

Figure 2.6 shows a typical example of defined flow pairs. After the definition of the three flow pair functions, we define three different sets of event pairs for any two different flows fi and fj in F, respectively. Assuming any two different flows fi and fj in F (fi ≠ fj) and two events ek, el in E, which are events of fi and fj, respectively, three different sets of event pairs, one for each flow pair, for these flows fi and fj are defined as, Pcs ( f i , f j )  (ek , el ) : 0  t sl  t sk  t max and fpcs ( f i , f j )  1

(2.8)

Pss ( f i , f j )  (ek , el ) : 0  t sl  t sk  t max and fp ss ( f i , f j )  1

(2.9)

54

Clients

DNSServ

WebServ

Clients

WebServ

DB

(b)

(a) WebServ

AppServ

DB

(c) Figure 2.6 – Flow Pairs: (a) cs flow pair, (b) ss flow pair, (c) ms flow pair

Pms ( f i , f j )  (ek , el ) : 0  t sl  t sk  t max and fp ms ( f i , f j )  1

(2.10)

As shown in the above definition, two events are considered an event pair if they satisfy one of the flow pair functions and the difference of their start timestamps is less than tmax. The main idea of using time difference as a criterion of dependency is that flow events close in time are likely really dependent. The value of the time window tmax is critical for the success of the algorithm. If it is too small, then it is possible not to mine real dependencies. On the other side, if it is too large, then the algorithm detects many false dependencies between hosts that do not represent meaningful relationships. Furthermore, the time window is strongly affected by the structure of an application. Some applications, i.e. if it is necessary to type a username and a password, may need to spend more time than other simpler services, i.e. just fetch the content of a web server. Hence it is obvious that choosing a value of time window which would suffice for all various applications is very difficult. Repeating our algorithm with various time window values, we conclude that the number of event pairs increases rapidly, but the number of different sets of event pairs does not increase similarly. This result shows that increasing the time window too much (more than 1 s) does not improve the performance our algorithm. 2.2.4 Confidence Variables The time difference of the start timestamps of two events is not a safe metric of dependency by itself. In heavily loaded networks there is a high probability of discovering many false event pairs. There are three main reasons for false associations. First, some event pairs are created clearly by chance. Another source of erroneous data is whether a host of AppList is a common part for two different applications. Finally there are some cases of very frequent flows (i.e. DNS queries) 55

that can create multiple event pairs with a flow that occurs only once. Even if there is one real event pair, many false pairs are identified and the strength of the relationship is distorted. Thus only the frequency of an event pair cannot reveal the correctness and strength of a relationship. To reduce the erroneously recognized dependencies, we define four confidence variables based on the following concepts: Confidence Variable 1:

c1 ( f1 ) 

E ( f1 p ) E ( f1 )

(2.11)

where E( f1p) is the number of events of f1 that participate in an event pair of set P( f1 , f 2 ) and the denominator is the number of elements of set E( f1). Confidence Variable 2:

c2 ( f 2 ) 

E( f 2 p ) E( f 2 )

(2.12)

where E( f2p) is the number of events of f2 that participate in an event pair of set P( f1 , f 2 ) and the denominator is the number of elements of set E( f2).These two confidence variables show whether events of flows f1 and f2 are related with a specific task and measure the strength of this relationship. Confidence Variable 3:

c3 ( f 1 , f 2 ) 

TNP( f1 , f 2 )  SNP( f1 , f 2 ) TNP( f1 , f 2 )

(2.13)

where TNP( f1 , f 2 ) is the total number of identical detected event pairs that flow f1 creates in a time window tmax and SNP( f1 , f 2 ) is the sequence number of the event pair (i.e. for the first detected event pair SNP( f1 , f 2 )1  0 , for the second

SNP( f1 , f 2 ) 2  1 ). Confidence Variable 4:

c 4 ( f1 , f 2 ) 

SNP( f1 , f 2 ) TNP( f1 , f 2 )

(2.14)

where TNP( f1 , f 2 ) is the total number of identical detected event pairs that flow f2 creates in a time window tmax and SNP( f1 , f 2 ) is the sequence number of the event pair (i.e. for the first detected event pair SNP( f1 , f 2 )1  0 , for the second

SNP( f1 , f 2 ) 2  1 ). We use c3 and c4 in order to reduce the multiple identical event 56

pairs that are generated from very frequent flows. Usually whether or not many identical pairs exist, the real one occurs earlier in the time window, which means c3  1 and c4  0 . The values of confidence variables are limited between zero and one. 2.2.5 Relationship Discovery In order to mine relationships in a network trace, we examine whether an event of a flow creates any event pair, as defined previously, in a time window of 1 s. The values of all confidence variables are calculated and then a set of fuzzy rules is used in order to evaluate whether a set of event pairs represents a real dependency and furthermore measure the strength of this association. In order to separate real from meaningless relationships we used four thresholds α and β1, β2, β3. If the dependency output of the fuzzy system for P( fi, fj) is less than threshold α then this set does not represent any real relationship. Thresholds α ,β1, β2, β3 classify relationships into four classes—low, medium, high and very high—according to their dependency value. Default values for thresholds α and β1, β2, β3 are 0.125, 0.375, 0.625 and 0.875, respectively. Algorithm 2.1 illustrates how all sets of event pairs Pcs are extracted. Algorithms for sets of event pairs Pss and Pms are similar and are omitted. Algorithm 2.1: Extract CS event pairs 1: Fill in AppList with dst (dst ∈ AppList) {Create outgoing matrix OM for each client} 2: for all clients (clIP  Applist) do 3: Create a matrix of events clIP ⇒ dst 4: Sort matrix by start timestamps ts 5: end for {Find event pairs in OM} 6: for all OM of clients (clIP  Applist) do 7: for all event ei in OM do 8: time difference between ei and next events ej dt = tsj − tsi 9: while dt ≤ 1s do 10: Update P(fi, fj) 11: end while 12: end for 13: end for 14: Compute all confidence variables {Examine the validity of sets of P(fi, fj)and measure the strength of relationships} 15: for all sets P(fi, fj) do 16: for all event pairs (ek, el) ∈ P(fi, fj) do 57

17: Use the set of Fuzzy Rules to calculate dependency class and its strength dep(ek, el). 18: end for 19: dep(P(fi, fj)) =Σ dep(ek, el) 20: if dep(P(fi, fj))≤ α then 21: None real relationship between (dsti, dstj). 22: else if α < dep(P(fi, f j)) ≤ 1 then 23: Classify relationship to the appropriate class according to the strength of it dep(P(fi, fj)) 24: end if 25: end for

To fully understand the underlying relationships of a network and minimize the generated event pairs, we aggregate pairs with the same meaning. The cs flow pair represents the interaction between clients and IT infrastructure for the completion of a specific task. We can aggregate event pairs that correspond to the same association regardless of the client, which creates this event pair. Similarly, we can aggregate event pairs of an ss flow pair, because it is unimportant which client is responsible for this event pair. We do not aggregate event pairs of an ms flow pair, because usually this does not happen, and whenever it occurs the host that causes this pair may have a special meaning.

2.3 Fuzzy Inference Mechanism It is obvious from the previous section that performance of our approach depends on the fuzzy inference mechanism (FIM), which is based on a small set of fuzzy rules (30 rules). In our previous work about multi-tier services [49], we used an inflexible FIM generated manually by human experts, which is not dynamic and has limited accuracy. On the contrary, a flexible and accurate FIM is proposed in this study which is automatically produced by a training process. FIM is a set of fuzzy rules which determine the existence and strength of a relationship. Because relationships and their strength vary a lot in a network with many different applications, a fuzzy system is an appropriate descriptive tool for the indistinct boundaries between strong, loose and no dependency. In our case we build a fuzzy system with five inputs: timestamp difference of an event pair and the four confidence variables. The output of the fuzzy system is the relationship strength, dep∈ [0, 1]. If dep → 0 there is no dependency; on the other hand dep → 1 indicates tight dependency between two network elements. Another important parameter of FIM is that each input is described by four linguistic values, such as ‘low’, ‘high’ and ‘medium’, and a special (‘don’t care’) linguistic value that helps us to aggregate similar rules. Similarly the output of FIM is described by four linguistic values (dependency classes). The rules set of an FIM is a set of fuzzy if–then rules like the following:

58

if dt is LOW and c1 is HIGH and c2 is HIGH and c3 is HIGH and c4 is LOW then dep is HIGH if dt is HIGH and c1 is LOW and c2 is LOW and c3 is LOW and c4 is HIGH then dep is LOW We produce the fuzzy rules of the flexible FIM using the training procedure of a hybrid genetic algorithm (HGA). Genetic algorithms (GA) are already used as classification tool and they can be applied to the dependency detection problem. FIM produced by GA has many advantages.Initially, it has a high classification rate, reducing false generated relationships. It is an automatic process that can be done offline. In a normal network situation, only the strength of a relationship changes during certain time period. Relationships do not appear or disappear suddenly, so the training process has to be done only if changes occur in IT infrastructure. Also, GA is suitable for handling high-dimensional datasets, where the number of all possible fuzzy rules increases dramatically. Finally, because the output of a genetic-based training process is a small fuzzy rule set, an online implementation of our approach is possible. There are many alternants of GAs that produce a set of rules to classify patterns from different classes. All versions of GAs are based on operations of selection, crossover and mutation. The population of GA consists of a rules set. One approach, the Pittsburgh approach, uses as population many different rules sets with specific number of fuzzy rules and has direct optimization ability. The Michigan approach uses as initial population a single set with fixed number of rules and has a high research ability to find good fuzzy rules. Our training procedure relies on a hybrid genetic algorithm (HGA) [60] that combines the benefits of both above approaches. We modify the presented algorithm to achieve a better classification rate. We feed HGA with patterns of real and false mined relationships. Table 2.2 illustrates the parameters of HGA training process. Rule Sets

Replaced Sets

Fuzzy Rules

Replaced Rules

70

40

30

6

Mutation Probability

Selection Probability

Crossover Probability

Michigan Probability

0.9

0.9

0.9

0.1

Table 2.2 –Parameters of HGA

59

The steps of HGA are described in Algorithm 2.2 below, Algorithm 2.2: Hybrid genetic algorithm 1: Generate Npop rule sets with Nrules fuzzy rules from training patterns. 2: Calculate the fitness value of each rule set in the current population. {Iterate the steps of the Pittsburgh approach for 150 generations.} 3: for generation = 1:150 do 4: Generate Nrepls rule sets by genetic operations of the Pittsburgh method. {Iterate twice the steps of the Michigan approach} 5: for i =1:2 do 6: Generate Nreplr rules by genetic operations of the Michigan method, with a prespecified probability. 7: Calculate the class of each new generated rule. 8: Calculate the fitness value of each new generated rule. 9: end for 10: Calculate the class of each rule in new generated rule sets. 11: Calculate the fitness value of each rule set in the current population. 12: end for 13: Save the rule set with the highest fitness value. Fitness values of the Pittsburgh and Michigan approaches are respectively calculated as,

FVPitt (seti )  1  1.5 * SFP(seti )  2 * SFN (seti )  1.5 * SFR(seti )

FVMich (seti )  RCP(rulei )  1.5 * RFP (rulei )  2 * RFN (rulei )

(2.15)

(2.16)

where SFP(seti) is the number of false positives (i.e. pairs that are not real but they are classified as real by rule set seti), SFN(seti) is the number of false negatives (pairs that are real but no dependency was unveiled by seti)and SFR(seti) is the number of pairs that seti cannot classify into a class. Similarly, RCP(rulei) is the number of correctly classified pairs by rule rulei, RFP(rulei) is the number of false positives by rulei and RFN(rulei) is the number of false negatives by rulei. After exhaustive running of HGA with different parameters, the dominant factors of HGA success are the following:  

Training patterns can be less than 10% of all patterns. We use the remaining patterns for evaluation. 25–35 rules are sufficient to succeed in classification rate higher than 90%; more rules increase the computation time with negligible improvement. 60

   



The larger the number of rule sets in the HGA population, the more likely it is to find better rule sets because HGA search space grows. 150 generations are sufficient for HGA to converge to an optimal solution. The number of replaced rules in the Michigan approach should be 20% of all rules in the rule set. The number of replaced rule sets in the Pittsburgh approach should be 60% of the total number of rule sets in the population. If it is larger, HGA has a low success rate. If it is smaller, HGA converges quickly in a solution and the population does not evolve to better rule sets. Usage of the special linguistic value ‘don’t care’ improves the success rate of HGA, because it aggregates rules into one.

2.4 Evaluation We tested our relationship discovery algorithm using network traces from two different locations. We collected data from the IBM Zurich Research Laboratory and the Department of Electrical Engineering and Computer Science from the University of Patras. Both traces are anonymized, so it is not possible to know the ground truth, although we could examine whether our results are correct, by service ports and from information that we gather from administrators of these networks. The first network is an enterprise network that hosts many different applications, so its infrastructure is complex and we expect to discover all kinds of dependencies. Conversely, the second network is a university network that hosts fewer applications (web hosting, mail, file servers) than an enterprise network, so we expect to mine fewer multi-tier relationships and stronger client-server (cs) flow pairs. Another interesting parameter is that a campus network has more users than an enterprise network, so the traffic load is higher in the Patras trace than in the IBM trace. Both traces show that the parameter of time window strongly affects the number of discovered event pairs and relationships. For the remainder of the paper time window is set to one second. 2.4.1 Simulations Our approach is evaluated by two different methods. First, we create and inject event pairs of artificial relationships in the real data traces, and then we apply our algorithm to check whether we can identify these artificial dependencies. Artificial dependencies emulate typical patterns of network traffic, such as web browsing, downloading files from a distributed file system and multi-tier applications, and are generated by normal distributions with different mean and standard deviation values inside a time window of 1 s. The benefit of these simulations is that ground truth is already known, thus we can extract safe conclusions about relationships over a network from the evaluation of our approach. We extract event pairs of real and false relationships and we feed HGA with them to generate a set of fuzzy rules (FIM). Then we generate different artificial event pairs to evaluate these rules. Figure 2.7 illustrates the classification rate of the HGA training process. It is obvious that HGA succeeds in producing a set of fuzzy rules that classifies correctly the 61

majority of the different types of the injected artificial relationships. After 150 generations of HGA the classification rate is around 92% for ms relationships, 94% for cs relationships and 97% for ss relationships. In order to evaluate the produced sets of fuzzy rules from the training procedure, we create a different set of artificial event pairs. Figure 2.8 depicts the percentage of correctly classified patterns over 30 runs. As we can see, HGA and produced fuzzy rules are accurate, because they identify artificial relationships and precisely measure their strength (tight or loose associations). The classification rate is similarly high as during the training procedure. The proposed relationship discovery algorithm succeeds in revealing 92% of the multi-server dependencies, 94.5% of the client server relationships and more than 95% of the multi server relationships.

Figure 2.7 – Training: Average Classification Rate of Simulations

62

Figure 2.8 – Evaluation: Average Classification Rate of Simulations

2.4.2 Real Data Furthermore our method is tested using data from real traces without having any prior knowledge about relationships among assets of these networks. Since information is anonymized, it is impossible to check the correctness of our results. However it is obvious that any of the discovered relationships correspond to well-known realistic dependencies can be classified as a true relationship. We use time window of one second and FIM determines 690 dependencies, as we can see in Table 2.3. Most of them are classified as poor relationships without any special meaning. There are 67 associations with medium dependency value that mainly represent true dependencies among DNS servers and other servers. There are 11 strong dependencies that correspond to tasks of specific applications. Furthermore, Table 2.3 shows that event pairs and associations increase as the time window increases but relationships with high dependency value are almost constant for all values of time window. As we 63

mentioned, we expect to discover more ms and ss relationships in the IBM network and simpler ones in the university campus network. In the following paragraphs we record the most typical examples of relationships that we detect in both network traces. Also we present characteristic dependencies of each network. HGA is fed with patterns to produce a set of 30 fuzzy rules and different data are used for training (about 10%) and evaluation. Since we do not know which dependencies are true, pairs of the most well-known discovered relationships are categorized as strong or loose and the rest as meaningless (poor relationships). Time window (sec)

0.25

0.5

1

2

3

4

5

Event pairs (105)

0.55

0.77

1.15

2.4

4.05

6.62

25

cs relationships

230

271

387

390

432

449

459

ms relationships

94

121

155

211

259

289

411

ss relationships

57

96

148

340

400

441

553

total relationships

381

488

690

941

1091

1179

1423

Medium Score

26

42

67

70

91

95

134

High Score

7

9

11

10

12

11

11

Table 2.3 – Effect of time window in real data

The classification rate of HGA during the training procedure is high. As it is demonstrated in Figure 2.9, the average classification rate of server-server (ss) and multi-server relationships (ms) is 90%. The classification rate of the client-server (ss) is higher (95%). It is worth mentioning that there are some fuzzy rules are present in every set over 30 runs of the training procedure. Figure 2.10 illustrates the results for the thirty different FIMs which are produced from the training process. The average classification rate of server-server (ss) and multi-server relationships (ms) is 90% and the classification rate of the client-server (ss) is higher, around 95%. It is obvious that the performance of the produced FIM is similar over the thirty runs of HGA. This happens because there are many common fuzzy rules among the different set of rules. This proves that HGA is capable of finding good rules and sets of rules regardless of the training patterns.

64

Figure 2.9 – Real Data: Average Classification Rate of Training Process

Figure 2.10 - Real Data: Evaluation of the produced FIMs

65

In the rest of this section, the most representative discovered relationships are presented. DNS queries Most cs sets of event pairs represent the relationship between a DNS query from a client and a flow from the same client and another network component. We assume that these relationships are real. The most often relationships represent an example between a DNS and a web server. We discover similar relationships between DNS servers and an SMTP server (port 25) or root server of a distributed file system (port 7000 only in IBM network). Wildcard (*) means that this value is random. Client.*: DNS.53 ⇒ Client.*:WebServ.80 Client.*: DNS.53 ⇒ Client.*:SMTPServ.25 Login and security Usually users have to communicate with a login server (SSH) before accessing an application server (file server system). We consider that the following pattern corresponds to these kinds of relationships, which are real and usually have high dependency value. Client.*:WebServ.80 ⇒ Client.*: LoginServ.22 Another example, from the trace of the University of Patras, shows a relationship between two web servers. Users, after visiting a web server, connect to another web server over TSL/SSL session (port 443). Client.*:WebServ.80 ⇒ Client.*:WebServ.443 AFS system In the IBM network we extract relationships between parts of a distributed file server system (AFS). Port 7000 corresponds to the file server itself, and port 7003 to the volume location database. A client talks to the root server at port 7000 to find which volume server contains the data and then communicates with the appropriate server at port 7003: Client.*: AFS1.7000 ⇒ Client.*: AFS2.7003 Also browsing to different parts of the AFS system initiates connections to an AFS/Kerberos authentication server (port 7004) and the users and groups database (port 7002): Client.*: AFS1.7004 ⇒ Client.*: AFS2.7002

66

Enterprise applications Another interesting ss relationship, which confirms that enterprise networks (IBM network) rely on distributed applications, shows the association between a web server and a couple of Lotus Notes applications (port 1352) servers. Client.*:WebServ.80 ⇒ WebServ.*: LotusNote1.1352 Client.*:WebServ.80 ⇒ WebServ.*: LotusNote2.1352 Servers updating Finally, looking for ms relationships, we do not find any in the trace of the University of Patras, as network administrators have already mentioned. On the contrary, in the IBM trace we find relationships that correspond to information exchange between a cluster of DNS servers like the following. Furthermore, we deduce that a master DNS server communicates with this cluster of servers via multicast DNS port 5353. DNSServ1.*:MasterDNS.53 ⇒ MasterDNS.5353: DNSServ2.53 Generally, after checking all kinds of relationships, we could say that the false positives (generated relationships that really do not exist) are about 12%. We conclude that co-location of many component of different services on a single physical machine is the main reason for false positives. However, our algorithm classifies relationships correctly and assigns them the appropriate dependency value. From our study, it is significant not only to discover a dependency but also to measure its strength, and fuzzy rules are appropriate to describe the indistinct strength of network relationships. We deduce that real relationships have low, medium or high dependency value. This happens because a flow usually is not created only for a single application. For example, flows to DNS or web servers are used from many different services; thus relationships between these network assets often do not have very high dependency value. Figure 5 illustrates the service overlay of IBM business network. The service overlay of university network is omitted since it is simpler and all the discovered relationships are already depicted on the service graph of IBM network.

2.5 Conclusions A novel general technique to infer relationships between IT infrastructure is presented in this chapter. The produced service overlay can be a useful tool for administrators to acquire a clear view of what is happening in their network. The main results of this study are the following: 

We discover relationships that correspond to the application structure, such as web and file servers, without any prior knowledge.

67

 



The generated erroneous relationships are few and our approach measures accurately the strength of every dependency. The hybrid genetic algorithm has a high classification rate and needs only a small fraction of training data (10%). Also it is an offline process, applicable to different networks. The set of fuzzy rules is not necessarily large: 30 rules suffice to classify correctly all candidate relationships.

Because of the simplicity of our algorithm and the flexibility of FIM, an online approach seems feasible. This overlay can be used for several administrative tasks, such as detecting misconfigurations or DoS attacks, monitoring service performance and design or expansion network infrastructure. Clients

116:7002

cs relationship ss relationship ms relationship

50:7004

Clients

Clients 56:53 Clients

Clients

Clients 116:7004 50:7002 16:88 2:111

2:755

1016:80 Clients

2:5353

56:659 91:728

1009:22 Clients

443:53 91:53

Clients 243:53

222:25 Clients

Clients

Clients 114:53

116:514 Clients

157:53

Clients

3538:80

24:1352

623:53

427:53

2:5353 218:53

Figure 2.11 – Graph of Relationships

68

69

CHAPTER 3 – Control of Distributed Web Services on Cloud Computing Infrastructure As it is mentioned in the first chapter, automated resource provisioning is one of the most challenging research problems of cloud computing. Meeting performance specifications of consolidated web services in a data center is critical issue, since the control of the underlying cloud computing infrastructure must be sophisticated enough to meet Service Level Agreements (SLA) requirements and system constraints. Virtualization technology allows the consolidation of many services on a server cluster. The co-hosted applications share the available resources according to their requirements. This implies that providers optimally allot the computational and network resources to the competing services and customers pay only for the consumed resources according to a usage-based price model. The performance of a web service is described in the SLA between the service provider and the customer; the Service Level Objectives (SLOs) are the network metrics, which prescribe exactly the desired Quality of Service (QoS) levels. The most common SLOs are the request response time and the served request rate during a fixed time interval. From the customer’s view, the goals are the satisfaction of some predefined nominal values of the SLOs and a guaranteed level of QoS with the minimum financial cost. These objectives usually conflict with the goals of the provider, who aims for a management system that optimally allocates the resources to each service in a manner that achieves SLOs, ensures availability of resources under any workload circumstances and minimizes the operational cost (e.g. the consumed power energy and machine failures). Due to the above contradictive targets between the customer and provider, managing the performance of such complex systems is a critical and challenging research problem. Furthermore, there are two dominant factors that make the automated resource provisioning even more complex; the workload of these web services is generally unpredictable and highly variant, while the available resources are subject to constraints. There are several emerging problems that an autonomic management framework must address in order to succeed all the above goals. Network controllers are implemented in two levels, namely, the local level and the global level. Global level controllers focus on the problems of load balancing, which shares the volume of requests among the replicas of an application, or Distributed Rate Limiting (DRL) problem that aims to equalize the performance at each of the underlying data centers according to a performance indicator. Also they arrange the Virtual Machines (VMs) placement, which determines the set of applications executed by each server. Local level controllers mainly concern application specific or server specific problems, i.d. admission control (AC) that rejects requests under peak workload conditions and resource allocation (RA), which determines the resources of each VM (CPU, memory, network bandwidth) either by changing their capacity or using the Dynamic Voltage 70

Local Level

Local Level AC+RA

AC+RA

Site 1

Site 2  12 ,  22 ,  ,  2n

11 , 12 ,  , 1n

1 ,  2 ,,  n Global level

 13 ,  32 ,  ,  3n

 14 ,  42 ,  ,  4n

Local Level

Local Level

AC+RA

AC+RA

Site 4

Site 3 Figure 3.1 – Control Architecture of Cloud Computing Infrastructure

and Frequency Scaling (DVFS) mechanism of servers to change their service rate. Figure 3.1 illustrates the general control architecture for the consolidation of web applications on a distributed cloud computing platform. An Infrastructure as a Service (IaaS) or Software as a Service (SaaS) provider has a set of server sites, which are heterogeneous in terms of resource capacity. The global level controller must share the total incoming workload of n applications (  i , i  1,n ) among their replicas ( ij , i  1,, n j  1,, m ) taking into consideration the available resource capacity on the local level. The local level controllers aim at satisfying the SLOs reference value, the physical constraints and minimizing the power consumption. Most of the existing solutions focus only either on local level or the global level controllers. Studies about local controllers adopt either admission control or resource allocation, while very few tackle both problems simultaneously. These approaches have two drawbacks, namely, they do not scale well and they do not guarantee 71

feasibility and stability of the desired operating point. On the other hand, the global level controllers solve the capacity allocation and load balancing problems by making strong, thus conservative, assumptions on the underlying infrastructure and its dynamic behavior. The existing approaches which concern the modeling and control of web services can be categorized according to the employed model and the control method. Initially, a categorization can be made according to the modeling method of the behavior of consolidated web services on a cloud computing infrastructure. Models based on queuing theory rely on the assumption that the system is at steady-state condition, which appears to be contradictive to the dynamic nature of it. On the other hand, linear time invariant (LTI) state-space models can efficiently describe the transient behavior of the system. However, the accuracy of the model is limited near an operation point. The most promising category of state space models employed are linear parameter varying (LPV) models, because they vary according to the value of a modeling parameter. This type of modeling is suitable to describe web services behavior, because it efficiently captures the effect of various parameters such as incoming request rate or service time. Table 3.1 summarizes the benefits and drawbacks of each modeling method which are used in the relevant studies. Strengths  Scalability.

Weaknesses  Valid only in steady state.  Specific workload distributions.

Related Studies [61] – [70]

LTI Models



Capture transient behavior.

 

Scalability. Accuracy.

[71] – [73]

LPV Models



Capture transient behavior. Embed system’s parameters. Scalability. Accuracy.



Identification Procedure.

[74] – [77]

Queuing Models

  

Table 3.1 – Comparison of existing modeling approaches. A second categorization of the existing approaches concerns the control method used. On the local level, admission control or resource allocation is employed. Furthermore, resource allocation approaches can be distinguished to those that change dynamically the CPU capacity of service VMs and those that consider as control variable the CPU frequency of the servers with fixed CPU capacity (DVFS technology). The above techniques can be combined with replicas addition using a pool of idle servers or migration of VM. Most of the resource allocation solutions assume that servers are 72

protected from overloading. Nevertheless admission control and resource allocation are dual problems and must be solved jointly towards ensuring the performance of the web service under any workload variations. Table 3.2 summarizes the benefits and drawbacks of each control method which are used in the following related studies. Strengths Admission  Overloading control Protection. (AC)

Weaknesses  Fixed Capacity.  Rejection of requests under heavy workload.

Related Studies [64], [69], [71], [77]

Resource Allocation (RA)



Dynamic Capacity.



No performance guarantee under heavy workload.

[61] – [63], [67], [70], [73], [75], [76], [78]

AC+RA



Stability Guarantee. Overloading Protection. Dynamic Capacity.



Complexity.

[64] – [66], [72], [74]

 

Table 3.2 – Comparison of existing control approaches. Nevertheless, independent of the modeling and control approach, it appears that no existing method considers the explicit determination of a desired operating point. The LTI or LPV models combined with feedback control infer an operation point from measurements or the system identification procedure. However, if the state variables are far away from that nominal operation point, the underlying servers are saturated and the model becomes non-linear. Similarly, queuing theory approaches combined with an optimization method suffer from the same limitation. In detail, queuing models assume specific distribution (eg. M/G/1) for the incoming request and service rate and determine the operating point only for steady state conditions. In the following paragraphs, we recall some representative studies of interest that are close to the approaches proposed in this chapter. Ardagna et al. [61] presented a distributed algorithm for capacity allocation and load balancing. They modeled the system’s dynamic behavior with a M/G/1 queue and solved the joint control problem as an optimization problem applying a decomposition technique for non-linear programming. Although the resource allocation is simplified, since they assume single tier web applications, VMs have predefined fixed capacity and are homogeneous in terms of resources (CPU, RAM). In [64], a joint admission control and resource allocation framework utilized queuing theory to capture systems dynamics. The two controllers were separated in two different optimization subproblems that were solved sequentially in an iterative fashion. However it was not examined whether the 73

sequence of solutions provides any performance guarantee. Kusic et al. [62], [63] derived an analytical mathematical model from queuing theory and tackled resource provisioning as an optimization problem solved with a predictive control method. Some system parameters (e.g. service rate) were empirically computed while the implementation considered the VM placement and capacity allocation problems separately. Also the scalability of the approach was not demonstrated. Urgaonkar et al. [65] presented a utility based solution, which maximizes the average application throughput and energy cost of the data center. Using queuing theory models and an optimization technique framework, they addressed the problems of admission control, server selection and resource allocation separately. In [66], the authors proposed a holistic approach for application placement, admission control and resource allocation. They used queuing theory models and a greedy algorithm to solve the placement problem and consequently they split admission control and capacity allocation as two decoupled subproblems. There, admission controller is designed separately from [79] ignoring the resource allocation solution. Wang et. al. [67] proposed a two level controller combined with queuing modeling. The global level determines if the CPU share is efficient for the total incoming workload and it activates a certain number of servers. The local controller solves the resource allocation problem in order to satisfy the predicted workload. However, it was assumed that the incoming load to each application replica is proportional to the CPU share and a linear relationship between CPU share and processing rate is proposed. Since all the above solutions use queuing models they are valid only for steady state conditions and they do not examine if the operational point is feasible or not. In [71], a feedback admission controller using an LTI model was proposed, which also takes into account also CPU utilization to regulate the admitted workload. ACRA [72] is an integrated framework that solves jointly the admission control and resource allocation problems using a group of LTI models, which is not scalable as the number of consolidated services increases. Furthermore it was shown that the determined equilibrium point is always feasible. Yaksha [68] presented an admission controller, which is based on an initial queuing model and an LTI model that is derived from the linearization of the queuing model while a proportional-integral (PI) feedback controller is designed. However the obtained linearized residual error model may be large. Liu et al. [69] addressed the above limitation by using an adaptive controller to design the feedback loop working together with the queuing model through on-line tuning of model parameters from measurements. This self-tuning approach lowers the need for system characterization experiments in designing the controller and makes the system more robust. However the range of admitting probability adjustment is limited because of the linearization model near the operation point. In [73], authors demonstrated a resource allocation framework that regulates CPU frequency and VM capacity to minimize the power consumption of server clusters. They used decoupled linear models and PI and MPC (Model Predictive Control) controllers. Qin et. al. [75], used an LPV model to design a robust controller on frequency domain. However, the convergence of the identification algorithm is very sensitive to some parameters. 74

Similarly in [76], the authors derived an LPV model from the linearization of a queuing model and they designed a robust resource allocator in frequency domain using DVFS technology. In [74], the authors combined the admission control and resource allocation using LPV system identification and Model Predictive Control, which calculates the admission probability and CPU frequency using DVFS technology. Although their controller offers a trade-off between power consumption and SLOs achievement, it does not guarantee the performance among different operation points. Giani et al. [77] presented an LPV modeling combined with a PI admission control, without addressing resource allocation. It is one of the few studies that provided a stability analysis of their controller. In [70] the authors presented a two level control framework. The global level decides how many servers are necessary and the local level optimizes the CPU share of VMs using an MPC optimization algorithm. The model is derived from analytical equations where the operation point was chosen arbitrary. Finally DynaQos [78] is a model-free self tuning fuzzy controller which on the global level computes the necessary aggregated capacity and the local level computes the capacity share in order to succeed the SLA requirements. Apart from [72], all the aforementioned papers do not examine if their solution guarantees the feasibility of steady-state operation, the stability and the consolidated applications’ performance. Contrary to most existing approaches, in this chapter we propose two different local level controllers that address the above limitations and guarantee the performance of the application under the system constraints and any variation of their incoming workload. These local controllers solve admission control and resource allocation simultaneously in a unified framework, thus, making the cooperation with existing global level controllers easier. The main contributions of the thesis are:        

Satisfaction of the defined targets by the SLA. Optimal usage of the computational and network resources. Design and implementation of dual controllers that address admission control and resource allocation simultaneously. Accurate modeling of the web service operation. Satisfaction of the underlying system constraints. Computation of feasible operation points according to SLA. Guarantee and analysis of the system stability. Application of novel methods of control theory to recent network research problems.

The first local level controller, named ACRA (Admission Control and Resource Allocation) is a modeling and control framework that addresses RA and AC jointly. The objective is to maximize the provider’s revenue (by maximizing admittance of customers to the provided service), while satisfying customer’s QoS requirements. Although ACRA does not consider power management as basic target, it includes on its objective the minimization of the necessary computing resources. Instead of a 75

single linear state space model, a group of linear state-space models with additive uncertainties is used in order to cover the variation of workload and quantify the system’s nonlinearities. For the design of the dual controller, we use a novel settheoretic control theory which provides stability and robustness guarantees. Finally ACRA has the ability to adapt between several operating points, because it always assures that the system will be driven to the neighborhood of a feasible equilibrium point. The second modeling and control framework presented in this chapter adopts a Linear Parameter Varying (LPV) state space models that capture the dynamic behavior of the underlying infrastructure. The operating conditions are determined according to an optimization criterion. By solving a linear program, a set of feasible operating points is calculated which satisfies the desired QoS nominal values. The resulting stabilizing state feedback control law is affine state-dependent control law that can be easily implemented. Indeed, the computational complexity of the controller implementation is small, since at every time instant only a linear program and a point location problem are solved. Finally, convergence to the feasible operating point and satisfaction of the system’s constraints are guaranteed, for a number of desired operating points of interest. The remainder of the chapter is structured as follows. Initially, all the essentials notion of modeling and control are described in the following section. Sections 3.2 describes ACRA framework in full details. The second autonomous modeling and control framework is presented in section 3.3.

3.1 Basic Definitions of Modeling and Control Theory 3.1.1 Linear Time Invariant State Space Models A state space model of a system is the mathematical description of the relationship between the cause and the effect or the inputs and the outputs of the system [80]. In this thesis, only discrete time state space models are used. The general form of a discrete time state space model is,

x  k  1  f ( x(k ), u(k )),

f:

n



m



n

(3.1)

where x(k )  n , u(k )  m are the state and the input vector respectively and k  is the time variable which values are integer numbers. The most widely used state space models are the linear time invariant (LTI) state space models, where the function f ( x(k ), u(k )) of (3.1) is linearly dependent on

x(t ), u(t ) ,

76

x(k  1)  Ax (k )  Bu (k ) y (k )  Cx(k )  Du(k ) where A 

nn

,B

nm

(3.2)

are constant time invariant matrices that describe the

system’s dynamics and y(k ) 

l

is the output vector of the system.

3.1.2 Linear Parameter Varying State Space Models Although the LTI models have become a strongly founded modeling framework, they cannot describe efficiently the dynamics of complex systems. The need to operate processes with higher accuracy/efficiency has soon resulted in the realization that the commonly non- linear and time-varying nature of many systems must be handled by the control designs [81]. This has led to the birth of linear parameter-varying (LPV) systems, which are dependent on the varying signal p, hence the name parametervarying, while the dynamic relation between the system signals is still linear. One particular type of model structure, which is used in some LPV identification approaches, originates from the input-output (IO) type of representation of the data generating system in the LTI prediction-error setting. These LPV-IO representations are commonly defined in a filter form, na

nb

i 1

j 0

y   a i ( p ) q i y   b j ( p ) q  j u

(3.3)

where na  nb , the coefficients ai ( p), b j ( p) are functions of p with constant dependence and q i y, q  j u are the past values of states and inputs respectively that are used in the model. Another type of model structure is inspired by the classical state space (SS) representation based LTI models. The so-called LPV-SS representations of the data generating system are often given by the following form,

x(k  1)  A( p) x(k )  B( p)u (k ) y (k )  C ( p) x(k )  D( p)u (k )

(3.4)

where A( p)  na na , B( p)  na nb are matrix functions with static dependence on p. Furthermore, the matrices are often considered with linear dependence. In case of A, such dependence is defined as, np

A  A0   Al f l (p) l 1

77

(3.5)

where Al 

na na

have real elements and { f1 ( p),, f n p ( p)} is a set of known basis

functions. This type of dependence is called affine, and used as a core assumption in many LPV control-design approaches. In general, the right basis functions may not be known. Usually basis functions for which an algorithmic implementation is simple are used. The most common basis functions are polynomial or periodic. 3.1.3 Stability The notion of stability is strongly connected with the dynamic behavior of systems in modern control theory. Many different kinds of stability are defined, i.e. input-output stability or stability of an equilibrium point. In general, a stable system means that the state or output variables of the system are inside a desired area and they remain there even if there are insisting or momentary disturbances. In this chapter, we focus on the stability of an equilibrium point xeq . The most general type of equilibrium point stability is Lyapunov stability, which guarantees that the system trajectories will remain close to xeq if they start from a neighborhood of the equilibrium point [82]. Furthermore, asymptotic stability is adopted in the following sections. Additionally, set-theoretic notions of stability analysis and control design problem, which identify and characterize subsets of the state space containing the desired equilibrium state with special properties: positively invariant sets or ultimately bounded sets, are introduced here. In the forthcoming paragraphs without loss of generality, all the necessary definitions are given assuming that xeq  0 . Definition 3.1.1: A sphere Bs with radius s  0 and the origin as its center is denoted, Bs  {x 

n

: x  s}

where  is any possible norm of vector x . Definition 3.1.2: Assuming a discrete time system of the following form

xk  1  f ( x(k ))

(3.6)

Then the equilibrium point xeq  0 is locally Lyapunov stable, if and only if

  0     ( )  0 then, x0  B ( )  x(t; x0 )  B , t  0

Definition 3.1.3: The zero equilibrium point of (3.6) is contractive in a region D  n if x0  D then

78

The region D is called Domain of Attraction (DoA) of the equilibrium point. Definition 3.1.4: The zero equilibrium point of (3.6) is asymptotically stable if and only if it is Lyapunov stable and contractive,  B ( )  D

where D is DoA. In the following definitions, we present the essentials results of Lyapunov theory (second or direct Lyapunov method) which connect the stability property with a specific type of functions. Definition 3.1.5: Assuming a continuous function V ( x), V : D 

where D

contains the origin. Then V (x) is positive (semi)definite in D if

V ( x)  ()0, x  D \ {0} V(0)  0 Definition 3.1.6: Function V (x) is negative (semi)definite if  V (x) is positive (semi)definite. Definition 3.1.7: For discrete time systems (3.6), the total difference of function V ( x), V : n  respectively to system (3.6) is

V ( x) (3.6)  V ( f ( x))  V ( x)

(3.7)

Then the Lyapunov theorem for discrete time system is formulated as, Theorem 3.1: ([82],[80]) Assuming a positive definite function V ( x), V : D 

then



If the total difference V (x) (3.6) of (3.7) is negative semidefinite x  D , then



the system is locally Lyapunov stable. If the total difference V (x) (3.6) of (3.7) is negative definite x  D , then the system is locally asymptotically stable.

Function V (x) , which satisfies the above theorem, is called Lyapunov Function (LF). From the previous analysis, the stability problem is equal to finding a positive definite function which is nonincreasing or decreasing along the trajectories of the system (3.6). Finding an LF allows us to define sets with special properties respectively to the equilibrium point. For example, if there is an LF V (x) that guarantees the stability or asymptotic stability in a region D and the sets R(V ;  )  {x  n : V ( x)   }  D are close and contain the zero equilibrium point then these sets consist an estimation of DoA. In the forthcoming lines, all the essential definitions are given. 79

Definition 3.1.8: [83] The set S  n is positively invariant to (3.6) if and only if x0  S the system trajectory will remain inside S for all the future moments

x(t; x0 )  S , t  0 . Definition 3.1.9: [83]For a convex compact set S  Minkowski function is,

n

which contains the origin, the

 S ( x)  inf{  :   0, x   S} Then, the set S is ε-contractive to (3.6), 0    1, if x0  S S ( x(t; x0 ))   t S ( x0 ),

t  t0

Now we can show the connection between the Lyapunov function and the positive invariant set, the DoA and the ε-contractive set. Remark 3.1.1: Assuming the system (3.6) and a candidate LF V (x) , if the set

R(V ;  )  {x  n : V ( x)   } is convex, compact and contains the origin, then the following conclusions apply: 

If V (x) is LF that guarantees Lyapunov stability, then the set R(V ;  ) is



positive invariant to system (3.6). If V (x) is LF that guarantees asymptotic stability, then the set R(V ;  )  D is



positive invariant and DoA to system (3.6). If V (x) is LF that guarantees Lyapunov stability and it applies that

V ( x) (3.6)  (  1)V ( x(t )) , 0    1 then the set R(V ;  )  D is ε-contractive to system (3.6). Definition 3.1.10: [84] Let subset of the state space X  n be a compact set containing the origin as an interior point. System (3.6) is said to be uniformly ultimately bounded in subset X 

n

if there exists a subset Δ, X   

n

such that

for any t 0  T and every initial condition x(t 0 )  x0   there exists a positive integer

N ( x0 ) such that x(t; t 0 , x0 )  X for all t  t 0  N ( x0 ) . Set Δ is set to be the domain of attraction of X . The most important benefits from Lyapunov theory is the characterization of the stability of the equilibrium point and the possibility of defining sets with interesting properties, e.g. if we can find a LF which ensures the asymptotic stability of the equilibrium point of a constrained system x(t )  S x  n , then any trajectory that

80

begins from a point inside R(V ;  )  S x will be driven to the equilibrium point without violating the system constraints.

3.2 ACRA: A Unified Admission Control and Resource Allocation Framework for Virtualized Environments This section presents ACRA (Admission Control and Resource Allocation) [72] an autonomic modeling and control framework that provides accurate modeling, joint admission control and capacity allocation among different consolidated cloud computing services. The objective is to maximize the provider’s revenue by maximizing admittance of customers to the provided service, while satisfying customers’ QoS requirements. Although ACRA does not consider power management as basic target, it includes on its optimization criteria the minimization of the necessarily allocated computing resources. Instead of a single linear state space model, a group of linear state-space models with additive uncertainties is used in order to cover the variation of workload and quantify the system’s nonlinearities. For the design of the dual controller, a set-theoretic controller is used which provides stability and satisfaction of the underlying system constraints. Finally ACRA has the ability to adapt between several operating points, because it always assures that the system will be driven in the neighborhood of a feasible equilibrium point. According to the SLA, the average response time (state variable) during a specific time interval is denotes as SLO and the admitted request rate (admission control) and the CPU capacity (resource allocation) are the control variables. The total CPU capacity of a machine is shared among VMs that are deployed on it. ACRA controller aims to preserve the average response time under its target value with the minimum CPU resources. The proposing modeling and control scheme are based on the following features of web services. Although the incoming request rate of web services is generally non-stationary and highly variant, it usually follows specific patterns [85], which allow us to predict the workload conditions for a period in a day. Secondly, internet applications have many types of transactions. For example browsing on a site needs less computing resources than buying a product on the same site. Also administrative tasks, such as saving the data of the day or executing particular scripts during a specific time interval, have varying service demands and time constraints. Apart from the distribution and the type of incoming requests, the constraints of the application’s infrastructure should be taken into account. CPU and memory resources of the underlying machines are limited, so the reserved capacity for the co-hosted VMs should never exceed the total capacity. Also there are constraints on the amount of the served incoming requests from the VMs. Finally there exists an upper limit of the response time of the incoming requests due to VM’s service rate and network restrictions. All the above features should be considered in the system identification and the controller design. Figure 3.2 illustrates the structure of ACRA framework. 81

System Constraints Feedback u Controller

LTI Models Equillibrium Point

Figure 12 – Structural Diagram of ACRA Framework

A single mathematical model cannot describe the system’s dynamics under any circumstances. Thus a set of linear time invariant (LTI) state-space mathematical models is identified in order to cover the whole range of workload. Furthermore, an additional term that models additive bounded uncertainties and nonlinearities of the system is introduced. This modeling approach is accurate and allows determining many different feasible operating points. Depending on the current workload conditions and the desired SLO values, a set of feasible operating points is computed by solving a multi-criterion optimization problem which takes into account the system constraints. This implies that the system trajectory will always be inside a desired operating area and it will be adaptive towards workload fluctuations. Finally ACRA addresses admission control and recourse allocation as a dual decision problem and it guarantees the system stability and the satisfaction of the system constraints. A settheoretic feedback controller is used which has the ability to ensure further properties of stability like positively invariant sets and ultimately bounded sets. 3.2.1 Modeling and System Identification In general, multi-tier services, which are deployed on many VMs, have complex and non-linear dynamic behavior. Modeling the system’s dynamics with a single linear state space model is not precise. A group of linear state space models is used to cover the total range of incoming workload. Assuming that there are m applications, n servers and each application has one VM on every server. For each application, the incoming request rate varies between a minimum ( Lmin,i ) and a maximal value ( Lmax,i ) . We suppose that Lmin,i corresponds to a small positive value and Lmax,i to the maximum served request rate when all VMs of the application get their maximum CPU capacity. Next, we divide each interval [ Lmin,i , Lmax,i ] for every application

i  1, m in p smaller equidistant parts, e.g. spanning a range of 50 requests each. Thus, combining all applications, a number of N  pm different regions are defined in the “request rate” space. For each region, we assign a LTI model affected by

82

additive disturbances denoted by M q , q  1, N which describes the system’s dynamics, when the request rates are inside region i

M q : rt (k  1)  Aq rt (k )  Bqu(k )   (k ), q  1,, N

(3.7)

where rt  m is the response time (state) vector of all m applications, u  ( n1) m is the input vector that contains the nm VM capacity inputs capij , i  1,, m, j  1,, n and the m admitted request rate variables lad i , i  1,, m . The effect of nonlinearities and uncertainties is represented by time varying vector   m which is considered here as unknown but bounded additive disturbance. Without deteriorating the accuracy of the modeling, Aq , q  1,, N are restricted to be diagonal matrices implying that response time of each application depends only from its own past values. Also we assume each application does not depend on the inputs (VM capacity and admitted rate) of other co-hosted applications, since this does not improve the model’s accuracy. Thus the elements of matrices Bq , q  1,,N , which do not correspond to the application’s inputs, will be zero. The input vector u (k ) and the matrices Aq , Bq are formulated as, u(k )  [cap11 cap ji lad1 ladi ]T , i  1,, m, j  1,, n

a11  0  Aq      , i  1,, m, q  1,, N  0  amm  b11  b1r  Bq      , r  1,, m, l  1,, (n  1)m br1  brl 

We use the Recursive Least Square (RLS) algorithm [86] to identify the nominal disturbance-free LTI models, M q* : rt(k  1)  Aq rt(k )  Bq u(k ), q  1, N

(3.8)

The bounds of vector  (k ) of each model (3.7) are determined by the minimal and maximum error of the RLS algorithm during the identification of the corresponding nominal model M q* . As inputs profiles, we use a mix of Gaussian, constant and Poisson distributions for incoming load and uniform distributions for VMs capacity. These typical traffic patterns correspond to normal conditions and sudden spikes of incoming requests. For each local model M q , q  1,, N , all the tests always satisfy the states and input constraints, which are analytically described in the next section. 83

3.2.2 State and Input Constraints There are some constraints of the state and input variables that must be satisfied under any circumstances. The average response time on a specific time interval varies from zero until the value when the system is saturated or request expired due to network constraints (TCP). Thus, states are bounded to the constraint set X, where X  {rt 

m

: 0  rti  rtmax , i  1,

, m}

(3.9)

Input constraints consider the restrictions on VM capacity and admitted request rate. Many VM can be deployed on a physical server and they share its CPU capacity. For each server j, j  1,, n , the CPU capacity of each VM varies from a minimum value cmin until a maximum cmax which depends from the total capacity of the server. Additionally the sum of VMs’ entitlement should not exceed the total capacity of the server cs max, j , j  1,, n .

cmin  cap ij  cmax , i  1,, m, j  1, n m

 cap i 1

ij

 c s max, j , i  1,, m, j  1, n

(3.10) (3.11)

There are also constraints regarding the admitted workload. As we mentioned in the previous subsection, the incoming load of each application varies from Lmin,i to Lmax,i . For each local model the admitted request rate is between Lmin,i and Lmax,i values,

lad min  lad i  lad max , i  1,, m (3.12) Putting together the input constraints, we can define the input constraint set u U as it follows,

U  {u 

( n 1) m

: cmin  capij  cmax , i  1, m

 cap i 1

ij

 cs max, j , i  1,

lad min  ladi  lad max , i  1,

, m, j  1,

n

, m, j  1,

n

, m } (3.13)

Finally the uncertainties of the model are bounded by the minimal and maximum values of modeling error of the system identification. Thus uncertainties constraints are defined as   N where, N  { 

m

: nmin  i  max , i  1,

84

, m}

(3.14)

3.2.3 Determination of the Equilibrium Point The target value of SLO is time varying according to workload profile and the specific tasks of the application during a day. For each desired operating value of average response time and admitted request rate, the corresponding equilibrium point has to be determined for our system, which is described by a local LTI state space model. After the definition of the systems constraints and given the optimization criteria such as maximization of admitted request rate and minimization of CPU resources, the feasibility of a desired operating (equilibrium) point has to be examined. Given a nominal model M q* , the desired response time vector RTref  X 

m

and

the

desired

admitted

request

rate

vector

Lref  [lad ref ,i ]T , i  1,, m , there is a feasible equilibrium point if there exists an

input

vector

U ref  [CAPref Lref ]T , U ref U 

( n 1) m

such

that

RTref  ARTref  BU ref . In order to relax the above definition of the equilibrium point

we reformulate it as follows, m  m n  min wc   cap ij  wd  d i  capij ,ladi , d i i 1  i 1 j 1  subject to RTref  ARTref  BU ref

(3.15a)

(3.15b)

RTref  X   m

(3.15c)

U ref  U   ( n 1) m

(3.15d)

 d i  lad i  Lref  d i

(3.15e)

di  0

(3.15f)

(3.15a) can be solved as an optimization problem with constraints. The constants wc and wd are the weights of the cost function and depending on their values we can have a trade-off between minimizing the VMs’ capacity and succeeding the desired admitted request rate. (3.15b) corresponds to the condition of the existence of the equilibrium point. (3.15c) and (3.15d) correspond to the state and input constraints respectively. The auxiliary variables di are defined in order to relax the definition of equilibrium point. (3.15e) and (3.15f) guarantee that if there is no solution that contains exactly the desired Lref , then the solution of the problem will be the closest value to Lref which satisfies all the constraints. The above formulation guarantees that the computed operating/equilibrium point always satisfy the system constraints. 3.2.4 ACRA Controller Design Admission control and resource allocation are two decision problems that must be considered jointly. A novel controller is presented here that computes the suitable 85

input vector to lead and stabilize the system near to the desired equilibrium point using the corresponding local system model. Most of the proposed solutions focus on stabilizing the system trajectory on a specific operating point, but this is impossible because the systems dynamics involve nonlinearities in the state space description as well as time-varying additive terms that represent uncertainties. Thus there is no feedback control law ensuring asymptotic stability of an equilibrium state. Instead settheoretic approaches deal with the stability analysis and control design problem, identifying and characterizing subsets of the state space containing the desired equilibrium state with special properties: positively invariant sets, disturbance invariant sets, or ultimately bounded sets. For more information, regarding these notions, readers can refer to the excellent monograph [83], summary paper [87], and articles [84], [88]. In our case, since we consider input and state constrained (3.9) and (3.13) systems affected by additive disturbances (3.7), and it seems straightforward to follow the settheoretic approach. Our goal is to design for each model M q , q  1,, N , corresponding to a given workload profile, an affine state feedback control law and compute a domain   m of the state space such that all trajectories starting from  are transferred to a target set R, which contains the equilibrium point, in a finite time and remain in it. In contrast to classical control problem formulations, we want to confine all trajectories in a target set R and not to drive them to the equilibrium point since the system is affected by additive disturbances. The resulting closed-loop system is said to be ultimately bounded in R from  [84], [88]. In order to apply these methods, initially we apply the essential coordinates transformation to obtain a transformed nominal linear model with the equilibrium point to the origin: z (k )  rt (k )  RTref v(k )  u (k )  U ref

(3.16)

M q : z (k  1)  Aq z (k )  Bq v(k ), q  1, N

Also state and input constraints are transformed accordingly, X  {z 

U  {v 

( n 1) m

m

:  RTref  z  rtmax  RTref }

(3.17)

: umin  U ref  v  umax  U ref ,

m

m

i 1

i 1

 vcapij  cs max, j   vcapref ,ij , i  1,

, m, j  1,

n} (3.18)

where vcapij  capij  capref ,ij transformed VMs capacity inputs. We will design a state feedback controller of the following form,

86

u (k )  K (rt (k )  RTref )  U ref or v(k )  Kz (k )

(3.19)

Secondly, we must compute the target set R, which must possess the robust invariance property [83], meaning that if the closed-loop system trajectory is in R, it remains in it for all future instances. There are many ways to obtain the target set, preferably small enough to guarantee that SLOs are met when the system’s trajectories are inside. Indeed, there are methods to approximate a minimal positively invariant set for this class of systems [89], although this is outside the scope of the current solution. In our case, we obtain a candidate target set S1 by applying the Jordan decomposition [90] of the closed-loop system A  BK , where K is the gain matrix that places its eigenvalues inside the unit rhombus [91], S1  {z 

where G1 

2 mm

V  : G1    , w1   V 

m

2m

: G1 z  w1}

(3.20)

and V is the real Jordan form of matrix

A  BK .

The state and input constraints set X and U are formulated below using the standard half-space representation of polytopes by Sz and Sv,

where Gz 

2 mm

, wz 

S z  {z 

m

Sv  {v 

( n 1) m

2m

and Gv 

: Gz z  wz }

(3.21)

: Gv v  wv }

2 n ( m1)mn

, wz 

2 n ( m 1)

.

The bounded uncertainties set S is, S  { 

where G 

2 mm

m

I  : G   mm  , w    I mm 

: G  w } 2m

(3.22)

.

Thirdly, our goal is to compute a control law of the form (3.19) and a set Δ such that the closed-loop system is ultimate bounded in R. We choose Δ to have the form,   {z 

m

: G1 z   w1}

(3.23)

According to the generalized Farkas’ lemma [92], we can compute a control law v  Kz and set Δ, which is the largest set, fitted in Sz and Sv. This can be described by the optimization problem, 87

max

K , H1 , H 2 , H 3 , ,

 

subject to G1 ( A  BK )  H 1G1

H 1 w 2  d   w 2 H 2 G1  G z

H 2 w 2  w z H 3G1  Gv K

H 3 w 2  w v 0 * The above optimization problem is non-linear because of the first inequality, thus we 1 define b  and reformulate the problem as linear,



min

K , H1 , H 2 , H 3 , ,b

b

(3.24a)

subject to G1 ( A  BK )  H1G1 H1w 2 bd   w 2

(3.24b) (3.24c)

H 2G1  Gz

(3.24d)

H 2 w 2  bw z

(3.24e)

H 3G1  Gv K

(3.24f)

H 3 w 2  bw v

(3.24g)

0 

*

(3.24h)

 max V i  where d   i , H1 , H 2 , H 3 are non-negative matrices, K is the feedback  V i   max i  gain matrix, and ε is positive between zero and a small value  * (i.e.  *  1 ) and it is a metric of convergence speed. (3.24b), (3.24c) guarantee the positive invariance and contractiveness. Equations (3.24d), (3.24e) guarantee that  satisfies the state constraints and (3.24f), (3.24g) guarantee that   S v ( z ) , where S z  {z  m : Gv Kz  wv } , satisfies the input constraints.

3.2.5 Evaluation In order to evaluate ACRA a testbed is built, which emulates the operation of an application in a virtualized environment. A server with 8 CPU cores (Intel(R) Xeon(R) CPU E5405 2.00GHz) and 8GB RAM is used. One CPU processor is dedicated for the client generators of each application, which produce requests of different distributions, such as constant, uniform and Poisson. Another core hosts the VMs of each application. Each VM runs a script which does some complex 88

mathematical calculations that are CPU intensive. On a third core we place a VM that hosts the controller of ACRA. We use XEN Hypervisor as a VMs hypervisor [93]. The testbed includes all the necessary monitoring tools to measure the average response time, incoming workload rate, served request rate and absolute and relative CPU utilization. We demonstrate an experiment, which shows ACRA’s performance and compares it with two well-known approaches. We assume two applications (m = 2), which are hosted on the same CPU core in two separated VMs (n = 1). The incoming workload for both applications varies between [175,325] req/s. Following the system identification method described in section 3.2.1, we divide the ”request rate” space into hypercubes (squares for two-dimensions space) with a step of 50 requests and generate a local linear model for each square, such as in (3.7). Figure 3.3 illustrates the created hypercubes (squares) and the produced models Mq. The measurements and control are updated every 20sec.

Figure 3.3 – Group of LTI Models

89

For both applications it is initially assumed that the admitted request rate is high and their computing resources for this type of requests (e.g. browsing) are low. Thus the equilibrium point corresponds to the reference value of average response time of 1s and the desired admitted request rate is 300req/s for both services (model M9 on Fig. 3.3). After a period there is need to change the SLO target value of both services, since the dominant type of transactions needs more CPU resources and has lower incoming request rate than the previous. This determines the target of average response time of to be 2s and desired admitted request rate 200req/s which corresponds to Model M1 on Figure 3.3. The modeling uncertainties vary inside the bounded area of [−0.4, 0.4] for both candidate equilibrium points. ACRA state feedback controller is applied to regulate the system and the controllers of [73] and [74] are used in order to compare our results with some well-established solutions. The first controller is a classical proportional-integral (PI) controller, which is used also in many other studies. The necessary eigenvalues vector of the PI controller is set

to eig  0.4 0.5 . The second one is an MPC scheme based on predictive control theory [94]. The predictive horizon of the MPC controller is set to H = 5 and the trade-off parameter of the approach a = 0.6. The cost function of the controller is T

J  a z (k )  (1  a)

K  H 1

 v( k ) .

k K

(a) Equilibrium Point 1

90

5 Intersection of Sz and Sv(z) Ser R Ultimate Bounded by Set  Positively Invariant Target Set R

4 3

rt2

2 1 0 -1 -2 -3 -4

-2

0

2 rt1

4

6

8

(b) Equilibrium Point 2 Figure 3.4 –State and Input Constraints set Sz, Sv(z), Positevely Invariant Set Δ, Target Set R.

Figures 3.4a and 3.4b are graphical representation of the set theoretic controller we used in our approach. On both graphs, the yellow set shows the intersection of the state and input constraints set S z  S v ( z ) , the red set depicts the positively invariant set Δ and the cyan set is the target set R. Set S z  S v ( z ) on figure 3.4a is identical with  , thus it is not shown. As we observe in both figures, R is subset of  ( R   ), which means that the trajectory of the system is always in the neighborhood of the target set and remains there. Figure 3.5 shows the average response time that each controller succeeds for both applications. The three controllers regulate efficiently the system near the first equilibrium point. They do not violate the target values. On the other hand, for the second equilibrium point, the figure highlights the difference between ACRA and the rest approaches. PI and MPC controllers fail to drive the system near the equilibrium point for different reasons. Initially, the PI controller does not take into account the additive disturbances and the constraints of the system. Furthermore, it implies asymptotic stability and tries to drive the system on the equilibrium point. Thus results either to oscillations like the response of the first application or to driving the system far from the second equilibrium point (application 2). 91

Figure 3.5 - Average Response Time of ACRA, PI, MPC

Also MPC controller fails to stabilize the system near the desired target values. This happens because this controller does not considered additive disturbances and does not provide any stability guarantee. Additionally its performance is very sensitive to the weights of the cost function. We conduct many trials to find a suitable value of trade-off parameter α and we conclude that even small changes of it have large impact on the performance and the stability of the controller. The percentage of time intervals where average response time violates the target value for ACRA, PI and MPC are 4.16%, 45.83%, 50.83% respectively.

92

Figure 3.6 – Reference and Admitted Request Rate

Figure 3.6 depicts the admitted and desired value of request rate. The graph shows that ACRA outperforms the two other approaches, since it admits the highest request rate, especially for the first equilibrium point. The average admitted request rate of the two equilibrium points for each approach are (314.38/202.67req/sec), (296.91/199.86req/sec), (301.65/202.66req/sec) respectively. 93

(a) Average Response Time

(b) Admitted Request Rate Figure 3.7 – Performance of MPC, PI controllers using a single LTI Model

94

Figures 3.7a and 3.7b show the performance of PI and MPC controllers when we use only one LTI model for any workload conditions. On figure 3.7a, the target values of the average response time and admitted request rate are the same like figure 3.5. However for the first equilibrium point of Application 2, both controllers fail to lead the system near the target value of admitted request rate as it is depicted on figure 3.6b. Furthermore both controllers do not perform sufficiently when the equilibrium point changes. As it is illustrated on figure 3.7a, on the second part of the experiment (eq. point 2) there are many violations of the target SLO value, which means that both controllers are inadequate. From these graphs, it is obvious that a unique LTI model cannot describe the system’s dynamic behavior under any workload circumstances. Although ACRA does not have as a primary goal the minimization of the system’s power consumption, it aims to minimize the reserved VMs’ capacity, which implies minimization of the consumed energy. In the following example, we show ACRA’s performance when on the first part of the experiment the desired average response time is 1s and the admitted request rate is 300req/s and on the second half the target values are 1s and 200req/s. Figure 3.8 illustrates that the controller reduces 18% the utilization of VMs capacity and consequently the power consumption of the system on the second part the resources demands are less than the first one.

Figure 3.8 – Average Response Time and VMs Capacity Allocation of ACRA

95

3.3 An Autonomous Admission Control and Resource Allocation Framework for Consolidated Web Applications Contrary to most existing approaches in the introduction of this chapter, an autonomous modeling and control framework [95] is presented in this section, which addresses admission control and resource allocation simultaneously in a unified manner. In specific, a Linear Parameter Varying (LPV) state space model is adopted to capture the dynamic behavior of the underlying infrastructure. The operating conditions are determined according to an optimization criterion. A feasible operating point, which satisfies the desired QoS nominal values, is computed. The resulting stabilizing state feedback control law is an affine state-dependent control law that can be easily implemented. The computational complexity of the controller implementation is small, since the calculation of the control elements is done offline and at every time instant only a linear program and a point location problem are solved. Finally, convergence to the feasible operating point and satisfaction of the system’s constraints are guaranteed, for a set of desired operating points of interest. The local level controllers on Figure 3.1 aim at satisfying the SLOs reference value, the physical constraints and at minimizing the power consumption. Figure 3.9 depicts the system’s architecture for the consolidation of two three-tier applications. Typically, web applications consist of many components such as web servers, application servers and databases. The local controller must solve admission control and resource allocation as a common decision problem; it determines the admittance

c1 ,1 ,..., c 2 , 3

Modeling p1 , p2 and Control Framework

VM1,1

VM1,2

VM1,3

VM2,1

VM2,2

VM2,3

Web Server

Application Server

Database

Figure 3.9 – System Architecture

96

System Constraints ~



 Predictor

Feedback u Controller

LPV Model Equillibrium Point

Figure 3.10 – Structural Diagram of Autonomous Framework.

probability ( p1 , p2 on Figure 3.9) for every service and the capacity (CPU) share for their VMs ( c1,1 ,

, c2,3 on Figure 3.9).

Figure 3.10 illustrates the modeling and control framework presented in this section. Initially, an LPV model is obtained in order to describe the systems dynamic behavior. This kind of modeling is suitable because it explicitly takes into account the systems parameters such as the prediction of the incoming request rate and service rate. Next, the existing physical system’s constraints are quantified. The CPU capacity of each VM varies between two limit values and the sum of the VMs capacities in each server cannot exceed the total capacity of the server. In addition, response time and incoming request rate vary between two values that can be extracted precisely by extreme experiments that lead servers near to their saturation point. These constraints are taken into account by the element of Figure 3.10 which determines the equilibrium point. The determination of the equilibrium point is addressed as a decision problem which depends on several objectives. Finally, a stabilizing state-feedback controller is designed and implemented which takes into account the SLOs target values, the system’s constraints and the determined equilibrium point from the previous step. Consequently, all consolidated applications have a certain level of QoS for a wide range of workloads. 3.3.1 Modeling and Identification Modeling the dynamic behavior of consolidated web services on a cluster of servers is challenging, since there are no analytical equations which capture the system dynamics. In previous work [72], a group of LTI models that covers the range of the incoming workload was used. This approach is more precise than a single LTI model 97

but it is not scalable with respect to the workload range and the number of co-hosted applications. However there are scalability and accuracy problems as the number of consolidated applications increases. In order to overcome this issue, we adopt an LPV model, which changes according to a system parameter. The LPV model substitutes the group of LTI models and it is scalable towards the range of incoming workload. Moreover it is proven to be more accurate. We consider n consolidated applications on m servers and each application has one VM on every server. We select as state variables xi  , i  1,

, n , the 90th percentile

of response time at the k th fixed time interval. The state vector at time k is denoted by

x

n

, i.e., x :  x1  xn  .

We consider as inputs the CPU capacities ci , j , i  1,

, n and j  1,

, m of each VM,

where i denotes the application and j the server, and the admittance probabilities

pi , i  1,

, n for each application. Consequently, the input vector u 

( m1) n

is

defined as follows, u : c1,1 ... c1,m

c2,1 ... c2,m ... cn,1 ... cn,m

p1 ...

pn 

(3.25)

The LPV model is of the following form,

xk 1  A(sk ) xk  B(sk )uk , (3.26) where A( sk ) 

nn

, B(sk ) 

The parameter vector s 

n

n( m 1) n

are the system matrices and k is time variable.

of the LPV model contains the prediction values of the

incoming request rate of each application i , i  1,

sk  [1k

2

k

, n , i.e,

 nk ] .

Table 3 summarizes the notation used throughout the section. The system (3.26) was identified using the algorithm of [96]. The parameter varying matrices A  sk  , B  sk  are chosen to have a linear dependence on vector sk , k  , i.e. n

n

i 1

i 1

A  sk   A0   si Ai , B  sk   B0   si Bi

98

(3.27)

where Ai 

nn

, i  1,

, n and Bi 

n( m1) n

, i  1,

, n are linear invariant matrices,

which elements are identified below. In specific, the following LPV input-output n

Number of consolidated applications.

m

Number of active servers.

xk

90th percentile of response time in the k th time interval

ci , jk

CPU capacity of VM i j that belongs to i th application on j th server in the k th time interval

pik

Admittance probability of i th application in the k th time interval

sk

LPV model parameter

i

Incoming request rate of i th application in the k th time interval

i

Prediction of Incoming request rate of i th application in the k th time interval

xref

Target response time vector

uref

Equilibrium point input vector

Tref

SLA highest acceptable response time vector

X

Set of state constraints

U

Set of input constraints

k

k

Table 3.3 - Notation of State, Input Variables and Parameters. (LPV-IO) representation method is used to derive the unknown coefficients from past measurements, xk  a1 (sk ) xk 1 

 ana (sk ) xk na  b1 (sk )uk 1 

 anb (sk )uk nb ,

where na , nb are the number of past values of the state and the input vector respectively. The coefficients a1 (sk ), i  1,

, na and b1 (sk ), i  1,

, nb induce the

elements of matrices Ai (sk ), Bi (sk ) of (3.27). In order to evaluate the extracted model, the standard Best Fit Rate (BFR) [81] score was used. In a typical testbed described analytically in the following evaluation 99

subsection, the identification experiments show that taking into account past values of states and inputs enhance the accuracy of the model. However, for na  3, nb  3 the improvement is negligible. Also, as expected, it was found that LPV-RLS algorithm has significantly higher performance than the classical RLS algorithm [86] for LTI modeling. These results are summarized in Table 4.

na  1, nb  1

na  2, nb  2

na  3, nb  3

LPV-RLS

76.72%− 82.61%

79.37%− 86.79%

80.29%− 87.17%

RLS

43.45%− 57.04%

45.08%− 59.23%

45.54%− 59.64%

Table 3.4 - Comparison of the BFR score of the LPV-RLS and RLS identification algorithms. In order to obtain a representation of the model that allows the development of the subsequent result, the matrices A  sk  , B  sk  can be described as the convex combination of the ”extreme” subsystems defined by the ( Aˆi , Bˆi ), i  1, , 2n , which correspond to the extreme values of sk ,

matrix pairs

2n

A  sk    mi Aˆi

(3.28a )

i 1 2n

B  sk    mi Bˆi

(3.28b)

i 1

mi  0, i  1, 2

, 2n

(3.28c)

n

m i 1

i

1

(3.28d )

This dual representation [83] corresponds to linear discrete time systems with polytopic uncertainties. 3.3.2 Request Rate Predictor In order to predict the incoming request rate of each application, Holt’s linear exponential smoothing (LES) filter is used [97], which can capture the linear trend in the time series. For example, during time step k, the estimated value k of incoming request rate k for a one-step prediction horizon is obtained as follows,

k  ˆk  bk , ˆk  k  (1   )(ˆk 1  bk 1 ), bk   (ˆk  ˆk 1 )  (1   )bk 1 , 100

where  ,  are smoothing constants, ˆk denotes the smoothed value for time step k and bk represents the linear trend in the measurement series. For initialization, we use a random value for ˆ0 inside the range of incoming request rate and b0  0.5 . 3.3.3 State and Input Constraints The computational resources of a server cluster, which hosts a group of applications, are inherently limited. Moreover, there are network constraints that bound the range of response time and incoming request rate. The 90 th percentile of response time xi of the application i in any time interval varies from a small positive value  (e.g 0.01ms) to the value when the system is saturated or the requests has expired due to network constraints. Thus, states are bounded to the constraint set X, where  {x 

n

:   xi  xmax , i  1,, n}.

(3.29)

On the other hand, input constraints consider the restrictions on VM capacity and admittance probability. For each VM of application i on server j, the CPU capacity ci , j varies from a minimum value cmin to a maximum cmax . Additionally, the sum of VMs entitlement should not exceed the total capacity of the server CT j . The admittance probability ranges from a minimum value pmin to pmax  1 . All the above constraints form the input constraint set U,  u 

( m 1) n

: cmin  ci , j  cmax , i  1,

, n, j  1,

, m,

 CT j , j  1,

, m,

pmin  pi  pmax , i  1,

, n .

n

c i 1

i, j

(3.30)

We remind that the input vector is defined by (3.25). 3.3.4 Determination of the Operating Point LPV model of (3.26) allows identifying a number of desired feasible operating points. From a set of candidate equilibrium points we choose one that satisfies the SLA requirements and the state and input constraints (3.29) and (3.30) respectively. The determination of this point is a decision problem that includes several competing objectives, such as strict response time target and minimization of power consumption. Thus depending on the priority of the goals set and selecting accordingly the corresponding cost function, an optimization problem can be formulated whose solution is the desired equilibrium point. In particular, we choose to guarantee the response time reference value of each service and maximize the admittance probability without violating the system constraints.

101

To this end, given a desired response time xref , an input vector uref is computed by the solution of the following linear optimization problem, n  n m  w c  wdi di  (3.31a)    ij i , jref ci , jref , piref , di i 1  i 1 j 1  subject to x  Aˆ x  Bˆ u , i  1, , 2n , (3.31b)

min

ref

l ref

l ref

  xref  xmax , i  1,

, n,

cmin  ci , jref  cmax , i  1, n

c i 1

i, j

 CT j , j  1,

, n.

, n, j  1,

, m,

pmax  piref  di , i  1, di  0, i  1,

(3.31c)

, n,

, m,

(3.31d)

(3.31e) (3.31f)

(3.31g)

Equation (3.31b) ensures that ( xref , uref ) is the equilibrium point for all possible parameter variations. Equations (3.31c)-(3.31f) guarantee state and input constraint satisfaction. The positive numbers wij , i  1, , n, j  1, , m and wdi , i  1, , n are weights of the optimization cost function and their values correspond to the trade-off between the competitive objectives. The auxiliary variables d i in the cost function and the last two equations are used to maximize the admittance probability. In accordance to (3.25), it follows for the input vector uref that, uref  c1,1ref

c1,mref

cn,1ref

... cn,mref

p1ref

pnref  .

3.3.5 Controller Design In this section, we focus on the controller synthesis and implementation, by addressing jointly the resource allocation and admission of requests. Due to the presence of state and input constraints, apart from computing stabilizing controllers for each desired operating point, it is important to characterize regions of attraction in the state-space for the closed-loop system. In other words, we are interested in computing the set of initial conditions in the state-space that can be driven (asymptotically) to the desired operating point. Thus, the problem to be investigated can be formulated as follows: Given the LPV system (3.26), a set of desired operating points xref ,i , i  1, , N is determined by the method presented in the previous subsection and the state and input constraints, Eq. (3.29) and (3.30), compute a state-feedback control strategy

102

gi   :

n



n

, i  1,

, N and a domain of attraction (DoA) Ri 

n

, such that the

closed-loop system is locally asymptotically stable with respect to xref ,i . Since the incoming workload is unpredictable and sudden changes occur, it is important to have different operating points in order to satisfy always the system constraints and to ensure that the response time violations, according to SLA, will be as less as possible. Thus, a second problem to be investigated is to compute a control strategy such that the closed-loop system can change efficiently between different operating points, without violating the state and input constraints. Assuming that a non-zero response time vector xref ,i is chosen to determine the feasible equilibrium point, we apply the following coordinates transformation in order to obtain a transformed nominal model that has its equilibrium point to the origin. Thus for each equilibrium point ( xref ,i , uref ,i ), i  1, , N , computed in subsection 3.3.4, we define the translated system zk  xk  xref ,i , vk  uk  uref ,i ,

(3.32)

zk 1  A( sk ) zk  B( sk )vk .

(3.33)

Also for the ith system (3.33), the state and input constraints are transformed accordingly to z  i

i



 z

i

,v

 z 

( m 1) n

i

n

, where

: cmin  ci , jref  tci , j  cmax  ci , jref , i  1,

, m,

 CT j   ci , jref , j  1,

, m,

pmin  piref  tpi  pmax  piref , i  1,

,n .

n

i 1

tci , j  ci , j  ci , jref

are

(3.34)

, n, j  1,

 tc

where

, n ,

:   xref ,i  z  xmax  xref ,i , i  1,

the

i, j

n

i 1

transformed

VMs

(3.35)



capacity

inputs

and

tpi , j  pi , j  pi , jref are the transformed admission probability inputs. The above

constraints can be rewritten in the following form of linear inequalities,

z

i

v

i

where Ci ( j ) 

 z 

n

 v 

( m 1) n

1n

: Ci ( j ) z  1, i  1,

, N , j  1,

: Di ( j )v  1, i  1,

and Di ( j ) 

1( m1) n

, p

, N , j  1,

(3.36) , q

(3.37)

are the rows of matrices Ci , Di respectively

which contain all the constraint inequalities for each system (3.33). One well known 103

approach in control engineering when trying to estimate simultaneously the DoA and a stabilizing control law is to apply Lyapunov based methods, see e.g. [83, chapter 2] and [98, chapter 4]. We consider a linear state-feedback control law gi   for each system i, i  1,

, N (3.33), of the form gi  z   Ki z, i  1,

,N ,

M i : zk 1  ( A(sk )  B(sk ) K j ) zk , i  1,, N . (3.38)

For each closed-loop system we consider a quadratic Lyapunov function (LF) Vi   , i  1, , N of the form Vi ( z )  z  Pz i

(3.39)

For each closed-loop system (3.38), we have to determine a matrix Pi

0 that

satisfies the one-step decrease along the trajectories of the system (3.38) requirement of the LF, as well as the feedback gain K i , i.e.,

 Ai  Bi Ki 

Pi  Ai  Bi Ki 

0

(3.40)

To eliminate the non-linearities in (3.40), we set Qi  Pi 1 and Yi  Ki Qi and by pre and post multiplying eq. (3.40) with Qi we get 1 ( AQ  BY  BY i i i ) Q ( AQ i i i )  Pi

0.

(3.41)

Applying the Schur complement [99] we get the following equivalent form Qi  ( AQ  B Y ) i i  i i

AQ i i  BiYi   Qi 

0,

(3.42)

which is a linear matrix inequality with respect to Qi , Yi . Then we can compute the gain matrix K i and Qi , Pi for each system (3.38) M i , i  1,..., N , by solving N convex optimization problems,

104

min trace(Qi )

Qi ,Yi ,

(3.43a)

subject to Qi Ai Qi  BiYi   ( A Q  B Y )  0, (3.43b) Qi i i  i i  1 Ci ( j )Qi   0, j  1, , p, (3.43c) (C ( j )Q ) Qi  i  i 1 Di ( j )Yi   ( D ( j )Y ) Qi  i  i Qi 0, (3.43e) 0   1.

0, j  1, , q,

(3.43d)

(3.43f)

In the above problem, inequalities (3.43b) and (3.43e) ensure local asympotic stability for the closed-loop system. Inequalities (3.43c) and (3.43d) guarantee state and input constraint satisfaction. Parameter is a measure of the speed of convergence of the closed loop system to the equilibrium point. The problem is convex and can be solved using off-the-shelf software, e.g the Matlab Robust Control Toolbox [100] or the SDPT 3.0 software toolbox [101]. The interested reader is referred to, e.g., [99], [102, Appendix A], for further details. The Lyapunov matrix and the gain matrix for each operating point are finally computed as Pi  Qi1 and Ki  Yi Q 1 respectively. Together with the controller, the set Siz  {z 

n

: z Pz i  1}

is an admissible DoA for the closed loop system , i.e. it is the set which contains all initial conditions that can be transferred asymptotically to each equilibrium point without violating the constraints. Transforming back to the initial state space, the domain of attraction for this system is Ri  {x 

n

: ( x  xref ,i )T Pi ( x  xref ,i )  1}.

(3.44)

Finally the control law for the initial system (3.26) is given by, uk  Ki ( xk  xref ,i )  uref ,i .

 3.45

Since (3.39) is a LF, Ri is a   contractive for the closed-loop system (3.38) [83]. This property of the set Ri ensures that for any x(0)  Ri the system will be driven to the desired operating point xref ,i without violating the input and state constraints. Figure 3.11 illustrates the domains of attraction of each operating point. For example the DoA of xref  [4 2.5] is highlighted. If any point inside the blue ellipsoid is the initial state then an admissible control law exists such that the system trajectory is 105

transferred to the equilibrium point xref  [4 2.5] . It is worth noting that the computation of the feedback gain and the domain of attraction Ri are performed offline using the dual representation of (3.27).

Figure 3.11 - DoAs Ri of each operating point xref,I , i=1,…,N.

From the set of desired operating points xref ,i , i  1,.., N , there is one operating point that is closest to the SLA requirements. According to the SLA, for each consolidated application there is a highest acceptable response time Tref* ,i , i  1,, n . Thus, an accepted operating area is implicitly shaped on the state space where the state variables should remain for all time intervals. The desired operation point is chosen to be the center of this area since then the corresponding DoA will include the largest part of the accepted operating area. We denote this desired operation point as * * ( xref , uref ) , its corresponding DoA R* and feedback gain matrix K * . For each * * application, we select as desired operation value xref ,i  Tref ,i / 2, i  1, , n and the * * * desired operating point of the system is formulated as xref  [ xref ,1  xref , n ] . Every

time step k the response time vector xk is measured. If this value is not in the set R* , then we have to choose the closest operating point that contains it according to the following Euclidean distance metric,

106

* min‖ xref ,i  xref ‖

(3.46a)

subject to xk  Ri , i  1,.., N

(3.46b)

xref ,i

which is a standard point location problem. There can be alternative methods of selecting the closest operating point, e.g. the operating point that its DoA has the * * largest overlap with the DoA of the desired operating point ( xref , uref ). As it is shown in Figure 3.11, there is always an overlap between the domains of attractions of each operating point. We can benefit from this overlap and initially select the closest set Ri to R* that contains the current response time vector in the state space area according to (3.46) and after few time steps the system trajectory we * * will be driven to the desired operating point ( xref , uref ). The following algorithms summarize all the necessary offline steps for the determination of the set of feasible operating points and the corresponding control laws and regions of attraction(Algorithm 1) and the online implementation of the control strategy (Algorithm 2), Algorithm 3.1 - Offline Controller Synthesis 1: for all xref ,i , i  1,, N do 2: compute ( xref ,i , uref ,i ), i  1,, N by solving (3.31) 3: compute

i

,

i

in (3.34), (3.35)

4: compute Ri , Ki , Pi by solving (3.43), ( xref ,i , uref ,i ), i  1,, N 5: end for * * 6: Select ( xref , uref ), R* , K * Algorithm 3.2 - Online Controller Implementation 1: if xk  R* then * * )  uref 2: uk  K * ( xk  xref

3: else 4: Find xref ,i , uref ,i by solving (3.46) 5: uk  Ki ( xk  xref ,i )  uref ,i 6: end if 7: xk  xk 1  A(sk ) xk  B(sk )uk 8: Go to line 1

107

Remark 3.1 - A modification to the offline controller synthesis and the online controller implementation must be made for the case where no feasible input vector uref ,i , j exists for a desired xref ,i , i.e., the problem (3.31) has no solution. Firstly for each extreme system ( Aˆ j , Bˆ j ), j  1,, 2n , problem (3.31) is solved in order to compute feasible input vectors uref ,i , j , j  1,, 2n for the desired equilibrium point xref ,i .

Next, for all extreme realizations ( Aˆ j , Bˆ j ) , the translated input constraints i, j

, j  1,, 2n are formulated as (3.35). We consider as the common input

constraint set

i ,c

in the translated space to be the intersection of each input

constraint set, i.e.,



i ,c

2n i, j

. Then for each operating point Lyapunov matrix Pi ,

j1

the feedback gain matrices K i and the DoA Ri are computed solving a similar problem to (3.43). Regarding the online controller implementation, a feasible input vector uref ,i ( sk ) has to be computed at each step for the desired operating point xref ,i ( sk )  xref ,i . In detail, the following linear programming problem is solved at each instant k. n  n m  w c  wdi di     ij i , j ci , jk , pik , di ,uref ,i ( sk ) i 1  i 1 j 1  subject to

min

xref ,i ( sk )  A( sk ) xref ,i ( sk )  B( sk )uref ,i ( sk ) ,

(3.47a)

(3.47b)

cmin  ci , jk  cmax , i  1, , n, j  1, , m, (3.47c) n

c

 CT j , j  1,, m,

(3.47d)

pmax  pik  di , i  1,, n,

(3.47e)

di  0, i  1,, n,

(3.47f)

uk  , uk  Ki ( xk  xref ,i ( sk ) )  uref ,i ( sk ) .

(3.47g)

i 1

i , jk

The positive numbers wij , i  1,, n, j  1,, m and wdi , i  1,, n, are weights of the optimization cost function. Equation (3.47b) ensures that ( xref ,i ( sk ) , uref ,i ( sk ) ) is the equilibrium point. Equations (3.47c)-(3.47e) ensure state and input constraint satisfaction. Equations (3.47d) and (3.47e) are used to maximize the admittance probability. Finally equation (3.47g) guarantees that the control law uk satisfies the input constraints at every time interval. 108

The above procedure allows computing a stabilization control law and a DoA that leads the system trajectory on the operating point. It is worth noting that for this case, the input constraint satisfaction is not guaranteed by the offline synthesis as before. However, if the optimization problem (3.47) is feasible, the input constraints are guaranteed to hold during the online implementation.

Figure 3.12 - Tested Diagram – VMi,j of ith application on jth server.

3.3.6 Evaluation We have built a real testbed in order to evaluate the performance of our system modeling and control framework. It includes a group of servers and each service consists of an application and a database tier. Each tier is hosted on a VM as shown in Figure 3.12. The CPU capacity of database tier is fixed and only the capacity of the application tier VMs changes dynamically. We used real, highly variant and nonstationary traces from [103] to create the incoming workload of every application. These traces are still used by recent studies [104], [105] since they are more realistic than benchmarks like TPC-W [106] and RUBis [107] that have stationary request generators. We demonstrate an experiment where the upper desired reference 90th 109

percentile value of response time, defined by SLA, is Tref  2sec , which implies that * the target value of the equilibrium point is xref  1sec . We assume the prediction of

the incoming request rate i (k ) as the LPV model parameter sk . Figures 3.13a-3.13d illustrate the performance of the proposed framework. Indicatively we present the performance of services App1 and App2 which VMs share the first server in the application tier of figure 3.12. Figure 3.13a shows the 90th percentile value of response time for both applications. The percentage of time * intervals that the response time is less than the target value xref (blue line), is 98.78%. Also there are very few time intervals whose response time violates the reference value Tref (green line).

(a) Response Time

110

(b) VMs Capacity

(c) Admittance Probability

111

(d) Incoming Request Rate Figure 3.13 - Overall Performance of Consolidated Applications.

Figures 3.13b and 3.13c indicate that it is essential to tackle resource allocation and admission control as a common decision problem. When the incoming workload is low the VM capacity is also low and it increases as the incoming load increases. On the other hand, admission probability is close to one when the workload is light whereas it decreases if more requests arrive. Figure 3.13d illustrates that although the incoming request rate is dynamic and unpredictable and the existed predictor efficiently estimates the future values of workload. Figure 3.14 illustrates a short example of the control strategy presented in this article. The red dots depict different operating points ( xref ,i , uref ,i ) and the red ellipsoids the * * , uref , R* . corresponding DoAs Ri and the blue dot and line correspond to the target xref

The black solid line corresponds to the system trajectory which begins from x0  [4.3 3.9]

*  1 . Applying far away from the desired operating point, xref

Algorithm 3.2, the controller choose the control law and by solving the point location problem (3.46) the response time vector enters the acceptable region (blue line) on state-space, x(k )  Tref , after few steps. This proves that the proposed control scheme is adaptive and able to react if there is any sudden change of the incoming request rate.

112

Figure 3.14 - DoA of operating points (red lines and circles), DoA of the desired operating point (blue line and square) and System Trajectory (black solid line).

Figure 3.15 – Alternative solution of point location problem.

Figure 3.15 shows an alternative method of selecting the equilibrium point depending on the current operating condition of the system. This method uses also the Euclidian distance but it is more conservative because it selects iteratively the same operating point until it reaches its center very closely and then it chooses a new one. We compare the proposed solution with study [74], which also uses LPV modeling and MPC control without guaranteeing the feasibility of the operating point. We select this particular study because is closest to our control perspective and it uses popular and a well-established control method. This study also provides a trade-off 113

between response time restrictions and energy consumption adjusting the parameter

 to the cost function J   || x(k ) || (1   )

K  H 1



| | u (k ) || . We used the same incoming

k K

workload in order to compare the two approaches and figures 3.16a – 3.16c show their performance. It can be seen that our method outperforms MPC controller. In particular, the MPC scheme results in a large percentage of response violations and it consumes more capacity resources and rejects a significant portion of the incoming requests. Table 3.5 summarizes the performance of the two control solutions. MPC controller does not perform very well because of the lack of a feasible equilibrium point that will assure system stability. Also it is very sensitive to the values of  parameter of the cost function. This results in the input oscillations in figure 3.16b and 3.16c. Accepted RT (%)

Capacity (%)

Probability (%)

AC+RA

98.55

36.08

96.69

MPC

69.92

45

60.32

Table 3.5 - Comparison of the performance of AC+RA and MPC control scheme.

(a) Response Time

114

(b) VMs Capacity

(c) Admittance Probability Figure 3.16 - Comparison with MPC Controller

115

3.4 Conclusions In this chapter two local level control frameworks for automated resource provisioning are presented. Both of them use novel techniques of modern control theory to address the problems of admission control and resource allocation simultaneously. The performance of both approaches is tested under extreme conditions and they are robust against the workload variation. Furthermore they are compared with other known control methods and it seems that they have better performance. The main conclusions about the performance of both controllers are the following,      

The requirements of the SLA are satisfied. The proposed modeling methods capture the dynamics of the consolidated web services. The computational resources of the system are optimally allocated to the VMs of each application according to their needs. The underlying system constraints are guaranteed. A set of feasible operation points according to SLA are computed. The system stability is guaranteed.

116

117

CHAPTER 4 – Conclusions and Future Work This chapter summarizes the conclusions of the previous chapters and the some potential aspects of the future work.

4.1 Conclusions The conclusions of this thesis concern two main research areas of distributed web services deployed on cloud computing infrastructure. The first category is about monitoring of cloud computing infrastructure. In chapter 2 a novel general technique is used to infer relationships between different service components in a data center. This approach relies on a small set of fuzzy rules, produced by a hybrid genetic algorithm with high classification rate. Furthermore, the strength of detected dependencies is measured. Although we do not know the ground truth about relationships in a network, the proposed method mines realistic relationships without having any previous information about network topology and infrastructure. This approach can be a useful monitoring tool for administrators to obtain a clear view of what is happening in the underlying network. Finally, because of the simplicity of our algorithm and the flexibility of FIM, an online approach seems feasible. The second major problem, which is addressed in chapter 3, is the automated resource control of consolidated web applications on cloud computing infrastructure. ACRA is an innovative modeling and controlling technique of distributed services that are colocated on server cluster. The system dynamics are modeled by a group of linear state space models, which cover all the range of workload conditions. Because of the variant workload conditions, there are non-linear term and uncertainties which are modeled by an additive term in the local linear models. Due to the several types of service transactions with varying time and resources demands there are many desired candidate reference values of the SLOs during a day. Due to these requirements and the workload circumstances, we choose the appropriate model and we compute the closest feasible operating point according to several optimization criteria. Then using a set-theoretic technique a state feedback controller is designed that successfully leads and stabilize the system in the region of the equilibrium point. ACRA controller computes a positively invariant set on the state-space, which includes the target set and drives the system trajectories in it. Thus provide stability guarantee and high levels of robustness against system disturbances and nonlinearities. Furthermore we compare ACRA with an MPC and a PI controller and the results are very promising, since our solution outperforms the two other approaches. Secondly, a unified local level modeling and control framework for consolidated web services in a server cluster was presented, which can be a vital element of a holistic distributed control platform. Admission control and resource allocation were addressed as a common decision problem. Stability and constraint satisfaction was 118

guaranteed. A real testbed was built and from a range of examples, in different operating conditions, we can conclude that both the identification scheme and controller provide high level of QoS. A novel component of this approach is the determination of a set of feasible operating (equilibrium) points which allows choosing the appropriate equilibrium point, depending only on what our objectives are, such as maximizing throughput, minimizing consumption or maximizing profit. Evaluation shows that our approach has high performance compared to well-known solutions, such as queuing models and measurement approach of equilibrium points. Both controllers succeed in their main targets respectively to the already proposed studies in literature. Firstly they satisfy the SLA requirements and the constraints of the underlying cloud computing infrastructure. To the best of our knowledge they are the only studies that calculate a set of feasible operating points that ensure system stability. Furthermore they adopt modern control theory and beyond the stability guarantee they introduce new control properties such as positively invariant sets , ultimate boundedness and e- contractive sets.

4.2 Future Work For future work, we intend to use the current work as an element of a general control framework, which addresses more problems in cloud computing area, i.e. VM migration techniques, application placement, load balancing between geographically dispersed data centers and energy management. This general control framework will include a global level controller that will distribute the incoming workload according to an optimization problem, either minimizing the number of active severs or equalizing the amount of load on all underlying servers and will determine where these applications should be hosted. Many different solutions have been proposed for the above problems. Assuming predefined capacity for each VM, they formulate and solve an optimization problem that computes how many VM are necessary. However these approaches do not concern the underlying system constraints. A combination of control and optimization theory or fuzzy logic can overcome this obstacle. ACRA or the autonomous modeling and control framework of chapter 3 can compute many different operating points on the local level of control. These particular operating points will be used as “targets – announcements” from the global level controller in order to address the problems of load balancing and application placement in such a manner that the underlying constraints are always satisfied. Another perspective of future work is to test the existing control frame to other kind of network application. Media distribution and online gaming are two emerging research problem where the proposed controllers can be implemented in order to achieve high performance. All the aforementioned research work assumes a centralized controller either on global or local level. It would be interesting to investigate if it is possible to establish a distributed control level, where there are many control nodes that exchange all the 119

necessary information between them in order to fulfill their requirements. This kind of control architecture will be very useful to many fundamental network research areas since most of them focus on the modeling and control of large scale dynamic systems, i.e congestion control.

120

121

Publications International Journals 1. Dechouniotis, D., Leontiou, N., Dimitropoulos, X., Kind, A., & Denazis, S., Unveiling the underlying relationships over a network for monitoring purposes. International Journal of Network Management, 19(6), 513526,2009.

International Journals under Revision 1. Dechouniotis D., Leontiou N., Athanasopoulos N., Christakidis A. and Denazis S., An Autonomous Admission Control and Resource Allocation Framework for Consolidated Web Applications, International Journal of Network Management, submitted 2013.

Patents 1. Daniel Nikolaus Bauer, Dimitrios Dechouniotis, Christos-Xenofontas Dimitropoulos, Andreas Kind, “Valley-free Shortest Path Method”, United States Patent, Patent No. 7907596 B2, US2009/0141637 A1, Date of Patent 15/3/2011.

International Conferences 1. Dechouniotis D., Leontiou N., Athanasopoulos N., Bitsoris G, Denazis S.,” ACRA: A Unified Admission Control and Resource Allocation Framework for Virtualized Enviroments”, 8th IEEE International Conference on Network and Service Management (CNSM), pages 145-149, 2012. 2. Leontiou, N. Dechouniotis, D. Denazis, S.,”Adaptive Admission Control of Distributed Cloud Services”, 6th IEEE International Conference on Network and Service Management (CNSM), pages 318-321, 2010, DOI : 10.1109/CNSM.2010.5691214. 3. Alexandru Caracas¸ Dimitrios Dechouniotis, Stefan Fussenegger, Dieter Gantenbein, Andreas Kind, "Mining Semantic Relations using NetFlow", 3th IEEE/IFIP International Workshop on Business-driven IT Management (BDIM 2008). Pages 110-11, 2008, DOI:10.1109/BDIM.2008.4540082. 4. D. Dechouniotis, X. Dimitropoulos, A. Kind, and S. Denazis. Dependency detection using a fuzzy engine. In Proc. of the 18th IFIP/IEEE International Workshop on Distributed Systems: Operations and Management, DSOM, pages 110–121, 2007.

122

123

Bibliography [1] [2]

[3]

[4] [5] [6] [7] [8] [9] [10] [11]

[12]

[13]

[14] [15] [16] [17] [18]

[19]

NIST Definition of Cloud Computing v15, csrc.nist.gov/groups/ SNS/cloudcomputing/cloud-def-v15.doc Vaquero L., Rodero-Merino L., Caceres J. and Lindner M., A break in the clouds: towards a cloud definition. ACM SIGCOMM computer communications review, 2009. Zhang Q., Cheng L. and Boutaba R., Cloud computing: state-of-the-art and research challenges. Journal of Internet Services and Applications, 1(1), 7-18, 2010. Amazon Elastic Computing Cloud, aws.amazon.com/ec2 Google App Engine, http://code.google.com/appengine Windows Azure, www.microsoft.com/azure Salesforce CRM, http://www.salesforce.com/platform Dedicated Server, Managed Hosting, Web Hosting by Rackspace Hosting, http://www.rackspace.com XenSource Inc, Xen, www.xensource.com VMWare ESX Server, www.vmware.com/products/esx Al-Fares M., Loukissas A. and Vahdat A., "A scalable, commodity data center network architecture." In ACM SIGCOMM Computer Communication Review, vol. 38, no. 4, pp. 63-74. ACM, 2008. A. Greenberg, J. R. Hamilton, N. Jain, S. Kandula, C. Kim, P. Lahiri, D. A. Maltz, P. Patel and S. Sengupta. "VL2: a scalable and flexible data center network." In ACM SIGCOMM Computer Communication Review, vol. 39, no. 4, pp. 51-62. ACM, 2009. R. Niranjan Mysore, A. Pamboris, N. Farrington, N. Huang, P. Miri, S. Radhakrishnan, V. Subramanya and A. Vahdat. "PortLand: a scalable faulttolerant layer 2 data center network fabric." In ACM SIGCOMM Computer Communication Review, vol. 39, no. 4, pp. 39-50. ACM, 2009. Ghemawat S., Gobioff H. and Leung S.T., The Google file system. In Proc. of SOSP, 2003. Hadoop Distributed File System, hadoop.apache.org/hdfs J. Dean and S. Ghemawat. "MapReduce: simplified data processing on large clusters." Communications of the ACM 51, no. 1 (2008): 107-113. Hadoop MapReduce, hadoop.apache.org/mapreduce B. Urgaonkar, P. Shenoy, A. Chandra, and P. Goyal. "Dynamic provisioning of multi-tier internet applications." In Autonomic Computing, 2005. ICAC 2005. Proceedings. Second International Conference on, pp. 217-228. IEEE, 2005. Q. Zhang, L. Cherkasova and E. Smirni. "A regression-based analytic model for dynamic resource provisioning of multi-tier applications." In Autonomic Computing, 2007. ICAC'07. Fourth International Conference on, pp. 27-27. IEEE, 2007

124

[20] S. Osman, D. Subhraveti, G. Su and J. Nieh. "The design and implementation

[21]

[22]

[23]

[24]

[25]

[26] [27]

[28]

[29]

[30]

[31]

[32] [33]

of Zap: A system for migrating computing environments." ACM SIGOPS Operating Systems Review 36, no. SI (2002): 361-376. C. Clark, K. Fraser, S. Hand, J. G. Hansen, E. Jul, C. Limpach, I. Pratt and A. Warfield. "Live migration of virtual machines." In Proc. of the 2nd conference on Symposium on Networked Systems Design & Implementation-Volume 2, pp. 273-286. USENIX Association, 2005. Chekuri C. and Khanna S., "On multi-dimensional packing problems." In Proc. of the tenth annual ACM-SIAM symposium on discrete algorithms, pp. 185194. Society for Industrial and Applied Mathematics, 1999. B. Li, J. Li, J. Huai, T. Wo, Q. Li and L. Zhong. "EnaCloud: an energy-saving application live placement approach for cloud computing environments." In Cloud Computing, 2009. CLOUD'09. IEEE International Conference on, pp. 17-24. IEEE, 2009. S. Srikantaiah, A. Kansal and F. Zhao. "Energy aware consolidation for cloud computing." In Proc. of the 2008 conference on Power aware computing and systems, vol. 10. USENIX Association, 2008. X. Meng, V. Pappas, and L. Zhang. "Improving the scalability of data center networks with traffic-aware virtual machine placement." In INFOCOM, 2010 Proceedings IEEE, pp. 1-9. IEEE, 2010. J. Sonnek and A. Chandra. "Virtual putty: Reshaping the physical footprint of virtual machines." Proc of HotCloud (2009). T. Wood, P. J. Shenoy, A. Venkataramani and M. S. Yousif. "Black-box and Gray-box Strategies for Virtual Machine Migration." In NSDI, vol. 7, pp. 229242. 2007. P. Padala, K.Y. Hou, K. G. Shin, X. Zhu, M. Uysal, Z. Wang, S. Singhal and A. Merchant. "Automated control of multiple virtualized resources." In Proceedings of the 4th ACM European conference on Computer systems, pp. 13-26. ACM, 2009. J. Hamilton. "Cooperative expendable micro-slice servers (CEMS): low cost, low power servers for internet-scale services." In Conference on Innovative Data Systems Research (CIDR’09)(January 2009). 2009. D. Brooks, M. P. Bose, S. E. Schuster, H. Jacobson, P. N. Kudva, A. Buyuktosunoglu, J. Wellman, V. Zyuban, M. Gupta and P. W. Cook. "Poweraware microarchitecture: Design and modeling challenges for next-generation microprocessors." Micro, IEEE 20, no. 6 (2000): 26-44. N. Vasić, M. Barisits, V. Salzgeber and D. Kostic. "Making cluster applications energy-aware." In Proceedings of the 1st workshop on Automated control for datacenters and clouds, pp. 37-42. ACM, 2009. IEEE P802.3az Energy Efficient Ethernet Task Force, www. ieee802.org/3/az S. Kumar, V. Talwar, V. Kumar, P. Ranganathan and K. Schwan. "vManage: loosely coupled platform and virtualization management in data centers." In Proceedings of the 6th international conference on Autonomic computing, pp. 127-136. ACM, 2009. 125

[34] E. Kalyvianaki, T. Charalambous and S. Hand. "Self-adaptive and self-

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44] [45]

[46]

configured cpu resource provisioning for virtualized servers using Kalman filters." In Proceedings of the 6th international conference on Autonomic computing, pp. 117-126. ACM, 2009. P. Bodık, R. Griffith, C. Sutton, A. Fox, M. Jordan and D. Patterson. "Statistical machine learning makes automatic control practical for internet datacenters." In Proceedings of the 2009 conference on Hot topics in cloud computing, pp. 1212. 2009. Benson, T., Anand, A., Akella, A. and Zhang, M., Understanding data centers traffic characteristics. ACM SIGCOMM Computer Communication Review, 40(1), 92-99, 2010. N. Santos, K. P. Gummadi and R. Rodrigues. "Towards trusted cloud computing." In Proceedings of the 2009 conference on Hot topics in cloud computing, pp. 3-3. 2009. F. J. Krautheim, "Private virtual infrastructure for cloud computing." In Proceedings of the 2009 conference on Hot topics in cloud computing, pp. 5-5. USENIX Association, 2009. K. Kambatla, A. Pathak and H. Pucha. "Towards optimizing hadoop provisioning in the cloud." In Proc. of the First Workshop on Hot Topics in Cloud Computing, p. 118. 2009. T. Sandholm and K. Lai. "MapReduce optimization using regulated dynamic prioritization." In Proc. of the eleventh international joint conference on Measurement and modeling of computer systems, pp. 299-310. ACM, 2009. M. Zaharia, A. Konwinski, A. D. Joseph, R. H. Katz and Ion Stoica. "Improving MapReduce Performance in Heterogeneous Environments." In OSDI, vol. 8, no. 4, p. 7. 2008. R. Ananthanarayanan, K. Gupta, P. Pandey, H. Pucha, P. Sarkar, M. Shah and R. Tewari. "Cloud analytics: Do we really need to reinvent the storage stack." In Proc. of the 1st USENIX Workshop on Hot Topics in Cloud Computing (HOTCLOUD’2009), San Diego, CA, USA. 2009. S. Patil, G. A. Gibson, G. R. Ganger, J. Lopez, M. Polte, W. Tantisiroj and Lin Xiao. "In search of an API for scalable file systems: under the table or above it?." In Proceedings of the 2009 conference on Hot topics in cloud computing, p. 13. USENIX Association, 2009. K. Church, A. Greenberg and J. Hamilton. "On delivering embarrassingly distributed cloud services." Hotnets VII 34 (2008). V. Valancius, N. Laoutaris, L. Massoulié, C. Diot and P. Rodriguez. "Greening the internet with nano data centers." In Proceedings of the 5th international conference on Emerging networking experiments and technologies, pp. 37-48. ACM, 2009. A. Chandra and J. Weissman. "Nebulas: using distributed voluntary resources to build clouds." In Proceedings of the 2009 conference on Hot topics in cloud computing, pp. 2-2. USENIX Association, 2009. 126

[47] [48]

[49] [50]

[51]

[52]

[53]

[54] [55] [56] [57] [58] [59]

[60]

[61]

[62]

Oetiker T., Rand D. MRTG—Multi Router Traffi c Grapher. http://oss.oetiker. ch/mrtg/ [5 April 2009]. Keller, A., Blumenthal, U., Kar, G.: Classification and computation of dependencies for distributed management. In: Proc. of the 5th IEEE Symposium on Computers and Communications ISCC. (2000) 78–83. Dechouniotis D, Dimitropoulos X, Kind A, Denazis, S. Dependency detection using a fuzzy engine. In Proceedings of IFIP/IEEE DSOM, 2007; 110–121. Agarwal M, Gupta M, Kar G, Neogi A, Sailer A. Mining activity data for dynamic dependency discovery in e-business systems. IEEE Transactions on Network and Service Management 2004; 1(2): 49–58. Kannan J, Jung J, Paxson V, Koksal CE. Semi-automated discovery of application session structure. In Proceedings of the 6th ACM IMC, 2005; 119– 132. Aiello B, Kalmanek C, McDaniel P, Sen S, Spatscheck O, Van der Merwe J. Analysis of communities of interest in data networks. In Proceedings of the 6th PAM, 2005; 83–96. Bahl P, Chandra R, Greenberg A, Kandula S, Maltz DA, Zhang M. Towards highly reliable enterprise network services via inference of multi-level dependencies. In Proceedings of ACM SIGCOMM, 2007; 13–24. Kandula S, Chandra R, Katabi D. What’s going on? Learning communication rules in edge networks. In Proceedings of ACM SIGCOMM, 2008; 87–98. Mendel J., Fuzzy logic systems for engineering: a tutorial. In Proceedings of the IEEE, 83(3):345{377, Mar 1995. Passino, K., Yurkovich, S.: FUZZY CONTROL. Addison-Wesley (1998) IBM Aurora: Network traffic analysis and visualization. http://www.zurich.ibm. com/aurora [5 April 2009]. IETF NetFlow/IPFIX. http://www.ietf.org/html.charters/ipfi x-charter.html [5 April 2009]. Dechouniotis, D., Leontiou, N., Dimitropoulos, X., Kind, A., & Denazis, S., Unveiling the underlying relationships over a network for monitoring purposes. International Journal of Network Management, 19(6), 513-526. Ishibuchi H, Yamamoto T, Nakashima, T. Hybridization of fuzzy gbml approaches for pattern classification problems. IEEE Transactions on Systems, Man and Cybernetics B 2005; 35(2): 359–365. Ardagna D., Ghezzi C., Panicucci B. and Trubian M., Service provisioning on the cloud: Distributed algorithms for joint capacity allocation and admission control. In Proc. of the 3rd European Conference ServiceWave, pages 1 – 12, 2010. Kusic D. and Kandasamy N., Risk-aware limited look-ahead control for dynamic resource provisioning in enterprise computing systems. In Proc. of the IEEE International Conference on Autonomic Computing (ICAC), pages 74–83, 2006.

127

[63] Kusic D., Kephart J.O., Hanson J.E., Kandasamy N. and Jiang G., Power and

[64]

[65]

[66]

[67]

[68]

[69]

[70]

[71]

[72]

[73]

[74]

[75]

performance management of virtualized computing environments via lookahead control. Springer Journal on Cluster Computing, 12(1):1–15, 2009. Almeida J., Almeida V., Ardagna D., Cuhna I. and Francalanci C., Joint admission control and resource allocation in virtualized servers. Elsevier Journal of Parallel and Distributed Computing, 70(4):344–362, 2010. Urgaonkar R., Kozat U., Igarashi K. and Neely M.J., Dynamic resource allocation and power management in virtualized data centers. In Proc. of the IEEE Network Operations and Management Symposium (NOMS), pages 479 – 486, 2010. Ardagna D., Panicucci B., Trubian M. and Zhang L., Energy-aware autonomic resource allocation in multitier virtualized environments. IEEE Transactions on Services Computing, 5(1):2–19, 2012. Wang R. and Kandasamy N., On the design of decentralized control architectures for workload consolidation in large-scale server clusters. In Proceedings of the 9th IEEE International Conference on Autonomic Computing (ICAC), pages 115–124, 2012. Kamra A., Misra V. and Nahum E., Yaksha: A self-tuning controller for managing the performance of 3-tiered web sites. In Proc. of the 12th IEEE International Workshop on Quality of Service (IWQOS), pages 47–56, 2004. Liu X., Heo J., Sha L. and Zhu X., Queuing-model-based adaptive control of multi-tiered web applications. IEEE Transactions on Network and Service Management, 5(3):157–167, 2008. Wang R., Kusic D. and Kandasamy N., A distributed control framework for performance management of virtualized computing environments. In Proc. of the 7th IEEE/ACM International Conference on Autonomic Computing (ICAC), pages 89–98, 2010. Leontiou N., Dechouniotis D. and Denazis S., Adaptive admission control of distributed cloud services. In Proc. of the IEEE International Conference Network and Service Management (CNSM), pages 318 – 321, 2010. Dechouniotis D., Leontiou N., Athanasopoulos N., Bitsoris G. and Denazis S., ACRA: A unified admission control and resource allocation framework for virtualized environments. In Proc. of the 8th IEEE International Conference on Network and Service Management (CNSM), pages 145–149, 2012. Wang X. and Wang Y., Coordinating power and performance management for virtualized server clusters. IEEE Transactions on Parallel and Distributed Systems, 22(2):245 – 259, 2011. Poussot-Vassal C., Tanelli M. and Lovera M., A control-theoretic approach for the combined management of the quality-of-service and energy is service centers. In Run-time Models for Self-managing Systems and Applications, pages 73–96. Springer, 2010. Qin W. and Wang Q., Feedback performance control for computer systems: an LPV approach. In Proceedings of the American Control Conference (ACC), pages 4760–4765, 2005. 128

[76] Qin W. and Wang Q., Modeling and control design for performance

[77]

[78]

[79]

[80] [81] [82]

[83] [84]

[85] [86] [87] [88]

[89]

[90] [91] [92] [93] [94]

management of web servers via an LPV approach. IEEE Transactions on Control Systems Technology, 15(2):259–275, 2007. Giani P., Tanelli M. and Lovera M., Controller design and close-loop stability analysis for admission control in web service systems. In Proc. of the 18th IFAC World Congress, pages 6709 – 6714, 2011. Rao J., Wei Y., Gong J. and Xu C., DynaQos: Model-free self-tuning fuzzy control of virtualized resources for qos provisioning. In Proc. of the 19th IEEE International Workshop on Quality of Service (IWQOS), pages 1–9, 2011. Cherkasova L. and Phaal P., Session-based admission control: A mechanism for peak load management of commercial web sites. IEEE Transactions on Computers, 51(6):669–685, 2002. Chen, C.T., Linear system theory and design. Oxford University Press, Inc., 1998. Tóth, R., Modeling and identification of linear parameter-varying systems. Vol. 403. Springer, 2010. Athanasopoulos N., Phd Thesis: Stability analysis and control of linear and nonlinear constrained systems via polyhedral Lyapunov functions. University of Patras, http://hdl.handle.net/10889/4005 , 2010. Blanchini F. and Miani S., Set-theoretic methods in control. Springer, 2008. Athanasopoulos N., Bitsoris G. and Vassilaki M., Ultimate boundedness and robust stabilization of bilinear discrete-time systems. In Decision and Control and European Control Conference (CDC-ECC), 2011 50th IEEE Conference on (pp. 4622-4627). A. Williams, M. Arlitt, C. Williamson, and K. Barker, Web Workload Characterization: Ten Years Later. Springer, 2005. P. E. Wellstead, E., and M. B. Zarrop, Self-Tuning Systems: Control and Signal Processing. John Wiley & Sons, Inc., 1991. F. Blanchini, Set invariance in control, Automatica, vol. 35, no. 11, pp. 1747 – 1767, 1999. F. Blanchini, Ultimate boundedness control for uncertain discretetime systems via set-induced lyapunov functions, in Proc. of the 30th IEEE Conference on Decision and Control(CDC), 1991, pp. 1755 – 1760 vol.2. S. Rakovic, E. Kerrigan, K. Kouramas, and D. Mayne, Invariant approximation of the minimal robust positively invariant set, IEEE Transactions on Automatic Control, vol. 50, no. 3, pp. 406 – 410, 2005. Ogata K., State space analysis of control systems. Prentice-Hall, 1967. Bitsoris G., Positively invariants polyhedral sets of discrete-time linear systems, International Journal of Control, vol. 47, no. 6, pp. 1713 – 1726, 1988. Bitsoris G. and Vassilaki M., Constrained regulation of linear systems, Automatica, vol. 31, no. 2, pp. 223 – 227, 1995. Xen hypervisor, Online Available: http://www.xen.org/. Maciejowski J., Predictive Control with Constraints. Prentice-Hall, 2001. 129

[95] Dechouniotis D., Leontiou N., Athanasopoulos N., Christakidis A. and Denazis

[96]

[97] [98] [99] [100] [101] [102] [103] [104]

[105]

[106] [107]

S., An Autonomous Admission Control and Resource Allocation Framework for Consolidated Web Applications, International Journal of Network Management, submitted 2013. Bamieh B. and Giarre L., Identification of linear parameter varying models. Wiley International Journal of Robust and Nonlinear Control, 12(9):841– 853, 2002. Wheelwright S.C. and Hyndman R. J., Forecasting: methods and applications, John Wiley & Sons Inc, 1998. Khalil H. K., Nonlinear systems, volume 3. Prentice hall Upper Saddle River, 2002. Boyd S., Linear matrix inequalities in system and control theory, volume 15. Siam, 1994. Matlab Robust Control Toolbox. http://www.mathworks.com/products/robust/. SDPT 3.0 - a Matlab software package for semidefinite-quadratic-linear programming. http://www.math.cmu.edu/~reha/sdpt3.html. Plugmers B., Robust Model Based Predictive Control - An Invariant Set Approach. PhD thesis, Katholieke Universiteit Lauven, 2006. World football cup 1998 network traces. http://ita.ee.lbl.gov/html/contrib/WorldCup.html. Xie T. and Sun Y., Understanding the relationship between energy conservation and reliability in parallel disk arrays. Elsevier Journal of Parallel and Distributed Computing, 71(2):198–210, 2011. Gandhi A., Chen Y., Gmach D., Arlitt M. and Marwah M., Hybrid resource provisioning for minimizing data center sla violations and power consumption. Elsevier Sustainable Computing: Informatics and Systems, 2(2):91–104, 2012. TPC-W: a transactional web e-commerce benchmark. http://www.tpc.org/tpcw/. RuBis: Rice University bidding system. http://rubis.ow2.org/.

130