Lecture notes 1

22 downloads 460 Views 2MB Size Report
Page 1. CS742 – Distributed & Parallel DBMS. M. Tamer Özsu. Page 1.1. Outline. ▫ Introduction & architectural issues. ○ What is a distributed DBMS.
Outline n  Introduction & architectural issues l  What is a distributed DBMS l  Problems l  Current state-of-affairs

q  Data distribution q  Distributed query processing q  Distributed query optimization q  Distributed transactions & concurrency control q  Distributed reliability q  Database replication q  Parallel database systems q  Database integration & querying q  Advanced topics CS742 – Distributed & Parallel DBMS

Page 1.1

M. Tamer Özsu

File Systems

program 1

File 1

data description 1 program 2

File 2

data description 2 program 3

File 3

data description 3

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 1

Page 1.2

Database Management Application program 1 (with data semantics)

DBMS

description manipulation control

Application program 2 (with data semantics)

database

Application program 3 (with data semantics) CS742 – Distributed & Parallel DBMS

Page 1.3

M. Tamer Özsu

Motivation Database Technology

Computer Networks

integration

distribution

Distributed Database Systems integration integration ≠ centralization CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 2

Page 1.4

Distributed Computing n  A number of autonomous processing elements

(not necessarily homogeneous) that are interconnected by a computer network and that cooperate in performing their assigned tasks. n  What is being distributed? l  Processing logic l  Function l  Data l  Control

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 1.5

What is a Distributed Database System? A distributed database (DDB) is a collection of multiple, logically interrelated databases distributed over a computer network. A distributed database management system (D–DBMS) is the software that manages the DDB and provides an access mechanism that makes this distribution transparent to the users. Distributed database system (DDBS) = DDB + D–DBMS

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 3

Page 1.6

What is not a DDBS? n  A timesharing computer system n  A loosely or tightly coupled multiprocessor

system n  A database system which resides at one of the

nodes of a network of computers - this is a centralized database on a network node

CS742 – Distributed & Parallel DBMS

Page 1.7

M. Tamer Özsu

Centralized DBMS on a Network Site 1 Site 2 Site 5 Communication Network

Site 3

Site 4

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 4

Page 1.8

Distributed DBMS Environment

Site 1 Site 2 Site 5 Communication Network

Site 4

CS742 – Distributed & Parallel DBMS

Site 3 M. Tamer Özsu

Page 1.9

Implicit Assumptions n  Data stored at a number of sites à each site

logically consists of a single processor. n  Processors at different sites are interconnected by a computer network à not a multiprocessor system l  Parallel database systems

n  Distributed database is a database, not a

collection of files à data logically related as exhibited in the users’ access patterns l  Relational data model

n  D-DBMS is a full-fledged DBMS l  Not remote file system, not a TP system

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 5

Page 1.10

Data Delivery Alternatives n  Delivery modes l  Pull-only l  Push-only l  Hybrid

n  Frequency l  Periodic l  Conditional l  Ad-hoc or irregular

n  Communication Methods l  Unicast l  One-to-many

n  Note: not all combinations make sense CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 1.11

Distributed DBMS Promises Œ Transparent management of distributed, fragmented, and replicated data  Improved reliability/availability through distributed transactions Ž Improved performance  Easier and more economical system expansion

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 6

Page 1.12

Ch.x/ 12

Transparency n  Transparency is the separation of the higher

level semantics of a system from the lower level implementation issues.

n  Fundamental issue is to provide data independence

in the distributed environment l  Network (distribution) transparency l  Replication transparency l  Fragmentation transparency

horizontal fragmentation: selection vertical fragmentation: projection u  hybrid u  u 

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 1.13

Ch.x/ 13

Example

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 7

Page 1.14

Transparent Access SELECT FROM WHERE AND AND

ENAME,SAL EMP,ASG,PAY DUR > 12 EMP.ENO = ASG.ENO PAY.TITLE = EMP.TITLE

Tokyo Paris

Boston

Communication Network

Paris projects Paris employees Paris assignments Boston employees

Boston projects Boston employees Boston assignments

Montreal New York Boston projects New York employees New York projects New York assignments CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Montreal projects Paris projects New York projects with budget > 200000 Montreal employees Montreal assignments Page 1.15

Distributed Database - User View

Distributed Database

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 8

Page 1.16

Distributed DBMS - Reality User Query

DBMS Software

DBMS Software

DBMS Software

User Application

DBMS Software

Communication Subsystem

User Query

User Application

DBMS Software

User Query

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 1.17

Types of Transparency n  Data independence n  Network transparency (or distribution

transparency)

l  Location transparency l  Fragmentation transparency

n  Replication transparency n  Fragmentation transparency

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 9

Page 1.18

Reliability Through Transactions n  Replicated components and data should make distributed

DBMS more reliable. n  Distributed transactions provide l  Concurrency transparency l  Failure atomicity

• Distributed transaction support requires implementation of l  Distributed concurrency control protocols l  Commit protocols

n  Data replication l  Great for read-intensive workloads, problematic for updates l  Replication protocols

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 1.19

Potentially Improved Performance n  Proximity of data to its points of use l  Requires some support for fragmentation and replication

n  Parallelism in execution l  Inter-query parallelism l  Intra-query parallelism

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 10

Page 1.20

Parallelism Requirements n  Have as much of the data required by each

application at the site where the application executes l  Full replication

n  How about updates? l  Mutual consistency l  Freshness of copies

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 1.21

System Expansion n  Issue is database scaling n  Emergence of microprocessor and workstation

technologies l  Demise of Grosh's law l  Client-server model of computing

n  Data communication cost vs

telecommunication cost

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 11

Page 1.22

Distributed DBMS Issues n  Distributed Database Design l  How to distribute the database l  Replicated & non-replicated database distribution l  A related problem in directory management

n  Query Processing l  Convert user transactions to data manipulation

instructions

l  Optimization problem u  min{cost

= data transmission + local processing}

l  General formulation is NP-hard

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 1.23

Distributed DBMS Issues n  Concurrency Control l  Synchronization of concurrent accesses l  Consistency and isolation of transactions' effects l  Deadlock management

n  Reliability l  How to make the system resilient to failures l  Atomicity and durability

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 12

Page 1.24

Relationship Between Issues Directory Management

Query Processing

Distribution Design

Reliability

Concurrency Control Deadlock Management CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 1.25

Related Issues n  Operating System Support l  Operating system with proper support for database

operations l  Dichotomy between general purpose processing requirements and database processing requirements

n  Open Systems and Interoperability l  Distributed Multidatabase Systems l  More probable scenario l  Parallel issues

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 13

Page 1.26

Architecture n  Defines the structure of the system l  components identified l  functions of each component defined l  interrelationships and interactions between components

defined

CS742 – Distributed & Parallel DBMS

Page 1.27

M. Tamer Özsu

ANSI/SPARC Architecture Users

External Schema

External view

Conceptual Schema

Internal Schema CS742 – Distributed & Parallel DBMS

External view

External view

Conceptual view

Internal view

M. Tamer Özsu

Page 14

Page 1.28

Generic DBMS Architecture

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 1.29

DBMS Implementation Alternatives

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 15

Page 1.30

Dimensions of the Problem n  Distribution

l  Whether the components of the system are located on the same

machine or not

n  Heterogeneity

l  Various levels (hardware, communications, operating system) l  DBMS important one u 

data model, query language,transaction management algorithms

n  Autonomy

l  Not well understood and most troublesome l  Various versions u  u  u 

Design autonomy: Ability of a component DBMS to decide on issues related to its own design. Communication autonomy: Ability of a component DBMS to decide whether and how to communicate with other DBMSs. Execution autonomy: Ability of a component DBMS to execute local operations in any manner it wants to.

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 1.31

Client/Server Architecture

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 16

Page 1.32

Advantages of Client-Server Architectures n  More efficient division of labor n  Horizontal and vertical scaling of resources n  Better price/performance on client machines n  Ability to use familiar tools on client machines n  Client access to remote data (via standards) n  Full DBMS functionality provided to client

workstations

n  Overall better system price/performance

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 1.33

Database Server

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 17

Page 1.34

Distributed Database Servers

CS742 – Distributed & Parallel DBMS

Page 1.35

M. Tamer Özsu

Datalogical Distributed DBMS Architecture ES1

ES2

...

ESn

GCS

LCS1

LCS2

...

LCSn

LIS1

LIS2

...

LISn

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 18

Page 1.36

Peer-to-Peer Component Architecture System Log

Local Internal Schema

Database

Local Query Processor

Local Recovery Manager

Local Conceptual Schema

GD/D

Global Execution Monitor

Global Query Optimizer

USER

Global Conceptual Schema

Semantic Data Controller

User requests

User Interface Handler

External Schema

DATA PROCESSOR

Runtime Support Processor

USER PROCESSOR

System responses

CS742 – Distributed & Parallel DBMS

Page 1.37

M. Tamer Özsu

Datalogical Multi-DBMS Architecture

LES11



GES1

GES2

LES1n

GCS

...

GESn

LESn1



LCS1

LCS2



LCSn

LIS1

LIS2



LISn

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 19

LESnm

Page 1.38

MDBS Components & Execution Global User Request

Local User Request

Local User Request

Multi-DBMS Layer Global Subrequest

DBMS1

CS742 – Distributed & Parallel DBMS

Global Subrequest

DBMS2

Global Subrequest

DBMS3

M. Tamer Özsu

Page 1.39

Mediator/Wrapper Architecture

CS742 – Distributed & Parallel DBMS

M. Tamer Özsu

Page 20

Page 1.40