Terminology and overview
Formally, "database" refers to the data themselves and supporting
data structures. Databases are created to operate large quantities of
information by inputting, storing, retrieving and managing that
information. Databases are set up so that one set of software programs
provides all users with access to all the data.
A "database management system" (DBMS) is a suite of computer software
providing the interface between users and a database or databases.
Because they are so closely related, the term "database" when used
casually often refers to both a DBMS and the data it manipulates.
Outside the world of professional
information technology, the term
database is sometimes used casually to refer to any collection of data (perhaps a
spreadsheet,
maybe even a card index). This article is concerned only with databases
where the size and usage requirements necessitate use of a database
management system.
[1]
The interactions catered for by most existing DBMSs fall into four main groups:
- Data definition – Defining new data structures for a
database, removing data structures from the database, modifying the
structure of existing data.
- Update – Inserting, modifying, and deleting data.
- Retrieval – Obtaining information either for end-user queries and reports or for processing by applications.
- Administration – Registering and monitoring users, enforcing
data security, monitoring performance, maintaining data integrity,
dealing with concurrency control, and recovering information if the
system fails.
A DBMS is responsible for maintaining the integrity and security of
stored data, and for recovering information if the system fails.
Both a database and its DBMS conform to the principles of a particular
database model.
[2] "Database system" refers collectively to the database model, database management system, and database.
[3]
Physically, database servers are dedicated computers that hold the
actual databases and run only the DBMS and related software. Database
servers are usually
multiprocessor computers, with generous memory and
RAID
disk arrays used for stable storage. RAID is used for recovery of data
if any of the disks fail. Hardware database accelerators, connected to
one or more servers via a high-speed channel, are also used in large
volume transaction processing environments. DBMSs are found at the heart
of most
database applications. DBMSs may be built around a custom
multitasking kernel with built-in
networking support, but modern DBMSs typically rely on a standard
operating system to provide these functions.
[citation needed] Since DBMSs comprise a significant
economical market, computer and storage vendors often take into account DBMS requirements in their own development plans.
[citation needed]
Databases and DBMSs can be categorized according to the database
model(s) that they support (such as relational or XML), the type(s) of
computer they run on (from a server cluster to a mobile phone), the
query language(s) used to access the database (such as SQL or
XQuery), and their internal engineering, which affects performance,
scalability, resilience, and security.
Applications and roles
Most organizations in developed countries today depend on databases for their
business operations.
Increasingly, databases are not only used to support the internal
operations of the organization, but also to underpin its online
interactions with customers and suppliers (see
Enterprise software).
Databases are not used only to hold administrative information, but are
often embedded within applications to hold more specialized data: for
example engineering data or economic models. Examples of database
applications include computerized
library systems,
flight reservation systems, and computerized
parts inventory systems.
Client-server or
transactional DBMSs are often complex to maintain high
performance,
availability and
security
when many users are querying and updating the database at the same
time. Personal, desktop-based database systems tend to be less complex.
For example,
FileMaker and
Microsoft Access come with built-in
graphical user interfaces.
General-purpose and special-purpose DBMSs
A DBMS has evolved into a complex software system and its development
typically requires thousands of person-years of development effort.
[4] Some general-purpose DBMSs such as
Adabas,
Oracle
and DB2 have been undergoing upgrades since the 1970s. General-purpose
DBMSs aim to meet the needs of as many applications as possible, which
adds to the complexity. However, the fact that their development cost
can be spread over a large number of users means that they are often the
most cost-effective approach. However, a general-purpose DBMS is not
always the optimal solution: in some cases a general-purpose DBMS may
introduce unnecessary overhead. Therefore, there are many examples of
systems that use special-purpose databases. A common example is an
email
system: email systems are designed to optimize the handling of email
messages, and do not need significant portions of a general-purpose DBMS
functionality.
Many databases have
application software
that accesses the database on behalf of end-users, without exposing the
DBMS interface directly. Application programmers may use a
wire protocol directly, or more likely through an
application programming interface.
Database designers and database administrators interact with the DBMS
through dedicated interfaces to build and maintain the applications'
databases, and thus need some more knowledge and understanding about how
DBMSs operate and the DBMSs' external interfaces and tuning parameters.
General-purpose databases are usually developed by one organization
or community of programmers, while a different group builds the
applications that use it. In many companies, specialized database
administrators maintain databases, run reports, and may work on code
that runs on the databases themselves (rather than in the client
application).
History
Following the technology progress in the areas of
processors,
computer memory,
computer storage and
computer networks,
the sizes, capabilities, and performance of databases and their
respective DBMSs have grown in orders of magnitude. The development of
database technology can be divided into three eras based on data model
or structure:
navigational,
[5] SQL/
relational, and post-relational.
The two main early navigational data models were the
hierarchical model, epitomized by IBM's IMS system, and the
CODASYL model (
network model), implemented in a number of products such as
IDMS.
The
relational model, first proposed in 1970 by
Edgar F. Codd,
departed from this tradition by insisting that applications should
search for data by content, rather than by following links. The
relational model employs sets of ledger-style tables, each used for a
different type of entity. Only in the mid-1980s did computing hardware
became powerful enough to allow the wide deployment of relational
systems (DBMSs plus applications). By the early 1990s, however,
relational systems dominated in all large-scale data processing
applications, and as of 2014 they remain dominant except in niche areas.
The dominant database language, standardised SQL for the relational
model, has influenced database languages for other data models.
[citation needed]
Object databases developed in the 1980s to overcome the inconvenience of
object-relational impedance mismatch, which led to the coining of the term "post-relational" and also the development of hybrid
object-relational databases.
The next generation of post-relational databases in the late 2000s became known as
NoSQL databases, introducing fast
key-value stores and
document-oriented databases. A competing "next generation" known as
NewSQL
databases attempted new implementations that retained the
relational/SQL model while aiming to match the high performance of NoSQL
compared to commercially available relational DBMSs.
1960s, navigational DBMS
Basic structure of navigational
CODASYL database model.
The introduction of the term
database coincided with the
availability of direct-access storage (disks and drums) from the
mid-1960s onwards. The term represented a contrast with the tape-based
systems of the past, allowing shared interactive use rather than daily
batch processing. The
Oxford English dictionary cites
[6]
a 1962 report by the System Development Corporation of California as
the first to use the term "data-base" in a specific technical sense.
As computers grew in speed and capability, a number of
general-purpose database systems emerged; by the mid-1960s a number of
such systems had come into commercial use. Interest in a standard began
to grow, and
Charles Bachman, author of one such product, the
Integrated Data Store (IDS), founded the "Database Task Group" within
CODASYL, the group responsible for the creation and standardization of
COBOL.
In 1971 theDatabase Task Group delivered their standard, which
generally became known as the "CODASYL approach", and soon a number of
commercial products based on this approach entered the market.
The CODASYL approach relied on the "manual" navigation of a linked
data set which was formed into a large network. Applications could find
records by one of three methods:
- use of a primary key (known as a CALC key, typically implemented by hashing)
- navigating relationships (called sets) from one record to another
- scanning all the records in a sequential order.
Later systems added B-Trees to provide alternate access paths. Many
CODASYL databases also added a very straightforward query language.
However, in the final tally, CODASYL was very complex and required
significant training and effort to produce useful applications.
IBM also had their own DBMS system in 1968, known as
IMS.
IMS was a development of software written for the
Apollo program on the
System/360.
IMS was generally similar in concept to CODASYL, but used a strict
hierarchy for its model of data navigation instead of CODASYL's network
model. Both concepts later became known as navigational databases due to
the way data was accessed, and Bachman's 1973
Turing Award presentation was
The Programmer as Navigator. IMS is classified
[by whom?] as a
hierarchical database. IDMS and
Cincom Systems'
TOTAL database are classified as network databases. IMS remains in use as of 2014.
[7]
1970s, relational DBMS
Edgar Codd worked at IBM in
San Jose, California, in one of their offshoot offices that was primarily involved in the development of
hard disk
systems. He was unhappy with the navigational model of the CODASYL
approach, notably the lack of a "search" facility. In 1970, he wrote a
number of papers that outlined a new approach to database construction
that eventually culminated in the groundbreaking
A Relational Model of Data for Large Shared Data Banks.
[8]
In this paper, he described a new system for storing and working with
large databases. Instead of records being stored in some sort of
linked list of free-form records as in CODASYL, Codd's idea was to use a "
table"
of fixed-length records, with each table used for a different type of
entity. A linked-list system would be very inefficient when storing
"sparse" databases where some of the data for any one record could be
left empty. The relational model solved this by splitting the data into a
series of normalized tables (or
relations), with optional
elements being moved out of the main table to where they would take up
room only if needed. Data may be freely inserted, deleted and edited in
these tables, with the DBMS doing whatever maintenance needed to present
a table view to the application/user.
The relational model also allowed the content of the database to
evolve without constant rewriting of links and pointers. The relational
part comes from entities referencing other entities in what is known as
one-to-many relationship, like a traditional hierarchical model, and
many-to-many relationship, like a navigational (network) model. Thus, a
relational model can express both hierarchical and navigational models,
as well as its native tabular model, allowing for pure or combined
modeling in terms of these three models, as the application requires.
For instance, a common use of a database system is to track
information about users, their name, login information, various
addresses and phone numbers. In the navigational approach all of these
data would be placed in a single record, and unused items would simply
not be placed in the database. In the relational approach, the data
would be
normalized into a user table, an address table and a
phone number table (for instance). Records would be created in these
optional tables only if the address or phone numbers were actually
provided.
Linking the information back together is the key to this system. In
the relational model, some bit of information was used as a "
key",
uniquely defining a particular record. When information was being
collected about a user, information stored in the optional tables would
be found by searching for this key. For instance, if the login name of a
user is unique, addresses and phone numbers for that user would be
recorded with the login name as its key. This simple "re-linking" of
related data back into a single collection is something that traditional
computer languages are not designed for.
Just as the navigational approach would require programs to loop in
order to collect records, the relational approach would require loops to
collect information about any
one record. Codd's solution to the
necessary looping was a set-oriented language, a suggestion that would
later spawn the ubiquitous SQL. Using a branch of mathematics known as
tuple calculus,
he demonstrated that such a system could support all the operations of
normal databases (inserting, updating etc.) as well as providing a
simple system for finding and returning
sets of data in a single operation.
Codd's paper was picked up by two people at Berkeley,
Eugene Wong and
Michael Stonebraker. They started a project known as
INGRES
using funding that had already been allocated for a geographical
database project and student programmers to produce code. Beginning in
1973, INGRES delivered its first test products which were generally
ready for widespread use in 1979. INGRES was similar to
System R in a number of ways, including the use of a "language" for data access, known as
QUEL. Over time, INGRES moved to the emerging SQL standard.
IBM itself did one test implementation of the relational model,
PRTV, and a production one,
Business System 12, both now discontinued.
Honeywell wrote
MRDS for
Multics, and now there are two new implementations:
Alphora Dataphor and
Rel. Most other DBMS implementations usually called
relational are actually SQL DBMSs.
In 1970, the University of Michigan began development of the
MICRO Information Management System[9] based on D.L. Childs' Set-Theoretic Data model.
[10][11][12] Micro was used to manage very large data sets by the
US Department of Labor, the
U.S. Environmental Protection Agency, and researchers from the
University of Alberta, the
University of Michigan, and
Wayne State University. It ran on IBM mainframe computers using the
Michigan Terminal System.
[13] The system remained in production until 1998.
Integrated approach
In the 1970s and 1980s attempts were made to build database systems
with integrated hardware and software. The underlying philosophy was
that such integration would provide higher performance at lower cost.
Examples were IBM
System/38, the early offering of
Teradata, and the
Britton Lee, Inc. database machine.
Another approach to hardware support for database management was
ICL's
CAFS
accelerator, a hardware disk controller with programmable search
capabilities. In the long term, these efforts were generally
unsuccessful because specialized database machines could not keep pace
with the rapid development and progress of general-purpose computers.
Thus most database systems nowadays are software systems running on
general-purpose hardware, using general-purpose computer data storage.
However this idea is still pursued for certain applications by some
companies like
Netezza and Oracle (
Exadata).
Late 1970s, SQL DBMS
IBM started working on a prototype system loosely based on Codd's concepts as
System R
in the early 1970s. The first version was ready in 1974/5, and work
then started on multi-table systems in which the data could be split so
that all of the data for a record (some of which is optional) did not
have to be stored in a single large "chunk". Subsequent multi-user
versions were tested by customers in 1978 and 1979, by which time a
standardized
query language – SQL
[citation needed]
– had been added. Codd's ideas were establishing themselves as both
workable and superior to CODASYL, pushing IBM to develop a true
production version of System R, known as
SQL/DS, and, later,
Database 2 (DB2).
Larry Ellison's
Oracle started from a different chain, based on IBM's papers on System
R, and beat IBM to market when the first version was released in 1978.
[citation needed]
Stonebraker went on to apply the lessons from INGRES to develop a new
database, Postgres, which is now known as PostgreSQL. PostgreSQL is
often used for global mission critical applications (the .org and .info
domain name registries use it as their primary data store, as do many
large companies and financial institutions).
In Sweden, Codd's paper was also read and
Mimer SQL was developed from the mid-1970s at
Uppsala University.
In 1984, this project was consolidated into an independent enterprise.
In the early 1980s, Mimer introduced transaction handling for high
robustness in applications, an idea that was subsequently implemented on
most other DBMSs.
Another data model, the
entity-relationship model, emerged in 1976 and gained popularity for
database design
as it emphasized a more familiar description than the earlier
relational model. Later on, entity-relationship constructs were
retrofitted as a data modeling construct for the relational model, and
the difference between the two have become irrelevant.
[citation needed]
1980s, on the desktop
The 1980s ushered in the age of
desktop computing.
The new computers empowered their users with spreadsheets like Lotus
1,2,3 and database software like dBASE. The dBASE product was
lightweight and easy for any computer user to understand out of the box.
C. Wayne Ratliff
the creator of dBASE stated: “dBASE was different from programs like
BASIC, C, FORTRAN, and COBOL in that a lot of the dirty work had already
been done. The data manipulation is done by dBASE instead of by the
user, so the user can concentrate on what he is doing, rather than
having to mess with the dirty details of opening, reading, and closing
files, and managing space allocation.“
[14] dBASE was one of the top selling software titles in the 1980s and early 1990s.
1980s, object-oriented
The 1980s, along with a rise in
object-oriented programming,
saw a growth in how data in various databases were handled. Programmers
and designers began to treat the data in their databases as objects.
That is to say that if a person's data were in a database, that person's
attributes, such as their address, phone number, and age, were now
considered to belong to that person instead of being extraneous data.
This allows for relations between data to be relations to objects and
their attributes and not to individual fields.
[15]
The term "object-relational impedance mismatch" described the
inconvenience of translating between programmed objects and database
tables. Object databases and object-relational databases attempt to
solve this problem by providing an object-oriented language (sometimes
as extensions to SQL) that programmers can use as alternative to purely
relational SQL. On the programming side, libraries known as
object-relational mappings (ORMs) attempt to solve the same problem.
2000s, NoSQL and NewSQL
The next generation of post-relational databases in the 2000s became
known as NoSQL databases, including fast key-value stores and
document-oriented databases.
XML databases are a type of structured document-oriented database that allows querying based on
XML document attributes.
NoSQL databases are often very fast, do not require fixed table schemas, avoid join operations by storing
denormalized data, and are designed to
scale horizontally.
In recent years there was a high demand for massively distributed databases with high partition tolerance but according to the
CAP theorem it is impossible for a
distributed system to simultaneously provide
consistency, availability and
partition tolerance
guarantees. A distributed system can satisfy any two of these
guarantees at the same time, but not all three. For that reason many
NoSQL databases are using what is called
eventual consistency to provide both availability and partition tolerance guarantees with a maximum level of data consistency.
The most popular NoSQL systems include:
MongoDB,
Couchbase,
Riak,
memcached,
Redis,
CouchDB,
Hazelcast,
Apache Cassandra and
HBase.
[16] Note that all are
open-source software products.
A number of new relational databases continuing use of SQL but aiming for performance comparable to NoSQL are known as NewSQL.
Research
Database technology has been an active research topic since the 1960s, both in
academia and in the research and development groups of companies (for example
IBM Research). Research activity includes
theory and development of
prototypes. Notable research topics have included
models, the atomic transaction concept and related
concurrency control techniques, query languages and
query optimization methods,
RAID, and more.
The database research area has several dedicated
academic journals (for example,
ACM Transactions on Database Systems-TODS,
Data and Knowledge Engineering-DKE) and annual
conferences (e.g.,
ACM SIGMOD, ACM
PODS,
VLDB,
IEEE ICDE).
Examples
One way to classify databases involves the type of their contents, for example:
bibliographic,
document-text, statistical, or multimedia objects. Another way is by
their application area, for example: accounting, music compositions,
movies, banking, manufacturing, or insurance. A third way is by some
technical aspect, such as the database structure or interface type. This
section lists a few of the adjectives used to characterize different
kinds of databases.
- An in-memory database is a database that primarily resides in main memory,
but is typically backed-up by non-volatile computer data storage. Main
memory databases are faster than disk databases, and so are often used
where response time is critical, such as in telecommunications network
equipment.[17]SAP HANA
platform is a very hot topic for in-memory database. By May 2012, HANA
was able to run on servers with 100TB main memory powered by IBM. The co
founder of the company claimed that the system was big enough to run
the 8 largest SAP customers.
- An active database
includes an event-driven architecture which can respond to conditions
both inside and outside the database. Possible uses include security
monitoring, alerting, statistics gathering and authorization. Many
databases provide active database features in the form of database triggers.
- A cloud database relies on cloud technology.
Both the database and most of its DBMS reside remotely, "in the cloud",
while its applications are both developed by programmers and later
maintained and utilized by (application's) end-users through a web browser and Open APIs.
- Data warehouses
archive data from operational databases and often from external sources
such as market research firms. The warehouse becomes the central source
of data for use by managers and other end-users who may not have access
to operational data. For example, sales data might be aggregated to
weekly totals and converted from internal product codes to use UPCs so that they can be compared with ACNielsen data. Some basic and essential components of data warehousing include retrieving, analyzing, and mining data, transforming, loading and managing data so as to make them available for further use.
- A document-oriented database is designed for storing, retrieving,
and managing document-oriented, or semi structured data, information.
Document-oriented databases are one of the main categories of NoSQL
databases.
- An embedded database
system is a DBMS which is tightly integrated with an application
software that requires access to stored data in such a way that the DBMS
is hidden from the application’s end-users and requires little or no
ongoing maintenance.[18]
- End-user databases consist of data developed by individual
end-users. Examples of these are collections of documents, spreadsheets,
presentations, multimedia, and other files. Several products exist to
support such databases. Some of them are much simpler than full fledged
DBMSs, with more elementary DBMS functionality.
- A federated database system
comprises several distinct databases, each with its own DBMS. It is
handled as a single database by a federated database management system
(FDBMS), which transparently integrates multiple autonomous DBMSs,
possibly of different types (in which case it would also be a heterogeneous database system), and provides them with an integrated conceptual view.
- Sometimes the term multi-database is used as a synonym to
federated database, though it may refer to a less integrated (e.g.,
without an FDBMS and a managed integrated schema) group of databases
that cooperate in a single application. In this case typically middleware is used for distribution, which typically includes an atomic commit protocol (ACP), e.g., the two-phase commit protocol, to allow distributed (global) transactions across the participating databases.
- In a hypertext or hypermedia
database, any word or a piece of text representing an object, e.g.,
another piece of text, an article, a picture, or a film, can be hyperlinked
to that object. Hypertext databases are particularly useful for
organizing large amounts of disparate information. For example, they are
useful for organizing online encyclopedias, where users can conveniently jump around the text. The World Wide Web is thus a large distributed hypertext database.
- Operational databases store detailed data about the operations of an organization. They typically process relatively high volumes of updates using transactions. Examples include customer databases
that record contact, credit, and demographic information about a
business' customers, personnel databases that hold information such as
salary, benefits, skills data about employees, enterprise resource
planning systems that record details about product components, parts
inventory, and financial databases that keep track of the organization's
money, accounting and financial dealings.
-
- The major parallel DBMS architectures which are induced by the underlying hardware architecture are:
- Shared memory architecture, where multiple processors share the main memory space, as well as other data storage.
- Shared disk architecture, where each processing unit
(typically consisting of multiple processors) has its own main memory,
but all units share the other storage.
- Shared nothing architecture, where each processing unit has its own main memory and other storage.
- Real-time databases process transactions fast enough for the result to come back and be acted on right away.
- A spatial database
can store the data with multidimensional features. The queries on such
data include location based queries, like "Where is the closest hotel in
my area?".
- A temporal database
has built-in time aspects, for example a temporal data model and a
temporal version of SQL. More specifically the temporal aspects usually
include valid-time and transaction-time.
- An unstructured data
database is intended to store in a manageable and protected way diverse
objects that do not fit naturally and conveniently in common databases.
It may include email messages, documents, journals, multimedia objects,
etc. The name may be misleading since some objects can be highly
structured. However, the entire possible object collection does not fit
into a predefined structured framework. Most established DBMSs now
support unstructured data in various ways, and new dedicated DBMSs are
emerging.
Design and modeling
The first task of a database designer is to produce a
conceptual data model
that reflects the structure of the information to be held in the
database. A common approach to this is to develop an entity-relationship
model, often with the aid of drawing tools. Another popular approach is
the
Unified Modeling Language.
A successful data model will accurately reflect the possible state of
the external world being modeled: for example, if people can have more
than one phone number, it will allow this information to be captured.
Designing a good conceptual data model requires a good understanding of
the application domain; it typically involves asking deep questions
about the things of interest to an organisation, like "can a customer
also be a supplier?", or "if a product is sold with two different forms
of packaging, are those the same product or different products?", or "if
a plane flies from New York to Dubai via Frankfurt, is that one flight
or two (or maybe even three)?". The answers to these questions establish
definitions of the terminology used for entities (customers, products,
flights, flight segments) and their relationships and attributes.
Producing the conceptual data model sometimes involves input from
business processes, or the analysis of
workflow
in the organization. This can help to establish what information is
needed in the database, and what can be left out. For example, it can
help when deciding whether the database needs to hold historic data as
well as current data.
Having produced a conceptual data model that users are happy with, the next stage is to translate this into a
schema
that implements the relevant data structures within the database. This
process is often called logical database design, and the output is a
logical data model
expressed in the form of a schema. Whereas the conceptual data model is
(in theory at least) independent of the choice of database technology,
the logical data model will be expressed in terms of a particular
database model supported by the chosen DBMS. (The terms
data model and
database model are often used interchangeably, but in this article we use
data model for the design of a specific database, and
database model for the modelling notation used to express that design.)
The most popular database model for general-purpose databases is the
relational model, or more precisely, the relational model as represented
by the SQL language. The process of creating a logical database design
using this model uses a methodical approach known as
normalization.
The goal of normalization is to ensure that each elementary "fact" is
only recorded in one place, so that insertions, updates, and deletions
automatically maintain consistency.
The final stage of database design is to make the decisions that
affect performance, scalability, recovery, security, and the like. This
is often called
physical database design. A key goal during this stage is
data independence,
meaning that the decisions made for performance optimization purposes
should be invisible to end-users and applications. Physical design is
driven mainly by performance requirements, and requires a good knowledge
of the expected workload and access patterns, and a deep understanding
of the features offered by the chosen DBMS.
Another aspect of physical database design is security. It involves both defining
access control to database objects as well as defining security levels and methods for the data itself.
Models
Collage of five types of database models.
Main article:
Database model
A database model is a type of data model that determines the logical
structure of a database and fundamentally determines in which manner
data
can be stored, organized, and manipulated. The most popular example of a
database model is the relational model (or the SQL approximation of
relational), which uses a table-based format.
Common logical data models for databases include:
An object-relational database combines the two related structures.
Physical data models include:
Other models include:
External, conceptual, and internal views
Traditional view of data
[21]
A database management system provides three views of the database data:
- The external level defines how each group of end-users sees
the organization of data in the database. A single database can have any
number of views at the external level.
- The conceptual level unifies the various external views into a compatible global view.[22]
It provides the synthesis of all the external views. It is out of the
scope of the various database end-users, and is rather of interest to
database application developers and database administrators.
- The internal level (or physical level) is the internal
organization of data inside a DBMS (see Implementation section below).
It is concerned with cost, performance, scalability and other
operational matters. It deals with storage layout of the data, using
storage structures such as indexes to enhance performance. Occasionally it stores data of individual views (materialized views),
computed from generic data, if performance justification exists for
such redundancy. It balances all the external views' performance
requirements, possibly conflicting, in an attempt to optimize overall
performance across all activities.
While there is typically only one conceptual (or logical) and
physical (or internal) view of the data, there can be any number of
different external views. This allows users to see database information
in a more business-related way rather than from a technical, processing
viewpoint. For example, a financial department of a company needs the
payment details of all employees as part of the company's expenses, but
does not need details about employees that are the interest of the
human resources department. Thus different departments need different
views of the company's database.
The three-level database architecture relates to the concept of
data independence
which was one of the major initial driving forces of the relational
model. The idea is that changes made at a certain level do not affect
the view at a higher level. For example, changes in the internal level
do not affect application programs written using conceptual level
interfaces, which reduces the impact of making physical changes to
improve performance.
The conceptual view provides a level of indirection between internal
and external. On one hand it provides a common view of the database,
independent of different external view structures, and on the other hand
it abstracts away details of how the data is stored or managed
(internal level). In principle every level, and even every external
view, can be presented by a different data model. In practice usually a
given DBMS uses the same data model for both the external and the
conceptual levels (e.g., relational model). The internal level, which is
hidden inside the DBMS and depends on its implementation (see
Implementation section below), requires a different level of detail and
uses its own types of data structure types.
Separating the
external,
conceptual and
internal levels was a major feature of the relational database model implementations that dominate 21st century databases.
[22]
Languages
Database languages are special-purpose languages, which do one or more of the following:
Database languages are specific to a particular data model. Notable examples include:
- XQuery is a standard XML query language implemented by XML database systems such as MarkLogic and eXist, by relational databases with XML capability such as Oracle and DB2, and also by in-memory XML processors such as Saxon.
A database language may also incorporate features like:
- DBMS-specific Configuration and storage engine management
- Computations to modify query results, like counting, summing, averaging, sorting, grouping, and cross-referencing
- Constraint enforcement (e.g. in an automotive database, only allowing one engine type per car)
- Application programming interface version of the query language, for programmer convenience
Performance, security, and availability
Because of the critical importance of database technology to the
smooth running of an enterprise, database systems include complex
mechanisms to deliver the required performance, security, and
availability, and allow database administrators to control the use of
these features.
Storage
Database storage is the container of the physical materialization of a database. It comprises the
internal (physical)
level in the database architecture. It also contains all the information needed (e.g.,
metadata, "data about the data", and internal
data structures) to reconstruct the
conceptual level and
external level from the internal level when needed. Putting data into permanent storage is generally the responsibility of the
database engine
a.k.a. "storage engine". Though typically accessed by a DBMS through
the underlying operating system (and often utilizing the operating
systems'
file systems
as intermediates for storage layout), storage properties and
configuration setting are extremely important for the efficient
operation of the DBMS, and thus are closely maintained by database
administrators. A DBMS, while in operation, always has its database
residing in several types of storage (e.g., memory and external
storage). The database data and the additional needed information,
possibly in very large amounts, are coded into bits. Data typically
reside in the storage in structures that look completely different from
the way the data look in the conceptual and external levels, but in ways
that attempt to optimize (the best possible) these levels'
reconstruction when needed by users and programs, as well as for
computing additional types of needed information from the data (e.g.,
when querying the database).
Some DBMSs support specifying which
character encoding was used to store data, so multiple encodings can be used in the same database.
Various low-level
database storage structures
are used by the storage engine to serialize the data model so it can be
written to the medium of choice. Techniques such as indexing may be
used to improve performance. Conventional storage is row-oriented, but
there are also
column-oriented and
correlation databases.
Materialized views
Often storage redundancy is employed to increase performance. A common example is storing
materialized views, which consist of frequently needed
external views
or query results. Storing such views saves the expensive computing of
them each time they are needed. The downsides of materialized views are
the overhead incurred when updating them to keep them synchronized with
their original updated database data, and the cost of storage
redundancy.
Replication
Occasionally a database employs storage redundancy by database
objects replication (with one or more copies) to increase data
availability (both to improve performance of simultaneous multiple
end-user accesses to a same database object, and to provide resiliency
in a case of partial failure of a distributed database). Updates of a
replicated object need to be synchronized across the object copies. In
many cases the entire database is replicated.
Security
Database security
deals with all various aspects of protecting the database content, its
owners, and its users. It ranges from protection from intentional
unauthorized database uses to unintentional database accesses by
unauthorized entities (e.g., a person or a computer program).
Database access control deals with controlling who (a person or a
certain computer program) is allowed to access what information in the
database. The information may comprise specific database objects (e.g.,
record types, specific records, data structures), certain computations
over certain objects (e.g., query types, or specific queries), or
utilizing specific access paths to the former (e.g., using specific
indexes or other data structures to access information). Database access
controls are set by special authorized (by the database owner)
personnel that uses dedicated protected security DBMS interfaces.
This may be managed directly on an individual basis, or by the assignment of individuals and
privileges
to groups, or (in the most elaborate models) through the assignment of
individuals and groups to roles which are then granted entitlements.
Data security prevents unauthorized users from viewing or updating the
database. Using passwords, users are allowed access to the entire
database or subsets of it called "subschemas". For example, an employee
database can contain all the data about an individual employee, but one
group of users may be authorized to view only payroll data, while others
are allowed access to only work history and medical data. If the DBMS
provides a way to interactively enter and update the database, as well
as interrogate it, this capability allows for managing personal
databases.
Data security
in general deals with protecting specific chunks of data, both
physically (i.e., from corruption, or destruction, or removal; e.g., see
physical security),
or the interpretation of them, or parts of them to meaningful
information (e.g., by looking at the strings of bits that they comprise,
concluding specific valid credit-card numbers; e.g., see
data encryption).
Change and access logging records who accessed which attributes, what
was changed, and when it was changed. Logging services allow for a
forensic
database audit
later by keeping a record of access occurrences and changes. Sometimes
application-level code is used to record changes rather than leaving
this to the database. Monitoring can be set up to attempt to detect
security breaches.
Transactions and concurrency
Database transactions can be used to introduce some level of
fault tolerance and
data integrity after recovery from a
crash.
A database transaction is a unit of work, typically encapsulating a
number of operations over a database (e.g., reading a database object,
writing, acquiring
lock,
etc.), an abstraction supported in database and also other systems.
Each transaction has well defined boundaries in terms of which
program/code executions are included in that transaction (determined by
the transaction's programmer via special transaction commands).
The acronym
ACID describes some ideal properties of a database transaction:
Atomicity,
Consistency,
Isolation, and
Durability.
Migration
- See also section Database migration in article Data migration
A database built with one DBMS is not portable to another DBMS (i.e.,
the other DBMS cannot run it). However, in some situations it is
desirable to move, migrate a database from one DBMS to another. The
reasons are primarily economical (different DBMSs may have different
total costs of ownership
or TCOs), functional, and operational (different DBMSs may have
different capabilities). The migration involves the database's
transformation from one DBMS type to another. The transformation should
maintain (if possible) the database related application (i.e., all
related application programs) intact. Thus, the database's conceptual
and external architectural levels should be maintained in the
transformation. It may be desired that also some aspects of the
architecture internal level are maintained. A complex or large database
migration may be a complicated and costly (one-time) project by itself,
which should be factored into the decision to migrate. This in spite of
the fact that tools may exist to help migration between specific DBMSs.
Typically a DBMS vendor provides tools to help importing databases from
other popular DBMSs.
Building, maintaining, and tuning
After designing a database for an application, the next stage is building the database. Typically an appropriate
general-purpose DBMS can be selected to be utilized for this purpose. A DBMS provides the needed
user interfaces
to be utilized by database administrators to define the needed
application's data structures within the DBMS's respective data model.
Other user interfaces are used to select needed DBMS parameters (like
security related, storage allocation parameters, etc.).
When the database is ready (all its data structures and other needed
components are defined) it is typically populated with initial
application's data (database initialization, which is typically a
distinct project; in many cases using specialized DBMS interfaces that
support bulk insertion) before making it operational. In some cases the
database becomes operational while empty of application data, and data
is accumulated during its operation.
After the database is created, initialised and populated it needs to
be maintained. Various database parameters may need changing and the
database may need to be tuned (
tuning)
for better performance; application's data structures may be changed or
added, new related application programs may be written to add to the
application's functionality, etc. Databases are often confused with
spreadsheets such as Microsoft Excel (Microsoft Access is a database
management system, Excel is a spreadsheet program). Both can be used to
store information, however a database is more efficient and flexible at
storing large amounts of data. Below is a simple comparison of
spreadsheets and databases.
| Spreadsheet strengths |
Spreadsheet Weaknesses |
| Very simple data storage |
Data integrity problems, including inaccurate, inconsistent and out of date data and formulas. |
| Relatively easy to use |
Difficult to validate data e.g. an incorrect formula |
| Require less planning |
|
| Database strengths |
Database Weaknesses |
| Methods for keeping data up to date and consistent |
Require more planning and designing |
| Data is of higher quality than data stored in spreadsheets |
Harder to change structure once database is built |
| Good for storing and organizing information. |
Requires more technical knowledge to administrate |
Backup and restore
Sometimes it is desired to bring a database back to a previous state
(for many reasons, e.g., cases when the database is found corrupted due
to a software error, or if it has been updated with erroneous data). To
achieve this a
backup operation is done occasionally or
continuously, where each desired database state (i.e., the values of its
data and their embedding in database's data structures) is kept within
dedicated backup files (many techniques exist to do this effectively).
When this state is needed, i.e., when it is decided by a database
administrator to bring the database back to this state (e.g., by
specifying this state by a desired point in time when the database was
in this state), these files are utilized to
restore that state.
Other
Other DBMS features might include:
- Database logs
- Graphics component for producing graphs and charts, especially in a data warehouse system
- Query optimizer – Performs query optimization on every query to choose for it the most efficient query plan
(a partial order (tree) of operations) to be executed to compute the
query result. May be specific to a particular storage engine.
- Tools or hooks for database design, application programming,
application program maintenance, database performance analysis and
monitoring, database configuration monitoring, DBMS hardware
configuration (a DBMS and related database may span computers, networks,
and storage units) and related database mapping (especially for a
distributed DBMS), storage allocation and database layout monitoring,
storage migration, etc.
See also
References
- Jeffrey Ullman 1997: First course in database systems, Prentice–Hall Inc., Simon & Schuster, Page 1, ISBN 0-13-861337-0.
- Tsitchizris, D. C. and F. H. Lochovsky (1982). Data Models. Englewood-Cliffs, Prentice–Hall.
- Beynon-Davies P. (2004). Database Systems 3rd Edition. Palgrave, Basingstoke, UK. ISBN 1-4039-1601-2
- Raul F. Chong, Michael Dang, Dwaine R. Snow, Xiaomei Wang (3 July 2008). "Introduction to DB2". Retrieved 17 March 2013.. This article quotes a development time of 5 years involving 750 people for DB2 release 9 alone
- C. W. Bachmann (November 1973), "The Programmer as Navigator", CACM (Turing Award Lecture 1973)
- "database, n". OED Online. Oxford University Press. June 2013. Retrieved July 12, 2013.
- IBM Corporation. "IBM
Information Management System (IMS) 13 Transaction and Database Servers
delivers high performance and low total cost of ownership". Retrieved Feb 20, 2014.
- Codd, E.F. (1970)."A Relational Model of Data for Large Shared Data Banks". In: Communications of the ACM 13 (6): 377–387.
- William Hershey and Carol Easthope, "A set theoretic data structure and retrieval language", Spring Joint Computer Conference, May 1972 in ACM SIGIR Forum, Volume 7, Issue 4 (December 1972), pp. 45–55, DOI=10.1145/1095495.1095500
- Ken North, "Sets, Data Models and Data Independence", Dr. Dobb's, 10 March 2010
- Description of a set-theoretic data structure,
D. L. Childs, 1968, Technical Report 3 of the CONCOMP (Research in
Conversational Use of Computers) Project, University of Michigan, Ann
Arbor, Michigan, USA
- Feasibility of a Set-Theoretic Data Structure : A General Structure Based on a Reconstituted Definition of Relation,
D. L. Childs, 1968, Technical Report 6 of the CONCOMP (Research in
Conversational Use of Computers) Project, University of Michigan, Ann
Arbor, Michigan, USA
- MICRO Information Management System (Version 5.0) Reference Manual,
M.A. Kahn, D.L. Rumelhart, and B.L. Bronson, October 1977, Institute of
Labor and Industrial Relations (ILIR), University of Michigan and Wayne
State University
- Interview with Wayne Ratliff. The FoxPro History. Retrieved on 2013-07-12.
- Development of an object-oriented DBMS; Portland, Oregon, United States; Pages: 472 – 482; 1986; ISBN 0-89791-204-7
- "DB-Engines Ranking". January 2013. Retrieved 22 January 2013.
- "TeleCommunication
Systems Signs up as a Reseller of TimesTen; Mobile Operators and
Carriers Gain Real-Time Platform for Location-Based Services". Business Wire. 2002-06-24.
- Graves, Steve. "COTS Databases For Embedded Systems", Embedded Computing Design magazine, January 2007. Retrieved on August 13, 2008.
- Argumentation in Artificial Intelligence by Iyad Rahwan, Guillermo R. Simari
- "OWL DL Semantics". Retrieved 10 December 2010.
- itl.nist.gov (1993) Integration Definition for Information Modeling (IDEFIX). 21 December 1993.
- Date 1990, pp. 31–32
- Chapple, Mike. "SQL Fundamentals". Databases. About.com. Retrieved 2009-01-28.
- "Structured Query Language (SQL)". International Business Machines. October 27, 2006. Retrieved 2007-06-10.
- Wagner, Michael (2010), "1. Auflage", SQL/XML:2006 – Evaluierung der Standardkonformität ausgewählter Datenbanksysteme, Diplomica Verlag, ISBN 3-8366-9609-6
Further reading
- Ling Liu and Tamer M. Özsu (Eds.) (2009). "Encyclopedia of Database Systems, 4100 p. 60 illus. ISBN 978-0-387-49616-0.
- Beynon-Davies, P. (2004). Database Systems. 3rd Edition. Palgrave, Houndmills, Basingstoke.
- Connolly, Thomas and Carolyn Begg. Database Systems. New York: Harlow, 2002.
- Date, C. J. (2003). An Introduction to Database Systems, Fifth Edition. Addison Wesley. ISBN 0-201-51381-1.
- Gray, J. and Reuter, A. Transaction Processing: Concepts and Techniques, 1st edition, Morgan Kaufmann Publishers, 1992.
- Kroenke, David M. and David J. Auer. Database Concepts. 3rd ed. New York: Prentice, 2007.
- Raghu Ramakrishnan and Johannes Gehrke, Database Management Systems
- Abraham Silberschatz, Henry F. Korth, S. Sudarshan, Database System Concepts
- Discussion on database systems, [1]
- Lightstone, S.; Teorey, T.; Nadeau, T. (2007). Physical Database Design: the database professional's guide to exploiting indexes, views, storage, and more. Morgan Kaufmann Press. ISBN 0-12-369389-6.
- Teorey, T.; Lightstone, S. and Nadeau, T. Database Modeling & Design: Logical Design, 4th edition, Morgan Kaufmann Press, 2005. ISBN 0-12-685352-5
External links
Blogspot
merupakan domain yang mennjukan dimana alamat blog anda berada. Namun,
domain .blogspot.com tersebut bisa anda ubah sesuai kebutuhan anda.
Dengan fitur custom domain tentunya. Misalnya saja anda mempunyai blog
dengan alamat di "namabloganda.blogspot.com" menjadi "namabloganda.com",
itu dapat dilakukan. Akan tetapi anda harus mengeluarkan biaya untuk
menyewa domain tersebut.
