<!-- doc/src/sgml/advanced.sgml -->
<chapter id="tutorial-advanced">
<title>Advanced Features</title>
<sect1 id="tutorial-advanced-intro">
<title>Introduction</title>
<para>
In the previous chapter we have covered the basics of using
<acronym>SQL</acronym> to store and access your data in
<productname>PostgreSQL</productname>. We will now discuss some
more advanced features of <acronym>SQL</acronym> that simplify
management and prevent loss or corruption of your data. Finally,
we will look at some <productname>PostgreSQL</productname>
extensions.
</para>
<para>
This chapter will on occasion refer to examples found in <xref
linkend="tutorial-sql"/> to change or improve them, so it will be
useful to have read that chapter. Some examples from
this chapter can also be found in
<filename>advanced.sql</filename> in the tutorial directory. This
file also contains some sample data to load, which is not
repeated here. (Refer to <xref linkend="tutorial-sql-intro"/> for
how to use the file.)
</para>
</sect1>
<sect1 id="tutorial-views">
<title>Views</title>
<indexterm zone="tutorial-views">
<primary>view</primary>
</indexterm>
<para>
Refer back to the queries in <xref linkend="tutorial-join"/>.
Suppose the combined listing of weather records and city location
is of particular interest to your application, but you do not want
to type the query each time you need it. You can create a
<firstterm>view</firstterm> over the query, which gives a name to
the query that you can refer to like an ordinary table:
<programlisting>
CREATE VIEW myview AS
SELECT name, temp_lo, temp_hi, prcp, date, location
FROM weather, cities
WHERE city = name;
SELECT * FROM myview;
</programlisting>
</para>
<para>
Making liberal use of views is a key aspect of good SQL database
design. Views allow you to encapsulate the details of the
structure of your tables, which might change as your application
evolves, behind consistent interfaces.
</para>
<para>
Views can be used in almost any place a real table can be used.
Building views upon other views is not uncommon.
</para>
</sect1>
<sect1 id="tutorial-fk">
<title>Foreign Keys</title>
<indexterm zone="tutorial-fk">
<primary>foreign key</primary>
</indexterm>
<indexterm zone="tutorial-fk">
<primary>referential integrity</primary>
</indexterm>
<para>
Recall the <classname>weather</classname> and
<classname>cities</classname> tables from <xref
linkend="tutorial-sql"/>. Consider the following problem: You
want to make sure that no one can insert rows in the
<classname>weather</classname> table that do not have a matching
entry in the <classname>cities</classname> table. This is called
maintaining the <firstterm>referential integrity</firstterm> of
your data. In simplistic database systems this would be
implemented (if at all) by first looking at the
<classname>cities</classname> table to check if a matching record
exists, and then inserting or rejecting the new
<classname>weather</classname> records. This approach has a
number of problems and is very inconvenient, so
<productname>PostgreSQL</productname> can do this for you.
</para>
<para>
The new declaration of the tables would look like this:
<programlisting>
CREATE TABLE cities (
name varchar(80) primary key,
location point
);
CREATE TABLE weather (
city varchar(80) references cities(name),
temp_lo int,
temp_hi int,
prcp real,
date date
);
</programlisting>
Now try inserting an invalid record:
<programlisting>
INSERT INTO weather VALUES ('Berkeley', 45, 53, 0.0, '1994-11-28');
</programlisting>
<screen>
ERROR: insert or update on table "weather" violates foreign key constraint "weather_city_fkey"
DETAIL: Key (city)=(Berkeley) is not present in table "cities".
</screen>
</para>
<para>
The behavior of foreign keys can be finely tuned to your
application. We will not go beyond this simple example in this
tutorial, but just refer you to <xref linkend="ddl"/>
for more information. Making correct use of
foreign keys will definitely improve the quality of your database
applications, so you are strongly encouraged to learn about them.
</para>
</sect1>
<sect1 id="tutorial-transactions">
<title>Transactions</title>
<indexterm zone="tutorial-transactions">
<primary>transaction</primary>
</indexterm>
<para>
<firstterm>Transactions</firstterm> are a fundamental concept of all database
systems. The essential point of a transaction is that it bundles
multiple steps into a single, all-or-nothing operation. The intermediate
states between the steps are not visible to other concurrent transactions,
and if some failure occurs that prevents the transaction from completing,
then none of the steps affect the database at all.
</para>
<para>
For example, consider a bank database that contains balances for various
customer accounts, as well as total deposit balances for branches.
Suppose that we want to record a payment of $100.00 from Alice's account
to Bob's account. Simplifying outrageously, the SQL commands for this
might look like:
<programlisting>
UPDATE accounts SET balance = balance - 100.00
WHERE name = 'Alice';
UPDATE branches SET balance = balance - 100.00
WHERE name = (SELECT branch_name FROM accounts WHERE name = 'Alice');
UPDATE accounts SET balance = balance + 100.00
WHERE name = 'Bob';
UPDATE branches SET balance = balance + 100.00
WHERE name = (SELECT branch_name FROM accounts WHERE name = 'Bob');
</programlisting>
</para>
<para>
The details of these commands are not important here; the important
point is that there are several separate updates involved to accomplish
this rather simple operation. Our bank's officers will want to be
assured that either all these updates happen, or none of them happen.
It would certainly not do for a system failure to result in Bob
receiving $100.00 that was not debited from Alice. Nor would Alice long
remain a happy customer if she was debited without Bob being credited.
We need a guarantee that if something goes wrong partway through the
operation, none of the steps executed so far will take effect. Grouping
the updates into a <firstterm>transaction</firstterm> gives us this guarantee.
A transaction is said to be <firstterm>atomic</firstterm>: from the point of
view of other transactions, it either happens completely or not at all.
</para>
<para>
We also want a
guarantee that once a transaction is completed and acknowledged by
the database system, it has indeed been permanently recorded
and won't be lost even if a crash ensues shortly thereafter.
For example, if we are recording a cash withdrawal by Bob,
we do not want any chance that the debit to his account will
disappear in a crash just after he walks out the bank door.
A transactional database guarantees that all the updates made by
a transaction are logged in permanent storage (i.e., on disk) before
the transaction is reported complete.
</para>
<para>
Another important property of transactional databases is closely
related to the notion of atomic updates: when multiple transactions
are running concurrently, each one should not be able to see the
incomplete changes made by others. For example, if one transaction
is busy totalling all the branch balances, it would not do for it
to include the debit from Alice's branch but not the credit to
Bob's branch, nor vice versa. So transactions must be all-or-nothing
not only in terms of their permanent effect on the database, but
also in terms of their visibility as they happen. The updates made
so far by an open transaction are invisible to other transactions
until the transaction completes, whereupon all the updates become
visible simultaneously.
</para>
<para>
In <productname>PostgreSQL</productname>, a transaction is set up by surrounding
the SQL commands of the transaction with
<command>BEGIN</command> and <command>COMMIT</command> commands. So our banking
transaction would actually look like:
<programlisting>
BEGIN;
UPDATE accounts SET balance = balance - 100.00
WHERE name = 'Alice';
-- etc etc
COMMIT;
</programlisting>
</para>
<para>
If, partway through the transaction, we decide we do not want to
commit (perhaps we just noticed that Alice's balance went negative),
we can issue the command <command>ROLLBACK</command> instead of
<command>COMMIT</command>, and all our updates so far will be canceled.
</para>
<para>
<productname>PostgreSQL</productname> actually treats every SQL statement as being
executed within a transaction. If you do not issue a <command>BEGIN</command>
command,
then each individual statement has an implicit <command>BEGIN</command> and
(if successful) <command>COMMIT</command> wrapped around it. A group of
statements surrounded by <command>BEGIN</command> and <command>COMMIT</command>
is sometimes called a <firstterm>transaction block</firstterm>.
</para>
<note>
<para>
Some client libraries issue <command>BEGIN</command> and <command>COMMIT</command>
commands automatically, so that you might get the effect of transaction
blocks without asking. Check the documentation for the interface
you are using.
</para>
</note>
<para>
It's possible to control the statements in a transaction in a more
granular fashion through the use of <firstterm>savepoints</firstterm>. Savepoints
allow you to selectively discard parts of the transaction, while
committing the rest. After defining a savepoint with
<command>SAVEPOINT</command>, you can if needed roll back to the savepoint
with <command>ROLLBACK TO</command>. All the transaction's database changes
between defining the savepoint and rolling back to it are discarded, but
changes earlier than the savepoint are kept.
</para>
<para>
After rolling back to a savepoint, it continues to be defined, so you can
roll back to it several times. Conversely, if you are sure you won't need
to roll back to a particular savepoint again, it can be released, so the
system can free some resources. Keep in mind that either releasing or
rolling back to a savepoint
will automatically release all savepoints that were defined after it.
</para>
<para>
All this is happening within the transaction block, so none of it
is visible to other database sessions. When and if you commit the
transaction block, the committed actions become visible as a unit
to other sessions, while the rolled-back actions never become visible
at all.
</para>
<para>
Remembering the bank database, suppose we debit $100.00 from Alice's
account, and credit Bob's account, only to find later that we should
have credited Wally's account. We could do it using savepoints like
this:
<programlisting>
BEGIN;
UPDATE accounts SET balance = balance - 100.00
WHERE name = 'Alice';
SAVEPOINT my_savepoint;
UPDATE accounts SET balance = balance + 100.00
WHERE name = 'Bob';
-- oops ... forget that and use Wally's account
ROLLBACK TO my_savepoint;
UPDATE accounts SET balance = balance + 100.00
WHERE name = 'Wally';
COMMIT;
</programlisting>
</para>
<para>
This example is, of course, oversimplified, but there's a lot of control
possible in a transaction block through the use of savepoints.
Moreover, <command>ROLLBACK TO</command> is the only way to regain control of a
transaction block that was put in aborted state by the
system due to an error, short of rolling it back completely and starting
again.
</para>
</sect1>
<sect1 id="tutorial-window">
<title>Window Functions</title>
<indexterm zone="tutorial-window">
<primary>window function</primary>
</indexterm>
<para>
A <firstterm>window function</firstterm> performs a calculation across a set of
table rows that are somehow related to the current row. This is comparable
to the type of calculation that can be done with an aggregate function.
However, window functions do not cause rows to become grouped into a single
output row like non-window aggregate calls would. Instead, the
rows retain their separate identities. Behind the scenes, the window
function is able to access more than just the current row of the query
result.
</para>
<para>
Here is an example that shows how to compare each employee's salary
with the average salary in his or her department:
<programlisting>
SELECT depname, empno, salary, avg(salary) OVER (PARTITION BY depname) FROM empsalary;
</programlisting>
<screen>
depname | empno | salary | avg
-----------+-------+--------+-----------------------
develop | 11 | 5200 | 5020.0000000000000000
develop | 7 | 4200 | 5020.0000000000000000
develop | 9 | 4500 | 5020.0000000000000000
develop | 8 | 6000 | 5020.0000000000000000
develop | 10 | 5200 | 5020.0000000000000000
personnel | 5 | 3500 | 3700.0000000000000000
personnel | 2 | 3900 | 3700.0000000000000000
sales | 3 | 4800 | 4866.6666666666666667
sales | 1 | 5000 | 4866.6666666666666667
sales | 4 | 4800 | 4866.6666666666666667
(10 rows)
</screen>
The first three output columns come directly from the table
<structname>empsalary</structname>, and there is one output row for each row in the
table. The fourth column represents an average taken across all the table
rows that have the same <structfield>depname</structfield> value as the current row.
(This actually is the same function as the non-window <function>avg</function>
aggregate, but the <literal>OVER</literal> clause causes it to be
treated as a window function and computed across the window frame.)
</para>
<para>
A window function call always contains an <literal>OVER</literal> clause
directly following the window function's name and argument(s). This is what
syntactically distinguishes it from a normal function or non-window
aggregate. The <literal>OVER</literal> clause determines exactly how the
rows of the query are split up for processing by the window function.
The <literal>PARTITION BY</literal> clause within <literal>OVER</literal>
divides the rows into groups, or partitions, that share the same
values of the <literal>PARTITION BY</literal> expression(s). For each row,
the window function is computed across the rows that fall into the
same partition as the current row.
</para>
<para>
You can also control the order in which rows are processed by
window functions using <literal>ORDER BY</literal> within <literal>OVER</literal>.
(The window <literal>ORDER BY</literal> does not even have to match the
order in which the rows are output.) Here is an example:
<programlisting>
SELECT depname, empno, salary,
row_number() OVER (PARTITION BY depname ORDER BY salary DESC)
FROM empsalary;
</programlisting>
<screen>
depname | empno | salary | row_number
-----------+-------+--------+------------
develop | 8 | 6000 | 1
develop | 10 | 5200 | 2
develop | 11 | 5200 | 3
develop | 9 | 4500 | 4
develop | 7 | 4200 | 5
personnel | 2 | 3900 | 1
personnel | 5 | 3500 | 2
sales | 1 | 5000 | 1
sales | 4 | 4800 | 2
sales | 3 | 4800 | 3
(10 rows)
</screen>
As shown here, the <function>row_number</function> window function
assigns sequential numbers to the rows within each partition,
in the order defined by the <literal>ORDER BY</literal> clause
(with tied rows numbered in an unspecified order).
<function>row_number</function> needs no explicit parameter,
because its behavior
is entirely determined by the <literal>OVER</literal> clause.
</para>
<para>
The rows considered by a window function are those of the <quote>virtual
table</quote> produced by the query's <literal>FROM</literal> clause as filtered by its
<literal>WHERE</literal>, <literal>GROUP BY</literal>, and <literal>HAVING</literal> clauses
if any. For example, a row removed because it does not meet the
<literal>WHERE</literal> condition is not seen by any window function.
A query can contain multiple window functions that slice up the data
in different ways using different <literal>OVER</literal> clauses, but
they all act on the same collection of rows defined by this virtual table.
</para>
<para>
We already saw that <literal>ORDER BY</literal> can be omitted if the ordering
of rows is not important. It is also possible to omit <literal>PARTITION
BY</literal>, in which case there is a single partition containing all rows.
</para>
<para>
There is another important concept associated with window functions:
for each row, there is a set of rows within its partition called its
<firstterm>window frame</firstterm>. Some window functions act only
on the rows of the window frame, rather than of the whole partition.
By default, if <literal>ORDER BY</literal> is supplied then the frame consists of
all rows from the start of the partition up through the current row, plus
any following rows that are equal to the current row according to the
<literal>ORDER BY</literal> clause. When <literal>ORDER BY</literal> is omitted the
default frame consists of all rows in the partition.
<footnote>
<para>
There are options to define the window frame in other ways, but
this tutorial does not cover them. See
<xref linkend="syntax-window-functions"/> for details.
</para>
</footnote>
Here is an example using <function>sum</function>:
</para>
<programlisting>
SELECT salary, sum(salary) OVER () FROM empsalary;
</programlisting>
<screen>
salary | sum
--------+-------
5200 | 47100
5000 | 47100
3500 | 47100
4800 | 47100
3900 | 47100
4200 | 47100
4500 | 47100
4800 | 47100
6000 | 47100
5200 | 47100
(10 rows)
</screen>
<para>
Above, since there is no <literal>ORDER BY</literal> in the <literal>OVER</literal>
clause, the window frame is the same as the partition, which for lack of
<literal>PARTITION BY</literal> is the whole table; in other words each sum is
taken over the whole table and so we get the same result for each output
row. But if we add an <literal>ORDER BY</literal> clause, we get very different
results:
</para>
<programlisting>
SELECT salary, sum(salary) OVER (ORDER BY salary) FROM empsalary;
</programlisting>
<screen>
salary | sum
--------+-------
3500 | 3500
3900 | 7400
4200 | 11600
4500 | 16100
4800 | 25700
4800 | 25700
5000 | 30700
5200 | 41100
5200 | 41100
6000 | 47100
(10 rows)
</screen>
<para>
Here the sum is taken from the first (lowest) salary up through the
current one, including any duplicates of the current one (notice the
results for the duplicated salaries).
</para>
<para>
Window functions are permitted only in the <literal>SELECT</literal> list
and the <literal>ORDER BY</literal> clause of the query. They are forbidden
elsewhere, such as in <literal>GROUP BY</literal>, <literal>HAVING</literal>
and <literal>WHERE</literal> clauses. This is because they logically
execute after the processing of those clauses. Also, window functions
execute after non-window aggregate functions. This means it is valid to
include an aggregate function call in the arguments of a window function,
but not vice versa.
</para>
<para>
If there is a need to filter or group rows after the window calculations
are performed, you can use a sub-select. For example:
<programlisting>
SELECT depname, empno, salary, enroll_date
FROM
(SELECT depname, empno, salary, enroll_date,
row_number() OVER (PARTITION BY depname ORDER BY salary DESC, empno) AS pos
FROM empsalary
) AS ss
WHERE pos &lt; 3;
</programlisting>
The above query only shows the rows from the inner query having
<literal>row_number</literal> less than 3 (that is, the first
two rows for each department).
</para>
<para>
When a query involves multiple window functions, it is possible to write
out each one with a separate <literal>OVER</literal> clause, but this is
duplicative and error-prone if the same windowing behavior is wanted
for several functions. Instead, each windowing behavior can be named
in a <literal>WINDOW</literal> clause and then referenced in <literal>OVER</literal>.
For example:
<programlisting>
SELECT sum(salary) OVER w, avg(salary) OVER w
FROM empsalary
WINDOW w AS (PARTITION BY depname ORDER BY salary DESC);
</programlisting>
</para>
<para>
More details about window functions can be found in
<xref linkend="syntax-window-functions"/>,
<xref linkend="functions-window"/>,
<xref linkend="queries-window"/>, and the
<xref linkend="sql-select"/> reference page.
</para>
</sect1>
<sect1 id="tutorial-inheritance">
<title>Inheritance</title>
<indexterm zone="tutorial-inheritance">
<primary>inheritance</primary>
</indexterm>
<para>
Inheritance is a concept from object-oriented databases. It opens
up interesting new possibilities of database design.
</para>
<para>
Let's create two tables: A table <classname>cities</classname>
and a table <classname>capitals</classname>. Naturally, capitals
are also cities, so you want some way to show the capitals
implicitly when you list all cities. If you're really clever you
might invent some scheme like this:
<programlisting>
CREATE TABLE capitals (
name text,
population real,
elevation int, -- (in ft)
state char(2)
);
CREATE TABLE non_capitals (
name text,
population real,
elevation int -- (in ft)
);
CREATE VIEW cities AS
SELECT name, population, elevation FROM capitals
UNION
SELECT name, population, elevation FROM non_capitals;
</programlisting>
This works OK as far as querying goes, but it gets ugly when you
need to update several rows, for one thing.
</para>
<para>
A better solution is this:
<programlisting>
CREATE TABLE cities (
name text,
population real,
elevation int -- (in ft)
);
CREATE TABLE capitals (
state char(2) UNIQUE NOT NULL
) INHERITS (cities);
</programlisting>
</para>
<para>
In this case, a row of <classname>capitals</classname>
<firstterm>inherits</firstterm> all columns (<structfield>name</structfield>,
<structfield>population</structfield>, and <structfield>elevation</structfield>) from its
<firstterm>parent</firstterm>, <classname>cities</classname>. The
type of the column <structfield>name</structfield> is
<type>text</type>, a native <productname>PostgreSQL</productname>
type for variable length character strings. The
<classname>capitals</classname> table has
an additional column, <structfield>state</structfield>, which shows its
state abbreviation. In
<productname>PostgreSQL</productname>, a table can inherit from
zero or more other tables.
</para>
<para>
For example, the following query finds the names of all cities,
including state capitals, that are located at an elevation
over 500 feet:
<programlisting>
SELECT name, elevation
FROM cities
WHERE elevation &gt; 500;
</programlisting>
which returns:
<screen>
name | elevation
-----------+-----------
Las Vegas | 2174
Mariposa | 1953
Madison | 845
(3 rows)
</screen>
</para>
<para>
On the other hand, the following query finds
all the cities that are not state capitals and
are situated at an elevation over 500 feet:
<programlisting>
SELECT name, elevation
FROM ONLY cities
WHERE elevation &gt; 500;
</programlisting>
<screen>
name | elevation
-----------+-----------
Las Vegas | 2174
Mariposa | 1953
(2 rows)
</screen>
</para>
<para>
Here the <literal>ONLY</literal> before <literal>cities</literal>
indicates that the query should be run over only the
<classname>cities</classname> table, and not tables below
<classname>cities</classname> in the inheritance hierarchy. Many
of the commands that we have already discussed &mdash;
<command>SELECT</command>, <command>UPDATE</command>, and
<command>DELETE</command> &mdash; support this <literal>ONLY</literal>
notation.
</para>
<note>
<para>
Although inheritance is frequently useful, it has not been integrated
with unique constraints or foreign keys, which limits its usefulness.
See <xref linkend="ddl-inherit"/> for more detail.
</para>
</note>
</sect1>
<sect1 id="tutorial-conclusion">
<title>Conclusion</title>
<para>
<productname>PostgreSQL</productname> has many features not
touched upon in this tutorial introduction, which has been
oriented toward newer users of <acronym>SQL</acronym>. These
features are discussed in more detail in the remainder of this
book.
</para>
<para>
If you feel you need more introductory material, please visit the PostgreSQL
<ulink url="https://www.postgresql.org">web site</ulink>
for links to more resources.
</para>
</sect1>
</chapter>