Home Explore Blog CI



postgresql

20th chunk of `doc/src/sgml/logicaldecoding.sgml`
3ee03e186f2079a8d6e42b4dce300c70d3e44863e1509d4e0000000100000fb0
 <structname>pg_class</structname> in
        a transaction.
       </para>
      </listitem>

      <listitem>
       <para>
        <command>PREPARE TRANSACTION</command> after <command>LOCK</command> command
        on <structname>pg_class</structname> and allow logical decoding of two-phase
        transactions.
       </para>
      </listitem>

      <listitem>
       <para>
        <command>PREPARE TRANSACTION</command> after <command>CLUSTER</command>
        command on <structname>pg_trigger</structname> and allow logical decoding of
        two-phase transactions. This will lead to deadlock only when published table
        have a trigger.
       </para>
      </listitem>

      <listitem>
       <para>
        Executing <command>TRUNCATE</command> on [user] catalog table in a
        transaction.
       </para>
      </listitem>
     </itemizedlist>

     Note that these commands can cause deadlocks not only for the system
     catalog tables listed above but for other catalog tables.
    </para>
   </sect2>
  </sect1>

  <sect1 id="logicaldecoding-streaming">
   <title>Streaming of Large Transactions for Logical Decoding</title>

   <para>
    The basic output plugin callbacks (e.g., <function>begin_cb</function>,
    <function>change_cb</function>, <function>commit_cb</function> and
    <function>message_cb</function>) are only invoked when the transaction
    actually commits. The changes are still decoded from the transaction
    log, but are only passed to the output plugin at commit (and discarded
    if the transaction aborts).
   </para>

   <para>
    This means that while the decoding happens incrementally, and may spill
    to disk to keep memory usage under control, all the decoded changes have
    to be transmitted when the transaction finally commits (or more precisely,
    when the commit is decoded from the transaction log). Depending on the
    size of the transaction and network bandwidth, the transfer time may
    significantly increase the apply lag.
   </para>

   <para>
    To reduce the apply lag caused by large transactions, an output plugin
    may provide additional callback to support incremental streaming of
    in-progress transactions. There are multiple required streaming callbacks
    (<function>stream_start_cb</function>, <function>stream_stop_cb</function>,
    <function>stream_abort_cb</function>, <function>stream_commit_cb</function>
    and <function>stream_change_cb</function>) and two optional callbacks
    (<function>stream_message_cb</function> and <function>stream_truncate_cb</function>).
    Also, if streaming of two-phase commands is to be supported, then additional
    callbacks must be provided. (See <xref linkend="logicaldecoding-two-phase-commits"/>
    for details).
   </para>

   <para>
    When streaming an in-progress transaction, the changes (and messages) are
    streamed in blocks demarcated by <function>stream_start_cb</function>
    and <function>stream_stop_cb</function> callbacks. Once all the decoded
    changes are transmitted, the transaction can be committed using the
    <function>stream_commit_cb</function> callback
    (or possibly aborted using the <function>stream_abort_cb</function> callback).
    If two-phase commits are supported, the transaction can be prepared using the
    <function>stream_prepare_cb</function> callback,
    <command>COMMIT PREPARED</command> using the
    <function>commit_prepared_cb</function> callback or aborted using the
    <function>rollback_prepared_cb</function>.
   </para>

   <para>
    One example sequence of streaming callback calls for one transaction may
    look like this:
<programlisting>
stream_start_cb(...);   &lt;-- start of first block of changes
  stream_change_cb(...);
  stream_change_cb(...);
  stream_message_cb(...);
  stream_change_cb(...);
  ...
  stream_change_cb(...);
stream_stop_cb(...);    &lt;-- end of first block of changes

stream_start_cb(...);   &lt;-- start of second block of changes
  stream_change_cb(...);

Title: Streaming of Large Transactions for Logical Decoding
Summary
This section explains how logical decoding can stream large transactions incrementally to reduce apply lag, and describes the required and optional callback functions that output plugins must provide to support streaming, including start, stop, commit, and abort callbacks, as well as callbacks for two-phase commits.