wenzelm@67904: (*:maxLineLen=78:*)
wenzelm@67904: 
wenzelm@67904: theory Server
wenzelm@67904: imports Base
wenzelm@67904: begin
wenzelm@67904: 
wenzelm@67904: chapter \<open>The Isabelle server\<close>
wenzelm@67904: 
wenzelm@67904: text \<open>
wenzelm@67904:   An Isabelle session requires at least two processes, which are both rather
wenzelm@67904:   heavy: Isabelle/Scala for the system infrastructure and Isabelle/ML for the
wenzelm@67904:   logic session (e.g.\ HOL). In principle, these processes can be invoked
wenzelm@67904:   directly on the command-line, e.g.\ via @{tool java}, @{tool scala}, @{tool
wenzelm@67904:   process}, @{tool console}, but this approach is inadequate for reactive
wenzelm@67904:   applications that require quick responses from the prover.
wenzelm@67904: 
wenzelm@67904:   In contrast, the Isabelle server exposes Isabelle/Scala as a
wenzelm@67904:   ``terminate-stay-resident'' application that manages multiple logic
wenzelm@67904:   \<^emph>\<open>sessions\<close> and concurrent tasks to use \<^emph>\<open>theories\<close>. This provides an
wenzelm@67904:   analogous to @{ML Thy_Info.use_theories} in Isabelle/ML, but with full
wenzelm@67904:   concurrency and Isabelle/PIDE markup.
wenzelm@67904: 
wenzelm@67904:   The client/server arrangement via TCP sockets also opens possibilities for
wenzelm@67904:   remote Isabelle services that are accessed by local applications, e.g.\ via
wenzelm@67904:   an SSH tunnel.
wenzelm@67904: \<close>
wenzelm@67904: 
wenzelm@67904: 
wenzelm@67904: section \<open>Command-line tools\<close>
wenzelm@67904: 
wenzelm@67904: subsection \<open>Server \label{sec:tool-server}\<close>
wenzelm@67904: 
wenzelm@67904: text \<open>
wenzelm@67904:   The @{tool_def server} tool manages resident server processes:
wenzelm@67904:   @{verbatim [display]
wenzelm@67904: \<open>Usage: isabelle server [OPTIONS]
wenzelm@67904: 
wenzelm@67904:   Options are:
wenzelm@67904:     -L FILE      logging on FILE
wenzelm@67904:     -c           console interaction with specified server
wenzelm@67904:     -l           list servers (alternative operation)
wenzelm@67904:     -n NAME      explicit server name (default: isabelle)
wenzelm@67904:     -p PORT      explicit server port
wenzelm@67904:     -s           assume existing server, no implicit startup
wenzelm@67904:     -x           exit specified server (alternative operation)
wenzelm@67904: 
wenzelm@67904:   Manage resident Isabelle servers.\<close>}
wenzelm@67904: 
wenzelm@67904:   The main operation of \<^verbatim>\<open>isabelle server\<close> is to ensure that a named server is
wenzelm@67904:   running, either by finding an already running process (according to the
wenzelm@67904:   central database file @{path "$ISABELLE_HOME_USER/servers.db"}) or by
wenzelm@67904:   becoming itself a new server that accepts connections on a particular TCP
wenzelm@67904:   socket. The server name and its address are printed as initial output line.
wenzelm@67904:   If another server instance is already running, the current
wenzelm@67904:   \<^verbatim>\<open>isabelle server\<close> process will terminate; otherwise, it keeps running as a
wenzelm@67904:   new server process until an explicit \<^verbatim>\<open>shutdown\<close> command is received.
wenzelm@67904:   Further details of the server socket protocol are explained in
wenzelm@67904:   \secref{sec:server-protocol}.
wenzelm@67904: 
wenzelm@67904:   Other server management operations are invoked via options \<^verbatim>\<open>-l\<close> and \<^verbatim>\<open>-x\<close>
wenzelm@67904:   (see below).
wenzelm@67904: 
wenzelm@67904:   \<^medskip>
wenzelm@67904:   Option \<^verbatim>\<open>-n\<close> specifies an alternative server name: at most one process for
wenzelm@67904:   each name may run, but each server instance supports multiple connections
wenzelm@67904:   and logic sessions.
wenzelm@67904: 
wenzelm@67904:   \<^medskip>
wenzelm@67904:   Option \<^verbatim>\<open>-p\<close> specifies an explicit TCP port for the server socket (which is
wenzelm@67904:   always on \<^verbatim>\<open>localhost\<close>): the default is to let the operating system assign a
wenzelm@67904:   free port number.
wenzelm@67904: 
wenzelm@67904:   \<^medskip>
wenzelm@67904:   Option \<^verbatim>\<open>-s\<close> strictly assumes that the specified server process is already
wenzelm@67904:   running, skipping the optional server startup phase.
wenzelm@67904: 
wenzelm@67904:   \<^medskip>
wenzelm@67904:   Option \<^verbatim>\<open>-c\<close> connects the console in/out channels after the initial check
wenzelm@67918:   for a suitable server process. Also note that the @{tool client} tool
wenzelm@67918:   (\secref{sec:tool-client}) provides a command-line editor to interact with
wenzelm@67918:   the server.
wenzelm@67904: 
wenzelm@67904:   \<^medskip>
wenzelm@67904:   Option \<^verbatim>\<open>-L\<close> specifies a log file for exceptional output of internal server
wenzelm@67904:   and session operations.
wenzelm@67918: 
wenzelm@67918:   \<^medskip>
wenzelm@67918:   Operation \<^verbatim>\<open>-l\<close> lists all active server processes with their connection
wenzelm@67918:   details.
wenzelm@67918: 
wenzelm@67918:   \<^medskip>
wenzelm@67918:   Operation \<^verbatim>\<open>-x\<close> exits the specified server process by sending it a
wenzelm@67918:   \<^verbatim>\<open>shutdown\<close> command.
wenzelm@67904: \<close>
wenzelm@67904: 
wenzelm@67904: 
wenzelm@67904: subsection \<open>Client \label{sec:tool-client}\<close>
wenzelm@67904: 
wenzelm@67904: text \<open>
wenzelm@67918:   The @{tool_def client} tool provides console interaction for Isabelle
wenzelm@67918:   servers:
wenzelm@67904:   @{verbatim [display]
wenzelm@67904: \<open>Usage: isabelle client [OPTIONS]
wenzelm@67904: 
wenzelm@67904:   Options are:
wenzelm@67904:     -n NAME      explicit server name
wenzelm@67904:     -p PORT      explicit server port
wenzelm@67904: 
wenzelm@67904:   Console interaction for Isabelle server (with line-editor).\<close>}
wenzelm@67904: 
wenzelm@67918:   This is a wrapper to \<^verbatim>\<open>isabelle server -s -c\<close> for interactive
wenzelm@67918:   experimentation, which uses @{setting ISABELLE_LINE_EDITOR} if available.
wenzelm@67904:   The server name is sufficient for identification, as the client can
wenzelm@67904:   determine the connection details from the local database of active servers.
wenzelm@67904: 
wenzelm@67904:   \<^medskip>
wenzelm@67904:   Option \<^verbatim>\<open>-n\<close> specifies an explicit server name as in @{tool server}.
wenzelm@67904: 
wenzelm@67904:   \<^medskip>
wenzelm@67904:   Option \<^verbatim>\<open>-p\<close> specifies an explicit server port as in @{tool server}.
wenzelm@67904: \<close>
wenzelm@67904: 
wenzelm@67904: 
wenzelm@67904: subsection \<open>Examples\<close>
wenzelm@67904: 
wenzelm@67904: text \<open>
wenzelm@67904:   Ensure that a particular server instance is running in the background:
wenzelm@67904:   @{verbatim [display] \<open>isabelle server -n test &\<close>}
wenzelm@67904: 
wenzelm@67918:   The first line of output presents the connection details:\<^footnote>\<open>This information
wenzelm@67918:   may be used in other TCP clients, without access to Isabelle/Scala and the
wenzelm@67918:   underlying database of running servers.\<close>
wenzelm@67904:   @{verbatim [display] \<open>server "test" = 127.0.0.1:4711 (password "XYZ")\<close>}
wenzelm@67904: 
wenzelm@67904:   \<^medskip>
wenzelm@67904:   List available server processes:
wenzelm@67904:   @{verbatim [display] \<open>isabelle server -l\<close>}
wenzelm@67904: 
wenzelm@67904:   \<^medskip>
wenzelm@67904:   Connect the command-line client to the above test server:
wenzelm@67904:   @{verbatim [display] \<open>isabelle client -n test\<close>}
wenzelm@67904: 
wenzelm@67904:   Interaction now works on a line-by-line basis, with commands like \<^verbatim>\<open>help\<close> or
wenzelm@67904:   \<^verbatim>\<open>echo\<close>. For example, some JSON values may be echoed like this:
wenzelm@67904: 
wenzelm@67904:   @{verbatim [display]
wenzelm@67904: \<open>echo 42
wenzelm@67904: echo [1, 2, 3]
wenzelm@67904: echo {"a": "text", "b": true, "c": 42}\<close>}
wenzelm@67904: 
wenzelm@67904:   Closing the connection (via CTRL-D) leaves the server running: it is
wenzelm@67904:   possible to reconnect again, and have multiple connections at the same time.
wenzelm@67904: 
wenzelm@67904:   \<^medskip>
wenzelm@67918:   Exit the named server on the command-line:
wenzelm@67904:   @{verbatim [display] \<open>isabelle server -n test -x\<close>}
wenzelm@67904: \<close>
wenzelm@67904: 
wenzelm@67904: 
wenzelm@67904: section \<open>Protocol messages \label{sec:server-protocol}\<close>
wenzelm@67904: 
wenzelm@67904: text \<open>
wenzelm@67904:   The Isabelle server listens on a regular TCP socket, using a line-oriented
wenzelm@67918:   protocol of structured messages. Input \<^emph>\<open>commands\<close> and output \<^emph>\<open>results\<close>
wenzelm@67904:   (via \<^verbatim>\<open>OK\<close> or \<^verbatim>\<open>ERROR\<close>) are strictly alternating on the toplevel, but
wenzelm@67904:   commands may also return a \<^emph>\<open>task\<close> identifier to indicate an ongoing
wenzelm@67904:   asynchronous process that is joined later (via \<^verbatim>\<open>FINISHED\<close> or \<^verbatim>\<open>FAILED\<close>).
wenzelm@67904:   Asynchronous \<^verbatim>\<open>NOTE\<close> messages may occur at any time: they are independent of
wenzelm@67918:   the main command-result protocol.
wenzelm@67904: 
wenzelm@67904:   For example, the synchronous \<^verbatim>\<open>echo\<close> command immediately returns its
wenzelm@67904:   argument as \<^verbatim>\<open>OK\<close> result. In contrast, the asynchronous \<^verbatim>\<open>session_build\<close>
wenzelm@67904:   command returns \<^verbatim>\<open>OK {"task":\<close>\<open>id\<close>\<^verbatim>\<open>}\<close> and continues in the background. It
wenzelm@67904:   will eventually produce \<^verbatim>\<open>FINISHED {"task":\<close>\<open>id\<close>\<^verbatim>\<open>,\<close>\<open>\<dots>\<close>\<^verbatim>\<open>}\<close> or
wenzelm@67904:   \<^verbatim>\<open>FAILED {"task":\<close>\<open>id\<close>\<^verbatim>\<open>,\<close>\<open>\<dots>\<close>\<^verbatim>\<open>}\<close> with the final result. Intermediately, it
wenzelm@67904:   may emit asynchronous messages of the form \<^verbatim>\<open>NOTE {"task":\<close>\<open>id\<close>\<^verbatim>\<open>,\<close>\<open>\<dots>\<close>\<^verbatim>\<open>}\<close>
wenzelm@67904:   to inform about its progress. Due to the explicit task identifier, the
wenzelm@67904:   client can show these messages in the proper context, e.g.\ a GUI window for
wenzelm@67918:   this particular session build job.
wenzelm@67904: 
wenzelm@67904:   \medskip Subsequently, the protocol message formats are described in further
wenzelm@67904:   detail.
wenzelm@67904: \<close>
wenzelm@67904: 
wenzelm@67904: 
wenzelm@67904: subsection \<open>Byte messages\<close>
wenzelm@67904: 
wenzelm@67904: text \<open>
wenzelm@67904:   The client-server connection is a raw byte-channel for bidirectional
wenzelm@67904:   communication, but the Isabelle server always works with messages of a
wenzelm@67904:   particular length. Messages are written as a single chunk that is flushed
wenzelm@67904:   immediately.
wenzelm@67904: 
wenzelm@67918:   Message boundaries are determined as follows:
wenzelm@67904: 
wenzelm@67904:     \<^item> A \<^emph>\<open>short message\<close> consists of a single line: it is a sequence of
wenzelm@67904:     arbitrary bytes excluding CR (13) and LF (10), and terminated by CR-LF or
wenzelm@67904:     just LF.
wenzelm@67904: 
wenzelm@67918:     \<^item> A \<^emph>\<open>long message\<close> starts with a single that consists only of decimal
wenzelm@67918:     digits: these are interpreted as length of the subsequent block of
wenzelm@67918:     arbitrary bytes. A final line-terminator (as above) may be included here,
wenzelm@67904:     but is not required.
wenzelm@67904: 
wenzelm@67904:   Messages in JSON format (see below) always fit on a single line, due to
wenzelm@67904:   escaping of newline characters within string literals. This is convenient
wenzelm@67904:   for interactive experimentation, but it can impact performance for very long
wenzelm@67904:   messages. If the message byte-length is given on the preceding line, the
wenzelm@67918:   server can read the message more efficiently as a single block.
wenzelm@67904: \<close>
wenzelm@67904: 
wenzelm@67904: 
wenzelm@67904: subsection \<open>Text messages\<close>
wenzelm@67904: 
wenzelm@67904: text \<open>
wenzelm@67904:   Messages are read and written as byte streams (with byte lengths), but the
wenzelm@67904:   content is always interpreted as plain text in terms of the UTF-8
wenzelm@67904:   encoding.\<^footnote>\<open>See also the ``UTF-8 Everywhere Manifesto''
wenzelm@67904:   \<^url>\<open>http://utf8everywhere.org\<close>.\<close>
wenzelm@67904: 
wenzelm@67904:   Note that line-endings and other formatting characters are invariant wrt.
wenzelm@67904:   UTF-8 representation of text: thus implementations are free to determine the
wenzelm@67904:   overall message structure before or after applying the text encoding.
wenzelm@67904: \<close>
wenzelm@67904: 
wenzelm@67904: 
wenzelm@67917: subsection \<open>Input and output messages \label{sec:input-output-messages}\<close>
wenzelm@67904: 
wenzelm@67904: text \<open>
wenzelm@67904:   Server input and output messages have a uniform format as follows:
wenzelm@67904: 
wenzelm@67904:     \<^item> \<open>name argument\<close> such that:
wenzelm@67904: 
wenzelm@67904:     \<^item> \<open>name\<close> is the longest prefix consisting of ASCII letters, digits,
wenzelm@67918:     ``\<^verbatim>\<open>_\<close>'', ``\<^verbatim>\<open>.\<close>'',
wenzelm@67904: 
wenzelm@67904:     \<^item> the separator between \<open>name\<close> and \<open>argument\<close> is the longest possible
wenzelm@67904:     sequence of ASCII blanks (it could be empty, e.g.\ when the argument
wenzelm@67904:     starts with a quote or bracket),
wenzelm@67904: 
wenzelm@67918:     \<^item> \<open>argument\<close> is the rest of the message without line terminator.
wenzelm@67904: 
wenzelm@67904:   \<^medskip>
wenzelm@67904:   Input messages are sent from the client to the server. Here the \<open>name\<close>
wenzelm@67904:   specifies a \<^emph>\<open>server command\<close>: the list of known commands may be
wenzelm@67904:   retrieved via the \<^verbatim>\<open>help\<close> command.
wenzelm@67904: 
wenzelm@67904:   \<^medskip>
wenzelm@67904:   Output messages are sent from the server to the client. Here the \<open>name\<close>
wenzelm@67904:   specifies the \<^emph>\<open>server reply\<close>, which always has a specific meaning as
wenzelm@67904:   follows:
wenzelm@67904: 
wenzelm@67904:     \<^item> synchronous results: \<^verbatim>\<open>OK\<close> or \<^verbatim>\<open>ERROR\<close>
wenzelm@67904:     \<^item> asynchronous results: \<^verbatim>\<open>FINISHED\<close> or \<^verbatim>\<open>FAILED\<close>
wenzelm@67904:     \<^item> intermediate notifications: \<^verbatim>\<open>NOTE\<close>
wenzelm@67904: 
wenzelm@67904:   \<^medskip>
wenzelm@67904:   The \<open>argument\<close> format is uniform for both input and output messages:
wenzelm@67904: 
wenzelm@67904:     \<^item> empty argument (Scala type \<^verbatim>\<open>Unit\<close>)
wenzelm@67904:     \<^item> XML element in YXML notation (Scala type \<^verbatim>\<open>XML.Elem\<close>)
wenzelm@67904:     \<^item> JSON value (Scala type \<^verbatim>\<open>JSON.T\<close>)
wenzelm@67904: 
wenzelm@67904:   JSON values may consist of objects (records), arrays (lists), strings,
wenzelm@67904:   numbers, bools, null.\<^footnote>\<open>See also the official specification
wenzelm@67904:   \<^url>\<open>https://www.json.org\<close> and unofficial explorations ``Parsing JSON is a
wenzelm@67904:   Minefield'' \<^url>\<open>http://seriot.ch/parsing_json.php\<close>.\<close> Since JSON requires
wenzelm@67904:   explicit quotes and backslash-escapes to represent arbitrary text, the YXML
wenzelm@67904:   notation for XML trees (\secref{sec:yxml-vs-xml}) works better
wenzelm@67904:   for large messages with a lot of PIDE markup.
wenzelm@67904: 
wenzelm@67904:   Nonetheless, the most common commands use JSON by default: big chunks of
wenzelm@67904:   text (theory sources etc.) are taken from the underlying file-system and
wenzelm@67904:   results are pre-formatted for plain-text output, without PIDE markup
wenzelm@67904:   information. This is a concession to simplicity: the server imitates the
wenzelm@67904:   appearance of command-line tools on top of the Isabelle/PIDE infrastructure.
wenzelm@67904: \<close>
wenzelm@67904: 
wenzelm@67904: 
wenzelm@67904: subsection \<open>Initial password exchange\<close>
wenzelm@67904: 
wenzelm@67904: text \<open>
wenzelm@67904:   Whenever a new client opens the server socket, the initial message needs to
wenzelm@67904:   be its unique password. The server replies either with \<^verbatim>\<open>OK\<close> (and some
wenzelm@67904:   information about the Isabelle version) or by silent disconnection of what
wenzelm@67918:   is considered an illegal connection attempt. Note that @{tool client}
wenzelm@67918:   already presents the correct password internally.
wenzelm@67904: 
wenzelm@67904:   Server passwords are created as Universally Unique Identifier (UUID) in
wenzelm@67904:   Isabelle/Scala and stored in a per-user database, with restricted
wenzelm@67904:   file-system access only for the current user. The Isabelle/Scala server
wenzelm@67904:   implementation is careful to expose the password only on private output
wenzelm@67904:   channels, and not on a process command-line (which is accessible to other
wenzelm@67904:   users, e.g.\ via the \<^verbatim>\<open>ps\<close> command).
wenzelm@67904: \<close>
wenzelm@67904: 
wenzelm@67904: 
wenzelm@67904: subsection \<open>Synchronous commands\<close>
wenzelm@67904: 
wenzelm@67904: text \<open>
wenzelm@67904:   A \<^emph>\<open>synchronous command\<close> corresponds to regular function application in
wenzelm@67904:   Isabelle/Scala, with single argument and result (regular or error). Both the
wenzelm@67904:   argument and the result may consist of type \<^verbatim>\<open>Unit\<close>, \<^verbatim>\<open>XML.Elem\<close>, \<^verbatim>\<open>JSON.T\<close>.
wenzelm@67904:   An error result typically consists of a JSON object with error message and
wenzelm@67918:   potentially further result fields (this resembles exceptions in Scala).
wenzelm@67904: 
wenzelm@67904:   These are the protocol exchanges for both cases of command execution:
wenzelm@67904:   \begin{center}
wenzelm@67904:   \begin{tabular}{rl}
wenzelm@67904:   \<^bold>\<open>input:\<close> & \<open>command argument\<close> \\
wenzelm@67904:   (a) regular \<^bold>\<open>output:\<close> & \<^verbatim>\<open>OK\<close> \<open>result\<close> \\
wenzelm@67904:   (b) error \<^bold>\<open>output:\<close> & \<^verbatim>\<open>ERROR\<close> \<open>result\<close> \\
wenzelm@67904:   \end{tabular}
wenzelm@67904:   \end{center}
wenzelm@67904: \<close>
wenzelm@67904: 
wenzelm@67904: 
wenzelm@67904: subsection \<open>Asynchronous commands\<close>
wenzelm@67904: 
wenzelm@67904: text \<open>
wenzelm@67904:   An \<^emph>\<open>asynchronous command\<close> corresponds to an ongoing process that finishes
wenzelm@67918:   or fails eventually, while emitting arbitrary notifications in between.
wenzelm@67918:   Formally, it starts as synchronous command with immediate result \<^verbatim>\<open>OK\<close>
wenzelm@67918:   giving the \<^verbatim>\<open>task\<close> identifier, or an immediate \<^verbatim>\<open>ERROR\<close> that indicates bad
wenzelm@67918:   command syntax. For a running task, the termination is indicated later by
wenzelm@67918:   \<^verbatim>\<open>FINISHED\<close> or \<^verbatim>\<open>FAILED\<close>, together with its ultimate result value.
wenzelm@67904: 
wenzelm@67904:   These are the protocol exchanges for various cases of command task
wenzelm@67904:   execution:
wenzelm@67904: 
wenzelm@67904:   \begin{center}
wenzelm@67904:   \begin{tabular}{rl}
wenzelm@67904:   \<^bold>\<open>input:\<close> & \<open>command argument\<close> \\
wenzelm@67904:   immediate \<^bold>\<open>output:\<close> & \<^verbatim>\<open>OK {"task":\<close>\<open>id\<close>\<^verbatim>\<open>}\<close> \\
wenzelm@67904:   intermediate \<^bold>\<open>output:\<close> & \<^verbatim>\<open>NOTE {"task":\<close>\<open>id\<close>\<^verbatim>\<open>,\<close>\<open>\<dots>\<close>\<^verbatim>\<open>}\<close> \\
wenzelm@67904:   (a) regular \<^bold>\<open>output:\<close> & \<^verbatim>\<open>FINISHED {"task":\<close>\<open>id\<close>\<^verbatim>\<open>,\<close>\<open>\<dots>\<close>\<^verbatim>\<open>}\<close> \\
wenzelm@67904:   (b) error \<^bold>\<open>output:\<close> & \<^verbatim>\<open>FAILED {"task":\<close>\<open>id\<close>\<^verbatim>\<open>,\<close>\<open>\<dots>\<close>\<^verbatim>\<open>}\<close> \\[3ex]
wenzelm@67904:   \<^bold>\<open>input:\<close> & \<open>command argument\<close> \\
wenzelm@67904:   immediate \<^bold>\<open>output:\<close> & \<^verbatim>\<open>ERROR\<close>~\<open>\<dots>\<close> \\
wenzelm@67904:   \end{tabular}
wenzelm@67904:   \end{center}
wenzelm@67904: 
wenzelm@67904:   All asynchronous messages are decorated with the task identifier that was
wenzelm@67918:   revealed in the immediate (synchronous) result. Thus the client can
wenzelm@67918:   invoke further asynchronous commands and still dispatch the resulting stream of
wenzelm@67904:   asynchronous messages properly.
wenzelm@67904: 
wenzelm@67904:   The synchronous command \<^verbatim>\<open>cancel\<close>~\<open>id\<close> tells the specified task to terminate
wenzelm@67917:   prematurely: usually causing a \<^verbatim>\<open>FAILED\<close> result, but this is not guaranteed:
wenzelm@67918:   the cancel event may come too late or the running process may just ignore
wenzelm@67918:   it.
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67918: section \<open>Types for JSON values \label{sec:json-types}\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67917:   In order to specify concrete JSON types for command arguments and result
wenzelm@67917:   messages, the following type definition language shall be used:
wenzelm@67917: 
wenzelm@67917:   \<^rail>\<open>
wenzelm@67917:     @{syntax type_def}: @'type' @{syntax name} '=' @{syntax type}
wenzelm@67917:     ;
wenzelm@67917:     @{syntax type}: @{syntax name} | @{syntax value} | 'any' | 'null' |
wenzelm@67917:       'bool' | 'int' | 'long' | 'double' | 'string' | '[' @{syntax type} ']' |
wenzelm@67917:       '{' (@{syntax field_type} ',' *) '}' |
wenzelm@67917:       @{syntax type} '\<oplus>' @{syntax type} |
wenzelm@67918:       @{syntax type} '|' @{syntax type} |
wenzelm@67918:       '(' @{syntax type} ')'
wenzelm@67917:     ;
wenzelm@67917:     @{syntax field_type}: @{syntax name} ('?'?) ':' @{syntax type}
wenzelm@67917:   \<close>
wenzelm@67917: 
wenzelm@67918:   This is a simplified variation of TypeScript
wenzelm@67918:   interfaces.\<^footnote>\<open>\<^url>\<open>https://www.typescriptlang.org/docs/handbook/interfaces.html\<close>\<close>
wenzelm@67918:   The meaning of these types is specified wrt. the Isabelle/Scala
wenzelm@67918:   implementation as follows.
wenzelm@67917: 
wenzelm@67917:   \<^item> A \<open>name\<close> refers to a type defined elsewhere. The environment of type
wenzelm@67917:   definitions is given informally: put into proper foundational order, it
wenzelm@67918:   needs to specify a strongly normalizing system of syntactic abbreviations;
wenzelm@67918:   type definitions may not be recursive.
wenzelm@67917: 
wenzelm@67917:   \<^item> A \<open>value\<close> in JSON notation represents the singleton type of the given
wenzelm@67918:   item. For example, the string \<^verbatim>\<open>"error"\<close> can be used as type for a slot that
wenzelm@67918:   is guaranteed to contain that constant.
wenzelm@67917: 
wenzelm@67917:   \<^item> Type \<open>any\<close> is the super type of all other types: it is an untyped slot in
wenzelm@67917:   the specification and corresponds to \<^verbatim>\<open>Any\<close> or \<^verbatim>\<open>JSON.T\<close> in Isabelle/Scala.
wenzelm@67917: 
wenzelm@67917:   \<^item> Type \<open>null\<close> is the type of the improper value \<open>null\<close>; it corresponds to
wenzelm@67917:   type \<^verbatim>\<open>Null\<close> in Scala and is normally not used in Isabelle/Scala.\<^footnote>\<open>See also
wenzelm@67917:   ``Null References: The Billion Dollar Mistake'' by Tony Hoare
wenzelm@67917:   \<^url>\<open>https://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare\<close>.\<close>
wenzelm@67917: 
wenzelm@67918:   \<^item> Type \<open>bool\<close> is the type of the truth values \<^verbatim>\<open>true\<close> and \<^verbatim>\<open>false\<close>; it
wenzelm@67917:   corresponds to \<^verbatim>\<open>Boolean\<close> in Scala.
wenzelm@67917: 
wenzelm@67917:   \<^item> Types \<open>int\<close>, \<open>long\<close>, \<open>double\<close> are specific versions of the generic
wenzelm@67917:   \<open>number\<close> type, corresponding to \<^verbatim>\<open>Int\<close>, \<^verbatim>\<open>Long\<close>, \<^verbatim>\<open>Double\<close> in Scala, but
wenzelm@67917:   \<^verbatim>\<open>Long\<close> is limited to 53 bit precision.\<^footnote>\<open>Implementations of JSON typically
wenzelm@67917:   standardize \<open>number\<close> to \<^verbatim>\<open>Double\<close>, which can absorb \<^verbatim>\<open>Int\<close> faithfully, but
wenzelm@67917:   not all of \<^verbatim>\<open>Long\<close>.\<close>
wenzelm@67917: 
wenzelm@67917:   \<^item> Type \<open>string\<close> represents Unicode text; it corresponds to type \<^verbatim>\<open>String\<close> in
wenzelm@67917:   Scala.
wenzelm@67917: 
wenzelm@67917:   \<^item> Type \<open>[t]\<close> is the array (or list) type over \<open>t\<close>; it corresponds to
wenzelm@67917:   \<^verbatim>\<open>List[t]\<close> in Scala. The list type is co-variant as usual (i.e.\ monotonic
wenzelm@67918:   wrt. the subtype relation).
wenzelm@67917: 
wenzelm@67917:   \<^item> Object types describe the possible content of JSON records, with field
wenzelm@67917:   names and types. A question mark after a field name means that it is
wenzelm@67918:   optional. In Scala this could refer to an explicit type \<^verbatim>\<open>Option[t]\<close>, e.g.\
wenzelm@67918:   \<open>{a: int, b?: string}\<close> corresponding to a Scala case class with arguments
wenzelm@67918:   \<^verbatim>\<open>a: Int\<close>, \<^verbatim>\<open>b: Option[String]\<close>.
wenzelm@67917: 
wenzelm@67917:   Alternatively, optional fields can have a default value. If nothing else is
wenzelm@67918:   specified, a standard ``empty value'' is used for each type, i.e.\ \<^verbatim>\<open>0\<close> for
wenzelm@67918:   the number types, \<^verbatim>\<open>false\<close> for \<open>bool\<close>, or the empty string, array, object
wenzelm@67918:   etc.
wenzelm@67917: 
wenzelm@67917:   Object types are \<^emph>\<open>permissive\<close> in the sense that only the specified field
wenzelm@67917:   names need to conform to the given types, but unspecified fields may be
wenzelm@67917:   present as well.
wenzelm@67917: 
wenzelm@67917:   \<^item> The type expression \<open>t\<^sub>1 \<oplus> t\<^sub>2\<close> only works for two object types with
wenzelm@67917:   disjoint field names: it is the concatenation of the respective @{syntax
wenzelm@67917:   field_type} specifications taken together. For example: \<open>{task: string} \<oplus>
wenzelm@67917:   {ok: bool}\<close> is the equivalent to \<open>{task: string, ok: bool}\<close>.
wenzelm@67917: 
wenzelm@67917:   \<^item> The type expression \<open>t\<^sub>1 | t\<^sub>2\<close> is the disjoint union of two types, either
wenzelm@67917:   one of the two cases may occur.
wenzelm@67917: 
wenzelm@67918:   \<^item> Parentheses \<open>(t)\<close> merely group type expressions syntactically.
wenzelm@67918: 
wenzelm@67917: 
wenzelm@67917:   These types correspond to JSON values in an obvious manner, which is not
wenzelm@67917:   further described here. For example, the JSON array \<^verbatim>\<open>[1, 2, 3]\<close> conforms to
wenzelm@67917:   types \<open>[int]\<close>, \<open>[long]\<close>, \<open>[double]\<close>, \<open>[any]\<close>, \<open>any\<close>.
wenzelm@67917: 
wenzelm@67917:   Note that JSON objects require field names to be quoted, but the type
wenzelm@67918:   language omits quotes for clarity. Thus the object \<^verbatim>\<open>{"a": 42, "b": "xyz"}\<close>
wenzelm@67918:   conforms to the type \<open>{a: int, b: string}\<close>, for example.
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67918:   The absence of an argument or result is represented by the Scala type
wenzelm@67918:   \<^verbatim>\<open>Unit\<close>: it is written as empty text in the message \<open>argument\<close>
wenzelm@67917:   (\secref{sec:input-output-messages}). This is not part of the JSON language.
wenzelm@67917: 
wenzelm@67917:   Server replies have name tags like \<^verbatim>\<open>OK\<close>, \<^verbatim>\<open>ERROR\<close>: these are used literally
wenzelm@67917:   together with type specifications to indicate the particular name with the
wenzelm@67917:   type of its argument, e.g.\ \<^verbatim>\<open>OK\<close>~\<open>[string]\<close> for a regular result that is a
wenzelm@67917:   list (JSON array) of strings.
wenzelm@67917: 
wenzelm@67917:   \<^bigskip>
wenzelm@67918:   Here are some common type definitions, for use in particular specifications
wenzelm@67918:   of command arguments and results.
wenzelm@67917: 
wenzelm@67917:   \<^item> \<^bold>\<open>type\<close>~\<open>position = {line?: int, offset?: int, end_offset?: int, file?:
wenzelm@67918:   string, id?: long}\<close> describes a source position within Isabelle text. Only
wenzelm@67918:   the \<open>line\<close> and \<open>file\<close> fields make immediate sense to external programs.
wenzelm@67917:   Detailed \<open>offset\<close> and \<open>end_offset\<close> positions are counted according to
wenzelm@67917:   Isabelle symbols, see @{ML_type Symbol.symbol} in Isabelle/ML @{cite
wenzelm@67918:   "isabelle-implementation"}. The position \<open>id\<close> belongs to the representation
wenzelm@67918:   of command transactions in the Isabelle/PIDE protocol: it normally does not
wenzelm@67918:   occur in externalized positions.
wenzelm@67917: 
wenzelm@67917:   \<^item> \<^bold>\<open>type\<close>~\<open>message = {kind?: string, message: string, pos?: position}\<close> where
wenzelm@67917:   the \<open>kind\<close> provides some hint about the role and importance of the message.
wenzelm@67917:   The main message kinds are \<^verbatim>\<open>writeln\<close> (for regular output), \<^verbatim>\<open>warning\<close>, and
wenzelm@67917:   \<^verbatim>\<open>error\<close>. The table \<^verbatim>\<open>Markup.messages\<close> in Isabelle/Scala defines further
wenzelm@67917:   message kinds for more specific applications.
wenzelm@67917: 
wenzelm@67917:   \<^item> \<^bold>\<open>type\<close>~\<open>error_message = {kind:\<close>~\<^verbatim>\<open>"error"\<close>\<open>, message: string}\<close> refers to
wenzelm@67917:   error messages in particular. These occur routinely with \<^verbatim>\<open>ERROR\<close> or
wenzelm@67917:   \<^verbatim>\<open>FAILED\<close> replies, but also as initial command syntax errors (which are
wenzelm@67917:   omitted in the command specifications below).
wenzelm@67917: 
wenzelm@67917:   \<^item> \<^bold>\<open>type\<close>~\<open>theory_progress = {kind:\<close>~\<^verbatim>\<open>"writeln"\<close>\<open>, message: string, theory:
wenzelm@67917:   string, session: string}\<close> reports formal progress in loading theories (e.g.\
wenzelm@67917:   when building a session image). Apart from a regular output message, it also
wenzelm@67918:   reveals the formal theory name (e.g.\ \<^verbatim>\<open>"HOL.Nat"\<close>) and session name (e.g.\
wenzelm@67918:   \<^verbatim>\<open>"HOL"\<close>). Note that some rare theory names lack a proper session prefix,
wenzelm@67918:   e.g. theory \<^verbatim>\<open>"Main"\<close> in session \<^verbatim>\<open>"HOL"\<close>.
wenzelm@67917: 
wenzelm@67917:   \<^item> \<^bold>\<open>type\<close>~\<open>timing = {elapsed: double, cpu: double, gc: double}\<close> refers to
wenzelm@67917:   common Isabelle timing information in seconds, usually with a precision of
wenzelm@67917:   three digits after the point (whole milliseconds).
wenzelm@67917: 
wenzelm@67917:   \<^item> \<^bold>\<open>type\<close>~\<open>uuid = string\<close> refers to a Universally Unique Identifier (UUID)
wenzelm@67917:   as plain text.\<^footnote>\<open>See \<^url>\<open>https://www.ietf.org/rfc/rfc4122.txt\<close> and
wenzelm@67917:   \<^url>\<open>https://docs.oracle.com/javase/8/docs/api/java/util/UUID.html\<close>.\<close> Such
wenzelm@67917:   identifiers are created as private random numbers of the server and only
wenzelm@67917:   revealed to the client that creates a certain resource (e.g.\ task or
wenzelm@67917:   session). A client may disclose this information for use in a different
wenzelm@67918:   client connection: this allows to share sessions between multiple
wenzelm@67917:   connections.
wenzelm@67917: 
wenzelm@67917:   Client commands need to provide syntactically wellformed UUIDs: this is
wenzelm@67917:   trivial to achieve by using only identifiers that have been produced by the
wenzelm@67917:   server beforehand.
wenzelm@67917: 
wenzelm@67917:   \<^item> \<^bold>\<open>type\<close>~\<open>task = {task: uuid}\<close> identifies a newly created asynchronous task
wenzelm@67917:   and thus allows the client to control it by the \<^verbatim>\<open>cancel\<close> command. The same
wenzelm@67918:   task identification is included in all messages produced by this task.
wenzelm@67918: 
wenzelm@67918:   \<^item> \<^bold>\<open>type\<close> \<open>node_status = {ok: bool, total: int, unprocessed: int, running:
wenzelm@67918:   int, warned: int, failed: int, finished: int, consolidated: bool}\<close>
wenzelm@67918:   represents a formal theory node status of the PIDE document model. Fields
wenzelm@67918:   \<open>total\<close>, \<open>unprocessed\<close>, \<open>running\<close>, \<open>warned\<close>, \<open>failed\<close>, \<open>finished\<close> account
wenzelm@67918:   for individual commands within a theory node; \<open>ok\<close> is an abstraction for
wenzelm@67918:   \<open>failed = 0\<close>. The \<open>consolidated\<close> flag indicates whether the outermost theory
wenzelm@67918:   command structure has finished (or failed) and the final \<^theory_text>\<open>end\<close> command has
wenzelm@67918:   been checked.
wenzelm@67918: \<close>
wenzelm@67918: 
wenzelm@67918: 
wenzelm@67918: section \<open>Server commands and results\<close>
wenzelm@67918: 
wenzelm@67918: text \<open>
wenzelm@67918:   Here follows an overview of particular Isabelle server commands with their
wenzelm@67918:   results, which are usually represented as JSON values with types according
wenzelm@67918:   to \secref{sec:json-types}. The general format of input and output messages
wenzelm@67918:   is described in \secref{sec:input-output-messages}. The relevant
wenzelm@67918:   Isabelle/Scala source files are:
wenzelm@67918: 
wenzelm@67918:   \<^medskip>
wenzelm@67918:   \begin{tabular}{l}
wenzelm@67918:   \<^file>\<open>$ISABELLE_HOME/src/Pure/Tools/server_commands.scala\<close> \\
wenzelm@67918:   \<^file>\<open>$ISABELLE_HOME/src/Pure/Tools/server.scala\<close> \\
wenzelm@67918:   \<^file>\<open>$ISABELLE_HOME/src/Pure/General/json.scala\<close> \\
wenzelm@67918:   \end{tabular}
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsection \<open>Command \<^verbatim>\<open>help\<close>\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67917:   \begin{tabular}{ll}
wenzelm@67917:   regular result: & \<^verbatim>\<open>OK\<close> \<open>[string]\<close> \\
wenzelm@67917:   \end{tabular}
wenzelm@67917:   \<^medskip>
wenzelm@67917: 
wenzelm@67918:   The \<^verbatim>\<open>help\<close> command has no argument and returns the list of server command
wenzelm@67918:   names. This is occasionally useful for interactive experimentation (see also
wenzelm@67918:   @{tool client} in \secref{sec:tool-client}).
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsection \<open>Command \<^verbatim>\<open>echo\<close>\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67917:   \begin{tabular}{ll}
wenzelm@67917:   argument: & \<open>any\<close> \\
wenzelm@67917:   regular result: & \<^verbatim>\<open>OK\<close> \<open>any\<close> \\
wenzelm@67917:   \end{tabular}
wenzelm@67917:   \<^medskip>
wenzelm@67917: 
wenzelm@67917:   The \<^verbatim>\<open>echo\<close> command is the identity function: it returns its argument as
wenzelm@67917:   regular result. This is occasionally useful for testing and interactive
wenzelm@67917:   experimentation (see also @{tool client} in \secref{sec:tool-client}).
wenzelm@67917: 
wenzelm@67918:   The Scala type of \<^verbatim>\<open>echo\<close> is actually more general than given above:
wenzelm@67918:   \<^verbatim>\<open>Unit\<close>, \<^verbatim>\<open>XML.Elem\<close>, \<^verbatim>\<open>JSON.T\<close> work uniformly. Note that \<^verbatim>\<open>XML.Elem\<close> might
wenzelm@67917:   be difficult to type on the console in its YXML syntax
wenzelm@67917:   (\secref{sec:yxml-vs-xml}).
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsection \<open>Command \<^verbatim>\<open>shutdown\<close>\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67917:   \begin{tabular}{ll}
wenzelm@67917:   regular result: & \<^verbatim>\<open>OK\<close> \\
wenzelm@67917:   \end{tabular}
wenzelm@67917:   \<^medskip>
wenzelm@67917: 
wenzelm@67917:   The \<^verbatim>\<open>shutdown\<close> command has no argument and result value. It forces a
wenzelm@67917:   shutdown of the connected server process, stopping all open sessions and
wenzelm@67917:   closing the server socket. This may disrupt pending commands on other
wenzelm@67917:   connections!
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67918:   The command-line invocation \<^verbatim>\<open>isabelle server -x\<close> opens a server connection
wenzelm@67918:   and issues a \<^verbatim>\<open>shutdown\<close> command (see also \secref{sec:tool-server}).
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsection \<open>Command \<^verbatim>\<open>cancel\<close>\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67917:   \begin{tabular}{ll}
wenzelm@67917:   argument: & \<open>uuid\<close> \\
wenzelm@67917:   regular result: & \<^verbatim>\<open>OK\<close> \\
wenzelm@67917:   \end{tabular}
wenzelm@67917:   \<^medskip>
wenzelm@67917: 
wenzelm@67918:   The command invocation \<^verbatim>\<open>cancel "\<close>\<open>uuid\<close>\<^verbatim>\<open>"\<close> attempts to cancel the
wenzelm@67918:   specified task.
wenzelm@67917: 
wenzelm@67917:   Cancellation is merely a hint that the client prefers an ongoing process to
wenzelm@67917:   be stopped. The command always succeeds formally, but it may get ignored by
wenzelm@67918:   a task that is still running; it might also refer to a non-existing or
wenzelm@67918:   no-longer existing task (without producing an error).
wenzelm@67917: 
wenzelm@67917:   Successful cancellation typically leads to an asynchronous failure of type
wenzelm@67918:   \<^verbatim>\<open>FAILED {\<close>\<open>task: uuid, message:\<close>~\<^verbatim>\<open>"Interrupt"}\<close>. A different message is
wenzelm@67918:   also possible, depending how the task handles the event.
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsection \<open>Command \<^verbatim>\<open>session_build\<close> \label{sec:command-session-build}\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67917:   \begin{tabular}{lll}
wenzelm@67917:   argument: & \<open>session_build_args\<close> \\
wenzelm@67917:   immediate result: & \<^verbatim>\<open>OK\<close> \<open>task\<close> \\
wenzelm@67917:   notifications: & \<^verbatim>\<open>NOTE\<close> \<open>task \<oplus> (theory_progress | message)\<close> \\
wenzelm@67917:   regular result: & \<^verbatim>\<open>FINISHED\<close> \<open>task \<oplus> session_build_results\<close> \\
wenzelm@67917:   error result: & \<^verbatim>\<open>FAILED\<close> \<open>task \<oplus> error_message \<oplus> session_build_results\<close> \\[2ex]
wenzelm@67917:   \end{tabular}
wenzelm@67917: 
wenzelm@67917:   \begin{tabular}{lll}
wenzelm@67917:   \<^bold>\<open>type\<close> \<open>session_build_args =\<close> \\
wenzelm@67917:   \quad\<open>{session: string,\<close> \\
wenzelm@67917:   \quad~~\<open>preferences?: string,\<close> & \<^bold>\<open>default:\<close> server preferences \\
wenzelm@67917:   \quad~~\<open>options?: [string],\<close> \\
wenzelm@67917:   \quad~~\<open>dirs?: [string],\<close> \\
wenzelm@67917:   \quad~~\<open>ancestor_session?: string,\<close> \\
wenzelm@67917:   \quad~~\<open>all_known?: bool,\<close> \\
wenzelm@67917:   \quad~~\<open>focus_session?: bool,\<close> \\
wenzelm@67917:   \quad~~\<open>required_session?: bool,\<close> \\
wenzelm@67917:   \quad~~\<open>system_mode?: bool,\<close> \\
wenzelm@67917:   \quad~~\<open>verbose?: bool}\<close> \\[2ex]
wenzelm@67917: 
wenzelm@67917:   \<^bold>\<open>type\<close> \<open>session_build_result =\<close> \\
wenzelm@67917:   \quad\<open>{session: string,\<close> \\
wenzelm@67917:   \quad~~\<open>ok: bool,\<close> \\
wenzelm@67917:   \quad~~\<open>return_code: int,\<close> \\
wenzelm@67917:   \quad~~\<open>timeout: bool,\<close> \\
wenzelm@67918:   \quad~~\<open>timing: timing}\<close> \\[2ex]
wenzelm@67918: 
wenzelm@67918:   \<^bold>\<open>type\<close> \<open>session_build_results =\<close> \\
wenzelm@67918:   \quad\<open>{ok: bool,\<close> \\
wenzelm@67918:   \quad~~\<open>return_code: int,\<close> \\
wenzelm@67918:   \quad~~\<open>sessions: [session_build_result]}\<close> \\
wenzelm@67917:   \end{tabular}
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67918:   The \<^verbatim>\<open>session_build\<close> command prepares a session image for interactive use of
wenzelm@67918:   theories. This is a limited version of command-line tool @{tool build}
wenzelm@67918:   (\secref{sec:tool-build}), with specific options to request a formal context
wenzelm@67918:   for an interactive PIDE session.
wenzelm@67917: 
wenzelm@67917:   The build process is asynchronous, with notifications that inform about the
wenzelm@67918:   progress of loaded theories. Some further informative messages are output as
wenzelm@67918:   well.
wenzelm@67917: 
wenzelm@67917:   Coordination of independent build processes is at the discretion of the
wenzelm@67918:   client (or end-user), just as for @{tool build} and @{tool jedit}. There is
wenzelm@67918:   no built-in coordination of conflicting builds with overlapping hierarchies
wenzelm@67918:   of session images. In the worst case, a session image produced by one task
wenzelm@67918:   may get overwritten by another task!
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsubsection \<open>Arguments\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67918:   The \<open>session\<close> field is mandatory. It specifies the target session name:
wenzelm@67918:   either directly (default) or indirectly (if \<open>required_session\<close> is \<^verbatim>\<open>true\<close>).
wenzelm@67918:   The build process will produce all required ancestor images for the
wenzelm@67918:   specified target.
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67917:   The environment of Isabelle system options is determined from \<open>preferences\<close>
wenzelm@67917:   that are augmented by \<open>options\<close>, which is a list individual updates of the
wenzelm@67917:   form the \<open>name\<close>\<^verbatim>\<open>=\<close>\<open>value\<close> or \<open>name\<close> (the latter abbreviates
wenzelm@67917:   \<open>name\<close>\<^verbatim>\<open>=true\<close>); see also command-line option \<^verbatim>\<open>-o\<close> for @{tool build}. The
wenzelm@67917:   preferences are loaded from the file
wenzelm@67917:   \<^path>\<open>$ISABELLE_HOME_USER/etc/preferences\<close> by default, but the client may
wenzelm@67917:   provide alternative contents for it (as text, not a file-name). This could
wenzelm@67917:   be relevant in situations where client and server run in different
wenzelm@67917:   operating-system contexts.
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67918:   The \<open>dirs\<close> field specifies additional directories for session ROOT and ROOTS
wenzelm@67918:   files (\secref{sec:session-root}). This augments the name space of available
wenzelm@67918:   sessions; see also option \<^verbatim>\<open>-d\<close> in @{tool build}.
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67917:   The \<open>ancestor_session\<close> field specifies an alternative image as starting
wenzelm@67917:   point for the target image. The default is to use the parent session
wenzelm@67917:   according to the ROOT entry; see also option \<^verbatim>\<open>-A\<close> in @{tool jedit}. This
wenzelm@67917:   can lead to a more light-weight build process, by skipping intermediate
wenzelm@67917:   session images of the hierarchy that are not used later on.
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67918:   The \<open>all_known\<close> field set to \<^verbatim>\<open>true\<close> includes all sessions from reachable
wenzelm@67918:   ROOT entries into the name space of theories. This is relevant for proper
wenzelm@67918:   session-qualified names, instead of referring to theory file names. The
wenzelm@67918:   option enables the \<^verbatim>\<open>use_theories\<close> command
wenzelm@67917:   (\secref{sec:command-use-theories}) to refer to arbitrary theories from
wenzelm@67918:   other sessions, but considerable time is required to explore all accessible
wenzelm@67918:   session directories and theory dependencies.
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67918:   The \<open>focus_session\<close> field set to \<^verbatim>\<open>true\<close> focuses on the target session:
wenzelm@67917:   the accessible name space of theories is restricted to sessions that are
wenzelm@67918:   connected to it in the imports graph.
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67917:   The \<open>required_session\<close> field set to \<^verbatim>\<open>true\<close> indicates that the target image
wenzelm@67917:   should not be the \<open>session\<close>, but its parent (or ancestor according to
wenzelm@67917:   \<open>ancestor_session\<close>). Thus it prepares a session context where theories from
wenzelm@67917:   the \<open>session\<close> itself can be loaded.
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67918:   The \<open>system_mode\<close> field set to \<^verbatim>\<open>true\<close> stores resulting session images and
wenzelm@67917:   log files in @{path "$ISABELLE_HOME/heaps"} instead of the default location
wenzelm@67917:   @{setting ISABELLE_OUTPUT} (which is normally in @{setting
wenzelm@67918:   ISABELLE_HOME_USER}). See also option \<^verbatim>\<open>-s\<close> in @{tool build}.
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67918:   The \<open>verbose\<close> field set to \<^verbatim>\<open>true\<close> yields extra verbosity. The effect is
wenzelm@67917:   similar to option \<^verbatim>\<open>-v\<close> in @{tool build}.
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsubsection \<open>Intermediate output\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67917:   The asynchronous notifications of command \<^verbatim>\<open>session_build\<close> mainly serve as
wenzelm@67917:   progress indicator: the output resembles that of the session build window of
wenzelm@67917:   Isabelle/jEdit after startup @{cite "isabelle-jedit"}.
wenzelm@67917: 
wenzelm@67917:   For the client it is usually sufficient to print the messages in plain text,
wenzelm@67918:   but note that \<open>theory_progress\<close> also reveals formal \<open>theory\<close> and
wenzelm@67918:   \<open>session\<close> names directly.
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsubsection \<open>Results\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67918:   The overall \<open>session_build_results\<close> contain both a summary and an entry
wenzelm@67918:   \<open>session_build_result\<close> for each session in the build hierarchy. The result
wenzelm@67918:   is always provided, independently of overall success (\<^verbatim>\<open>FINISHED\<close> task) or
wenzelm@67918:   failure (\<^verbatim>\<open>FAILED\<close> task).
wenzelm@67917: 
wenzelm@67917:   The \<open>ok\<close> field tells abstractly, whether all required session builds came
wenzelm@67918:   out as \<open>ok\<close>, i.e.\ with zero \<open>return_code\<close>. A non-zero \<open>return_code\<close>
wenzelm@67918:   indicates an error according to usual POSIX conventions for process exit.
wenzelm@67918: 
wenzelm@67918:   The individual \<open>session_build_result\<close> entries provide extra fields:
wenzelm@67917: 
wenzelm@67918:   \<^item> \<open>timeout\<close> tells if the build process was aborted after running too long,
wenzelm@67918: 
wenzelm@67918:   \<^item> \<open>timing\<close> gives the overall process timing in the usual Isabelle format
wenzelm@67918:   with elapsed, CPU, GC time.
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsubsection \<open>Examples\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67917:   Build of a session image from the Isabelle distribution:
wenzelm@67917:   @{verbatim [display] \<open>session_build {"session": "HOL-Word"}\<close>}
wenzelm@67917: 
wenzelm@67917:   Build a session image from the Archive of Formal Proofs:
wenzelm@67917:   @{verbatim [display] \<open>session_build {"session": "Coinductive", "dirs": ["$AFP_BASE/thys"]}\<close>}
wenzelm@67917: 
wenzelm@67917:   Build of a session image of \<^verbatim>\<open>HOL-Number_Theory\<close> directly on top of \<^verbatim>\<open>HOL\<close>:
wenzelm@67917:   @{verbatim [display] \<open>session_build {"session": "HOL-Number_Theory", "ancestor_session": "HOL"}\<close>}
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsection \<open>Command \<^verbatim>\<open>session_start\<close> \label{sec:command-session-start}\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67917:   \begin{tabular}{lll}
wenzelm@67917:   argument: & \<open>session_build_args \<oplus> {print_mode?: [string]}\<close> \\
wenzelm@67917:   immediate result: & \<^verbatim>\<open>OK\<close> \<open>task\<close> \\
wenzelm@67917:   notifications: & \<^verbatim>\<open>NOTE\<close> \<open>task \<oplus> (theory_progress | message)\<close> \\
wenzelm@67917:   regular result: & \<^verbatim>\<open>FINISHED\<close> \<open>task \<oplus> session_start_result\<close> \\
wenzelm@67917:   error result: & \<^verbatim>\<open>FAILED\<close> \<open>task \<oplus> error_message\<close> \\[2ex]
wenzelm@67917:   \end{tabular}
wenzelm@67917: 
wenzelm@67917:   \begin{tabular}{l}
wenzelm@67917:   \<^bold>\<open>type\<close> \<open>session_start_result = {session: string, session_id: uuid}\<close>
wenzelm@67917:   \end{tabular}
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67917:   The \<^verbatim>\<open>session_start\<close> command starts a new Isabelle/PIDE session with
wenzelm@67917:   underlying Isabelle/ML process, based on a session image that it produces on
wenzelm@67918:   demand using \<^verbatim>\<open>session_build\<close>. Thus it accepts all \<open>session_build_args\<close> and
wenzelm@67918:   produces similar notifications, but the detailed \<open>session_build_results\<close> are
wenzelm@67918:   omitted.
wenzelm@67917: 
wenzelm@67917:   The session build and startup process is asynchronous: when the task is
wenzelm@67918:   finished, the session remains active for commands, until a \<^verbatim>\<open>session_stop\<close>
wenzelm@67918:   or \<^verbatim>\<open>shutdown\<close> command is sent to the server.
wenzelm@67917: 
wenzelm@67917:   Sessions are independent of client connections: it is possible to start a
wenzelm@67917:   session and later apply \<^verbatim>\<open>use_theories\<close> on different connections, as long as
wenzelm@67918:   the internal session identifier is known: shared theory imports will be used
wenzelm@67918:   only once (and persist until purged explicitly).
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsubsection \<open>Arguments\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67918:   Most arguments are shared with \<^verbatim>\<open>session_build\<close>
wenzelm@67917:   (\secref{sec:command-session-build}).
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67917:   The \<open>print_mode\<close> field adds identifiers of print modes to be made active for
wenzelm@67917:   this session. For example, \<^verbatim>\<open>"print_mode": ["ASCII"]\<close> prefers ASCII
wenzelm@67917:   replacement syntax over mathematical Isabelle symbols. See also option \<^verbatim>\<open>-m\<close>
wenzelm@67917:   in @{tool process} (\secref{sec:tool-process}).
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsubsection \<open>Results\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67917:   The \<open>session\<close> field provides the active session name for clarity.
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67917:   The \<open>session_id\<close> field provides the internal identification of the session
wenzelm@67917:   object within the sever process. It can remain active as long as the server
wenzelm@67917:   is running, independently of the current client connection.
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsection \<open>Examples\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67917:   Start a default Isabelle/HOL session:
wenzelm@67917:   @{verbatim [display] \<open>session_start {"session": "HOL"}\<close>}
wenzelm@67918: 
wenzelm@67918:   Start a session from the Archive of Formal Proofs:
wenzelm@67918:   @{verbatim [display] \<open>session_start {"session": "Coinductive", "dirs": ["$AFP_BASE/thys"]}\<close>}
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsection \<open>Command \<^verbatim>\<open>session_stop\<close>\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67917:   \begin{tabular}{ll}
wenzelm@67917:   argument: & \<open>{session_id?: uuid}\<close> \\
wenzelm@67917:   immediate result: & \<^verbatim>\<open>OK\<close> \<open>task\<close> \\
wenzelm@67917:   regular result: & \<^verbatim>\<open>FINISHED\<close> \<open>task \<oplus> session_stop_result\<close> \\
wenzelm@67917:   error result: & \<^verbatim>\<open>FAILED\<close> \<open>task \<oplus> error_message \<oplus> session_stop_result\<close> \\[2ex]
wenzelm@67917:   \end{tabular}
wenzelm@67917: 
wenzelm@67917:   \begin{tabular}{l}
wenzelm@67917:   \<^bold>\<open>type\<close> \<open>session_stop_result = {ok: bool, return_code: int}\<close>
wenzelm@67917:   \end{tabular}
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67918:   The \<^verbatim>\<open>session_stop\<close> command forces a shutdown of the identified PIDE
wenzelm@67918:   session. This asynchronous tasks usually finishes quickly. Failure only
wenzelm@67918:   happens in unusual situations, according to the return code of the
wenzelm@67917:   underlying Isabelle/ML process.
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsubsection \<open>Arguments\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67918:   The \<open>session_id\<close> field provides the UUID originally created by the server
wenzelm@67918:   for this session.
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsubsection \<open>Results\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67918:   The \<open>ok\<close> field tells abstractly, whether the Isabelle/ML process has
wenzelm@67918:   terminated properly.
wenzelm@67918: 
wenzelm@67918:   The \<open>return_code\<close> field expresses this information according to usual POSIX
wenzelm@67918:   conventions for process exit.
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsection \<open>Command \<^verbatim>\<open>use_theories\<close> \label{sec:command-use-theories}\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67917:   \begin{tabular}{ll}
wenzelm@67917:   argument: & \<open>use_theories_arguments\<close> \\
wenzelm@67917:   regular result: & \<^verbatim>\<open>OK\<close> \<open>use_theories_results\<close> \\
wenzelm@67917:   \end{tabular}
wenzelm@67917: 
wenzelm@67917:   \begin{tabular}{ll}
wenzelm@67917:   \<^bold>\<open>type\<close> \<open>theory_name = string | {name: string, pos: position}\<close> \\
wenzelm@67917:   \end{tabular}
wenzelm@67917: 
wenzelm@67917:   \begin{tabular}{ll}
wenzelm@67917:   \<^bold>\<open>type\<close> \<open>use_theories_arguments =\<close> \\
wenzelm@67917:   \quad\<open>{session_id: uuid,\<close> \\
wenzelm@67917:   \quad~~\<open>theories: [theory_name],\<close> \\
wenzelm@67917:   \quad~~\<open>qualifier?: string,\<close> \\
wenzelm@67917:   \quad~~\<open>master_dir?: string,\<close> \\
wenzelm@67917:   \quad~~\<open>pretty_margin?: double\<close> & \<^bold>\<open>default:\<close> \<^verbatim>\<open>76\<close> \\
wenzelm@67917:   \quad~~\<open>unicode_symbols?: bool}\<close> \\[2ex]
wenzelm@67918:   \end{tabular}
wenzelm@67917: 
wenzelm@67918:   \begin{tabular}{ll}
wenzelm@67917:   \<^bold>\<open>type\<close> \<open>node_result =\<close> \\
wenzelm@67917:   \quad\<open>{node_name: string,\<close> \\
wenzelm@67917:   \quad~~\<open>theory: string,\<close> \\
wenzelm@67917:   \quad~~\<open>status: node_status,\<close> \\
wenzelm@67917:   \quad~~\<open>messages: [message]}\<close> \\[2ex]
wenzelm@67917: 
wenzelm@67917:   \<^bold>\<open>type\<close> \<open>use_theories_results =\<close> \\
wenzelm@67917:   \quad\<open>{ok: bool,\<close> \\
wenzelm@67917:   \quad~~\<open>errors: [message],\<close> \\
wenzelm@67917:   \quad~~\<open>nodes: [node_result]}\<close> \\[2ex]
wenzelm@67917:   \end{tabular}
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67918:   The \<^verbatim>\<open>use_theories\<close> command updates the identified session by adding the
wenzelm@67918:   current version of theory files to it, while dependencies are resolved
wenzelm@67918:   implicitly. The command succeeds eventually, when all theories have been
wenzelm@67918:   \<^emph>\<open>consolidated\<close> in the sense the formal \<open>node_status\<close>
wenzelm@67918:   (\secref{sec:json-types}): the outermost command structure has finished (or
wenzelm@67918:   failed) and the final \<^theory_text>\<open>end\<close> command of each theory has been checked.
wenzelm@67917: 
wenzelm@67918:   Already used theories persist in the session until purged explicitly. This
wenzelm@67918:   also means that repeated invocations of \<^verbatim>\<open>use_theories\<close> are idempotent: it
wenzelm@67918:   could make sense to do that with different values for \<open>pretty_margin\<close> or
wenzelm@67918:   \<open>unicode_symbols\<close> to get different formatting for \<open>errors\<close> or \<open>messages\<close>.
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsubsection \<open>Arguments\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67917:   The \<open>session_id\<close> is the identifier provided by the server, when the session
wenzelm@67917:   was created (possibly on a different client connection).
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67918:   The \<open>theories\<close> field specifies theory names as in theory \<^theory_text>\<open>imports\<close> or in
wenzelm@67918:   ROOT \<^bold>\<open>theories\<close>. An explicit source position for these theory name
wenzelm@67918:   specifications may be given, which is occasionally useful for precise error
wenzelm@67918:   locations.
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67917:   The \<open>qualifier\<close> field provides an alternative session qualifier for theories
wenzelm@67917:   that are not formally recognized as a member of a particular session. The
wenzelm@67917:   default is \<^verbatim>\<open>Draft\<close> as in Isabelle/jEdit. There is rarely a need to change
wenzelm@67917:   that, as theory nodes are already uniquely identified by their physical
wenzelm@67918:   file-system location. This already allows reuse of theory base names with
wenzelm@67918:   the same session qualifier --- as long as these theories are not used
wenzelm@67918:   together (e.g.\ in \<^theory_text>\<open>imports\<close>).
wenzelm@67917: 
wenzelm@67918:   \<^medskip> The \<open>master_dir\<close> field explicit specifies the formal master directory of
wenzelm@67918:   the imported theory. Normally this is determined implicitly from the parent
wenzelm@67918:   directory of the theory file.
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67917:   The \<open>pretty_margin\<close> field specifies the line width for pretty-printing. The
wenzelm@67918:   default is suitable for classic console output. Formatting happens at the
wenzelm@67918:   end of \<^verbatim>\<open>use_theories\<close>, when all prover messages are exported to the client.
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67918:   The \<open>unicode_symbols\<close> field set to \<^verbatim>\<open>true\<close> renders message output for direct
wenzelm@67918:   output on a Unicode capable channel, ideally with the Isabelle fonts as in
wenzelm@67918:   Isabelle/jEdit. The default is to keep the symbolic representation of
wenzelm@67918:   Isabelle text, e.g.\ \<^verbatim>\<open>\<forall>\<close> instead of its rendering as \<open>\<forall>\<close>. This means the
wenzelm@67918:   client needs to perform its own rendering before presenting it to the
wenzelm@67918:   end-user.
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67918: 
wenzelm@67917: subsubsection \<open>Results\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67917:   The \<open>ok\<close> field indicates overall success of processing the specified
wenzelm@67917:   theories with all their dependencies.
wenzelm@67917: 
wenzelm@67918:   When \<open>ok\<close> is \<^verbatim>\<open>false\<close>, the \<open>errors\<close> field lists all errors cumulatively
wenzelm@67918:   (including imported theories). The messages contain position information for
wenzelm@67918:   the original theory nodes.
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67918:   The \<open>nodes\<close> field provides detailed information about each imported theory
wenzelm@67918:   node. The individual fields are as follows:
wenzelm@67917: 
wenzelm@67917:   \<^item> \<open>node_name\<close>: the physical name for the theory node, based on its
wenzelm@67918:   file-system location;
wenzelm@67917: 
wenzelm@67918:   \<^item> \<open>theory\<close>: the logical theory name, e.g.\ \<^verbatim>\<open>HOL-Library.AList\<close>;
wenzelm@67917: 
wenzelm@67917:   \<^item> \<open>status\<close>: the overall node status, e.g.\ see the visualization in the
wenzelm@67917:   \<open>Theories\<close> panel of Isabelle/jEdit @{cite "isabelle-jedit"};
wenzelm@67917: 
wenzelm@67918:   \<^item> \<open>messages\<close>: the main bulk of prover messages produced in this theory
wenzelm@67917:   (\<^verbatim>\<open>writeln\<close>, \<^verbatim>\<open>warning\<close>, \<^verbatim>\<open>error\<close>, etc.).
wenzelm@67917: \<close>
wenzelm@67917: 
wenzelm@67917: 
wenzelm@67917: subsubsection \<open>Examples\<close>
wenzelm@67917: 
wenzelm@67917: text \<open>
wenzelm@67917:   Process some example theory from the Isabelle distribution, within the
wenzelm@67917:   context of an already started session for Isabelle/HOL (see also
wenzelm@67917:   \secref{sec:command-session-start}):
wenzelm@67917:   @{verbatim [display] \<open>use_theories {"session_id": ..., "theories": ["~~/src/HOL/Isar_Examples/Drinker"]}\<close>}
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67917:   Process some example theories that import other theories via
wenzelm@67917:   session-qualified theory names:
wenzelm@67917: 
wenzelm@67917:   @{verbatim [display] \<open>session_start {"session": "HOL", "all_known": true}
wenzelm@67917: use_theories {"session_id": ..., "theories": ["~~/src/HOL/Unix/Unix"]}
wenzelm@67917: session_stop {"session_id": ...}\<close>}
wenzelm@67917: 
wenzelm@67917:   The option \<open>all_known\<close> has been used here to the full name space of
wenzelm@67917:   session-qualified theory names accessible in this session. This is also the
wenzelm@67917:   default in Isabelle/jEdit, although it requires significant startup time.
wenzelm@67917: 
wenzelm@67917:   \<^medskip>
wenzelm@67917:   Process some example theories in the context of their (single) parent
wenzelm@67917:   session:
wenzelm@67917: 
wenzelm@67917:   @{verbatim [display] \<open>session_start {"session": "HOL-Library"}
wenzelm@67917: use_theories {"session_id": ..., "theories": ["~~/src/HOL/Unix/Unix"]}
wenzelm@67917: session_stop {"session_id": ...}\<close>}
wenzelm@67904: \<close>
wenzelm@67904: 
wenzelm@67904: end