src/Pure/Concurrent/lazy.ML
author wenzelm
Mon, 19 Feb 2018 10:05:37 +0100
changeset 67657 ef20d4961f86
parent 66904 d9783ea1160c
child 67661 a5ab9ea36cd5
permissions -rw-r--r--
more operations;
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
32815
1a5e364584ae separate concurrent/sequential versions of lazy evaluation;
wenzelm
parents: 32738
diff changeset
     1
(*  Title:      Pure/Concurrent/lazy.ML
1a5e364584ae separate concurrent/sequential versions of lazy evaluation;
wenzelm
parents: 32738
diff changeset
     2
    Author:     Makarius
28673
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
     3
34278
228f27469139 do not memoize interrupts;
wenzelm
parents: 33023
diff changeset
     4
Lazy evaluation with memoing of results and regular exceptions.
40389
511e5263f5c6 tuned comments;
wenzelm
parents: 39232
diff changeset
     5
Parallel version based on (passive) futures, to avoid critical or
511e5263f5c6 tuned comments;
wenzelm
parents: 39232
diff changeset
     6
multiple evaluation (unless interrupted).
28673
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
     7
*)
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
     8
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
     9
signature LAZY =
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
    10
sig
28971
300ec36a19af renamed type Lazy.T to lazy;
wenzelm
parents: 28673
diff changeset
    11
  type 'a lazy
66904
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    12
  val value: 'a -> 'a lazy
66167
1bd268ab885c more informative task_statistics;
wenzelm
parents: 66166
diff changeset
    13
  val lazy_name: string -> (unit -> 'a) -> 'a lazy
28971
300ec36a19af renamed type Lazy.T to lazy;
wenzelm
parents: 28673
diff changeset
    14
  val lazy: (unit -> 'a) -> 'a lazy
59193
59f1591a11cb eliminated Document.execution frontier (again, see 627fb639a2d9): just run into older execution, potentially stalling worker thread, but without global delay due to long-running tasks (notably sledgehammer);
wenzelm
parents: 59191
diff changeset
    15
  val peek: 'a lazy -> 'a Exn.result option
59f1591a11cb eliminated Document.execution frontier (again, see 627fb639a2d9): just run into older execution, potentially stalling worker thread, but without global delay due to long-running tasks (notably sledgehammer);
wenzelm
parents: 59191
diff changeset
    16
  val is_running: 'a lazy -> bool
59f1591a11cb eliminated Document.execution frontier (again, see 627fb639a2d9): just run into older execution, potentially stalling worker thread, but without global delay due to long-running tasks (notably sledgehammer);
wenzelm
parents: 59191
diff changeset
    17
  val is_finished: 'a lazy -> bool
29422
fdf396a24a9f added force_result;
wenzelm
parents: 28971
diff changeset
    18
  val force_result: 'a lazy -> 'a Exn.result
28971
300ec36a19af renamed type Lazy.T to lazy;
wenzelm
parents: 28673
diff changeset
    19
  val force: 'a lazy -> 'a
66168
fcd09fc36d7f consolidate proofs more simultaneously;
wenzelm
parents: 66167
diff changeset
    20
  val force_list: 'a lazy list -> 'a list
67657
ef20d4961f86 more operations;
wenzelm
parents: 66904
diff changeset
    21
  val forced: 'a lazy -> 'a lazy
44386
4048ca2658b7 some support for toplevel printing wrt. editor perspective (still inactive);
wenzelm
parents: 44298
diff changeset
    22
  val map: ('a -> 'b) -> 'a lazy -> 'b lazy
44427
c4a86d72a5cc tuned signature;
wenzelm
parents: 44387
diff changeset
    23
  val future: Future.params -> 'a lazy -> 'a future
32815
1a5e364584ae separate concurrent/sequential versions of lazy evaluation;
wenzelm
parents: 32738
diff changeset
    24
end;
28673
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
    25
32815
1a5e364584ae separate concurrent/sequential versions of lazy evaluation;
wenzelm
parents: 32738
diff changeset
    26
structure Lazy: LAZY =
28673
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
    27
struct
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
    28
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
    29
(* datatype *)
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
    30
32815
1a5e364584ae separate concurrent/sequential versions of lazy evaluation;
wenzelm
parents: 32738
diff changeset
    31
datatype 'a expr =
66167
1bd268ab885c more informative task_statistics;
wenzelm
parents: 66166
diff changeset
    32
  Expr of string * (unit -> 'a) |
37183
dae865999db5 force_result within the current execution context -- avoids overhead of potential thread context switch and robustifies Interrupt handling;
wenzelm
parents: 34278
diff changeset
    33
  Result of 'a future;
28673
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
    34
66904
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    35
abstype 'a lazy = Value of 'a | Lazy of 'a expr Synchronized.var
32815
1a5e364584ae separate concurrent/sequential versions of lazy evaluation;
wenzelm
parents: 32738
diff changeset
    36
with
28673
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
    37
66904
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    38
fun value a = Value a;
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    39
66167
1bd268ab885c more informative task_statistics;
wenzelm
parents: 66166
diff changeset
    40
fun lazy_name name e = Lazy (Synchronized.var "lazy" (Expr (name, e)));
1bd268ab885c more informative task_statistics;
wenzelm
parents: 66166
diff changeset
    41
fun lazy e = lazy_name "lazy" e;
59193
59f1591a11cb eliminated Document.execution frontier (again, see 627fb639a2d9): just run into older execution, potentially stalling worker thread, but without global delay due to long-running tasks (notably sledgehammer);
wenzelm
parents: 59191
diff changeset
    42
66904
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    43
fun peek (Value a) = SOME (Exn.Res a)
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    44
  | peek (Lazy var) =
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    45
      (case Synchronized.value var of
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    46
        Expr _ => NONE
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    47
      | Result res => Future.peek res);
28673
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
    48
66904
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    49
fun pending (Value _) = false
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    50
  | pending (Lazy var) =
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    51
      (case Synchronized.value var of
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    52
        Expr _ => true
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    53
      | Result _ => false);
66168
fcd09fc36d7f consolidate proofs more simultaneously;
wenzelm
parents: 66167
diff changeset
    54
67657
ef20d4961f86 more operations;
wenzelm
parents: 66904
diff changeset
    55
fun is_value (Value _) = true
ef20d4961f86 more operations;
wenzelm
parents: 66904
diff changeset
    56
  | is_value (Lazy _) = false;
66168
fcd09fc36d7f consolidate proofs more simultaneously;
wenzelm
parents: 66167
diff changeset
    57
59193
59f1591a11cb eliminated Document.execution frontier (again, see 627fb639a2d9): just run into older execution, potentially stalling worker thread, but without global delay due to long-running tasks (notably sledgehammer);
wenzelm
parents: 59191
diff changeset
    58
59f1591a11cb eliminated Document.execution frontier (again, see 627fb639a2d9): just run into older execution, potentially stalling worker thread, but without global delay due to long-running tasks (notably sledgehammer);
wenzelm
parents: 59191
diff changeset
    59
(* status *)
59f1591a11cb eliminated Document.execution frontier (again, see 627fb639a2d9): just run into older execution, potentially stalling worker thread, but without global delay due to long-running tasks (notably sledgehammer);
wenzelm
parents: 59191
diff changeset
    60
66904
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    61
fun is_running (Value _) = false
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    62
  | is_running (Lazy var) =
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    63
      (case Synchronized.value var of
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    64
        Expr _ => false
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    65
      | Result res => not (Future.is_finished res));
59193
59f1591a11cb eliminated Document.execution frontier (again, see 627fb639a2d9): just run into older execution, potentially stalling worker thread, but without global delay due to long-running tasks (notably sledgehammer);
wenzelm
parents: 59191
diff changeset
    66
66904
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    67
fun is_finished (Value _) = true
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    68
  | is_finished (Lazy var) =
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    69
      (case Synchronized.value var of
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    70
        Expr _ => false
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    71
      | Result res =>
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    72
          Future.is_finished res andalso not (Exn.is_interrupt_exn (Future.join_result res)));
59191
682aa538c5c0 more thorough Lazy.is_finished;
wenzelm
parents: 59147
diff changeset
    73
28673
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
    74
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
    75
(* force result *)
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
    76
66904
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    77
fun force_result (Value a) = Exn.Res a
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    78
  | force_result (Lazy var) =
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    79
      (case peek (Lazy var) of
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    80
        SOME res => res
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    81
      | NONE =>
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    82
          Thread_Attributes.uninterruptible (fn restore_attributes => fn () =>
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    83
            let
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    84
              val (expr, x) =
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    85
                Synchronized.change_result var
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    86
                  (fn Expr (name, e) =>
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    87
                        let val x = Future.promise_name name I
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    88
                        in ((SOME (name, e), x), Result x) end
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    89
                    | Result x => ((NONE, x), Result x));
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    90
            in
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    91
              (case expr of
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    92
                SOME (name, e) =>
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    93
                  let
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    94
                    val res0 = Exn.capture (restore_attributes e) ();
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    95
                    val _ = Exn.capture (fn () => Future.fulfill_result x res0) ();
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    96
                    val res = Future.join_result x;
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    97
                    (*semantic race: some other threads might see the same
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    98
                      interrupt, until there is a fresh start*)
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
    99
                    val _ =
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
   100
                      if Exn.is_interrupt_exn res then
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
   101
                        Synchronized.change var (fn _ => Expr (name, e))
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
   102
                      else ();
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
   103
                  in res end
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
   104
              | NONE => Exn.capture (restore_attributes (fn () => Future.join x)) ())
d9783ea1160c minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
wenzelm
parents: 66168
diff changeset
   105
            end) ());
28673
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
   106
d746a8c12c43 renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
wenzelm
parents:
diff changeset
   107
end;
44386
4048ca2658b7 some support for toplevel printing wrt. editor perspective (still inactive);
wenzelm
parents: 44298
diff changeset
   108
66168
fcd09fc36d7f consolidate proofs more simultaneously;
wenzelm
parents: 66167
diff changeset
   109
fun force x = Exn.release (force_result x);
fcd09fc36d7f consolidate proofs more simultaneously;
wenzelm
parents: 66167
diff changeset
   110
fcd09fc36d7f consolidate proofs more simultaneously;
wenzelm
parents: 66167
diff changeset
   111
fun force_list xs =
fcd09fc36d7f consolidate proofs more simultaneously;
wenzelm
parents: 66167
diff changeset
   112
  (List.app (fn x => if pending x then ignore (force_result x) else ()) xs;
fcd09fc36d7f consolidate proofs more simultaneously;
wenzelm
parents: 66167
diff changeset
   113
   map force xs);
fcd09fc36d7f consolidate proofs more simultaneously;
wenzelm
parents: 66167
diff changeset
   114
67657
ef20d4961f86 more operations;
wenzelm
parents: 66904
diff changeset
   115
fun forced x = if is_value x then x else value (force x);
ef20d4961f86 more operations;
wenzelm
parents: 66904
diff changeset
   116
66167
1bd268ab885c more informative task_statistics;
wenzelm
parents: 66166
diff changeset
   117
fun map f x = lazy_name "Lazy.map" (fn () => f (force x));
44386
4048ca2658b7 some support for toplevel printing wrt. editor perspective (still inactive);
wenzelm
parents: 44298
diff changeset
   118
4048ca2658b7 some support for toplevel printing wrt. editor perspective (still inactive);
wenzelm
parents: 44298
diff changeset
   119
4048ca2658b7 some support for toplevel printing wrt. editor perspective (still inactive);
wenzelm
parents: 44298
diff changeset
   120
(* future evaluation *)
4048ca2658b7 some support for toplevel printing wrt. editor perspective (still inactive);
wenzelm
parents: 44298
diff changeset
   121
4048ca2658b7 some support for toplevel printing wrt. editor perspective (still inactive);
wenzelm
parents: 44298
diff changeset
   122
fun future params x =
4048ca2658b7 some support for toplevel printing wrt. editor perspective (still inactive);
wenzelm
parents: 44298
diff changeset
   123
  if is_finished x then Future.value_result (force_result x)
4048ca2658b7 some support for toplevel printing wrt. editor perspective (still inactive);
wenzelm
parents: 44298
diff changeset
   124
  else (singleton o Future.forks) params (fn () => force x);
4048ca2658b7 some support for toplevel printing wrt. editor perspective (still inactive);
wenzelm
parents: 44298
diff changeset
   125
62663
bea354f6ff21 clarified modules;
wenzelm
parents: 59348
diff changeset
   126
bea354f6ff21 clarified modules;
wenzelm
parents: 59348
diff changeset
   127
(* toplevel pretty printing *)
bea354f6ff21 clarified modules;
wenzelm
parents: 59348
diff changeset
   128
bea354f6ff21 clarified modules;
wenzelm
parents: 59348
diff changeset
   129
val _ =
62819
d3ff367a16a0 careful export of type-dependent functions, without losing their special status;
wenzelm
parents: 62663
diff changeset
   130
  ML_system_pp (fn depth => fn pretty => fn x =>
62663
bea354f6ff21 clarified modules;
wenzelm
parents: 59348
diff changeset
   131
    (case peek x of
62823
751bcf0473a7 tuned signature;
wenzelm
parents: 62819
diff changeset
   132
      NONE => PolyML_Pretty.PrettyString "<lazy>"
751bcf0473a7 tuned signature;
wenzelm
parents: 62819
diff changeset
   133
    | SOME (Exn.Exn _) => PolyML_Pretty.PrettyString "<failed>"
62663
bea354f6ff21 clarified modules;
wenzelm
parents: 59348
diff changeset
   134
    | SOME (Exn.Res y) => pretty (y, depth)));
bea354f6ff21 clarified modules;
wenzelm
parents: 59348
diff changeset
   135
32815
1a5e364584ae separate concurrent/sequential versions of lazy evaluation;
wenzelm
parents: 32738
diff changeset
   136
end;
28971
300ec36a19af renamed type Lazy.T to lazy;
wenzelm
parents: 28673
diff changeset
   137
300ec36a19af renamed type Lazy.T to lazy;
wenzelm
parents: 28673
diff changeset
   138
type 'a lazy = 'a Lazy.lazy;