| author | wenzelm | 
| Tue, 08 May 2018 15:41:52 +0200 | |
| changeset 68115 | 23c6ae3dd3a0 | 
| parent 67669 | ad8ca85f13e2 | 
| child 68130 | 6fb85346cb79 | 
| permissions | -rw-r--r-- | 
| 32815 
1a5e364584ae
separate concurrent/sequential versions of lazy evaluation;
 wenzelm parents: 
32738diff
changeset | 1 | (* Title: Pure/Concurrent/lazy.ML | 
| 
1a5e364584ae
separate concurrent/sequential versions of lazy evaluation;
 wenzelm parents: 
32738diff
changeset | 2 | Author: Makarius | 
| 28673 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 3 | |
| 34278 | 4 | Lazy evaluation with memoing of results and regular exceptions. | 
| 40389 | 5 | Parallel version based on (passive) futures, to avoid critical or | 
| 6 | multiple evaluation (unless interrupted). | |
| 28673 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 7 | *) | 
| 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 8 | |
| 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 9 | signature LAZY = | 
| 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 10 | sig | 
| 28971 | 11 | type 'a lazy | 
| 66904 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 12 | val value: 'a -> 'a lazy | 
| 66167 | 13 | val lazy_name: string -> (unit -> 'a) -> 'a lazy | 
| 28971 | 14 | val lazy: (unit -> 'a) -> 'a lazy | 
| 59193 
59f1591a11cb
eliminated Document.execution frontier (again, see 627fb639a2d9): just run into older execution, potentially stalling worker thread, but without global delay due to long-running tasks (notably sledgehammer);
 wenzelm parents: 
59191diff
changeset | 15 | val peek: 'a lazy -> 'a Exn.result option | 
| 67661 | 16 | val is_pending: 'a lazy -> bool | 
| 59193 
59f1591a11cb
eliminated Document.execution frontier (again, see 627fb639a2d9): just run into older execution, potentially stalling worker thread, but without global delay due to long-running tasks (notably sledgehammer);
 wenzelm parents: 
59191diff
changeset | 17 | val is_running: 'a lazy -> bool | 
| 
59f1591a11cb
eliminated Document.execution frontier (again, see 627fb639a2d9): just run into older execution, potentially stalling worker thread, but without global delay due to long-running tasks (notably sledgehammer);
 wenzelm parents: 
59191diff
changeset | 18 | val is_finished: 'a lazy -> bool | 
| 29422 | 19 | val force_result: 'a lazy -> 'a Exn.result | 
| 28971 | 20 | val force: 'a lazy -> 'a | 
| 67661 | 21 | val force_value: 'a lazy -> 'a lazy | 
| 22 | val trim_value: 'a lazy -> 'a lazy | |
| 44386 
4048ca2658b7
some support for toplevel printing wrt. editor perspective (still inactive);
 wenzelm parents: 
44298diff
changeset | 23 |   val map: ('a -> 'b) -> 'a lazy -> 'b lazy
 | 
| 67661 | 24 |   val map_finished: ('a -> 'b) -> 'a lazy -> 'b lazy
 | 
| 67668 | 25 | val consolidate: 'a lazy list -> 'a lazy list | 
| 44427 | 26 | val future: Future.params -> 'a lazy -> 'a future | 
| 32815 
1a5e364584ae
separate concurrent/sequential versions of lazy evaluation;
 wenzelm parents: 
32738diff
changeset | 27 | end; | 
| 28673 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 28 | |
| 32815 
1a5e364584ae
separate concurrent/sequential versions of lazy evaluation;
 wenzelm parents: 
32738diff
changeset | 29 | structure Lazy: LAZY = | 
| 28673 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 30 | struct | 
| 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 31 | |
| 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 32 | (* datatype *) | 
| 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 33 | |
| 32815 
1a5e364584ae
separate concurrent/sequential versions of lazy evaluation;
 wenzelm parents: 
32738diff
changeset | 34 | datatype 'a expr = | 
| 66167 | 35 | Expr of string * (unit -> 'a) | | 
| 37183 
dae865999db5
force_result within the current execution context -- avoids overhead of potential thread context switch and robustifies Interrupt handling;
 wenzelm parents: 
34278diff
changeset | 36 | Result of 'a future; | 
| 28673 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 37 | |
| 66904 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 38 | abstype 'a lazy = Value of 'a | Lazy of 'a expr Synchronized.var | 
| 32815 
1a5e364584ae
separate concurrent/sequential versions of lazy evaluation;
 wenzelm parents: 
32738diff
changeset | 39 | with | 
| 28673 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 40 | |
| 66904 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 41 | fun value a = Value a; | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 42 | |
| 66167 | 43 | fun lazy_name name e = Lazy (Synchronized.var "lazy" (Expr (name, e))); | 
| 44 | fun lazy e = lazy_name "lazy" e; | |
| 59193 
59f1591a11cb
eliminated Document.execution frontier (again, see 627fb639a2d9): just run into older execution, potentially stalling worker thread, but without global delay due to long-running tasks (notably sledgehammer);
 wenzelm parents: 
59191diff
changeset | 45 | |
| 66904 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 46 | fun peek (Value a) = SOME (Exn.Res a) | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 47 | | peek (Lazy var) = | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 48 | (case Synchronized.value var of | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 49 | Expr _ => NONE | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 50 | | Result res => Future.peek res); | 
| 28673 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 51 | |
| 67661 | 52 | |
| 53 | (* status *) | |
| 66168 | 54 | |
| 67657 | 55 | fun is_value (Value _) = true | 
| 56 | | is_value (Lazy _) = false; | |
| 66168 | 57 | |
| 67661 | 58 | fun is_pending (Value _) = false | 
| 59 | | is_pending (Lazy var) = | |
| 60 | (case Synchronized.value var of | |
| 61 | Expr _ => true | |
| 62 | | Result _ => false); | |
| 59193 
59f1591a11cb
eliminated Document.execution frontier (again, see 627fb639a2d9): just run into older execution, potentially stalling worker thread, but without global delay due to long-running tasks (notably sledgehammer);
 wenzelm parents: 
59191diff
changeset | 63 | |
| 66904 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 64 | fun is_running (Value _) = false | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 65 | | is_running (Lazy var) = | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 66 | (case Synchronized.value var of | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 67 | Expr _ => false | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 68 | | Result res => not (Future.is_finished res)); | 
| 59193 
59f1591a11cb
eliminated Document.execution frontier (again, see 627fb639a2d9): just run into older execution, potentially stalling worker thread, but without global delay due to long-running tasks (notably sledgehammer);
 wenzelm parents: 
59191diff
changeset | 69 | |
| 66904 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 70 | fun is_finished (Value _) = true | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 71 | | is_finished (Lazy var) = | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 72 | (case Synchronized.value var of | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 73 | Expr _ => false | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 74 | | Result res => | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 75 | Future.is_finished res andalso not (Exn.is_interrupt_exn (Future.join_result res))); | 
| 59191 | 76 | |
| 28673 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 77 | |
| 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 78 | (* force result *) | 
| 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 79 | |
| 66904 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 80 | fun force_result (Value a) = Exn.Res a | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 81 | | force_result (Lazy var) = | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 82 | (case peek (Lazy var) of | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 83 | SOME res => res | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 84 | | NONE => | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 85 | Thread_Attributes.uninterruptible (fn restore_attributes => fn () => | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 86 | let | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 87 | val (expr, x) = | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 88 | Synchronized.change_result var | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 89 | (fn Expr (name, e) => | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 90 | let val x = Future.promise_name name I | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 91 | in ((SOME (name, e), x), Result x) end | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 92 | | Result x => ((NONE, x), Result x)); | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 93 | in | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 94 | (case expr of | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 95 | SOME (name, e) => | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 96 | let | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 97 | val res0 = Exn.capture (restore_attributes e) (); | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 98 | val _ = Exn.capture (fn () => Future.fulfill_result x res0) (); | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 99 | val res = Future.join_result x; | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 100 | (*semantic race: some other threads might see the same | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 101 | interrupt, until there is a fresh start*) | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 102 | val _ = | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 103 | if Exn.is_interrupt_exn res then | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 104 | Synchronized.change var (fn _ => Expr (name, e)) | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 105 | else (); | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 106 | in res end | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 107 | | NONE => Exn.capture (restore_attributes (fn () => Future.join x)) ()) | 
| 
d9783ea1160c
minor performance tuning: avoid mutable variable for plain value, e.g. relevant for GC;
 wenzelm parents: 
66168diff
changeset | 108 | end) ()); | 
| 28673 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 109 | |
| 
d746a8c12c43
renamed structure Susp to Lazy, and Susp.delay to Lazy.lazy;
 wenzelm parents: diff
changeset | 110 | end; | 
| 44386 
4048ca2658b7
some support for toplevel printing wrt. editor perspective (still inactive);
 wenzelm parents: 
44298diff
changeset | 111 | |
| 66168 | 112 | fun force x = Exn.release (force_result x); | 
| 113 | ||
| 67661 | 114 | fun force_value x = if is_value x then x else value (force x); | 
| 115 | fun trim_value x = if is_pending x then x else force_value x; | |
| 116 | ||
| 117 | ||
| 118 | (* map *) | |
| 67657 | 119 | |
| 66167 | 120 | fun map f x = lazy_name "Lazy.map" (fn () => f (force x)); | 
| 44386 
4048ca2658b7
some support for toplevel printing wrt. editor perspective (still inactive);
 wenzelm parents: 
44298diff
changeset | 121 | |
| 67661 | 122 | fun map_finished f x = if is_finished x then value (f (force x)) else map f x; | 
| 123 | ||
| 44386 
4048ca2658b7
some support for toplevel printing wrt. editor perspective (still inactive);
 wenzelm parents: 
44298diff
changeset | 124 | |
| 67668 | 125 | (* consolidate in parallel *) | 
| 126 | ||
| 127 | fun consolidate xs = | |
| 128 | let | |
| 129 | val pending = | |
| 130 | xs |> map_filter (fn x => if is_pending x then SOME (fn () => force_result x) else NONE); | |
| 131 | val _ = | |
| 132 | if Multithreading.relevant pending then | |
| 133 |         ignore (Future.forked_results {name = "Lazy.consolidate", deps = []} pending)
 | |
| 134 | else List.app (fn e => ignore (e ())) pending; | |
| 135 | in xs end; | |
| 136 | ||
| 137 | ||
| 44386 
4048ca2658b7
some support for toplevel printing wrt. editor perspective (still inactive);
 wenzelm parents: 
44298diff
changeset | 138 | (* future evaluation *) | 
| 
4048ca2658b7
some support for toplevel printing wrt. editor perspective (still inactive);
 wenzelm parents: 
44298diff
changeset | 139 | |
| 
4048ca2658b7
some support for toplevel printing wrt. editor perspective (still inactive);
 wenzelm parents: 
44298diff
changeset | 140 | fun future params x = | 
| 
4048ca2658b7
some support for toplevel printing wrt. editor perspective (still inactive);
 wenzelm parents: 
44298diff
changeset | 141 | if is_finished x then Future.value_result (force_result x) | 
| 
4048ca2658b7
some support for toplevel printing wrt. editor perspective (still inactive);
 wenzelm parents: 
44298diff
changeset | 142 | else (singleton o Future.forks) params (fn () => force x); | 
| 
4048ca2658b7
some support for toplevel printing wrt. editor perspective (still inactive);
 wenzelm parents: 
44298diff
changeset | 143 | |
| 62663 | 144 | |
| 145 | (* toplevel pretty printing *) | |
| 146 | ||
| 147 | val _ = | |
| 62819 
d3ff367a16a0
careful export of type-dependent functions, without losing their special status;
 wenzelm parents: 
62663diff
changeset | 148 | ML_system_pp (fn depth => fn pretty => fn x => | 
| 62663 | 149 | (case peek x of | 
| 62823 | 150 | NONE => PolyML_Pretty.PrettyString "<lazy>" | 
| 151 | | SOME (Exn.Exn _) => PolyML_Pretty.PrettyString "<failed>" | |
| 62663 | 152 | | SOME (Exn.Res y) => pretty (y, depth))); | 
| 153 | ||
| 32815 
1a5e364584ae
separate concurrent/sequential versions of lazy evaluation;
 wenzelm parents: 
32738diff
changeset | 154 | end; | 
| 28971 | 155 | |
| 156 | type 'a lazy = 'a Lazy.lazy; |