src/Pure/System/benchmark.scala
author wenzelm
Wed, 08 Nov 2023 13:14:59 +0100
changeset 78916 e97fa2edf4b2
parent 78915 90756ad4d8d7
child 79291 e9a788a75775
permissions -rw-r--r--
clarified signature;
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
78840
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
     1
/*  Title:      Pure/System/benchmark.scala
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
     2
    Author:     Fabian Huch, TU Muenchen
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
     3
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
     4
Host platform benchmarks for performance estimation.
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
     5
*/
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
     6
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
     7
package isabelle
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
     8
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
     9
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    10
object Benchmark {
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    11
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    12
  def benchmark_command(
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    13
    host: Build_Cluster.Host,
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    14
    ssh: SSH.System = SSH.Local,
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    15
    isabelle_home: Path = Path.current,
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    16
  ): String = {
78916
e97fa2edf4b2 clarified signature;
wenzelm
parents: 78915
diff changeset
    17
    val options = Options.Spec.eq("build_hostname", host.name) :: host.options
78840
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    18
    ssh.bash_path(isabelle_home + Path.explode("bin/isabelle")) + " benchmark" +
78915
90756ad4d8d7 more accurate treatment of surrounding whitespace;
wenzelm
parents: 78908
diff changeset
    19
      Options.Spec.bash_strings(options, bg = true)
78840
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    20
  }
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    21
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    22
  def benchmark(options: Options, progress: Progress = new Progress()): Unit = {
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    23
    val hostname = options.string("build_hostname")
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    24
    val store = Store(options)
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    25
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    26
    using(store.open_server()) { server =>
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    27
      val db = store.open_build_database(path = Host.private_data.database, server = server)
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    28
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    29
      progress.echo("Starting benchmark...")
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    30
      val start = Time.now()
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    31
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    32
      // TODO proper benchmark
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    33
      def fib(n: Long): Long = if (n < 2) 1 else fib(n - 2) + fib(n - 1)
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    34
      val result = fib(42)
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    35
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    36
      val stop = Time.now()
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    37
      val timing = stop - start
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    38
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    39
      val score = Time.seconds(100).ms.toDouble / (1 + timing.ms)
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    40
      progress.echo(
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    41
        "Finished benchmark in " + timing.message + ". Score: " + String.format("%.2f", score))
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    42
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    43
      Host.write_info(db, Host.Info.gather(hostname, score = Some(score)))
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    44
    }
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    45
  }
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    46
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    47
  val isabelle_tool = Isabelle_Tool("benchmark", "run system benchmark",
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    48
    Scala_Project.here,
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    49
    { args =>
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    50
      var options = Options.init()
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    51
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    52
      val getopts = Getopts("""
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    53
Usage: isabelle benchmark [OPTIONS]
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    54
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    55
  Options are:
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    56
    -o OPTION    override Isabelle system OPTION (via NAME=VAL or NAME)
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    57
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    58
  Run a system benchmark.
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    59
""",
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    60
        "o:" -> (arg => options = options + arg))
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    61
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    62
      val more_args = getopts(args)
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    63
      if (more_args.nonEmpty) getopts.usage()
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    64
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    65
      val progress = new Console_Progress()
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    66
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    67
      benchmark(options, progress = progress)
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    68
    })
4b528ca25573 added initial version of benchmark module, e.g., to compare performance of different hosts;
Fabian Huch <huch@in.tum.de>
parents:
diff changeset
    69
}