author | wenzelm |
Mon, 25 Mar 2024 20:05:40 +0100 | |
changeset 79990 | 34420f5f2e92 |
parent 79988 | 36e33d227bf0 |
child 80030 | 5c00c04f09fb |
permissions | -rw-r--r-- |
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
1 |
# Isabelle system components # |
35610 | 2 |
|
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
3 |
## Multi-platform support of Isabelle ## |
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
4 |
|
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
5 |
### Preamble ### |
35610 | 6 |
|
7 |
The general programming model is that of a stylized ML + Scala + POSIX |
|
79987 | 8 |
environment, with a minimum of system-specific code in user-space tools. |
35610 | 9 |
|
79987 | 10 |
The Isabelle system infrastructure provides some facilities to make this work, |
11 |
e.g. see the ML and Scala modules `File` and `Path`, or functions like |
|
12 |
`Isabelle_System.bash`. The settings environment also provides some means for |
|
13 |
portability, e.g. the `bash` function `platform_path` to keep the impression |
|
79988 | 14 |
that Windows/Cygwin adheres to Isabelle/POSIX standards, although most |
15 |
executables are running natively on Windows. |
|
35610 | 16 |
|
79987 | 17 |
When producing add-on tools, it is important to stay within this clean room of |
18 |
Isabelle, and refrain from non-portable access to operating system functions. |
|
79988 | 19 |
The Isabelle environment uses Isabelle/Scala as portable system |
20 |
infrastructure, and Isabelle/ML refers to that for anything non-trivial. |
|
35610 | 21 |
|
22 |
||
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
23 |
### Supported platforms ### |
35610 | 24 |
|
79987 | 25 |
A broad range of hardware and operating system platforms are supported by |
79990 | 26 |
building executables on base-line versions that are neither too old nor too |
27 |
new. Common OS families should work: Linux, macOS, Windows. Exotic platforms |
|
28 |
are unsupported: NixOS, BSD, Solaris etc. |
|
72366 | 29 |
|
79990 | 30 |
The official platforms, with **base-line operating systems**, and reference |
79988 | 31 |
machines are as follows: |
35610 | 32 |
|
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
33 |
* `x86_64-linux` |
79988 | 34 |
- **Ubuntu 18.04 LTS** |
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
35 |
* `arm64-linux` |
79988 | 36 |
- **Ubuntu 18.04 LTS** (e.g. via `docker run -it ubuntu:18.04 bash`) |
72366 | 37 |
|
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
38 |
* `x86_64-darwin` |
79988 | 39 |
- **macOS 11 Big Sur** (`mini1` Macmini8,1) |
40 |
- macOS 12 Monterey (untested) |
|
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
41 |
- macOS 13 Ventura (`mini3` Mac14,12 -- MacMini M2 Pro, 6+4 cores) |
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
42 |
- macOS 14 Sonoma (`mini2` Macmini8,1) |
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
43 |
* `arm64-darwin` |
79988 | 44 |
- **macOS 11 Big Sur** (`assur` Macmini9,1 -- MacMini M1, 4+4 cores) |
45 |
- macOS 12 Monterey (untested) |
|
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
46 |
- macOS 13 Ventura (`mini3` Mac14,12 -- MacMini M2 Pro, 6+4 cores) |
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
47 |
- macOS 14 Sonoma (`studio1` Mac13,2 M1 Ultra, 16+4 cores) |
73646 | 48 |
|
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
49 |
* `x86_64-windows` |
79988 | 50 |
- **Windows Server 2012 Rev 2** (`vmnipkow9`) |
51 |
- **Windows 10** |
|
52 |
- Windows 11 |
|
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
53 |
* `x86_64-cygwin` |
79988 | 54 |
- **Cygwin 3.5.x** |
55 |
https://isabelle.sketis.net/cygwin_2024 (`x86_64/release`) |
|
72366 | 56 |
|
36204 | 57 |
|
79988 | 58 |
### Multiple platform personalities ### |
36204 | 59 |
|
79988 | 60 |
Isabelle works with current 64 bit hardware and 64 bit operating systems, |
61 |
which usually means Intel (`x86_64`) and very often ARM (`arm64`). Windows and |
|
62 |
macOS provide `x86_64` emulation on their ARM versions, so that is in theory |
|
63 |
sufficient, but native `arm64` is more efficient. Linux lacks proper |
|
64 |
emulation, so tools should be provided for `x86_64-linux` and `arm64-linux` |
|
65 |
whenever possible. Also note that `arm64-linux` is the standard platform for |
|
66 |
Docker on ARM hardware (e.g. Apple Silicon). |
|
67 |
||
68 |
For extra performance on macOS, Isabelle tools are usually included in both |
|
69 |
variants: `x86_64-darwin` and `arm64-darwin` (or as hybrid executable that |
|
70 |
pretends to be `x86_64-darwin`, the default platform). Windows support is only |
|
71 |
for Intel so far: this could mean `x86_64-windows` or `x86_64-cygwin`, but |
|
72 |
also `x86-windows` for old binary-only tools. |
|
65073 | 73 |
|
79987 | 74 |
The Isabelle settings environment provides variable `ISABELLE_PLATFORM64` to |
79988 | 75 |
refer to the standard POSIX platform personality (Linux/ARM, Linux/Intel, |
76 |
macOS/Intel, Windows/Cygwin64/Intel). Alternative settings are available for |
|
77 |
native platforms as show below. In summary, the symbolic platform names from |
|
78 |
the settings environment are as follows: |
|
79 |
||
80 |
* Linux (Intel) |
|
81 |
- `ISABELLE_PLATFORM64` is `x86_64-linux` |
|
82 |
||
83 |
* Linux (ARM) |
|
84 |
- `ISABELLE_PLATFORM64` is `arm64-linux` |
|
65073 | 85 |
|
79988 | 86 |
* Windows |
87 |
- `ISABELLE_PLATFORM64` is `x86_64-cygwin` |
|
88 |
- `ISABELLE_WINDOWS_PLATFORM64` is `x86_64-windows` |
|
89 |
- `ISABELLE_WINDOWS_PLATFORM32` is `x86-windows` |
|
90 |
||
91 |
* macOS (Intel) |
|
92 |
- `ISABELLE_PLATFORM64` is `x86_64-darwin` |
|
65073 | 93 |
|
79988 | 94 |
* macOS (ARM) |
95 |
- `ISABELLE_PLATFORM64` is `x86_64-darwin` |
|
96 |
- `ISABELLE_APPLE_PLATFORM64` is `arm64-darwin` |
|
97 |
||
98 |
When used outside their proper system context, platform settings remain empty. |
|
99 |
This allows to refer symbolically to various combinations, using conditional |
|
100 |
expressions in GNU `bash` like this: |
|
72895 | 101 |
|
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
102 |
* `"${ISABELLE_WINDOWS_PLATFORM64:-$ISABELLE_PLATFORM64}"` |
79988 | 103 |
-- native Windows, or default POSIX platform (always Intel on macOS) |
72895 | 104 |
|
79988 | 105 |
* `"${ISABELLE_WINDOWS_PLATFORM64:-${ISABELLE_APPLE_PLATFORM64:-$ISABELLE_PLATFORM64}}"` |
106 |
-- native Windows platform, native Apple Silicon platform, or default/native Linux platform |
|
73646 | 107 |
|
65073 | 108 |
|
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
109 |
### Dependable system tools ### |
35610 | 110 |
|
111 |
The following portable system tools can be taken for granted: |
|
112 |
||
79987 | 113 |
* Scala on top of Java. Isabelle/Scala irons out many fine points of the Java |
114 |
platform to make it fully portable as described above. |
|
64339 | 115 |
|
79987 | 116 |
* GNU `bash` as uniform shell on all platforms. The POSIX "standard" shell |
117 |
`/bin/sh` does not work portably -- there are too many non-standard |
|
118 |
implementations of it. On Debian and Ubuntu `/bin/sh` is actually |
|
79988 | 119 |
`/bin/dash` and causes many problems. |
35610 | 120 |
|
121 |
||
79988 | 122 |
### Common problems ### |
123 |
||
124 |
* The traditional `uname` Unix tool only tells about its own executable |
|
125 |
format, not the underlying platform! There are special tricks to get |
|
126 |
underlying platform details, depending on OS versions: Isabelle/Scala and |
|
127 |
the Isabelle settings environment provide sanitized versions of that. |
|
128 |
Isabelle tools should not attempt anything their own account. |
|
129 |
||
130 |
* Common Unix tools like `/bin/sh`, `/bin/kill`, `sed`, `ulimit` are |
|
131 |
notoriously non-portable an should be avoided. |
|
35610 | 132 |
|
79987 | 133 |
* macOS: If Homebrew or MacPorts is installed, there is some danger that |
134 |
accidental references to its shared libraries are created (e.g. `libgmp`). |
|
135 |
Use `otool -L` to check if compiled binaries also work without MacPorts. |
|
41668 | 136 |
|
79988 | 137 |
* macOS as SSH server: The target user shell needs to be set to `/bin/bash` |
138 |
instead of the default `/bin/zsh`, to make shell script escapes work |
|
139 |
reliably. |
|
79985 | 140 |
|
141 |
||
79988 | 142 |
## The Isabelle component repository at TUM and sketis.net ## |
143 |
||
144 |
Isabelle repository versions and administrative tools download components via |
|
145 |
HTTPS from `ISABELLE_COMPONENT_REPOSITORY`, the default is |
|
146 |
`https://isabelle.sketis.net/components`, and alternative is |
|
147 |
`https://isabelle.in.tum.de/components`. |
|
148 |
||
149 |
Isabelle releases have all required components bundled, but additional |
|
150 |
components may be included via suitable manual configuration. |
|
151 |
||
79985 | 152 |
|
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
153 |
### Quick reference ### |
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
154 |
|
79988 | 155 |
The subsequent steps serve as a reminder of how to maintain components: |
79985 | 156 |
|
157 |
* local setup (and test) of component directory, e.g. in |
|
158 |
||
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
159 |
- `screwdriver-3.14/` |
79985 | 160 |
|
161 |
* packaging (with associated SHA1 digest), e.g. |
|
162 |
||
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
163 |
- `$ isabelle components_build screwdriver-3.14` |
79985 | 164 |
|
165 |
* publishing, e.g. |
|
166 |
||
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
167 |
- `$ isabelle components_build -P screwdriver-3.14.tar.gz` |
79985 | 168 |
|
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
169 |
* manual editing of `Admin/components/main`: `screwdriver-3.14` |
79985 | 170 |
|
171 |
||
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
172 |
### Unique names ### |
79985 | 173 |
|
79987 | 174 |
Component names are globally unique over time and space: names of published |
79988 | 175 |
components are never re-used! If some component needs to be re-packaged, extra |
79987 | 176 |
indices may be added to the official version number like this: |
79985 | 177 |
|
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
178 |
* `screwdriver-3.14` -- default packaging/publishing, no index |
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
179 |
* `screwdriver-3.14-1` -- another refinement of the same |
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
180 |
* `screwdriver-3.14-2` -- yet another refinement of the same |
79985 | 181 |
|
79987 | 182 |
There is no standard format for the structure of component names: they are |
79988 | 183 |
compared for equality only, without any guess at an ordering (notions of |
184 |
"older", "newer", "better" etc. are irrelevant). |
|
79985 | 185 |
|
79988 | 186 |
Components are registered in `Admin/components/main` (or similar) for use of |
79987 | 187 |
that particular Isabelle repository version, subject to regular Mercurial |
188 |
history. This allows to bisect Isabelle versions with full record of the |
|
189 |
required components for testing. |
|
79985 | 190 |
|
191 |
||
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
192 |
### Authentic archives ### |
79985 | 193 |
|
79988 | 194 |
TUM provides the shared administrative directory `/p/home/isabelle/components` |
195 |
where the single source of all components is located as authentic `.tar.gz` |
|
196 |
archives. The file `Admin/components/components.sha1` contains SHA1 |
|
197 |
identifiers within the Isabelle repository, for integrity checking of the |
|
198 |
archives that are exposed to the public file-system. The command-line tool |
|
199 |
`isabelle components_build` maintains these hash-keys automatically. |
|
79985 | 200 |
|
79988 | 201 |
Components are published on https://isabelle.sketis.net/components and |
202 |
https://isabelle.in.tum.de/components --- visibility on the web server depends |
|
203 |
on local Unix file permission: nonfree components should omit "read" mode for |
|
204 |
the Unix group/other; regular components should be world-readable. |
|
79985 | 205 |
|
206 |
||
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
207 |
### Unpacked copy ### |
79985 | 208 |
|
79988 | 209 |
A second unpacked copy is provided in `/p/home/isabelle/contrib/`. This allows |
79985 | 210 |
users and administrative services within the TUM network to activate arbitrary |
211 |
snapshots of the repository with all standard components being available, |
|
79988 | 212 |
without extra copying or unpacking of the authentic archives. |
79985 | 213 |
|
214 |
The command-line tool `isabelle components_build -P` takes care of uploading |
|
79986
980cefd8ff9b
more accurate Markdown formatting, both for VSCode and Phabricator;
wenzelm
parents:
79985
diff
changeset
|
215 |
the `.tar.gz` archive and unpacking it, unless it is a special component (e.g. |
79985 | 216 |
for multiplatform application bundling). |
79988 | 217 |
|
218 |
||
219 |
### Repeatable component builds ### |
|
220 |
||
221 |
Historically, Isabelle components have often been assembled manually, packaged |
|
222 |
as `.tar.gz` and uploaded to the administrative directory. This model no |
|
223 |
longer fits the typical complexity of multi-platform tools. |
|
224 |
||
225 |
The current quality standard demands a separate tool in Isabelle/Scala, to |
|
226 |
build a component in a repeatable manner: e.g. see `isabelle component_jdk` or |
|
227 |
`isabelle component_e` with sources in `src/Pure/Admin`. Such tools often |
|
228 |
require a Unix platform (Linux or macOS), or the specific platform for which |
|
229 |
the target is built. In the latter case, the component build tool is run |
|
230 |
manually in each operating-system context, using the base-line versions |
|
231 |
specified above (e.g. via Docker); all results are assembled into one big |
|
232 |
`.tar.gz` archive. |
|
233 |
||
234 |
Multi-platform tools also require thorough testing on all platforms: base-line |
|
235 |
and latest versions. It "works for me on my system" is not sufficient for the |
|
236 |
general public. |