Loading HACKING.md +197 −55 Original line number Diff line number Diff line # Road Map There are a few personal goals for this repo. If other people join in, they might have more to add. For now the road map for this repo looks like this: 1. (currently here) Gather the files. So far the only source seems to be decuslib.com. However there might be others so I'll send some more emails over the next few weeks to see. 2. Build a timeline from the various versions that have been acquired. Use that to populate a `historical` branch and the `master` branch as well. 3. Try to document how bulletin looked. One way would be to get it running on an emulator. So far I've only found VMS 7.3. A copy of VMS 4 or 5 would be better. Emulation is really only useful for getting an idea of what it looked like - for me I don't know VMS well enough to secure it for exposure to the net. Alternatively, it might be possible to port it. This gets explored further down, but it would not be easy. 4. Assuming a port isn't done (likely), reimplement in Go. Just the local bulletin functionality initially. And make it simple. Just use the local fs and setuid/gid-type functionality. Mimic the old ISAM VMS stores with protobufs and some sort of file locking. Creating protobuf services to exchange boards could come later. But generally no web, not client-server, just plain-old file-based IPC. ## Emulation * [Step by step guide](http://www.wherry.com/gadgets/retrocomputing/vax-simh.html) for getting OpenVMS 7.3 up and running on simh. * [Notes on networking](https://vanalboom.org/node/18) and clustering on simh. Might not need this. ## Port to Linux While a rewrite is preferred, a port might be possible. Some interesting links on getting VMS FORTRAN running on a Linux/Unix system: * A [script](http://www.simconglobal.com/vms.html) to do this using a proprietary tool called FTP. Need to contact them about licensing. * Things to [look out for](http://labs.hoffmanlabs.com/node/225). ## Recreating the timeline Some thoughts on this. Use announcements for the commit messages. Review the code to figure out something like a changelog. Commit the converted/unpacked raw code in one commit (with author set to Mark London, given the right date, etc); add additional files like a README.md and a ChangeLog in a follow-on commit with the author of those files. The code commits should have the following elements. * Source: zip file name, decuslib url, original path in that zipfile. * File listing with date info. * Announce text. * Any additional notes. # Hacking See [godoc](http://localhost:6060/) for the code layout. Run `make docs` to start the local godoc server. ## Getting started Prerequisites: Go 1.25+ (the project uses tool dependencies via `go tool`). ```sh make # build + all checks (fmt, vet, lint, sec, test) make build # just build make test # tests with coverage make check # fmt, vet, lint, security, test ``` The binary is `./bulletin`. It requires a `-u USER` flag to identify the current user (there's no system-level auth — it's designed to be invoked via an SSH forced command). First-time setup: ```sh ./bulletin -u SYSTEM -b install # create DB, seed SYSTEM user + GENERAL folder ./bulletin -u SYSTEM -b new-user ALICE "Alice Smith" ./bulletin -u ALICE # enter the REPL as ALICE ``` Batch commands (`-b reboot`, `-b expire`) are meant to run from cron. The install command sets up a crontab for the SYSTEM user automatically. ## Project layout ``` main.go CLI entry point (urfave/cli) ask/ User input helpers (readline-based prompts) batch/ Batch/maintenance commands (install, expire, reboot, new-user) dclish/ DCL-like command parser (types, builder, completer, tests) editor/ Built-in text editor (tview-based) folders/ Higher-level folder and message operations key/ SSH authorized_keys management pager/ Terminal pager for long output repl/ REPL loop, command definitions, and all action handlers storage/ Database layer (SQLite, sqlc-generated queries, migrations) this/ Global session state (current user, folder, message pointer) users/ Login name validation decus/ Original VMS Fortran source (historical reference only) ``` ## Architecture ### Session model Each bulletin process serves a single user. The `this` package holds global session state: `this.User`, `this.Folder`, `this.MsgID`, `this.Q` (query handle), `this.Store` (DB connection). The `StartThis` function in `this/this.go` initialises everything — opens the DB, loads or creates the user, selects GENERAL, and runs login alerts. ### Command processing Commands are defined as map literals in `repl/command.go` using the `dclish.Commands` type (a `map[string]*dclish.Command`). At startup, `repl.Loop()` calls `dclish.BuildCommandTable()` to convert the map into a sorted `CommandTable` (slice of entries). This table is used for both parsing (binary search + prefix matching) and tab completion. The flow for a command like `ADD/SUBJECT="hello" myarg`: 1. `Split()` tokenises: `["ADD", "/SUBJECT=hello", "myarg"]` 2. `CommandTable.Find("ADD")` does binary search 3. Flags are looked up via `FlagTable.Find()` — `/NO` entries are auto-generated for toggle flags by the builder 4. Args are collected and validated against `MinArgs`/`MaxArgs` 5. `cmd.Action(cmd)` is called with the parsed `*Command` Subcommands (e.g. `SET BRIEF`, `SHOW FOLDER`, `USER ADD`) work by recursive lookup — the parent command's `CommandTable` is searched for the second word. ### The mail sub-app The `MAIL` command can either send directly (`MAIL username`) or enter a nested REPL with its own prompt (`MAIL>`), history file, command table, and completer. The sub-app is defined in `repl/mail_commands.go`, with actions in `repl/mail_actions.go` and the loop in `repl/mail.go`. It exits by returning an `ErrExitMail` sentinel, similar to how the main REPL uses `ErrExit`. ### Database SQLite via `modernc.org/sqlite` (pure Go, no CGO required for the driver itself — though tests use `-race` which needs CGO). **Migrations** live in `storage/migrations/` and are embedded into the binary with `//go:embed`. They run automatically on `storage.Open()`. To add a migration, create `N_name.up.sql` and `N_name.down.sql` where N is the next sequence number. **Queries** live in `storage/queries/*.sql` using [sqlc](https://docs.sqlc.dev/) annotations. After editing queries, run: ```sh go generate ./... ``` Or just `make` (it runs `go generate ./...` as part of the `build` target) The generated `*.sql.go` files are committed to the repo. The `query_parameter_limit` in `sqlc.yaml` is set to 3, so queries with more than 3 parameters get a params struct. **Key tables**: `users`, `folders`, `messages`, `seen`, `marks`, `folder_access`, `folder_configs`, `system`, `broadcast`, `mail`. ### Adding a new command 1. Add the entry to `repl/command.go` in the `commands` map with description, flags, arg limits, and `Action` function reference. 2. Write the action function in the appropriate `repl/*.go` file. Action functions receive a `*dclish.Command` with parsed `Args` and `Flags`. Access the DB via `this.Q` and `storage.Context()`. 3. If you need new queries, add them to `storage/queries/*.sql` and run `go generate ./...`. If you need schema changes, add a new migration. 4. Run `make` to verify everything. ### Adding a new flag to an existing command Add the flag to the command's `Flags` map in `repl/command.go`: ```go "/MYFLAG": { Description: `Description shown in help.`, }, ``` For a flag that takes a value, set `OptArg: true`. For a flag that defaults to on (user can `/NOMYFLAG` to disable), set `Default: "true"`. The `/NO` negated form is auto-generated by the builder for all non-OptArg flags. In your action function, check `cmd.Flags["/MYFLAG"].Value` (it will be `"true"`, `"false"`, or the assigned value for OptArg flags) and `cmd.Flags["/MYFLAG"].Set` to see if the user explicitly provided it. ## sqlite * sqlite trigger tracing: `.trace stdout --row --profile --stmt --expanded --plain --close` * https://sqlite.org/cli.html * https://www.sqlitetutorial.net/ The database file lives at `$XDG_DATA_HOME/BULLETIN/bulletin.db` (typically `~/.local/share/BULLETIN/bulletin.db`). History files live at `$XDG_CONFIG_HOME/BULLETIN/` (one per user, plus `.mail.history` files for the mail sub-app). ## Testing Run `make test` for tests with coverage, `make coverage` for an HTML report. The `storage.ConfigureTestDB(t)` helper creates a temporary SQLite database with all migrations applied. Use it to write tests that need a real database: ```go func TestSomething(t *testing.T) { storage.ConfigureTestDB(t) defer storage.RemoveTestDB() db, err := storage.Open() // ... set up this.Q, this.User, etc. and test } ``` The `dclish` package has thorough unit tests for the command parser, flag handling, prefix matching, and completion. These don't require a database. ## Code style and conventions * `make fmt` enforces `gofmt`. `make lint` runs `staticcheck`. `make sec` runs `gosec`. * Error messages printed to the user follow the pattern `fmt.Printf("ERROR: %s.\n", err)` — capital ERROR, period at end. * Action functions that encounter user-facing errors (bad input, missing messages) generally print the error and `return nil` rather than returning the error, to avoid the REPL printing it again with a redundant prefix. Return errors for unexpected/internal failures. * The `this` package is intentionally global state — it mirrors the VMS model where one process = one user session. Don't fight it. ## Module links * cli - [docs](https://cli.urfave.org/v3/getting-started/) * readline - [docs](https://pkg.go.dev/github.com/chzyer/readline) * xdg - [docs](https://pkg.go.dev/github.com/adrg/xdg) * sqlite - [docs](https://modernc.org/sqlite) * sqlc - [docs](https://docs.sqlc.dev/) * tview - [docs](https://pkg.go.dev/github.com/rivo/tview) * migrate - [docs](https://pkg.go.dev/github.com/golang-migrate/migrate/v4) HISTORY.md 0 → 100644 +189 −0 Original line number Diff line number Diff line # History This covers the research behind this repository, what it was based on and the decisions made. ## Research Road Map There are a few personal goals for this repo. If other people join in, they might have more to add. For now the road map for this repo looks like this: 1. (currently here) Gather the files. So far the only source seems to be decuslib.com. However there might be others so I'll send some more emails over the next few weeks to see. 2. Build a timeline from the various versions that have been acquired. Use that to populate a `historical` branch and the `master` branch as well. 3. Try to document how bulletin looked. One way would be to get it running on an emulator. So far I've only found VMS 7.3. A copy of VMS 4 or 5 would be better. Emulation is really only useful for getting an idea of what it looked like - for me I don't know VMS well enough to secure it for exposure to the net. Alternatively, it might be possible to port it. This gets explored further down, but it would not be easy. 4. Assuming a port isn't done (likely), reimplement in Go. Just the local bulletin functionality initially. And make it simple. Just use the local fs and setuid/gid-type functionality. Mimic the old ISAM VMS stores with protobufs and some sort of file locking. Creating protobuf services to exchange boards could come later. But generally no web, not client-server, just plain-old file-based IPC. ## Emulation * [Step by step guide](http://www.wherry.com/gadgets/retrocomputing/vax-simh.html) for getting OpenVMS 7.3 up and running on simh. * [Notes on networking](https://vanalboom.org/node/18) and clustering on simh. Might not need this. ## Port to Linux While a rewrite is preferred, a port might be possible. Some interesting links on getting VMS FORTRAN running on a Linux/Unix system: * A [script](http://www.simconglobal.com/vms.html) to do this using a proprietary tool called FTP. Need to contact them about licensing. * Things to [look out for](http://labs.hoffmanlabs.com/node/225). ## Recreating the timeline Some thoughts on this. Use announcements for the commit messages. Review the code to figure out something like a changelog. Commit the converted/unpacked raw code in one commit (with author set to Mark London, given the right date, etc); add additional files like a README.md and a ChangeLog in a follow-on commit with the author of those files. The code commits should have the following elements. * Source: zip file name, decuslib url, original path in that zipfile. * File listing with date info. * Announce text. * Any additional notes. ## Why rewrite? The original BULLETIN is ~25,000 lines of Fortran-77 across 13 source files (bulletin.for through bulletin11.for), plus a VAX MACRO assembly file (allmacs.mar). A direct port to gfortran on Linux was considered but rejected for several reasons: ### It's a VMS application, not a Fortran program The code makes over 1,000 calls to VMS-specific system services. These aren't portable library calls -- they're deep hooks into the VMS kernel, RMS file system, process model, and security architecture: * `SYS$BRKTHRU` for broadcasting to terminals * `SYS$GETJPIW` for process information * `CLI$DCL_PARSE` for the DCL command parser itself * `SYS$CMKRNL` (in the MACRO assembly) for kernel-mode process control block manipulation * `$PRVDEF`, `$UAIDEF` for direct access to VMS authorization database structures * RMS indexed files with record-level locking for all data storage Every single one of these would need a replacement. At that point you're not porting, you're rewriting. ### The data storage is primitive and fragile The original uses RMS indexed files (`*.BULLDIR`, `*.BULLFIL`) with spin-lock retry for file contention: ```fortran DO WHILE (FILE_LOCK(IER,IER1)) OPEN (UNIT=12,FILE='filename',...) END DO ``` No transactions, no crash recovery, corruption-prone under contention. ### The code structure is poor * **231 subroutines across 13 arbitrarily-numbered files** with no logical grouping. DELETE_MSG, DIRECTORY, FILE, and LOGIN all share bulletin0.for for no apparent reason. * **29+ COMMON blocks** for global state, creating invisible coupling between every routine. * **The main loop is a 200-line IF/ELSE IF chain** matching on `INCMD(:3)` and `INCMD(:4)` -- literal string prefix matching on raw command text. * **Date encoding hacks**: deleted messages are flagged by changing the expiration year to `18xx` or `1900`. Permanent messages use `5-NOV-2000`. Shutdown messages encode the node name in the time fields. * **Multi-purpose subroutines**: RESPOND handles four different commands (REPLY, RESPOND, POST, and NEWS posting) in a single 300-line routine differentiated by checking `INCMD(:4)`. * **Minimal error recovery**: most errors just print a message and RETURN, potentially leaving state inconsistent. ### It includes VAX MACRO assembly `allmacs.mar` contains kernel-mode routines for manipulating process control blocks directly. The announcement file explicitly warns that reassembling this file is required when changing VMS versions, and that running the wrong version "can cause your system to crash." ### The license is restrictive The source carries an MIT Plasma Fusion Center copyright that prohibits copying or distribution "for non-MIT use without specific written approval." Even if a port were practical, redistribution would be unclear. ## What the rewrite covers The Go reimplementation preserves the core user experience while dropping features that were VMS-specific or tied to dead protocols. ### Preserved from the original * **The full command set**: ADD, BACK, CHANGE, COPY, CREATE, CURRENT, DELETE, DIRECTORY, EXIT, FIRST, HELP, INDEX, LAST, MARK/UNMARK, MODIFY, MOVE, NEXT, PRINT, READ, REMOVE, REPLY, SEARCH, SEEN/UNSEEN, SELECT, and the SET/SHOW hierarchies. * **DCL command-line semantics**: prefix matching (SH -> SHOW), `/FLAG` and `/FLAG=value` syntax, `/NO` negation for toggle flags, quoted strings, the "type a number to READ" shortcut. * **The folder model**: GENERAL as the default folder, per-folder alert modes (brief/shownew/readnew), visibility levels (public/semiprivate/private), access control, default expiration. * **The message model**: permanent, system, and shutdown messages, per-user seen tracking, marks. * **The broadcast system** for real-time notifications. * **Login alerts** with the original brief/shownew/readnew behavior. ### Intentionally dropped * **DECnet remote folders** (~2,000 lines of REMOTE_* subroutines for reading and writing across DECnet). Dead protocol. * **NEWS/USENET support** (bulletin10.for is entirely NNTP, ~2,300 lines). Dead feature for this use case. * **BBOARD** -- the mail-to-bulletin gateway that polled a VMS MAIL account. * **BULLCP** -- the background daemon for cleanup, notification, and remote folder sync. * **POST** -- a separate command for posting to mailing lists (overlapped with RESPOND). * **FILE/EXTRACT** -- copy a bulletin to a file. * **ATTACH** -- attach to parent process (VMS-specific). * **SET KEYPAD/NOKEYPAD** -- VMS terminal keypad mode. * **SET PRIVILEGES** -- VMS privilege management. * **SUBSCRIBE/UNSUBSCRIBE** -- NEWS-specific subscription management. ### New or improved * **Internal mail system**: the original MAIL command just shells out to VMS MAIL via `LIB$SPAWN('$MAIL SYS$LOGIN:BULL.SCR ...')`. It copies the current bulletin to a temp file and invokes the system mailer as a subprocess. The rewrite implements a proper internal mail system with its own REPL, persistent storage, inbox management, send/read/list/delete/reply/navigate commands, and unread notifications. * **SQLite storage** replacing RMS indexed files, with proper transactions, foreign keys, and migration support. * **SSH key management** for user authentication. * **Structured command parser** (dclish package) replacing the monolithic IF/ELSE IF dispatch with sorted-slice binary search, auto-generated `/NO` toggles, and tab completion including flag completion. NOTES.mddeleted 100644 → 0 +0 −20 Original line number Diff line number Diff line # Development notes These are the development notes for the Go version. The idea is to use the help files to implement BULLETIN. See [godoc](http://localhost:6060/) for the code layout. ## sqlite * sqlite trigger tracing: `.trace stdout --row --profile --stmt --expanded --plain --close` * https://sqlite.org/cli.html * https://www.sqlitetutorial.net/ ## Module links * cli - [docs](https://cli.urfave.org/v3/getting-started/) * readline - [docs](https://pkg.go.dev/github.com/chzyer/readline) * xdg - [docs](https://pkg.go.dev/github.com/adrg/xdg) * sqlite - [docs](https://modernc.org/sqlite) Loading
HACKING.md +197 −55 Original line number Diff line number Diff line # Road Map There are a few personal goals for this repo. If other people join in, they might have more to add. For now the road map for this repo looks like this: 1. (currently here) Gather the files. So far the only source seems to be decuslib.com. However there might be others so I'll send some more emails over the next few weeks to see. 2. Build a timeline from the various versions that have been acquired. Use that to populate a `historical` branch and the `master` branch as well. 3. Try to document how bulletin looked. One way would be to get it running on an emulator. So far I've only found VMS 7.3. A copy of VMS 4 or 5 would be better. Emulation is really only useful for getting an idea of what it looked like - for me I don't know VMS well enough to secure it for exposure to the net. Alternatively, it might be possible to port it. This gets explored further down, but it would not be easy. 4. Assuming a port isn't done (likely), reimplement in Go. Just the local bulletin functionality initially. And make it simple. Just use the local fs and setuid/gid-type functionality. Mimic the old ISAM VMS stores with protobufs and some sort of file locking. Creating protobuf services to exchange boards could come later. But generally no web, not client-server, just plain-old file-based IPC. ## Emulation * [Step by step guide](http://www.wherry.com/gadgets/retrocomputing/vax-simh.html) for getting OpenVMS 7.3 up and running on simh. * [Notes on networking](https://vanalboom.org/node/18) and clustering on simh. Might not need this. ## Port to Linux While a rewrite is preferred, a port might be possible. Some interesting links on getting VMS FORTRAN running on a Linux/Unix system: * A [script](http://www.simconglobal.com/vms.html) to do this using a proprietary tool called FTP. Need to contact them about licensing. * Things to [look out for](http://labs.hoffmanlabs.com/node/225). ## Recreating the timeline Some thoughts on this. Use announcements for the commit messages. Review the code to figure out something like a changelog. Commit the converted/unpacked raw code in one commit (with author set to Mark London, given the right date, etc); add additional files like a README.md and a ChangeLog in a follow-on commit with the author of those files. The code commits should have the following elements. * Source: zip file name, decuslib url, original path in that zipfile. * File listing with date info. * Announce text. * Any additional notes. # Hacking See [godoc](http://localhost:6060/) for the code layout. Run `make docs` to start the local godoc server. ## Getting started Prerequisites: Go 1.25+ (the project uses tool dependencies via `go tool`). ```sh make # build + all checks (fmt, vet, lint, sec, test) make build # just build make test # tests with coverage make check # fmt, vet, lint, security, test ``` The binary is `./bulletin`. It requires a `-u USER` flag to identify the current user (there's no system-level auth — it's designed to be invoked via an SSH forced command). First-time setup: ```sh ./bulletin -u SYSTEM -b install # create DB, seed SYSTEM user + GENERAL folder ./bulletin -u SYSTEM -b new-user ALICE "Alice Smith" ./bulletin -u ALICE # enter the REPL as ALICE ``` Batch commands (`-b reboot`, `-b expire`) are meant to run from cron. The install command sets up a crontab for the SYSTEM user automatically. ## Project layout ``` main.go CLI entry point (urfave/cli) ask/ User input helpers (readline-based prompts) batch/ Batch/maintenance commands (install, expire, reboot, new-user) dclish/ DCL-like command parser (types, builder, completer, tests) editor/ Built-in text editor (tview-based) folders/ Higher-level folder and message operations key/ SSH authorized_keys management pager/ Terminal pager for long output repl/ REPL loop, command definitions, and all action handlers storage/ Database layer (SQLite, sqlc-generated queries, migrations) this/ Global session state (current user, folder, message pointer) users/ Login name validation decus/ Original VMS Fortran source (historical reference only) ``` ## Architecture ### Session model Each bulletin process serves a single user. The `this` package holds global session state: `this.User`, `this.Folder`, `this.MsgID`, `this.Q` (query handle), `this.Store` (DB connection). The `StartThis` function in `this/this.go` initialises everything — opens the DB, loads or creates the user, selects GENERAL, and runs login alerts. ### Command processing Commands are defined as map literals in `repl/command.go` using the `dclish.Commands` type (a `map[string]*dclish.Command`). At startup, `repl.Loop()` calls `dclish.BuildCommandTable()` to convert the map into a sorted `CommandTable` (slice of entries). This table is used for both parsing (binary search + prefix matching) and tab completion. The flow for a command like `ADD/SUBJECT="hello" myarg`: 1. `Split()` tokenises: `["ADD", "/SUBJECT=hello", "myarg"]` 2. `CommandTable.Find("ADD")` does binary search 3. Flags are looked up via `FlagTable.Find()` — `/NO` entries are auto-generated for toggle flags by the builder 4. Args are collected and validated against `MinArgs`/`MaxArgs` 5. `cmd.Action(cmd)` is called with the parsed `*Command` Subcommands (e.g. `SET BRIEF`, `SHOW FOLDER`, `USER ADD`) work by recursive lookup — the parent command's `CommandTable` is searched for the second word. ### The mail sub-app The `MAIL` command can either send directly (`MAIL username`) or enter a nested REPL with its own prompt (`MAIL>`), history file, command table, and completer. The sub-app is defined in `repl/mail_commands.go`, with actions in `repl/mail_actions.go` and the loop in `repl/mail.go`. It exits by returning an `ErrExitMail` sentinel, similar to how the main REPL uses `ErrExit`. ### Database SQLite via `modernc.org/sqlite` (pure Go, no CGO required for the driver itself — though tests use `-race` which needs CGO). **Migrations** live in `storage/migrations/` and are embedded into the binary with `//go:embed`. They run automatically on `storage.Open()`. To add a migration, create `N_name.up.sql` and `N_name.down.sql` where N is the next sequence number. **Queries** live in `storage/queries/*.sql` using [sqlc](https://docs.sqlc.dev/) annotations. After editing queries, run: ```sh go generate ./... ``` Or just `make` (it runs `go generate ./...` as part of the `build` target) The generated `*.sql.go` files are committed to the repo. The `query_parameter_limit` in `sqlc.yaml` is set to 3, so queries with more than 3 parameters get a params struct. **Key tables**: `users`, `folders`, `messages`, `seen`, `marks`, `folder_access`, `folder_configs`, `system`, `broadcast`, `mail`. ### Adding a new command 1. Add the entry to `repl/command.go` in the `commands` map with description, flags, arg limits, and `Action` function reference. 2. Write the action function in the appropriate `repl/*.go` file. Action functions receive a `*dclish.Command` with parsed `Args` and `Flags`. Access the DB via `this.Q` and `storage.Context()`. 3. If you need new queries, add them to `storage/queries/*.sql` and run `go generate ./...`. If you need schema changes, add a new migration. 4. Run `make` to verify everything. ### Adding a new flag to an existing command Add the flag to the command's `Flags` map in `repl/command.go`: ```go "/MYFLAG": { Description: `Description shown in help.`, }, ``` For a flag that takes a value, set `OptArg: true`. For a flag that defaults to on (user can `/NOMYFLAG` to disable), set `Default: "true"`. The `/NO` negated form is auto-generated by the builder for all non-OptArg flags. In your action function, check `cmd.Flags["/MYFLAG"].Value` (it will be `"true"`, `"false"`, or the assigned value for OptArg flags) and `cmd.Flags["/MYFLAG"].Set` to see if the user explicitly provided it. ## sqlite * sqlite trigger tracing: `.trace stdout --row --profile --stmt --expanded --plain --close` * https://sqlite.org/cli.html * https://www.sqlitetutorial.net/ The database file lives at `$XDG_DATA_HOME/BULLETIN/bulletin.db` (typically `~/.local/share/BULLETIN/bulletin.db`). History files live at `$XDG_CONFIG_HOME/BULLETIN/` (one per user, plus `.mail.history` files for the mail sub-app). ## Testing Run `make test` for tests with coverage, `make coverage` for an HTML report. The `storage.ConfigureTestDB(t)` helper creates a temporary SQLite database with all migrations applied. Use it to write tests that need a real database: ```go func TestSomething(t *testing.T) { storage.ConfigureTestDB(t) defer storage.RemoveTestDB() db, err := storage.Open() // ... set up this.Q, this.User, etc. and test } ``` The `dclish` package has thorough unit tests for the command parser, flag handling, prefix matching, and completion. These don't require a database. ## Code style and conventions * `make fmt` enforces `gofmt`. `make lint` runs `staticcheck`. `make sec` runs `gosec`. * Error messages printed to the user follow the pattern `fmt.Printf("ERROR: %s.\n", err)` — capital ERROR, period at end. * Action functions that encounter user-facing errors (bad input, missing messages) generally print the error and `return nil` rather than returning the error, to avoid the REPL printing it again with a redundant prefix. Return errors for unexpected/internal failures. * The `this` package is intentionally global state — it mirrors the VMS model where one process = one user session. Don't fight it. ## Module links * cli - [docs](https://cli.urfave.org/v3/getting-started/) * readline - [docs](https://pkg.go.dev/github.com/chzyer/readline) * xdg - [docs](https://pkg.go.dev/github.com/adrg/xdg) * sqlite - [docs](https://modernc.org/sqlite) * sqlc - [docs](https://docs.sqlc.dev/) * tview - [docs](https://pkg.go.dev/github.com/rivo/tview) * migrate - [docs](https://pkg.go.dev/github.com/golang-migrate/migrate/v4)
HISTORY.md 0 → 100644 +189 −0 Original line number Diff line number Diff line # History This covers the research behind this repository, what it was based on and the decisions made. ## Research Road Map There are a few personal goals for this repo. If other people join in, they might have more to add. For now the road map for this repo looks like this: 1. (currently here) Gather the files. So far the only source seems to be decuslib.com. However there might be others so I'll send some more emails over the next few weeks to see. 2. Build a timeline from the various versions that have been acquired. Use that to populate a `historical` branch and the `master` branch as well. 3. Try to document how bulletin looked. One way would be to get it running on an emulator. So far I've only found VMS 7.3. A copy of VMS 4 or 5 would be better. Emulation is really only useful for getting an idea of what it looked like - for me I don't know VMS well enough to secure it for exposure to the net. Alternatively, it might be possible to port it. This gets explored further down, but it would not be easy. 4. Assuming a port isn't done (likely), reimplement in Go. Just the local bulletin functionality initially. And make it simple. Just use the local fs and setuid/gid-type functionality. Mimic the old ISAM VMS stores with protobufs and some sort of file locking. Creating protobuf services to exchange boards could come later. But generally no web, not client-server, just plain-old file-based IPC. ## Emulation * [Step by step guide](http://www.wherry.com/gadgets/retrocomputing/vax-simh.html) for getting OpenVMS 7.3 up and running on simh. * [Notes on networking](https://vanalboom.org/node/18) and clustering on simh. Might not need this. ## Port to Linux While a rewrite is preferred, a port might be possible. Some interesting links on getting VMS FORTRAN running on a Linux/Unix system: * A [script](http://www.simconglobal.com/vms.html) to do this using a proprietary tool called FTP. Need to contact them about licensing. * Things to [look out for](http://labs.hoffmanlabs.com/node/225). ## Recreating the timeline Some thoughts on this. Use announcements for the commit messages. Review the code to figure out something like a changelog. Commit the converted/unpacked raw code in one commit (with author set to Mark London, given the right date, etc); add additional files like a README.md and a ChangeLog in a follow-on commit with the author of those files. The code commits should have the following elements. * Source: zip file name, decuslib url, original path in that zipfile. * File listing with date info. * Announce text. * Any additional notes. ## Why rewrite? The original BULLETIN is ~25,000 lines of Fortran-77 across 13 source files (bulletin.for through bulletin11.for), plus a VAX MACRO assembly file (allmacs.mar). A direct port to gfortran on Linux was considered but rejected for several reasons: ### It's a VMS application, not a Fortran program The code makes over 1,000 calls to VMS-specific system services. These aren't portable library calls -- they're deep hooks into the VMS kernel, RMS file system, process model, and security architecture: * `SYS$BRKTHRU` for broadcasting to terminals * `SYS$GETJPIW` for process information * `CLI$DCL_PARSE` for the DCL command parser itself * `SYS$CMKRNL` (in the MACRO assembly) for kernel-mode process control block manipulation * `$PRVDEF`, `$UAIDEF` for direct access to VMS authorization database structures * RMS indexed files with record-level locking for all data storage Every single one of these would need a replacement. At that point you're not porting, you're rewriting. ### The data storage is primitive and fragile The original uses RMS indexed files (`*.BULLDIR`, `*.BULLFIL`) with spin-lock retry for file contention: ```fortran DO WHILE (FILE_LOCK(IER,IER1)) OPEN (UNIT=12,FILE='filename',...) END DO ``` No transactions, no crash recovery, corruption-prone under contention. ### The code structure is poor * **231 subroutines across 13 arbitrarily-numbered files** with no logical grouping. DELETE_MSG, DIRECTORY, FILE, and LOGIN all share bulletin0.for for no apparent reason. * **29+ COMMON blocks** for global state, creating invisible coupling between every routine. * **The main loop is a 200-line IF/ELSE IF chain** matching on `INCMD(:3)` and `INCMD(:4)` -- literal string prefix matching on raw command text. * **Date encoding hacks**: deleted messages are flagged by changing the expiration year to `18xx` or `1900`. Permanent messages use `5-NOV-2000`. Shutdown messages encode the node name in the time fields. * **Multi-purpose subroutines**: RESPOND handles four different commands (REPLY, RESPOND, POST, and NEWS posting) in a single 300-line routine differentiated by checking `INCMD(:4)`. * **Minimal error recovery**: most errors just print a message and RETURN, potentially leaving state inconsistent. ### It includes VAX MACRO assembly `allmacs.mar` contains kernel-mode routines for manipulating process control blocks directly. The announcement file explicitly warns that reassembling this file is required when changing VMS versions, and that running the wrong version "can cause your system to crash." ### The license is restrictive The source carries an MIT Plasma Fusion Center copyright that prohibits copying or distribution "for non-MIT use without specific written approval." Even if a port were practical, redistribution would be unclear. ## What the rewrite covers The Go reimplementation preserves the core user experience while dropping features that were VMS-specific or tied to dead protocols. ### Preserved from the original * **The full command set**: ADD, BACK, CHANGE, COPY, CREATE, CURRENT, DELETE, DIRECTORY, EXIT, FIRST, HELP, INDEX, LAST, MARK/UNMARK, MODIFY, MOVE, NEXT, PRINT, READ, REMOVE, REPLY, SEARCH, SEEN/UNSEEN, SELECT, and the SET/SHOW hierarchies. * **DCL command-line semantics**: prefix matching (SH -> SHOW), `/FLAG` and `/FLAG=value` syntax, `/NO` negation for toggle flags, quoted strings, the "type a number to READ" shortcut. * **The folder model**: GENERAL as the default folder, per-folder alert modes (brief/shownew/readnew), visibility levels (public/semiprivate/private), access control, default expiration. * **The message model**: permanent, system, and shutdown messages, per-user seen tracking, marks. * **The broadcast system** for real-time notifications. * **Login alerts** with the original brief/shownew/readnew behavior. ### Intentionally dropped * **DECnet remote folders** (~2,000 lines of REMOTE_* subroutines for reading and writing across DECnet). Dead protocol. * **NEWS/USENET support** (bulletin10.for is entirely NNTP, ~2,300 lines). Dead feature for this use case. * **BBOARD** -- the mail-to-bulletin gateway that polled a VMS MAIL account. * **BULLCP** -- the background daemon for cleanup, notification, and remote folder sync. * **POST** -- a separate command for posting to mailing lists (overlapped with RESPOND). * **FILE/EXTRACT** -- copy a bulletin to a file. * **ATTACH** -- attach to parent process (VMS-specific). * **SET KEYPAD/NOKEYPAD** -- VMS terminal keypad mode. * **SET PRIVILEGES** -- VMS privilege management. * **SUBSCRIBE/UNSUBSCRIBE** -- NEWS-specific subscription management. ### New or improved * **Internal mail system**: the original MAIL command just shells out to VMS MAIL via `LIB$SPAWN('$MAIL SYS$LOGIN:BULL.SCR ...')`. It copies the current bulletin to a temp file and invokes the system mailer as a subprocess. The rewrite implements a proper internal mail system with its own REPL, persistent storage, inbox management, send/read/list/delete/reply/navigate commands, and unread notifications. * **SQLite storage** replacing RMS indexed files, with proper transactions, foreign keys, and migration support. * **SSH key management** for user authentication. * **Structured command parser** (dclish package) replacing the monolithic IF/ELSE IF dispatch with sorted-slice binary search, auto-generated `/NO` toggles, and tab completion including flag completion.
NOTES.mddeleted 100644 → 0 +0 −20 Original line number Diff line number Diff line # Development notes These are the development notes for the Go version. The idea is to use the help files to implement BULLETIN. See [godoc](http://localhost:6060/) for the code layout. ## sqlite * sqlite trigger tracing: `.trace stdout --row --profile --stmt --expanded --plain --close` * https://sqlite.org/cli.html * https://www.sqlitetutorial.net/ ## Module links * cli - [docs](https://cli.urfave.org/v3/getting-started/) * readline - [docs](https://pkg.go.dev/github.com/chzyer/readline) * xdg - [docs](https://pkg.go.dev/github.com/adrg/xdg) * sqlite - [docs](https://modernc.org/sqlite)
