Antonym for 'serial', as in 'a serial publication'

"One-off" is, as you say, slightly British.

"Stand-alone" is frequently used in Britain and North America.

"One-shot" is also used for comics.

A publication that is not serial is a one-time publication. This usage beats out stand-alone even in Ngram’s American corpus. It is official terminology for Canada Post, though lumped together for postal purposes with annual publication. The term is also sometimes used in a slightly different sense, though, associated with the granting or selling of publication rights by an author to a publisher.

For a one time publication of either a scholarly or casual work you can use the term Monograph.

For a magazine, I would use Special edition

Example: We are currently working on a special edition of The Lancet to promote the latest thinking on this critical issue.