Differences from the reference compiler

The goal of this rewrite is to be byte-identical to the reference C compiler (CompuPhase Pawn 4.1.7599). Validating that means running both against the same inputs and comparing the results, which surfaces two kinds of difference — and both are recorded here, so the picture is symmetric.

Part 1 — bugs in the reference: defects found in the upstream compiler, up to and including a segfault on every multi-source invocation. Some are reproduced on purpose (imitating them is what keeps the bytecode identical), others are not (imitating a crash buys nothing).

Part 2 — divergences on this side: places where this port behaves differently from upstream, whether because Rust does not have the construct the C code relies on, or because a subsystem is not ported yet. These are ours, not upstream's.

Short excerpts of the upstream source are quoted where they are needed to follow the argument; the reference compiler is © CompuPhase and licensed under Apache-2.0 — see NOTICE.md.

Part 1 — bugs in the reference compiler

Each report names the file and function in the upstream tree, describes the defect, gives a reproducer, states the impact and suggests a fix. None of these have been reported upstream by this project yet.

How this port treats each one

Reproduced deliberately — the defect changes the bytes of the .amx, and imitating it is what keeps the output byte-identical. Applies to Bugs 1 and 2, both in the amxdbg debug section emitted under -d2. The places in the Rust source that imitate them carry an [[atualizar-apos-fix-upstream]] marker naming the bug, so they can be found when upstream fixes it.
Not reproduced — the defect is a hang, a crash or a wrong diagnostic, with no effect on the .amx of a valid program. Imitating it would buy nothing, so this port does the right thing and diverges. Applies to Bugs 3, 4, 5, 6 and 7.

Bug 1 — `AMX_DBG_HDR.size` undercounts the `AMX_DBG_SYMDIM` entries

Where: compiler/sc6.c, append_dbginfo() (the sizing pass, ~lines 1205-1207).

When sizing the debug block, the space taken by array dimensions (AMX_DBG_SYMDIM) is counted by looking for a ':' in the symbol's name:

if ((prevstr=strchr(name,'['))!=NULL)
  while ((prevstr=strchr(prevstr+1,':'))!=NULL)
    dbghdr.size+=sizeof(AMX_DBG_SYMDIM);

But the debug string built by insert_dbgsymbol() (sclist.c) writes the dimensions as space-separated values, with no ':':

strcat(string," [ ");
for (sub=sym; sub!=NULL; sub=finddepend(sub))
  sprintf(string+strlen(string),"%x ",(unsigned)sub->dim.array.length);
strcat(string,"]");

With no ':' to find, the counting loop never runs and no SYMDIM is added to dbghdr.size — while the writing pass (~lines 1337-1344) does emit one AMX_DBG_SYMDIM per dimension.

Impact: the size field of the header is smaller than the bytes actually written, by sizeof(AMX_DBG_SYMDIM) (6 bytes) per array dimension, whenever a program with arrays is compiled with -d2. A consumer that trusts size to find the end of the section reads the wrong offset.

Suggested fix: count the SYMDIM entries the same way they are written — by the values after the '[', separated by spaces — instead of looking for ':'.

Bug 2 — `AMX_DBG_SYMDIM.tag` is written from uninitialized memory

Where: compiler/sc6.c, append_dbginfo() (the writing pass, ~lines 1315-1344).

Each symbol's dimensions are read into the local array AMX_DBG_SYMDIM dbgidxtag[sDIMEN_MAX];, but only the .size field is ever filled in:

if (*str=='[') {
  while (*(str=skipwhitespace(str+1))!=']') {
    dbgidxtag[dbgsym.dim].size=(uint32_t)hex2ucell(str,&str);
    dbgsym.dim++;
  }
}

.tag is never assigned. Since dbgidxtag is an uninitialized stack array, the bytes written for each dimension's tag (~line 1342, pc_writebin(fout,&dbgidxtag[dim].tag,...)) are stack garbage — undefined behaviour.

Impact: .amx files built with -d2 for programs with arrays contain non-deterministic bytes in the dimension tag field. In practice, for this particular binary, the value is stable (slot 0 holds 1, the rest 0), but that depends on the stack layout of the compiler and platform that built it, so it is not portably reproducible.

Suggested fix: initialize dbgidxtag[].tag, either with the dimension's real index tag or explicitly with 0.

Bug 3 — `stgopt` aborts on an inconsistent staging buffer during error recovery

Where: compiler/sc7.c, stgopt() (line 777).

The peephole optimizer opens with an assertion:

assert(start!=NULL && end!=NULL && end>=start);

Compiling certain invalid inputs involving state variables (new x <a>;) that produce errors 021/017 lets the staging buffer reach stgopt() inconsistent (end < start, or a null pointer), firing the assertion and aborting with SIGABRT.

Reproducer (the trigger is non-deterministic — it depends on the heap state along the error-recovery path):

new count <a>;
new count <b>;
foo() <a> { return count; }
foo() <b> { return count; }
main() { state a; count=5; return foo(); }

Observed output:

sc7.c:777: stgopt: Assertion `start!=NULL && end!=NULL && end>=start' failed.

Impact: in a debug build (assertions on) the compiler aborts instead of reporting the diagnostics and stopping cleanly. In a release build (NDEBUG) the assertion is gone, but the underlying inconsistency (end < start) likely leads to undefined behaviour in the peephole loop. It points at a latent bug in the error-recovery path for state variables.

Suggested fix: when code generation is abandoned because of an error — especially in invalid state-variable declarations — leave the staging buffer consistent (start == end), or skip stgopt while errors are pending.

Bug 4 — infinite loop when a directory is opened as a source or include file

Where: compiler/libpawnc.c (pc_opensrc/pc_readsrc/pc_eofsrc) and compiler/sc2.c (readline).

On POSIX, fopen(path, "rt") on a directory succeeds (it returns a non-null pointer). The following read fails with EISDIR, which surfaces as fgets → NULL and feof → 0 — it is an error, not end of file.

Chained with the source:

void *pc_opensrc(const char *f) { return fopen(f,"rt"); }  /* opens the FOLDER successfully */
char *pc_readsrc(void *h,...)   { return fgets(...,h); }    /* → NULL forever               */
int   pc_eofsrc(void *h)        { return feof(h); }         /* → 0 forever (it is an error) */

And the end-of-file test in readline (sc2.c) only consults feof:

if (inpf==NULL || pc_eofsrc(inpf))   /* NEVER true for a directory */

Since feof never becomes true, the compiler never pops the file: it keeps reading "empty lines" forever without making progress.

Two defects compound here:

pc_opensrc does not reject directories (on POSIX, fopen on a folder succeeds).
The reader consults feof() but ignores ferror() — it treats a read error as "not EOF yet, keep trying", and never terminates.

Impact: any input that is a directory — the main source or an #include that resolves to a folder — hangs the compiler. Confirmed by test.

Suggested fix: in the equivalent of pc_opensrc, check for a directory and treat it as "not found"; and/or treat a read error (not EOF) as a failure. This port is immune by construction, since Rust's file API returns Err for a directory instead of a "valid" handle that reads empty. As a side effect, name.inc and a name/ folder can coexist here, which upstream does not allow.

Bug 5 — `double free` when aborting over an exceeded size limit (errors 106/112)

Where: compiler/sc1.c, pc_compile() (~lines 800-808), in the #if !defined PAWN_LIGHT memory-report block.

Errors 112 (overlay function exceeds the limit) and 106 (script exceeds the maximum memory size) are emitted after the .amx has been assembled and the report printed. Both are fatal (numbers 100-199), and a fatal error longjmps to the cleanup label. The jump lands with structures already freed — or about to be freed again — along the normal shutdown path, and the release happens twice.

Reproducers (both deterministic, 3 out of 3 runs):

/* error 106 */
#pragma amxlimit 200
main() {
    new arr[512] = {1, ...};
    return arr[0];
}

/* error 112: compile with -V32 */
big() {
    new a = 1, b = 2, c = 3, d = 4, e = 5;
    a = a + b * c - d / e;
    b = a * a + b * b + c * c;
    c = a - b + c - d + e;
    d = (a + b) * (c + d) - e;
    e = a * b * c * d * e;
    return a + b + c + d + e;
}
main() { return big(); }

Observed output (both cases):

... : fatal error 106: compiled script exceeds the maximum memory size (200 bytes)

Compilation aborted.free(): double free detected in tcache 2

Exit status: 134 (SIGABRT).

Impact: the compiler aborts with heap corruption instead of exiting cleanly. The diagnostic does get printed, but the summary line (1 Error.) does not, and the exit code is a crash (134) rather than a compilation failure — a build tool checking the exit status cannot tell "memory limit exceeded" from "the compiler broke". The double free is undefined behaviour: on another libc or platform it may not abort at all and instead keep corrupting memory silently.

Suggested fix: free each resource exactly once along cleanup (nulling the pointers after free), or emit 106/112 before the release sequence starts. This port reports the same diagnostics, with the same numbers, and exits normally with the summary and a failure status.

Bug 6 — the name set by `#file` leaks into later passes

Where: compiler/sc2.c, the tpFILE case of command() (~lines 1079-1096), together with the pass loop in pc_compile() (sc1.c).

#file "name" replaces the global inpfname (free(inpfname); inpfname=duplicatestring(pathname);). The compiler makes several passes over the same source, and when restarting a pass it restores the input file (inpf=inpf_org; pc_resetsrc(...)) but not inpfname. The name set by the last #file of the previous pass still applies from the first line of the next one.

Reproducer:

main() {
    return nosuch1;
}
#file "renamed.pwn"
foo() {
    return nosuch2;
}

Observed output:

renamed.pwn(2) : error 017: undefined symbol "nosuch1"

The error is on line 2, before the #file on line 4, and is still attributed to renamed.pwn. (This same input also triggers Bug 3: the stgopt assertion aborts the compiler right after.)

Impact: diagnostics point at the wrong file. Since #file exists precisely so that code generators can map messages back to the original source, the effect is the opposite of the intent: the part before the remapping gets attributed to the remapped file.

Suggested fix: save inpfname before the first pass and restore it along with inpf when each pass restarts. This port keeps the name in the input-file frame, which is rebuilt per pass, and reports fchk.pwn(2) and renamed.pwn(5) correctly.

Bug 7 — any multi-source invocation segfaults

Where: compiler/sc1.c, pc_compile() (~lines 543-566), the POSIX branch that builds the temporary file.

Given two or more source files, upstream concatenates them into a temporary file, separating them with #file/#line so diagnostics keep pointing at the original source. On POSIX the temporary's name is obtained like this:

char mask[] = "pawn.XXXXXX";
int tmpfd = mkstemp(mask);
char *procname = alloca(64);
tmpname = alloca(PATH_MAX);
sprintf(procname, "/proc/self/fd/%d", tmpfd);
readlink(procname, tmpname, PATH_MAX);
close(tmpfd);

readlink() does not NUL-terminate its output, and its return value — the number of bytes written, or -1 — is discarded. tmpname comes from alloca, so it is uninitialized: whatever follows the path is garbage. Every subsequent use of tmpname (pc_createsrc(tmpname), strcpy(inpfname,tmpname), remove(tmpname)) reads past the end of the string.

Reproducer — two perfectly valid files, no compilation error involved:

/* lib.pwn */
helper(x) { return x * 2; }

/* app.pwn */
main() { return helper(21); }

$ pawncc lib.pwn app.pwn
Segmentation fault
$ echo $?
139

Deterministic: 3 out of 3 runs, with and without errors in the sources.

Impact: the multi-source feature is unusable on POSIX — the compiler crashes before producing anything. Since #file/#line exist precisely to support it, the feature they serve cannot be reached. Being undefined behaviour, on another platform it may not crash and instead write to, or delete, a path built from stack garbage.

Suggested fix: terminate the string with the byte count readlink() returns, and check that return value:

ssize_t n = readlink(procname, tmpname, PATH_MAX - 1);
if (n < 0)
  error(103);           /* or a dedicated diagnostic */
tmpname[n] = '\0';

This port does not need a temporary file at all: the sources are concatenated in memory, with the same #file/#line separators, so there is nothing to name, race on or clean up. Because upstream crashes, it cannot act as the oracle here; the equivalence is checked instead — N sources must produce exactly the bytecode of one source #includeing the others, which is validated against the reference (tests/multi_source.rs).

Note — the reference build has assertions enabled

The reference-c/build/pawncc used for validation is built with assert() active, so some inputs (assert 1==1; in the source, for one) raise SIGABRT instead of producing a diagnostic. That is partly expected of a debug build — but Bug 3 shows that at least one of those assertions is reporting a real state inconsistency along the error-recovery path.

Part 2 — divergences on this side

Ways in which this port does not match upstream. None of them affect the bytecode of a program that compiles on both, which is why the golden suite is green — but they are real differences and a user is entitled to know about them.

Out-of-memory reports `fatal error 103` in C, and aborts here

Upstream checks the result of every allocation:

sym = (symbol*)malloc(sizeof(symbol));
if (sym == NULL)
  error(103);   /* insufficient memory */

That pattern needs an allocator that returns something inspectable. Rust's global allocator does not: on failure it calls handle_alloc_error, which aborts the process. A Vec::push has no failure path to hand back, so there is no point at which a fatal error 103 could be emitted — by the time control would return, the process is gone.

Rust does offer fallible allocation (try_reserve, allocator_api), but using it would mean replacing every allocation in the crate with a fallible variant and threading the error through the whole call tree.

Observable difference: under genuine OOM, upstream prints fatal error 103 and exits with a failure status; this port aborts with memory allocation of N bytes failed and SIGABRT. Both fail; the message and exit status differ.

Unsupported command-line options are accepted and ignored

This is the one worth knowing about, because the compiler currently misleads by silence. The option parser ends with a catch-all that swallows anything it does not recognize, and -C accepts only 64 as a special case:

} else if let Some(v) = arg.strip_prefix("-C") {
    opts.cell_size = if v == "64" { CellSize::Bits64 } else { CellSize::Bits32 };
…
} else if arg.starts_with('-') {
    // unsupported flag: ignored

The practical consequences:

invocation	reference	this port
`-C16`	compiles with 16-bit cells	compiles with 32-bit cells, silently
`-C99` (invalid)	rejects, prints the usage	compiles with 32-bit cells, silently
`-c<name>` (codepage)	loads the mapping file, or `fatal error 108` if missing	ignored, silently

So a build asking for 16-bit cells or a codepage gets something else and is told nothing. Two diagnostics are unreachable as a direct result:

error 104 (incompatible options) fires upstream for -k together with cells narrower than 32 bits (pawncc -C16 -k1234 → "incompatible options: \"-k\" versus \"cell size < 32-bits\""). Without 16-bit cells there is no way to reach it. Its other call site is inside a #if defined(MACOS) && !defined(__MACH__) block — classic Mac OS, dead code in any current build.
errors 108 and 109 (codepage mapping file not found, invalid path) belong to the codepage file loader. frontend/codepage.rs implements the translation itself, but nothing loads a mapping file from disk: there is no -c, no #pragma codepage, and no install-path lookup. Porting the diagnostics without the loader would mean emitting errors that can never fire.

Neither 16-bit cells nor the codepage loader is a diagnostic — each is a subsystem that changes what the compiler accepts as input, and each deserves to be its own piece of work. Rejecting the options that are not supported, instead of ignoring them, does not depend on either and is the fix that removes the misleading part.