# Speakmac 27-clip licensed real-speech reproducibility bundle

This is the licensed real-human subset of the fixed corpus used for the July
28, 2026 Speakmac/Wispr Flow comparison. It contains the exact 27 normalized
CC BY 4.0 WAV files, one reference file per clip, source attribution, file
hashes, the historical paired outputs, and a zero-dependency runner/scorer.

The runner verifies every pinned file and WAV format before doing any work. It
then requires Speakmac 5.5.0 build 61, Parakeet, and the on-device Clean Up
Dictation model; runs each file with `--stdout --clean --no-history`; saves only
stdout; and compares the result with the published Speakmac 5.2.1 build 55
baseline.

From this directory, reproduce the release regression with one command:

```sh
python3 scripts/benchmark.py
```

The command uses only Python's standard library. Progress and CLI diagnostics
stay on stderr and are not written into result files. New output is written to
`results/local-speakmac-5.5.0-build-61*.{json,csv}`. Existing result files are
never overwritten.

## Bundle contents

- `audio/`: 27 exact 16 kHz, mono, 16-bit PCM WAV inputs.
- `reference/`: one UTF-8 reference transcript per WAV.
- `manifest.json`: relative paths, SHA-256 hashes, selection metadata, and
  source-level provenance.
- `checksums.sha256`: hashes for every pinned static input and script.
- `paired-outputs.json`, `per-sample.csv`, `summary.json`: privacy-clean
  historical Speakmac 5.2.1 / Wispr Flow 1.6.224 results. These historical
  files retain the original 37 text rows, including 10 controlled rows whose
  Apple System Voice audio cannot be redistributed.
- `results/`: the dated Speakmac 5.5.0 build 61 stdout and release-regression
  scores.
- `ATTRIBUTION.md`: licenses, creator credits, change notices, and provenance
  boundaries.

## Scoring contract

Word error rate is micro-averaged across each segment. Text is lowercased,
ASCII punctuation is removed, repeated whitespace is collapsed, and the word
sequence is scored with Levenshtein distance. The script first reproduces the
published 22 errors across 349 words (6.3% WER) for Speakmac 5.2.1 on the 27
licensed real-human clips; a mismatch
stops the run.

## Why the controlled audio is absent

The original experiment also used 10 stress clips rendered with macOS System
Voices. Apple's macOS license permits System Voice output for personal,
non-commercial projects and does not permit public or commercial publishing or
redistribution. Those WAVs, their reference files, and their Speakmac 5.5.0
outputs are therefore excluded from this public bundle and release regression.
The 37-row historical text tables remain for audit context only; they are not a
fully reproducible public audio corpus.

The saved 5.5.0 result is a same-corpus Speakmac release regression. It is not a
new Wispr Flow run, a current price check, or evidence that either app is a
universal accuracy winner.

The dated release result returned 27 of 27 non-empty outputs and scored 22
errors across 349 words (6.3% WER), the same aggregate as 5.2.1 build 55. Three
per-clip error counts changed—two improved and one regressed—with zero net
error change.