# Local AI Sprite Generation

This project uses a local ComfyUI workflow to generate and install baseball
player sprites. The current production path is Flux2 Klein 4B FP8 plus the
Limbicnation pixel-art LoRA, then a repo script cleans the keyed background,
builds 128 px cells, patches runtime sprite sheets, and annotates
`public/assets/generated/manifest.json`.

Use this file when replacing or extending player sprite rows.

## Current Model Stack

The code expects ComfyUI at:

```bash
/home/jetson/ComfyUI
```

The runtime row generator uses these model filenames:

| Purpose | Expected ComfyUI name | Current local target |
| --- | --- | --- |
| Flux2 image model | `models/diffusion_models/flux-2-klein-4b-fp8.safetensors` | `/home/jetson/ComfyUI/models/_downloads/flux2-klein-fp8/flux-2-klein-4b-fp8.safetensors` |
| Qwen text encoder | `models/text_encoders/qwen_3_4b_fp4_flux2.safetensors` | `/home/jetson/ComfyUI/models/_downloads/flux2-klein/split_files/text_encoders/qwen_3_4b_fp4_flux2.safetensors` |
| Flux2 VAE | `models/vae/flux2-vae.safetensors` | `/home/jetson/ComfyUI/models/_downloads/flux2-klein/split_files/vae/flux2-vae.safetensors` |
| Pixel art LoRA | `models/loras/pixel-art-lora.safetensors` | `/home/jetson/ComfyUI/models/loras/Limbicnation_pixel-art-lora/pytorch_lora_weights.comfyui.safetensors` |
| Background removal fallback | `models/background_removal/birefnet.safetensors` | `/home/jetson/ComfyUI/models/background_removal/birefnet.safetensors` |

The LoRA is from:

```text
https://huggingface.co/Limbicnation/pixel-art-lora
```

The local LoRA README says the intended base is
`black-forest-labs/FLUX.2-klein-4B`, with 4 inference steps, CFG 1.0, Euler
sampling, and the trigger phrase `pixel art sprite`.

## One-Time Setup Checks

From the repo root:

```bash
cd /home/jetson/benchmarks/baseball-classic
```

Confirm the files ComfyUI will load:

```bash
find /home/jetson/ComfyUI/models/diffusion_models \
  /home/jetson/ComfyUI/models/text_encoders \
  /home/jetson/ComfyUI/models/vae \
  /home/jetson/ComfyUI/models/loras \
  -maxdepth 2 -printf '%M %p -> %l\n' | sort
```

If the LoRA needs to be reinstalled:

```bash
huggingface-cli download Limbicnation/pixel-art-lora \
  pytorch_lora_weights.comfyui.safetensors \
  README.md LICENSE config.json config.yaml comfyui_workflow.json \
  --local-dir /home/jetson/ComfyUI/models/loras/Limbicnation_pixel-art-lora

ln -sf \
  /home/jetson/ComfyUI/models/loras/Limbicnation_pixel-art-lora/pytorch_lora_weights.comfyui.safetensors \
  /home/jetson/ComfyUI/models/loras/pixel-art-lora.safetensors
```

The base model, text encoder, and VAE must keep the exact symlink names in the
table above. The Python scripts reference those names directly.

## Primary Workflow: Patch One Runtime Animation Row

Use `tools/generate_flux2_runtime_pose_row_with_retries.py` for production
work. It generates each of the six frames one at a time, retries failed frames
at alternate sizes, builds a contact sheet, and optionally installs the row.

Example:

```bash
python3 tools/generate_flux2_runtime_pose_row_with_retries.py \
  --clip blueLFRunBack \
  --widths 320,256 \
  --timeout 900 \
  --install
```

What this does:

- Starts ComfyUI on `127.0.0.1:8188` unless `--no-start-comfy` is passed.
- Uses `steps=4`, `cfg=1.0`, and `lora-strength=1.12` by default.
- Writes frame artifacts to `qa/player-sprite-ai-rows/<role>/<clip>/`.
- Writes `raw.png`, `alpha.png`, `cell.png`, and `prompt.json` for each frame.
- Writes `contact-sheet.png` for the row.
- Patches the correct cells in `public/assets/generated/<role>FullSheet.png`.
- Adds `aiPoseRow` metadata to the matching animation clip in the manifest.

For a preview without patching runtime assets, generate a selected frame:

```bash
python3 tools/generate_flux2_runtime_pose_row.py \
  --clip blueLFRunBack \
  --frames 1 \
  --width 320 \
  --height 320 \
  --start-comfy
```

Selected-frame runs are QA previews only. Full-row installs require all six
frames and `--install` on the retry wrapper.

## Clip Naming

Runtime row clip ids follow:

```text
<role><Motion><Angle>
```

Examples:

```text
bluePIdleFront
bluePWalkSideRight
blueCFRunBack
blueLFRunSideLeft
```

Core motions and angles are:

```text
motions: idle, walk, run
angles: front, sideRight, back, sideLeft
frames per row: 6
cell size: 128 x 128
sheet columns: 6
```

Pitchers, catchers, fielders, batters, and umpires also have role-specific
action rows. The row definitions live in `tools/build_flux2_runtime_player_sheets.py`.

## Initial Candidate Workflow

The older candidate workflow generates one approved source actor per role under
`qa/player-sprite-revamp/`. Use it when replacing the base actor identity for a
role, not when patching a specific animation row.

Generate candidates:

```bash
python3 tools/generate_flux2_player_sprite_revamp.py \
  --roles blueP,blueCF,blueLF \
  --poses idle \
  --start-comfy
```

Build the QA page:

```bash
python3 tools/build_player_sprite_revamp_qa.py
```

Open:

```text
qa/player-sprite-revamp/index.html
```

Promote approved idle candidates into runtime full sheets:

```bash
python3 tools/build_flux2_runtime_player_sheets.py --roles blueP,blueCF,blueLF
```

This promotion creates `public/assets/generated/<role>FullSheet.png` and
`<role>FullSheet.json`, then retargets manifest clips to the full sheets.

## Prompt Ownership

Do not hand-edit generated prompts inside `qa/` except for forensic review.
Change prompt behavior in code:

- Role identity, uniforms, and likeness cues:
  `tools/generate_flux2_player_sprite_revamp.py`
- Runtime frame prompt assembly:
  `tools/generate_flux2_runtime_pose_row.py`
- Runtime sheet sequence definitions:
  `tools/build_flux2_runtime_player_sheets.py`

The founding-father likeness cues are in `ROLE_SPECS`. For example, the blue
team currently encodes George Washington, John Hancock, Samuel Adams, James
Madison, Alexander Hamilton, John Adams, Thomas Jefferson, Paul Revere, and
Benjamin Franklin as distinct roles.

## QA Artifacts

For a patched runtime row, inspect:

```text
qa/player-sprite-ai-rows/<role>/<clip>/contact-sheet.png
qa/player-sprite-ai-rows/<role>/<clip>/<frame>/raw.png
qa/player-sprite-ai-rows/<role>/<clip>/<frame>/alpha.png
qa/player-sprite-ai-rows/<role>/<clip>/<frame>/cell.png
qa/player-sprite-ai-rows/<role>/<clip>/<frame>/prompt.json
```

The generated contact sheet shows all six cells with pose labels and pass/review
status. If any frame is marked `review`, do not patch it unless you are making a
deliberate exception with `--allow-review` on the base row tool.

## Runtime Review Tool

Start the dev server:

```bash
npm run dev -- --host 127.0.0.1 --port 5192 --strictPort
```

Open:

```text
http://127.0.0.1:5192/tools/sprite-review/
```

Use the character picker and sequence picker to review individual frames and the
animated row. The manifest metadata should show:

- `review.tool` as `/tools/sprite-review/`
- `review.sequencePoseLabels` with six labels
- `aiPoseRow.patched` as `true` for installed AI rows
- `aiPoseRow.contactSheet` pointing at the QA contact sheet

## Validation

Run the focused unit tests:

```bash
python3 -m unittest \
  tests/test_flux2_runtime_pose_row.py \
  tests/test_flux2_runtime_pose_row_retry.py \
  tests/test_flux2_runtime_builder.py \
  tests/test_sprite_art_proof.py \
  tests/test_sprite_generation_policy.py
```

Validate the manifest proof metadata:

```bash
python3 tools/validate_sprite_art_proof.py public/assets/generated/manifest.json
```

Refresh the roster review matrix:

```bash
python3 tools/review_roster_sprites.py
```

Run the JS manifest and animation tests:

```bash
npm test -- \
  tests/assetManifest.test.js \
  tests/playerMotion.test.js \
  tests/spriteReviewTool.test.js \
  tests/spriteAnimator.test.js \
  tests/spriteSheetAnimator.test.js \
  tests/loadSprites.test.js
```

For a quick coverage query:

```bash
python3 - <<'PY'
import json
from pathlib import Path

m = json.loads(Path('public/assets/generated/manifest.json').read_text())
for role in ['blueP', 'blueCF', 'blueLF']:
    clips = []
    for clip_id, clip in sorted(m['animations'].items()):
        review = clip.get('review', {})
        if not clip_id.startswith(role):
            continue
        if review.get('motion') in {'idle', 'walk', 'run'} and review.get('angle') in {'front', 'sideRight', 'sideLeft', 'back'}:
            clips.append((clip_id, clip.get('aiPoseRow', {}).get('patched') is True))
    print(role, f'{sum(ok for _, ok in clips)}/{len(clips)}', 'remaining:', ', '.join([clip for clip, ok in clips if not ok]) or 'none')
print('ai_patched_total', sum(1 for clip in m['animations'].values() if clip.get('aiPoseRow', {}).get('patched') is True))
PY
```

At the time this file was written, the current local coverage was:

```text
blueP 12/12 remaining: none
blueCF 12/12 remaining: none
blueLF 10/12 remaining: blueLFRunBack, blueLFRunSideLeft
ai_patched_total 36
```

## Troubleshooting

If ComfyUI is already running, use the retry wrapper with:

```bash
--no-start-comfy
```

If a generated frame is too wide, cropped, duplicated, or has background
fragments, rerun that frame with `--overwrite` and keep the width list at
`320,256`:

```bash
python3 tools/generate_flux2_runtime_pose_row_with_retries.py \
  --clip blueLFRunBack \
  --frames 3 \
  --widths 320,256 \
  --timeout 900 \
  --overwrite
```

If ComfyUI stalls, interrupt or stop the local prompt before restarting:

```bash
pgrep -af 'ComfyUI/main.py|main.py --listen 127.0.0.1|generate_flux2_runtime_pose_row'
```

The row generator starts ComfyUI with low-memory settings:

```text
--lowvram --cpu-vae --cache-none --preview-method none
```

Slow throughput is expected on this machine. Prefer one row at a time, inspect
the contact sheet, then run validation before moving to the next row.