OpenFloodAI — Your First End-to-End Review

Before you start

This page is a guide. Run the commands in a terminal on your computer, then use the Home UI. Reading this page does not start validation.

Machine-only review

Needs a site config, a video, and a watched area. Produces machine evidence. Without a human label, there is no human comparison.

Human comparison review

Add human labels and confirm the manifest. The report compares what the machine measured with what a person saw.

Human labels are optional for running validation, but required for comparing the machine result with human review.

Use synthetic or approved practice videos. Keep sharing unchecked. This workflow does not send public warnings or train a model. A successful run does not prove flood detection accuracy.

Check the boxes as you go. They are a temporary aid, not a saved test record. Save your notes separately before closing this page.

1. Start the Home UI

Open a terminal in your OpenFloodAI repository folder. For first-time setup on macOS or Linux:

python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install -e ".[dev]"
python3 scripts/run_openfloodai_home_ui.py

Windows PowerShell commands

py -3.12 -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install -e ".[dev]"
python scripts/run_openfloodai_home_ui.py

If your computer blocks script activation, ask your system administrator or use the Python executable inside the virtual environment directly.

If the environment is already installed, activate it and run only the last command.

Open the local Home UI. Keep the terminal running. The page needs this server.

I can see the Home UI and its guided workflow.

2. Create a practice site

Choose Create Site. Use these example values:

Site Name: MVP Rehearsal
Folder Name: mvp-rehearsal — filled automatically
Site ID: site-mvp-rehearsal
Camera ID: camera-mvp-rehearsal
Privacy Notes: Practice videos only.

Choose Create Folder Structure. If that folder already exists, choose another name. Select your new site in the workflow.

The site appears and its config is found.

3. Choose a video and enter its details

Choose Add Video to open Add Video To Site. Select your practice site and a short local video.

Video ID: use a unique name such as practice-01.
Video purpose: choose why this video is useful.
Dataset group: choose practice.
Difficult case type: leave No difficult case unless it applies.
Video notes: write one sentence, such as “Clear view of the bridge; water looks steady.”

Keep Safe to share in repository unchecked. This form copies the file locally and updates its manifest details. Human labels are added separately.

Do not save yet if the watched area is still missing. Complete step 4 in the same form.

I need safe synthetic practice files

With the environment activated, run:

python3 -m pytest tests/ui/test_home_server.py::test_mvp_rehearsal_setup_to_five_video_result_review -q -s

The test prints Synthetic rehearsal files: followed by a temporary folder. It contains five source AVI files. Select those files for a new manual practice site. They are simple grey scenes, not flood footage. Use “I cannot judge from this video” if you cannot identify water behavior.

Some browsers cannot preview AVI. Record that issue or use an approved browser-compatible local MP4. Do not skip watched-area selection and call the rehearsal complete.

I selected the right video and entered its details.

4. Choose the watched area and save

Use the video controls to choose a clear frame. Drag a rectangle around a fixed riverbank, bridge pillar, or other useful reference area. Avoid a black opening frame.

Choose Copy Video And Update Manifest. Confirm the watched-area step is complete.

The watched area is saved in the site's config. Use clips from the same camera view. Choosing an area for a later video can replace the site's saved area.

Repeat steps 3–4 until the site contains five videos. Give each a different Video ID. You can repeat with ten videos later.

Five videos are listed and the watched area is saved.

5. Confirm the manifest

The manifest.jsonl file is the list of video details. You should not need to edit it by hand.

Open the Manifest workflow step. Check that all five videos are tracked. If it says missing or incomplete, use its create/repair action and read the result message.

Existing notes should remain, and sharing should stay off. If repair reports a conflict, record it instead of ignoring it.

The manifest tracks all five videos.

6. Try a machine-only run

Before adding labels, read the Run validation readiness summary. Confirm the site, videos, watched area, and local output destination.

Run validation. With no labels, the report should clearly say human comparison is unavailable. Cannot compare is expected; it does not mean the software failed to run.

A machine-only run exists and missing labels are explained.

7. Add a human label for each video

Choose Add Label. Select the site and an existing Video ID. Enter a start and end time in seconds, within the video's actual duration.

Example: You review seconds 0–10. If the water looks steady, choose No clear water change. If you cannot see clearly, choose I cannot judge from this video.

Choose what you actually see, add useful reviewer notes, and save the label. Repeat for all five videos. A period ending at 10 seconds uses frames before 10 seconds; the end time is excluded.

A human label is a comparison reference. It does not train the machine or force agreement.

Each of my five videos has a human-reviewed time period.

8. Run with human labels

Read the readiness summary again. Check config, video count, watched area, human labels, manifest, and output location. Resolve blockers or record them.

Choose Run Validation and wait. This creates a new folder under outputs/runs/. The earlier run should remain unchanged.

The new run finished and the earlier run is still available.

9. Review the result, not just the completion message

Agree: the human label and current machine evidence match. This is not proof of flood accuracy or water direction.
Disagree: they do not match. Review the label period, watched area, and images.
Cannot compare: evidence or a label is missing or unclear. Do not count it as success.

Check the Home UI counts and open the report, scorecard, comparison notes, and images. If the UI shows a path without an open action, open that path in your editor or file manager.

data/sites/mvp-rehearsal/outputs/runs/<run-id>/
  validation-report.md
  scorecard.json
  run-metadata.json
  records/<video-id>.jsonl
  review-images/<video-id>/
  videos/<video-id>/summary.md
  videos/<video-id>/label-comparison.md

Check the image time captions against the label period. A dark clip may have no comparison images; its report should explain why. Even with labels, the generated grey clips may all remain cannot_compare.

I can explain one result and one unclear case using the saved evidence.

10. Record what worked and what was confusing

Do not change labels just to increase the Agree count. Record failures and questions so the next contributor has a better workflow.

Your rehearsal notes — copy these into a local text file before closing:

No information in this box is sent anywhere. This guide has no network scripts or save service. Use your browser's Print command if you want a paper checklist.

Stop the Home UI server with Ctrl+C. Leave practice media and output files out of commits.

I saved my notes and recorded any unfinished steps honestly.

If something goes wrong

Page will not open: check that the server terminal is still running and use the local address from step 1.
Video will not preview: try an approved browser-compatible file and record the original format/browser.
Run is blocked: read the missing items in readiness, especially the watched area.
Labels exist but comparison is unclear: check exact Video ID, time period, frame quality, and detailed notes.
“Nones to Nones” in a report: this is a known wording problem for an empty usable range. It means there were no usable frame times.

The workflow is not independently verified until a contributor completes these steps without help. The automated rehearsal alone does not prove that.