Skip to content

Optional Hosted SAM Segmentation

OpenFloodAI can optionally send images you choose to Meta's hosted SAM 3.1 service. SAM outlines a concept you name, such as river water or riverbank, so a person can review an image faster. It is off by default. Ingestion, validation, and review all work offline with no key and no SAM.

A SAM outline is an unreviewed machine prediction. It is not a human label, not a detected waterline, not calibrated accuracy, and not a flood decision. Zero matches, a failed request, and an unreviewed result never mean safe, normal, or no water-level change.

Set up

  1. Create an account and arrange billing with Meta: SAM hosted API: signup and pricing. You pay Meta through your own account. OpenFloodAI does not bill you.
  2. Open Settings in the console and turn on Hosted SAM segmentation.
  3. Paste your API key and select Save key. The key field is masked. Only the last four characters are shown afterward. Use Replace key or Remove key to change it.
  4. Install Meta's SAM parser, which reads replies and decodes masks. It is an optional extra, because it is distributed under Meta's own SAM License rather than this project's MIT license: pip install "openfloodai[sam]" (or pip install meta_sam_parser). Without it, the Start button stays unavailable and nothing is sent.
  5. Optional, for a local operator: set the OPENFLOODAI_SAM_API_KEY environment variable instead. A key saved in Settings wins over it.

Run segmentation

  1. Open a run in Review. The Image to segment box in the Hosted SAM panel shows exactly which image will be sent: a picture, its file name, and its capture time in UTC.
  2. Choose the image:
  3. Image selected on the region change chart (the default): click any point on the charts or an event. That point gets a purple ring labelled SAM, and the same image is marked "Will be segmented" in the image card.
  4. Baseline image: the reference the watched area and guide were drawn on. It is not a point on the chart, so no ring is shown; its picture is marked instead. Then choose river water, riverbank, or both. Each concept is its own paid request. The panel shows the exact number of requests before you start.
  5. The first time, confirm that the selected image, cropped to the watched area, will be uploaded to Meta and your account may be charged.
  6. Select Start segmentation. Nothing is uploaded before this step, and reloading the page never starts one.
  7. The predicted mask is shown over the original image with the unchanged human guide. Mark each result Accepted, Rejected, or Needs correction. These marks are saved separately and do not create human labels.

Batches are limited to 10 images and 2 concepts. Requests are sent one at a time, with a 60 second timeout and no automatic retries: a request that may have reached Meta could already be billed, so it is never resent for you. After an invalid key, no quota, a rate limit, a timeout, or a network failure, the rest of the batch is not sent.

Fetch masks for a whole sequence

On Sequences & runs, a checkbox beside Run validation says Also fetch water masks. It saves you from selecting images one by one when you want masks for every image of a sequence (the baseline included).

  • It is off and explained until hosted SAM is ready. The checkbox is disabled, with the reason, unless the plugin is on, a key is set and the mask decoder is installed. Nothing is ticked by default.
  • Ticking it only plans. It checks locally which images already have a mask and shows, for example, "4 of 6 images have no mask and would be sent (paid). 2 already have masks and are not sent again." Nothing is uploaded yet.
  • You confirm the exact count. When you press Run validation, a confirmation names how many images will be cropped to the watched area, uploaded and possibly billed, and shows the provider's price note. Cancel and nothing runs, not even the validation. The server also checks the count and refuses a different one.
  • Validation runs first, then the masks. Validation is free and local, so a failure there costs nothing. The masks then go out in batches of 10 images using the concept river water, each batch a normal hosted-SAM run. One click is limited to 100 paid requests. The first quota, key, rate-limit or network error stops the remaining batches, which are reported as not sent.
  • Results are unreviewed drafts. Accept or reject them in Review as usual. They never become labels by themselves, and only accepted water masks can be used by dataset curation or the water-change measurement.

If every image already has a mask

Nothing is sent, nothing is charged, no new run is created, and no confirmation is asked for. The validation simply runs. Existing results keep their place and their human review decisions, because they are not copied or re-run. A mask is reused only when the image's bytes, the model, the prompt and the watched-area crop are all identical. If you changed the watched area or an image, those images count as missing and are sent. An earlier failed attempt is not reused, so it is retried (and counts toward the confirmed number); an earlier "no match" result is reused, because the provider did find nothing.

If only some images have masks, only the others are sent, and the new run holds only those.

How your key is handled

  • Requests are made by the local backend. The browser never receives the key after you submit it, only a "configured" flag and a masked hint.
  • The key is held in the app's memory for the session. It is never written to a site folder, manifest, run folder, settings file, log, or browser storage. Removing it clears it. Restarting the app clears it and the upload confirmation.
  • Provider error text and parser warnings are redacted before they are kept or shown, and every saved record is scrubbed of the key as a last step.
  • Authorization is checked again before every paid request in a batch. If you turn the plugin off or remove the key partway through, the remaining items are not sent and are saved as "not attempted". A request already on its way cannot be recalled.
  • Disabling the plugin stops new uploads. Removing the key clears it from the app. Neither reverses charges for requests already sent.

What is saved

Each run is its own folder under outputs/hosted-sam-runs/<run_id>/ and is not edited afterward. A rerun is a new run.

  • Image SHA-256 and capture time, provider, requested model, the prompt, and the crop used (watched area as percentages, the pixel rectangle, source size, and JPEG quality), request status, and processing time.
  • The watched area and human guides in force when the run started (inputs-used/guides.snapshot.json), so an old result's overlay always draws the guide it was made with.
  • Raw full-size masks as PNG files in original-image pixels, kept apart from the overlay, which is drawn on request. An overlay is only drawn over the exact image the result came from; if that file has since changed, no overlay is shown and the saved masks are left as they were.
  • Human review decisions in an append-only reviews.jsonl.
  • The model Meta returns is not yet read from the reply, so only the requested model (sam-3.1) is recorded. A confidence is saved only when the service sends one, exactly as sent. It is not calibrated accuracy.

An identical request (same image bytes, model, prompt, and crop) reuses the saved result without a new paid request. Any change to the image, the watched area, or the prompt makes a new request. Failed attempts are never reused.

Coordinates

The service accepts text concepts, not boxes or points. OpenFloodAI crops the image to the site's watched area, sends the crop, then shifts each returned box and mask by the crop origin (a mask is the size of its box and anchored at the box's top-left corner) to place it on the original image. A mask that does not fit inside the crop is rejected rather than clipped, so a mask can never land outside the watched area or on the wrong image. The crop is a rectangle: a bank-region outline is not a waterline, and its vegetation-facing edge is not a detected water boundary.

Privacy and limits

  • Selected images leave your device. The provider page does not state how long uploaded images are kept or whether they are used for training. Read Meta's terms and privacy policy before uploading. Service terms are separate from the downloadable SAM model license.
  • The provider page listed $2.50/1k images when this was written. Check the current price with Meta.
  • Replies are read with Meta's own meta_sam_parser package, which defines the stream format, the inclusive box corners, and the mask codec. The HTTP status codes the service uses for quota and rate limits could not be confirmed without a paid key, so any unexpected reply is reported as a malformed response or provider error rather than guessed. A reply that ends early, fails, is refused, or has an unreadable object is never shown as "no match".
  • The meaning of an empty successful reply is also unconfirmed against the live service. Only a finished reply with no objects is shown as "no match".
  • A small pilot on representative images, authorized separately, should record review effort, processing time, and cost before any claim about usefulness. One successful image says nothing about accuracy.