Spaces:

build-small-hackathon
/

Forager-Field-Notes

Sleeping

Forager-Field-Notes / pipeline /session.py

Safety fix: never return an edible verdict from a single photo

6f38f56 verified 16 days ago

2.74 kB

"""session.py — the two-photo safety policy that sits over the resolver.
The core safety rule, stated plainly:
    You are never shown an edible verdict on the strength of a single photo.
A single photo can show DEADLY (over-warning is always safe) or abstain. A
SAFE/CAUTION identification is only surfaced when a SECOND photo agrees with the
first (geometric-mean fusion + consensus, in infer.identify_many). So the second
photo is requested exactly, and only, when the system is leaning toward telling
you something is edible — the one moment extra rigor is worth the friction.
This is what closes the ~1% single-shot deadly-shown-edible residual to 0.00%
(scripts/multiview_smoketest.py).
"""

from PIL import Image

from .convergence import ForagerResult, build_result
from .infer import Pipeline

# There is no confidence bar that lets one photo commit edible. A 0.85 bar used to:
# the val sets had put every single-photo deadly-as-edible miss at <= 0.772. Over
# the full val sets, two toxic mushroom photos commit as SAFE above it (lion's
# mane at 0.91, chanterelle at 0.87), and val is the optimistic case. A confident
# wrong answer looks exactly like a confident right one, so no threshold on one
# photo's confidence can hold; only a second look can.


def resolve_session(pipe: Pipeline, images: list[Image.Image]) -> ForagerResult:
    """Run the K-photo pipeline and apply the single-photo edible gate.
    - 0 photos -> abstain.
    - DEADLY (any photo count) -> surface it. Over-warning is safe.
    - 1 photo, SAFE/CAUTION, at any confidence -> withhold; ask for a second photo.
    - 2+ photos -> the fused/consensus verdict from identify_many stands.
    """
    result = build_result(pipe.identify_many(images))

    if (len(images) == 1
            and not result.abstained
            and result.safety in ("SAFE", "CAUTION")):
        return _need_second_photo(result)
    return result


def _need_second_photo(provisional: ForagerResult) -> ForagerResult:
    """Turn a single-photo edible lean into a 'add a second photo' refusal,
    carrying the provisional species so the UI can say what it's leaning toward
    without committing to it."""
    return ForagerResult(
        domain=provisional.domain,
        species="unknown",
        scientific_name=provisional.scientific_name,
        confidence=provisional.confidence,
        safety="UNKNOWN",
        lookalike=provisional.lookalike,
        key_diff=(f"Leaning {provisional.species.replace('_', ' ')}, but an edible call "
                  f"needs a second photo from a different angle to confirm."),
        low_confidence=True,
        expert_model=provisional.expert_model,
        abstained=True,
        reason="need_second_photo",
    )

	"""session.py — the two-photo safety policy that sits over the resolver.

	The core safety rule, stated plainly:

	You are never shown an edible verdict on the strength of a single photo.

	A single photo can show DEADLY (over-warning is always safe) or abstain. A
	SAFE/CAUTION identification is only surfaced when a SECOND photo agrees with the
	first (geometric-mean fusion + consensus, in infer.identify_many). So the second
	photo is requested exactly, and only, when the system is leaning toward telling
	you something is edible — the one moment extra rigor is worth the friction.

	This is what closes the ~1% single-shot deadly-shown-edible residual to 0.00%
	(scripts/multiview_smoketest.py).
	"""

	from PIL import Image

	from .convergence import ForagerResult, build_result
	from .infer import Pipeline

	# There is no confidence bar that lets one photo commit edible. A 0.85 bar used to:
	# the val sets had put every single-photo deadly-as-edible miss at <= 0.772. Over
	# the full val sets, two toxic mushroom photos commit as SAFE above it (lion's
	# mane at 0.91, chanterelle at 0.87), and val is the optimistic case. A confident
	# wrong answer looks exactly like a confident right one, so no threshold on one
	# photo's confidence can hold; only a second look can.


	def resolve_session(pipe: Pipeline, images: list[Image.Image]) -> ForagerResult:
	"""Run the K-photo pipeline and apply the single-photo edible gate.

	- 0 photos -> abstain.
	- DEADLY (any photo count) -> surface it. Over-warning is safe.
	- 1 photo, SAFE/CAUTION, at any confidence -> withhold; ask for a second photo.
	- 2+ photos -> the fused/consensus verdict from identify_many stands.
	"""
	result = build_result(pipe.identify_many(images))

	if (len(images) == 1
	and not result.abstained
	and result.safety in ("SAFE", "CAUTION")):
	return _need_second_photo(result)
	return result


	def _need_second_photo(provisional: ForagerResult) -> ForagerResult:
	"""Turn a single-photo edible lean into a 'add a second photo' refusal,
	carrying the provisional species so the UI can say what it's leaning toward
	without committing to it."""
	return ForagerResult(
	domain=provisional.domain,
	species="unknown",
	scientific_name=provisional.scientific_name,
	confidence=provisional.confidence,
	safety="UNKNOWN",
	lookalike=provisional.lookalike,
	key_diff=(f"Leaning {provisional.species.replace('_', ' ')}, but an edible call "
	f"needs a second photo from a different angle to confirm."),
	low_confidence=True,
	expert_model=provisional.expert_model,
	abstained=True,
	reason="need_second_photo",
	)