| """Unpack JL1-CD into npy: jl_pre/jl_post (N,512,512,3) uint8, jl_chg (N,512,512) uint8 {0,1}. Heuristic folder roles.""" |
| import glob, os, sys, zipfile |
| from collections import defaultdict |
| import cv2, numpy as np |
| from huggingface_hub import hf_hub_download |
| z = hf_hub_download("circleLZY/JL1-CD", "JL1-CD.zip", repo_type="dataset", local_dir="/workspace/hfd") |
| zipfile.ZipFile(z).extractall("/workspace/jl1"); os.remove(z) |
| imgs = [p for p in glob.glob("/workspace/jl1/**/*", recursive=True) if p.lower().endswith((".png", ".jpg", ".jpeg", ".tif", ".tiff", ".bmp"))] |
| by_dir = defaultdict(list) |
| for p in imgs: by_dir[os.path.dirname(p)].append(p) |
| for d, v in sorted(by_dir.items()): print("DIR", d, len(v), flush=True) |
| PRE = {"a", "t1", "im1", "time1", "before", "pre", "image1", "img1", "imagea"} |
| POST = {"b", "t2", "im2", "time2", "after", "post", "image2", "img2", "imageb"} |
| def role(d): |
| b = os.path.basename(d).lower() |
| if any(k in b for k in ("label", "gt", "mask", "change")): return "lab" |
| if b in PRE: return "pre" |
| if b in POST: return "post" |
| return None |
| groups = defaultdict(dict) |
| for d, v in by_dir.items(): |
| r = role(d) |
| if r: groups[os.path.dirname(d)][r] = {os.path.splitext(os.path.basename(p))[0]: p for p in v} |
| pre, post, chg = [], [], [] |
| for g, roles in sorted(groups.items()): |
| if not {"pre", "post", "lab"} <= set(roles): print("skip group", g, list(roles)); continue |
| keys = sorted(set(roles["pre"]) & set(roles["post"]) & set(roles["lab"])) |
| print("GROUP", g, len(keys), flush=True) |
| for k in keys: |
| a = cv2.cvtColor(cv2.imread(roles["pre"][k], cv2.IMREAD_COLOR), cv2.COLOR_BGR2RGB) |
| b = cv2.cvtColor(cv2.imread(roles["post"][k], cv2.IMREAD_COLOR), cv2.COLOR_BGR2RGB) |
| l = cv2.imread(roles["lab"][k], cv2.IMREAD_GRAYSCALE) |
| if a.shape[:2] != (512, 512) or b.shape != a.shape or l.shape != a.shape[:2]: continue |
| pre.append(a); post.append(b); chg.append((l > 0).astype(np.uint8)) |
| pre, post, chg = np.stack(pre), np.stack(post), np.stack(chg) |
| os.makedirs("/workspace/prep", exist_ok=True) |
| np.save("/workspace/prep/jl_pre.npy", pre); np.save("/workspace/prep/jl_post.npy", post); np.save("/workspace/prep/jl_chg.npy", chg) |
| print("JL1_DONE", pre.shape, "change frac", round(float(chg.mean()), 4), "pairs with change", int((chg.reshape(len(chg), -1).sum(1) >= 20).sum()), flush=True) |
|
|
