v0.5: GeoRas2Vec model files, figures and card (code moved to GitHub)
Browse files- .gitattributes +6 -0
- README.md +69 -73
- exemple/carte_geologique.gpkg +2 -2
- exemple/rendu_qgis.png +2 -2
- figures/method_geometry.png +3 -0
- figures/method_naming.png +3 -0
- figures/naming_region.png +3 -0
- figures/naming_zoom.png +3 -0
- figures/results_continents.png +3 -0
- figures/results_real_scans.png +3 -0
- figures/results_test_scores.png +0 -0
- gc_model.py +0 -127
- leader.py +0 -79
- legend_ranker.pkl +2 -2
- naming_model.py +0 -203
- pipeline.py +0 -288
- prop_model.pkl +3 -0
- qgis_test.py +0 -55
- qml_faults.xml +0 -17
- qml_units.xml +0 -41
- requirements.txt +0 -13
- training/train_final.py +0 -471
- vectorize.py +0 -56
- water_model.pkl +3 -0
.gitattributes
CHANGED
|
@@ -36,3 +36,9 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 36 |
exemple/carte_geologique.gpkg filter=lfs diff=lfs merge=lfs -text
|
| 37 |
exemple/rendu_qgis.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
training/test_examples.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
exemple/carte_geologique.gpkg filter=lfs diff=lfs merge=lfs -text
|
| 37 |
exemple/rendu_qgis.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
training/test_examples.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
figures/results_real_scans.png filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
figures/results_continents.png filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
figures/naming_region.png filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
figures/naming_zoom.png filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
figures/method_geometry.png filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
figures/method_naming.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -3,7 +3,7 @@ license: cc-by-sa-4.0
|
|
| 3 |
library_name: pytorch
|
| 4 |
pipeline_tag: image-segmentation
|
| 5 |
datasets:
|
| 6 |
-
- PerfectWin/
|
| 7 |
tags:
|
| 8 |
- geology
|
| 9 |
- geological-map
|
|
@@ -13,89 +13,85 @@ tags:
|
|
| 13 |
- qgis
|
| 14 |
---
|
| 15 |
|
| 16 |
-
#
|
| 17 |
|
| 18 |
-
**Research preview (v0.
|
| 19 |
-
QGIS or ArcGIS: one polygon layer of geological units (with code, name, age, lithology when they can be read),
|
| 20 |
-
one line layer of faults, the legend table and default QGIS styles.
|
| 21 |
|
| 22 |
-
|
| 23 |
-
real USGS GMNA scans + realistic synthetic maps).
|
| 24 |
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
|
| 38 |
## Usage
|
| 39 |
```bash
|
|
|
|
| 40 |
pip install -r requirements.txt
|
| 41 |
-
python vectorize.py
|
| 42 |
```
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
Attribute table of `unites_geologiques`: `id, code, nom, age, lithologie, source_nom` (code read on the map / legend
|
| 49 |
-
colour / not identified), `confiance, texte_lu, suggestions, couleur_hex` (legend colour), `couleur_carte` (colour
|
| 50 |
-
measured on the scan), `surface_km2`.
|
| 51 |
-
|
| 52 |
-
## Results (held-out test split, never seen in training, 608 tiles)
|
| 53 |
-
Recall = share of reference polygons recovered with IoU ≥ 0.5. *Small* < 1000 px, *large* ≥ 1000 px (512 px tiles).
|
| 54 |
-
*Correct area* = share of the map area that falls in a correctly recovered polygon.
|
| 55 |
-
|
| 56 |
-
| group | ARAND ↓ | small recall | large recall | correct area | fault F1 (2 px) |
|
| 57 |
-
|---|---|---|---|---|---|
|
| 58 |
-
| Africa (synthetic) | 0.030 | 89.6 % | 97.9 % | 99.3 % | – |
|
| 59 |
-
| Europe (synthetic) | 0.099 | 82.2 % | 90.4 % | 92.0 % | – |
|
| 60 |
-
| North America (all) | 0.091 | 72.5 % | 90.5 % | 95.1 % | 0.95 |
|
| 61 |
-
| Oceania (synthetic) | 0.176 | 63.9 % | 75.6 % | 83.5 % | 0.99 |
|
| 62 |
-
| **GMNA real scans** | 0.052 | **74.7 %** | **93.4 %** | **98.3 %** | 0.63 |
|
| 63 |
-
| **full test set** | 0.107 | 70.5 % | 86.2 % | 92.1 % | 0.97 |
|
| 64 |
-
| trap set: neighbouring units with the same colour | 0.151 | 62.8 % | 76.9 % | 84.9 % | 0.97 |
|
| 65 |
-
| trap set: patterned units | 0.099 | 74.3 % | 89.8 % | 93.9 % | 0.98 |
|
| 66 |
-
|
| 67 |
-
Full table: `training/test_metrics.csv`; ablation of the training choices: `training/lab_round4_ablation.csv`
|
| 68 |
-
(small-polygon loss re-weighting and OHEM were tested and rejected: they over-segment real maps).
|
| 69 |
-
|
| 70 |
-
### Naming (real GMNA scans, map sheet never used to train the ranker)
|
| 71 |
-
End-to-end test (segmentation by this model + OCR + naming) on 9 real 2048 × 2048 px windows of the GMNA north sheet;
|
| 72 |
-
legend = all units of the sheet, swatches measured outside the tested window; ranker trained on the south sheet only.
|
| 73 |
-
|
| 74 |
-
| | value |
|
| 75 |
-
|---|---|
|
| 76 |
-
| area that receives a name (confidence ≥ 0.8) | **30.0 %** (per window 4 % – 57 %) |
|
| 77 |
-
| name correct, on the named area | **91.6 %** (per window 76 % – 98 %) |
|
| 78 |
-
| area correctly named, over the whole map | 27.5 % |
|
| 79 |
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
(
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 84 |
|
| 85 |
## Limitations
|
| 86 |
-
*
|
| 87 |
-
*
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
* Fault detection on real scans (F1 0.63) is much weaker than on synthetic maps.
|
| 92 |
-
* Large maps are processed in one piece for the watershed: keep inputs below ~6000 × 6000 px or split them.
|
| 93 |
|
| 94 |
-
##
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
|
|
|
|
|
|
|
| 99 |
|
| 100 |
## Citation
|
| 101 |
-
|
|
|
|
|
|
| 3 |
library_name: pytorch
|
| 4 |
pipeline_tag: image-segmentation
|
| 5 |
datasets:
|
| 6 |
+
- PerfectWin/georas2vec
|
| 7 |
tags:
|
| 8 |
- geology
|
| 9 |
- geological-map
|
|
|
|
| 13 |
- qgis
|
| 14 |
---
|
| 15 |
|
| 16 |
+
# GeoRas2Vec model – scanned geological map → GIS polygons, faults and unit names
|
| 17 |
|
| 18 |
+
**Research preview (v0.5).** **Code, installation and documentation: [GeoRas2Vec on GitHub](https://github.com/PerfectWin7777/georas2vec).**
|
|
|
|
|
|
|
| 19 |
|
| 20 |
+
GeoRas2Vec turns a scanned or rasterised geological map into a GeoPackage that opens directly in QGIS or ArcGIS. The GeoPackage holds three things:
|
|
|
|
| 21 |
|
| 22 |
+
- a polygon layer of geological units, with code, name, age and lithology when they can be read;
|
| 23 |
+
- a line layer of faults;
|
| 24 |
+
- the legend table, with default QGIS styles.
|
| 25 |
+
|
| 26 |
+
Unit names come **only from text printed on the map and from its legend**.
|
| 27 |
+
|
| 28 |
+
This repository holds the model files. The command-line tool downloads them automatically.
|
| 29 |
+
|
| 30 |
+

|
| 31 |
+
|
| 32 |
+
## Files
|
| 33 |
+
| file | role |
|
| 34 |
+
|---|---|
|
| 35 |
+
| `best_fp16.pt` | segmentation network (U-Net++ / EfficientNet-B4, RGB + Lab): 10 pixel affinities + fault map |
|
| 36 |
+
| `naming_model.pkl` | scores (text read on the map, polygon) pairs |
|
| 37 |
+
| `legend_ranker.pkl` | ranks the candidate codes of each polygon (text evidence, leader lines, legend swatch) |
|
| 38 |
+
| `prop_model.pkl` | decides when a code read on the map can be propagated to polygons of the same colour |
|
| 39 |
+
| `water_model.pkl` | recognises water / outside-the-map regions |
|
| 40 |
+
| `exemple/` | example output GeoPackage, legend CSV, QGIS render |
|
| 41 |
+
| `training/` | training curves and metrics of the network |
|
| 42 |
+
|
| 43 |
+
The `.pkl` files are scikit-learn pickles: only load files from this repository.
|
| 44 |
|
| 45 |
## Usage
|
| 46 |
```bash
|
| 47 |
+
git clone https://github.com/PerfectWin7777/georas2vec.git && cd georas2vec
|
| 48 |
pip install -r requirements.txt
|
| 49 |
+
python scripts/vectorize.py my_map.tif my_legend.csv my_map.gpkg
|
| 50 |
```
|
| 51 |
+
See the [GitHub README](https://github.com/PerfectWin7777/georas2vec#quick-start) for the legend CSV format.
|
| 52 |
+
|
| 53 |
+
## Results (never seen during training)
|
| 54 |
+

|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
|
| 56 |
+
| official test split | large units | small units | map area in the correct polygon |
|
| 57 |
+
|---|---|---|---|
|
| 58 |
+
| real scans (78 tiles, USGS GMNA 300 dpi) | **94.5 %** | **77.0 %** | **98.5 %** |
|
| 59 |
+
| synthetic maps, 4 continents (60 tiles) | 85.2 % | 73.6 % | 90.4 % |
|
| 60 |
+
|
| 61 |
+
Faults: line-level F1 at 2 px is **0.80** on the real-scan test split and **0.85** on a held-out region.
|
| 62 |
+
|
| 63 |
+

|
| 64 |
+
|
| 65 |
+
### Names
|
| 66 |
+
End to end on a held-out 6144 × 6144 px region of the GMNA north sheet. Polygons and names are both predicted.
|
| 67 |
+
The ranker and the propagation model were trained without this region.
|
| 68 |
+
|
| 69 |
+
| | area named | names correct | area correctly named |
|
| 70 |
+
|---|---|---|---|
|
| 71 |
+
| names from text only | 65.9 % | 94.1 % | 62.0 % |
|
| 72 |
+
| **+ colour propagation** | **73.9 %** | **94.2 %** | **69.6 %** |
|
| 73 |
+
|
| 74 |
+

|
| 75 |
+
|
| 76 |
+
## How it works
|
| 77 |
+

|
| 78 |
+

|
| 79 |
|
| 80 |
## Limitations
|
| 81 |
+
* A unit whose code is never printed (or never readable) stays *Unité non identifiée*, with 3 suggestions.
|
| 82 |
+
* Sibling units with almost identical colours (e.g. `lK` / `K`) cause most propagation errors.
|
| 83 |
+
* Real-scan validation is so far on the USGS Geologic Map of North America. The other continents are evaluated
|
| 84 |
+
on realistic renderings of official vector maps.
|
| 85 |
+
* On large maps, unnamed areas can show straight cuts at the 2048 px block edges.
|
|
|
|
|
|
|
| 86 |
|
| 87 |
+
## Versions
|
| 88 |
+
| version | change |
|
| 89 |
+
|---|---|
|
| 90 |
+
| **v0.5** | two-scale mutex watershed; fault lines split at junctions; topological contours (no gap, no overlap); water detection; OCR restricted to the legend alphabet; decoding of age prefixes, Ʀ, Є and superscripts; colour propagation over the whole map; block processing of large maps |
|
| 91 |
+
| v0.2 | first public release (`revision="v0.2"`) |
|
| 92 |
+
|
| 93 |
+
The repository was renamed from `PerfectWin/geocarto-model` to `PerfectWin/georas2vec-model` (old links redirect).
|
| 94 |
|
| 95 |
## Citation
|
| 96 |
+
Winter Tony, *GeoRas2Vec: from scanned geological maps to vector units, faults and names*, 2026 –
|
| 97 |
+
[GitHub](https://github.com/PerfectWin7777/georas2vec). Please also cite the data sources listed in the [dataset card](https://huggingface.co/datasets/PerfectWin/georas2vec).
|
exemple/carte_geologique.gpkg
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:52d72d064a6607c334abc8ecaaf8d8d192b195e04ee1dc845334d92c2f350bf1
|
| 3 |
+
size 2760704
|
exemple/rendu_qgis.png
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|
figures/method_geometry.png
ADDED
|
Git LFS Details
|
figures/method_naming.png
ADDED
|
Git LFS Details
|
figures/naming_region.png
ADDED
|
Git LFS Details
|
figures/naming_zoom.png
ADDED
|
Git LFS Details
|
figures/results_continents.png
ADDED
|
Git LFS Details
|
figures/results_real_scans.png
ADDED
|
Git LFS Details
|
figures/results_test_scores.png
ADDED
|
gc_model.py
DELETED
|
@@ -1,127 +0,0 @@
|
|
| 1 |
-
# Définitions du modèle GeoCarto (extraites de train_final.py) pour l'inférence hors Kaggle.
|
| 2 |
-
import numpy as np, torch, torch.nn as nn
|
| 3 |
-
from scipy import ndimage as ndi
|
| 4 |
-
from numba import njit
|
| 5 |
-
import segmentation_models_pytorch as smp
|
| 6 |
-
|
| 7 |
-
CFG = dict(arch="unetpp", encoder="efficientnet-b4", input="rgb+lab",
|
| 8 |
-
offsets=[(1, 0), (0, 1), (3, 0), (0, 3), (9, 0), (0, 9), (9, 9), (9, -9), (27, 0), (0, 27)],
|
| 9 |
-
n_short=2, mws_stride=1, mws_min_size=40, tile=512)
|
| 10 |
-
OFFSETS, N_SHORT, K = CFG["offsets"], CFG["n_short"], len(CFG["offsets"])
|
| 11 |
-
|
| 12 |
-
# ------------------------------------------------------------------ affinités + mutex watershed
|
| 13 |
-
def fill_holes(inst):
|
| 14 |
-
"""Chaque zone sans polygone (mer, lac, halo) devient sa propre région."""
|
| 15 |
-
holes, n = ndi.label(inst == 0)
|
| 16 |
-
if n == 0:
|
| 17 |
-
return inst
|
| 18 |
-
out = inst.copy()
|
| 19 |
-
out[holes > 0] = holes[holes > 0] + inst.max()
|
| 20 |
-
return out
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
def affinity_targets(inst):
|
| 24 |
-
H, W = inst.shape
|
| 25 |
-
aff, mask = np.zeros((K, H, W), np.float32), np.zeros((K, H, W), np.float32)
|
| 26 |
-
for k, (dy, dx) in enumerate(OFFSETS):
|
| 27 |
-
ys, ye, xs, xe = max(0, -dy), H - max(0, dy), max(0, -dx), W - max(0, dx)
|
| 28 |
-
a, b = inst[ys:ye, xs:xe], inst[ys + dy:ye + dy, xs + dx:xe + dx]
|
| 29 |
-
aff[k, ys:ye, xs:xe] = a == b
|
| 30 |
-
mask[k, ys:ye, xs:xe] = 1 # après fill_holes, tous les pixels ont une région
|
| 31 |
-
return aff, mask
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
@njit
|
| 35 |
-
def _find(parent, x):
|
| 36 |
-
while parent[x] != x:
|
| 37 |
-
parent[x] = parent[parent[x]]
|
| 38 |
-
x = parent[x]
|
| 39 |
-
return x
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
@njit
|
| 43 |
-
def _mws_core(n, u, v, rep):
|
| 44 |
-
parent = np.arange(n); size = np.ones(n, np.int64); m = len(u)
|
| 45 |
-
mutex_a = np.empty(m, np.int64); mutex_b = np.empty(m, np.int64); nm = 0
|
| 46 |
-
head = -np.ones(n, np.int64); nxt = np.empty(2 * m, np.int64); val = np.empty(2 * m, np.int64); nl = 0
|
| 47 |
-
for e in range(m):
|
| 48 |
-
ra, rb = _find(parent, u[e]), _find(parent, v[e])
|
| 49 |
-
if ra == rb:
|
| 50 |
-
continue
|
| 51 |
-
if rep[e]:
|
| 52 |
-
val[nl] = nm; nxt[nl] = head[ra]; head[ra] = nl; nl += 1
|
| 53 |
-
val[nl] = nm; nxt[nl] = head[rb]; head[rb] = nl; nl += 1
|
| 54 |
-
mutex_a[nm] = ra; mutex_b[nm] = rb; nm += 1
|
| 55 |
-
else:
|
| 56 |
-
small, other = (ra, rb) if size[ra] <= size[rb] else (rb, ra)
|
| 57 |
-
blocked = False; p = head[small]
|
| 58 |
-
while p != -1:
|
| 59 |
-
k = val[p]; x = _find(parent, mutex_a[k]); y = _find(parent, mutex_b[k])
|
| 60 |
-
if (x == small and y == other) or (x == other and y == small):
|
| 61 |
-
blocked = True; break
|
| 62 |
-
p = nxt[p]
|
| 63 |
-
if blocked:
|
| 64 |
-
continue
|
| 65 |
-
parent[small] = other; size[other] += size[small]; p = head[small]
|
| 66 |
-
if p != -1:
|
| 67 |
-
while nxt[p] != -1:
|
| 68 |
-
p = nxt[p]
|
| 69 |
-
nxt[p] = head[other]; head[other] = head[small]
|
| 70 |
-
out = np.empty(n, np.int64)
|
| 71 |
-
for i in range(n):
|
| 72 |
-
out[i] = _find(parent, i)
|
| 73 |
-
return out
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
def mutex_watershed(aff, stride=CFG["mws_stride"], min_size=CFG["mws_min_size"]):
|
| 77 |
-
_, H, W = aff.shape
|
| 78 |
-
idx = np.arange(H * W).reshape(H, W)
|
| 79 |
-
us, vs, ws, rep = [], [], [], []
|
| 80 |
-
for k, (dy, dx) in enumerate(OFFSETS):
|
| 81 |
-
a = idx[max(0, -dy):H - max(0, dy), max(0, -dx):W - max(0, dx)]
|
| 82 |
-
b = idx[max(0, dy):H - max(0, -dy), max(0, dx):W - max(0, -dx)]
|
| 83 |
-
w = aff[k, max(0, -dy):H - max(0, dy), max(0, -dx):W - max(0, dx)].copy()
|
| 84 |
-
if k >= N_SHORT:
|
| 85 |
-
a, b, w = a[::stride, ::stride], b[::stride, ::stride], 1 - w[::stride, ::stride]
|
| 86 |
-
us.append(a.ravel()); vs.append(b.ravel()); ws.append(w.ravel()); rep.append(np.full(a.size, k >= N_SHORT))
|
| 87 |
-
u, v, w, rep = map(np.concatenate, (us, vs, ws, rep))
|
| 88 |
-
order = np.argsort(-w, kind="stable")
|
| 89 |
-
lab = _mws_core(H * W, u[order], v[order], rep[order]).reshape(H, W)
|
| 90 |
-
_, lab = np.unique(lab, return_inverse=True); lab = lab.reshape(H, W) + 1
|
| 91 |
-
sizes = np.bincount(lab.ravel()); tiny = sizes < min_size; tiny[0] = False
|
| 92 |
-
if tiny[lab].any():
|
| 93 |
-
_, (iy, ix) = ndi.distance_transform_edt(tiny[lab], return_indices=True)
|
| 94 |
-
lab = lab[iy, ix]; _, lab = np.unique(lab, return_inverse=True); lab = lab.reshape(H, W) + 1
|
| 95 |
-
return lab.astype(np.int32)
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
# ------------------------------------------------------------------ modèle
|
| 99 |
-
def rgb2lab(x):
|
| 100 |
-
"""x : B x 3 x H x W dans [0,1] -> Lab normalisé (L/50-1, a/64, b/64)."""
|
| 101 |
-
lin = torch.where(x <= 0.04045, x / 12.92, ((x + 0.055) / 1.055) ** 2.4)
|
| 102 |
-
M = torch.tensor([[0.4124, 0.3576, 0.1805], [0.2126, 0.7152, 0.0722], [0.0193, 0.1192, 0.9505]], device=x.device)
|
| 103 |
-
xyz = torch.einsum("ij,bjhw->bihw", M, lin) / torch.tensor([0.95047, 1.0, 1.08883], device=x.device).view(1, 3, 1, 1)
|
| 104 |
-
f = torch.where(xyz > 0.008856, xyz.clamp(min=1e-6) ** (1 / 3), 7.787 * xyz + 16 / 116)
|
| 105 |
-
L, a, b = 116 * f[:, 1] - 16, 500 * (f[:, 0] - f[:, 1]), 200 * (f[:, 1] - f[:, 2])
|
| 106 |
-
return torch.stack([L / 50 - 1, a / 64, b / 64], 1)
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
class GeoCartoNet(nn.Module):
|
| 110 |
-
"""Entrée : image uint8 B x 3 x H x W (H, W multiples de 32). Sortie : K affinités + 1 carte de failles (logits)."""
|
| 111 |
-
|
| 112 |
-
def __init__(self, cfg, pretrained=True):
|
| 113 |
-
super().__init__()
|
| 114 |
-
self.cfg = cfg
|
| 115 |
-
self.register_buffer("mean", torch.tensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1))
|
| 116 |
-
self.register_buffer("std", torch.tensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1))
|
| 117 |
-
Arch = {"unet": smp.Unet, "unetpp": smp.UnetPlusPlus}[cfg["arch"]]
|
| 118 |
-
self.net = Arch(cfg["encoder"], encoder_weights="imagenet" if pretrained else None, in_channels=6,
|
| 119 |
-
classes=len(cfg["offsets"]) + 1)
|
| 120 |
-
|
| 121 |
-
def forward(self, x8):
|
| 122 |
-
with torch.autocast(x8.device.type, enabled=False): # conversion de couleur en précision complète
|
| 123 |
-
x = x8.float() / 255
|
| 124 |
-
x = torch.cat([(x - self.mean) / self.std, rgb2lab(x)], 1)
|
| 125 |
-
return self.net(x)
|
| 126 |
-
|
| 127 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
leader.py
DELETED
|
@@ -1,79 +0,0 @@
|
|
| 1 |
-
# Détection des traits de rappel : trait sombre fin qui part du bord d'une étiquette, dans n'importe quelle direction.
|
| 2 |
-
import numpy as np
|
| 3 |
-
from scipy import ndimage as ndi
|
| 4 |
-
from skimage.segmentation import find_boundaries
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
def prepare(reg):
|
| 8 |
-
"""Masque des pixels 'trait candidat' : sombres, hors contacts, hors failles. Calculé une fois par zone."""
|
| 9 |
-
if "lead_mask" in reg:
|
| 10 |
-
return
|
| 11 |
-
lum, inst = reg["lum"], reg["inst"]
|
| 12 |
-
bd = find_boundaries(inst, mode="thick")
|
| 13 |
-
nearbd = ndi.binary_dilation(bd, iterations=1)
|
| 14 |
-
flt = reg.get("flt")
|
| 15 |
-
nf = ndi.binary_dilation(flt > 0, iterations=3) if flt is not None else np.zeros_like(bd)
|
| 16 |
-
rgb = reg["rgb"].astype(np.int16)
|
| 17 |
-
sat = rgb.max(-1) - rgb.min(-1)
|
| 18 |
-
dark = (lum < 100) & (sat < 70) # trait noir/gris (pas de texte ou de contour coloré)
|
| 19 |
-
reg["lead_dark"] = dark
|
| 20 |
-
reg["lead_mask"] = dark & ~nearbd & ~nf
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
def trace(reg, wd, max_len=160, gap=6):
|
| 24 |
-
"""Renvoie la liste des extrémités (y, x, longueur) des traits partant de la boîte du mot."""
|
| 25 |
-
prepare(reg)
|
| 26 |
-
M, D, inst = reg["lead_mask"], reg["lead_dark"], reg["inst"]
|
| 27 |
-
N = inst.shape[0]
|
| 28 |
-
x0, y0, x1, y1 = int(max(wd.x0, 0)), int(max(wd.y0, 0)), int(min(wd.x1, N - 1)), int(min(wd.y1, N - 1))
|
| 29 |
-
P = 14
|
| 30 |
-
Y0, X0, Y1, X1 = max(y0 - P, 0), max(x0 - P, 0), min(y1 + P, N - 1), min(x1 + P, N - 1)
|
| 31 |
-
sub = M[Y0:Y1 + 1, X0:X1 + 1].copy()
|
| 32 |
-
# on enlève la boîte de texte (+1 px) : l'encre des lettres n'est pas un trait
|
| 33 |
-
sub[max(y0 - 1 - Y0, 0):y1 + 2 - Y0, max(x0 - 1 - X0, 0):x1 + 2 - X0] = False
|
| 34 |
-
lab, n = ndi.label(sub, structure=np.ones((3, 3)))
|
| 35 |
-
if n == 0:
|
| 36 |
-
return []
|
| 37 |
-
yy, xx = np.nonzero(lab)
|
| 38 |
-
ll = lab[yy, xx]
|
| 39 |
-
gy, gx = yy + Y0, xx + X0
|
| 40 |
-
# distance à la boîte
|
| 41 |
-
db = np.hypot(np.maximum(np.maximum(y0 - gy, gy - y1), 0), np.maximum(np.maximum(x0 - gx, gx - x1), 0))
|
| 42 |
-
out = []
|
| 43 |
-
for c in range(1, n + 1):
|
| 44 |
-
s = ll == c
|
| 45 |
-
if s.sum() < 5 or db[s].min() > 2.5: # doit toucher (presque) la boîte
|
| 46 |
-
continue
|
| 47 |
-
py, px, d = gy[s].astype(float), gx[s].astype(float), db[s]
|
| 48 |
-
# un trait est fin et allongé
|
| 49 |
-
cov = np.cov(np.stack([py, px]))
|
| 50 |
-
ev, evec = np.linalg.eigh(cov)
|
| 51 |
-
v = evec[:, 1]
|
| 52 |
-
if ev[1] < 9: # trop court (< ~10 px)
|
| 53 |
-
continue
|
| 54 |
-
res = np.abs((py - py.mean()) * v[1] - (px - px.mean()) * v[0])
|
| 55 |
-
if np.percentile(res, 90) > 1.3: # pas droit ou trop épais
|
| 56 |
-
continue
|
| 57 |
-
a = np.argmin(d); b = np.argmax(d)
|
| 58 |
-
if d[b] - d[a] < 4: # doit s'éloigner de la boîte
|
| 59 |
-
continue
|
| 60 |
-
if np.dot(v, [py[b] - py[a], px[b] - px[a]]) < 0:
|
| 61 |
-
v = -v
|
| 62 |
-
# suivi dans la direction, au-delà du morceau visible, en tolérant les coupures aux contacts
|
| 63 |
-
y, x = py[b], px[b]; miss = 0; L = float(np.hypot(py[b] - py[a], px[b] - px[a]))
|
| 64 |
-
ey, ex = y, x
|
| 65 |
-
while L < max_len:
|
| 66 |
-
y += v[0]; x += v[1]; L += 1
|
| 67 |
-
iy, ix = int(round(y)), int(round(x))
|
| 68 |
-
if not (0 <= iy < N and 0 <= ix < N):
|
| 69 |
-
break
|
| 70 |
-
if D[max(iy - 1, 0):iy + 2, max(ix - 1, 0):ix + 2].any():
|
| 71 |
-
miss = 0; ey, ex = y, x
|
| 72 |
-
else:
|
| 73 |
-
miss += 1
|
| 74 |
-
if miss > gap:
|
| 75 |
-
break
|
| 76 |
-
# pointe : on avance de 2 px au-delà de la fin (le trait se termine DANS le polygone visé)
|
| 77 |
-
ty, tx = int(np.clip(round(ey + 2 * v[0]), 0, N - 1)), int(np.clip(round(ex + 2 * v[1]), 0, N - 1))
|
| 78 |
-
out.append((ty, tx, float(np.hypot(ty - py[a], tx - px[a]))))
|
| 79 |
-
return out
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
legend_ranker.pkl
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:34cf13b45af05cc52e897f5bc5203f181c430e3b59306e4d02e2f59320acb069
|
| 3 |
+
size 550493
|
naming_model.py
DELETED
|
@@ -1,203 +0,0 @@
|
|
| 1 |
-
# =====================================================================================
|
| 2 |
-
# MODULE DE NOMMAGE : à quel polygone appartient chaque code lu sur la carte ?
|
| 3 |
-
# Module séparé du modèle de segmentation. Pour chaque code lu (OCR), on note chaque polygone
|
| 4 |
-
# voisin avec des indices simples (distance, trait de rappel sombre, place disponible pour le
|
| 5 |
-
# texte, recouvrement...) et un petit modèle appris sur de VRAIES étiquettes GMNA choisit.
|
| 6 |
-
# Apprentissage : zones "south" ; test : zones "north" (jamais vues).
|
| 7 |
-
# =====================================================================================
|
| 8 |
-
import sys, glob, pickle, re, difflib, time
|
| 9 |
-
import numpy as np, pandas as pd
|
| 10 |
-
from scipy import ndimage as ndi
|
| 11 |
-
from sklearn.ensemble import HistGradientBoostingClassifier
|
| 12 |
-
|
| 13 |
-
RADIUS = 110 # rayon de recherche des polygones candidats autour d'une étiquette (px)
|
| 14 |
-
norm = lambda c: re.sub(r"[\*\?\s]", "", str(c))
|
| 15 |
-
import os
|
| 16 |
-
if os.path.exists("naming_cache/vocab.pkl"):
|
| 17 |
-
_codes = pickle.load(open("naming_cache/vocab.pkl", "rb"))
|
| 18 |
-
else: # lecture encore en cours : vocabulaire = codes des zones déjà lues
|
| 19 |
-
_codes = [c for f in glob.glob("naming_cache/*_*.pkl") for c in pickle.load(open(f, "rb"))["codes"]]
|
| 20 |
-
VOCAB = sorted({norm(c) for c in _codes} - {""})
|
| 21 |
-
VSET = set(VOCAB)
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
def to_code(txt):
|
| 25 |
-
t = re.sub(r"[^A-Za-z0-9€Є³]", "", txt).replace("€", "Є")
|
| 26 |
-
if not t:
|
| 27 |
-
return None, 0.0
|
| 28 |
-
if t in VSET:
|
| 29 |
-
return t, 1.0
|
| 30 |
-
m = difflib.get_close_matches(t, VOCAB, n=1, cutoff=0.75)
|
| 31 |
-
return (m[0], 0.75) if m else (None, 0.0)
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
FEATS = ["dist_centre", "dist_boite", "sous_centre", "part_boite", "rang_dist", "n_cand", "aire_log", "rayon_int",
|
| 35 |
-
"rayon_sur_texte", "trait_sombre", "trait_lum", "limites_croisees", "long_trait", "h_texte", "l_texte",
|
| 36 |
-
"conf", "qualite_code", "dist_rel", "plus_proche_non_sous", "part_boite_max_autre", "vise_trait", "trait_L"]
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
def candidates(reg, wd):
|
| 40 |
-
"""Lignes de caractéristiques (une par polygone candidat) pour un mot lu."""
|
| 41 |
-
inst, lum = reg["inst"], reg["lum"]
|
| 42 |
-
N = inst.shape[0]
|
| 43 |
-
x0, y0, x1, y1 = int(max(wd.x0, 0)), int(max(wd.y0, 0)), int(min(wd.x1, N - 1)), int(min(wd.y1, N - 1))
|
| 44 |
-
if x1 - x0 < 2 or y1 - y0 < 2:
|
| 45 |
-
return []
|
| 46 |
-
cx, cy = (x0 + x1) / 2, (y0 + y1) / 2
|
| 47 |
-
X0, Y0, X1, Y1 = max(x0 - RADIUS, 0), max(y0 - RADIUS, 0), min(x1 + RADIUS, N - 1), min(y1 + RADIUS, N - 1)
|
| 48 |
-
sub, slum = inst[Y0:Y1 + 1, X0:X1 + 1], lum[Y0:Y1 + 1, X0:X1 + 1]
|
| 49 |
-
yy, xx = np.mgrid[Y0:Y1 + 1, X0:X1 + 1]
|
| 50 |
-
dc = np.hypot(yy - cy, xx - cx) # distance au centre de la boîte
|
| 51 |
-
db = np.hypot(np.maximum(np.maximum(y0 - yy, yy - y1), 0), np.maximum(np.maximum(x0 - xx, xx - x1), 0)) # à la boîte
|
| 52 |
-
ids = np.unique(sub); ids = ids[ids > 0]
|
| 53 |
-
if not len(ids):
|
| 54 |
-
return []
|
| 55 |
-
box = sub[y0 - Y0:y1 - Y0 + 1, x0 - X0:x1 - X0 + 1]
|
| 56 |
-
light = slum[y0 - Y0:y1 - Y0 + 1, x0 - X0:x1 - X0 + 1] >= 120 # hors encre du texte
|
| 57 |
-
nb = max(light.sum(), 1)
|
| 58 |
-
under = inst[int(cy), int(cx)]
|
| 59 |
-
th, tw = y1 - y0, x1 - x0
|
| 60 |
-
dmin_c = ndi.minimum(dc, sub, ids); dmin_b = ndi.minimum(db, sub, ids)
|
| 61 |
-
pos_b = ndi.minimum_position(db, sub, ids)
|
| 62 |
-
edt = ndi.distance_transform_edt(sub > 0) if False else None
|
| 63 |
-
from leader import trace
|
| 64 |
-
tips = trace(reg, wd)
|
| 65 |
-
rows = []
|
| 66 |
-
order = np.argsort(dmin_b)
|
| 67 |
-
rank = np.empty(len(ids)); rank[order] = np.arange(len(ids))
|
| 68 |
-
part = np.array([(box[light] == i).sum() / nb for i in ids])
|
| 69 |
-
for j, i in enumerate(ids):
|
| 70 |
-
m = sub == i
|
| 71 |
-
# rayon intérieur du polygone (dans la fenêtre) : peut-il contenir le texte ?
|
| 72 |
-
rin = ndi.distance_transform_edt(np.pad(m, 1))[1:-1, 1:-1].max()
|
| 73 |
-
# segment droit boîte -> point le plus proche du polygone : un trait de rappel est sombre et droit
|
| 74 |
-
py, px = pos_b[j][0] + Y0, pos_b[j][1] + X0
|
| 75 |
-
qy, qx = min(max(py, y0), y1), min(max(px, x0), x1)
|
| 76 |
-
L = float(np.hypot(py - qy, px - qx))
|
| 77 |
-
if L >= 3:
|
| 78 |
-
n = int(L) + 1
|
| 79 |
-
sy = np.clip(np.round(np.linspace(qy, py, n)).astype(int), 0, N - 1)
|
| 80 |
-
sx = np.clip(np.round(np.linspace(qx, px, n)).astype(int), 0, N - 1)
|
| 81 |
-
ls = np.minimum.reduce([lum[np.clip(sy + a, 0, N - 1), np.clip(sx + b, 0, N - 1)] for a in (-1, 0, 1) for b in (-1, 0, 1)])
|
| 82 |
-
dark, lmean = float((ls[1:-1] < 120).mean()) if n > 2 else 0.0, float(ls.mean())
|
| 83 |
-
cross = int((np.diff(inst[sy, sx]) != 0).sum())
|
| 84 |
-
else:
|
| 85 |
-
dark, lmean, cross = 0.0, 255.0, 0
|
| 86 |
-
others = np.delete(part, j)
|
| 87 |
-
tl = [t[2] for t in tips if inst[t[0], t[1]] == i]
|
| 88 |
-
rows.append(dict(poly=int(i), dist_centre=float(dmin_c[j]), dist_boite=float(dmin_b[j]), sous_centre=float(i == under),
|
| 89 |
-
part_boite=float(part[j]), rang_dist=float(rank[j]), n_cand=float(len(ids)),
|
| 90 |
-
aire_log=float(np.log1p(reg["area"][i])), rayon_int=float(rin), rayon_sur_texte=float(rin / max(th / 2, 1)),
|
| 91 |
-
trait_sombre=dark, trait_lum=lmean, limites_croisees=float(cross), long_trait=L, h_texte=float(th),
|
| 92 |
-
l_texte=float(tw), conf=float(wd.conf), qualite_code=float(wd.q),
|
| 93 |
-
dist_rel=float(dmin_b[j] / max(th, 1)), plus_proche_non_sous=float(rank[j] == (1 if under in ids else 0) and i != under),
|
| 94 |
-
part_boite_max_autre=float(others.max() if len(others) else 0),
|
| 95 |
-
vise_trait=float(len(tl)), trait_L=float(min(tl)) if tl else -1.0))
|
| 96 |
-
return rows
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
def load_region(f):
|
| 100 |
-
r = pickle.load(open(f, "rb"))
|
| 101 |
-
r["lum"] = (r["rgb"].astype(np.float32) @ np.array([0.299, 0.587, 0.114], np.float32))
|
| 102 |
-
r["area"] = np.bincount(r["inst"].ravel())
|
| 103 |
-
r["true"] = {i + 1: norm(c) for i, c in enumerate(r["codes"])}
|
| 104 |
-
return r
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
def build(files):
|
| 108 |
-
out = []
|
| 109 |
-
for f in files:
|
| 110 |
-
reg = load_region(f)
|
| 111 |
-
W_ = reg["words"]
|
| 112 |
-
for k, wd in enumerate(W_.itertuples()):
|
| 113 |
-
code, q = to_code(wd.text)
|
| 114 |
-
if code is None:
|
| 115 |
-
continue
|
| 116 |
-
wd = wd._replace(conf=wd.conf) if False else wd
|
| 117 |
-
class _W: pass
|
| 118 |
-
w = _W(); w.__dict__.update(wd._asdict()); w.q = q
|
| 119 |
-
for row in candidates(reg, w):
|
| 120 |
-
row.update(region=f, word=k, code=code, y=float(reg["true"][row["poly"]] == code),
|
| 121 |
-
vrai=reg["true"][row["poly"]], aire=float(reg["area"][row["poly"]]))
|
| 122 |
-
out.append(row)
|
| 123 |
-
return pd.DataFrame(out)
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
def assign(D, score):
|
| 127 |
-
"""Choix final : pour chaque mot le polygone le mieux noté ; un polygone ne garde qu'UN code
|
| 128 |
-
(celui qui a la meilleure note), les mots perdants passent à leur candidat suivant."""
|
| 129 |
-
D = D.assign(s=score)
|
| 130 |
-
choice = {}
|
| 131 |
-
words = {k: g.sort_values("s", ascending=False) for k, g in D.groupby(["region", "word"])}
|
| 132 |
-
ptr = {k: 0 for k in words}
|
| 133 |
-
for _ in range(4):
|
| 134 |
-
for k, g in words.items():
|
| 135 |
-
if ptr[k] < len(g):
|
| 136 |
-
r = g.iloc[ptr[k]]
|
| 137 |
-
choice[k] = (r.region, int(r.poly), r.code, float(r.s))
|
| 138 |
-
else:
|
| 139 |
-
choice.pop(k, None)
|
| 140 |
-
best = {}
|
| 141 |
-
for k, (reg, p, c, s) in choice.items():
|
| 142 |
-
key = (reg, p)
|
| 143 |
-
if key not in best or s > best[key][1]:
|
| 144 |
-
best[key] = (c, s)
|
| 145 |
-
moved = False
|
| 146 |
-
for k, (reg, p, c, s) in list(choice.items()):
|
| 147 |
-
if best[(reg, p)][0] != c and ptr[k] + 1 < len(words[k]) and words[k].iloc[ptr[k] + 1].s > 0.15:
|
| 148 |
-
ptr[k] += 1; moved = True
|
| 149 |
-
if not moved:
|
| 150 |
-
break
|
| 151 |
-
return choice
|
| 152 |
-
|
| 153 |
-
|
| 154 |
-
def report(D, choice, title):
|
| 155 |
-
key = D.set_index(["region", "word", "poly"])
|
| 156 |
-
evaluable = D.groupby(["region", "word"]).y.max() > 0 # le bon code existe parmi les candidats (lecture juste)
|
| 157 |
-
ok = n = 0
|
| 158 |
-
polys = {}
|
| 159 |
-
for k, (reg, p, c, s) in choice.items():
|
| 160 |
-
r = key.loc[(k[0], k[1], p)]
|
| 161 |
-
good = bool(np.atleast_1d(r.y)[0])
|
| 162 |
-
if evaluable[k]:
|
| 163 |
-
n += 1; ok += good
|
| 164 |
-
cur = polys.get((reg, p))
|
| 165 |
-
if cur is None or s > cur[1]:
|
| 166 |
-
polys[(reg, p)] = (good, s, float(np.atleast_1d(r.aire)[0]))
|
| 167 |
-
pg = np.array([v[0] for v in polys.values()]); pa = np.array([v[2] for v in polys.values()])
|
| 168 |
-
print(f"{title:<46} codes bien rattachés : {ok / max(n, 1):5.1%} ({n} codes) | polygones nommés : {len(polys)}, "
|
| 169 |
-
f"noms justes : {pg.mean():5.1%} (en surface : {(pa * pg).sum() / pa.sum():5.1%})")
|
| 170 |
-
return ok / max(n, 1), polys
|
| 171 |
-
|
| 172 |
-
|
| 173 |
-
if __name__ == "__main__":
|
| 174 |
-
t0 = time.time()
|
| 175 |
-
tr_files, te_files = sorted(glob.glob("naming_cache/south_*.pkl")), sorted(glob.glob("naming_cache/north_*.pkl"))
|
| 176 |
-
if len(te_files) < 3: # provisoire : pas encore de zones nord -> 1 zone sud sur 3 en test
|
| 177 |
-
allf = tr_files + te_files
|
| 178 |
-
tr_files, te_files = [f for i, f in enumerate(allf) if i % 3], [f for i, f in enumerate(allf) if i % 3 == 0]
|
| 179 |
-
Dtr, Dte = build(tr_files), build(te_files)
|
| 180 |
-
print(f"apprentissage : {len(tr_files)} zones, {Dtr.groupby(['region', 'word']).ngroups} codes lus, {len(Dtr)} paires | "
|
| 181 |
-
f"test : {len(te_files)} zones, {Dte.groupby(['region', 'word']).ngroups} codes lus ({time.time() - t0:.0f} s)")
|
| 182 |
-
ev = Dtr.groupby(["region", "word"]).y.transform("max") > 0 # on n'apprend que sur les mots correctement lus
|
| 183 |
-
clf = HistGradientBoostingClassifier(max_iter=300, learning_rate=0.06, max_leaf_nodes=24, l2_regularization=1.0,
|
| 184 |
-
class_weight="balanced", random_state=0)
|
| 185 |
-
clf.fit(Dtr.loc[ev, FEATS], Dtr.loc[ev, "y"])
|
| 186 |
-
pickle.dump({"model": clf, "features": FEATS, "radius": RADIUS}, open("naming_model.pkl", "wb"))
|
| 187 |
-
print("\nSUR LES ZONES DE TEST (north, jamais vues) :")
|
| 188 |
-
naive = Dte.sous_centre + 0.001 * Dte.part_boite
|
| 189 |
-
a0, _ = report(Dte, {k: (g.region.iloc[0], int(g.poly.iloc[np.argmax(n_)]), g.code.iloc[0], 1.0)
|
| 190 |
-
for (k, g), n_ in ((kg, naive.loc[kg[1].index].values) for kg in Dte.groupby(["region", "word"]))},
|
| 191 |
-
"Règle naïve (polygone sous l'étiquette)")
|
| 192 |
-
p = clf.predict_proba(Dte[FEATS])[:, 1]
|
| 193 |
-
a1, _ = report(Dte, {k: (g.region.iloc[0], int(g.poly.iloc[np.argmax(p[Dte.index.get_indexer(g.index)])]), g.code.iloc[0],
|
| 194 |
-
float(p[Dte.index.get_indexer(g.index)].max())) for k, g in Dte.groupby(["region", "word"])},
|
| 195 |
-
"Module appris (meilleure note)")
|
| 196 |
-
ch = assign(Dte, p)
|
| 197 |
-
a2, polys = report(Dte, ch, "Module appris + un seul code par polygone")
|
| 198 |
-
# avec seuil de confiance : on ne nomme que si la note est élevée
|
| 199 |
-
for thr in (0.5, 0.7, 0.85):
|
| 200 |
-
report(Dte, {k: v for k, v in ch.items() if v[3] >= thr}, f" ... en ne gardant que les notes >= {thr}")
|
| 201 |
-
from sklearn.inspection import permutation_importance
|
| 202 |
-
imp = permutation_importance(clf, Dte[FEATS], Dte["y"], n_repeats=3, random_state=0, scoring="average_precision")
|
| 203 |
-
print("\nIndices les plus utiles :", ", ".join(f"{FEATS[i]} ({imp.importances_mean[i]:.3f})" for i in np.argsort(-imp.importances_mean)[:8]))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
pipeline.py
DELETED
|
@@ -1,288 +0,0 @@
|
|
| 1 |
-
# =====================================================================================
|
| 2 |
-
# GEOCARTO – PIPELINE COMPLET : raster de carte géologique -> GeoPackage QGIS / ArcGIS
|
| 3 |
-
# 1. modèle (affinités + failles) par fenêtres glissantes 2. mutex watershed -> polygones
|
| 4 |
-
# 3. vectorisation (unités + failles) 4. OCR des textes de la carte
|
| 5 |
-
# 5. nommage : code écrit (module appris) > case de légende (si sûr) > "Unité non identifiée" + 3 suggestions
|
| 6 |
-
# 6. export GeoPackage (couches unites_geologiques, failles, table legende + styles QGIS par défaut)
|
| 7 |
-
# =====================================================================================
|
| 8 |
-
import os, sys, re, sqlite3, pickle, time
|
| 9 |
-
import numpy as np, pandas as pd, torch, geopandas as gpd
|
| 10 |
-
from scipy import ndimage as ndi
|
| 11 |
-
from rasterio.features import shapes
|
| 12 |
-
from rasterio.transform import Affine
|
| 13 |
-
from shapely.geometry import shape, LineString
|
| 14 |
-
from skimage.color import rgb2lab as sk_rgb2lab
|
| 15 |
-
from skimage.morphology import skeletonize
|
| 16 |
-
|
| 17 |
-
HERE = os.path.dirname(os.path.abspath(__file__))
|
| 18 |
-
sys.path.insert(0, HERE)
|
| 19 |
-
import gc_model as M
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
# ------------------------------------------------------------------ 1. modèle
|
| 23 |
-
def load_model(path):
|
| 24 |
-
ck = torch.load(path, map_location="cpu", weights_only=True)
|
| 25 |
-
cfg = ck["cfg"]
|
| 26 |
-
M.CFG.update(cfg); M.OFFSETS[:] = cfg["offsets"]
|
| 27 |
-
model = M.GeoCartoNet(cfg, pretrained=False)
|
| 28 |
-
model.load_state_dict({k: v.float() for k, v in ck["state_dict"].items()})
|
| 29 |
-
return model.eval()
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
@torch.no_grad()
|
| 33 |
-
def _run(model, im):
|
| 34 |
-
x = torch.from_numpy(np.ascontiguousarray(im).transpose(2, 0, 1).copy())[None]
|
| 35 |
-
return torch.sigmoid(model(x).float())[0].numpy()
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
def _tta(model, im, offsets):
|
| 39 |
-
"""Moyenne sur 4 miroirs ; chaque décalage est ramené dans le repère d'origine (testé : exact)."""
|
| 40 |
-
H, W = im.shape[:2]
|
| 41 |
-
Y, X = np.mgrid[:H, :W]
|
| 42 |
-
idx = {tuple(o): k for k, o in enumerate(offsets)}
|
| 43 |
-
K = len(offsets)
|
| 44 |
-
acc = 0
|
| 45 |
-
for fy in (0, 1):
|
| 46 |
-
for fx in (0, 1):
|
| 47 |
-
v = im[::-1] if fy else im
|
| 48 |
-
v = v[:, ::-1] if fx else v
|
| 49 |
-
o = _run(model, v)
|
| 50 |
-
A, pf = o[:K], o[K]
|
| 51 |
-
py, px = (H - 1 - Y) if fy else Y, (W - 1 - X) if fx else X
|
| 52 |
-
out = np.empty_like(o)
|
| 53 |
-
for k, (dy, dx) in enumerate(offsets):
|
| 54 |
-
t = (-dy if fy else dy, -dx if fx else dx)
|
| 55 |
-
if t in idx:
|
| 56 |
-
out[k] = A[idx[t]][py, px]
|
| 57 |
-
else:
|
| 58 |
-
out[k] = A[idx[(-t[0], -t[1])]][np.clip(py + t[0], 0, H - 1), np.clip(px + t[1], 0, W - 1)]
|
| 59 |
-
out[K] = pf[py, px]
|
| 60 |
-
acc = acc + out
|
| 61 |
-
return acc / 4
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
def infer(model, rgb, tile=512, stride=384, tta=False):
|
| 65 |
-
"""Fenêtres glissantes avec pondération de Hann -> affinités (K x H x W) + probabilité de faille."""
|
| 66 |
-
H, W = rgb.shape[:2]
|
| 67 |
-
ph, pw = max(tile - H, 0), max(tile - W, 0)
|
| 68 |
-
img = np.pad(rgb, ((0, ph), (0, pw), (0, 0)), mode="edge")
|
| 69 |
-
Hp, Wp = img.shape[:2]
|
| 70 |
-
K = len(M.OFFSETS)
|
| 71 |
-
acc = np.zeros((K + 1, Hp, Wp), np.float32); ws = np.zeros((Hp, Wp), np.float32)
|
| 72 |
-
hw = np.maximum(np.outer(np.hanning(tile), np.hanning(tile)), 1e-3).astype(np.float32)
|
| 73 |
-
pos = lambda n: sorted(set(list(range(0, n - tile + 1, stride)) + [n - tile]))
|
| 74 |
-
for y in pos(Hp):
|
| 75 |
-
for x in pos(Wp):
|
| 76 |
-
im = img[y:y + tile, x:x + tile]
|
| 77 |
-
o = _tta(model, im, M.OFFSETS) if tta else _run(model, im)
|
| 78 |
-
acc[:, y:y + tile, x:x + tile] += o * hw; ws[y:y + tile, x:x + tile] += hw
|
| 79 |
-
o = acc / ws
|
| 80 |
-
return o[:K, :H, :W], o[K, :H, :W]
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
# ------------------------------------------------------------------ 3. vectorisation
|
| 84 |
-
def vectorize_units(lab, transform, crs):
|
| 85 |
-
geoms, ids = [], []
|
| 86 |
-
for g, v in shapes(lab.astype(np.int32), mask=lab > 0, transform=transform):
|
| 87 |
-
geoms.append(shape(g)); ids.append(int(v))
|
| 88 |
-
P = gpd.GeoDataFrame({"pid": ids}, geometry=geoms, crs=crs).dissolve("pid").reset_index()
|
| 89 |
-
P["geometry"] = P.geometry.simplify(abs(transform.a) * 0.75, preserve_topology=True)
|
| 90 |
-
return P
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
def vectorize_faults(pf, transform, crs, thr=0.5, min_px=15):
|
| 94 |
-
sk = skeletonize(pf > thr)
|
| 95 |
-
lab, n = ndi.label(sk, structure=np.ones((3, 3)))
|
| 96 |
-
sizes = np.bincount(lab.ravel())
|
| 97 |
-
nb = ndi.convolve(sk.astype(np.uint8), np.ones((3, 3), np.uint8), mode="constant") - 1
|
| 98 |
-
lines = []
|
| 99 |
-
seen = np.zeros_like(sk)
|
| 100 |
-
for c in range(1, n + 1):
|
| 101 |
-
if sizes[c] < min_px:
|
| 102 |
-
continue
|
| 103 |
-
ys, xs = np.nonzero(lab == c)
|
| 104 |
-
pts = set(zip(ys.tolist(), xs.tolist()))
|
| 105 |
-
ends = [p for p in pts if nb[p] == 1] or [next(iter(pts))]
|
| 106 |
-
for st in ends:
|
| 107 |
-
if seen[st]:
|
| 108 |
-
continue
|
| 109 |
-
path, cur = [st], st
|
| 110 |
-
seen[st] = True
|
| 111 |
-
while True:
|
| 112 |
-
nxt = [(cur[0] + a, cur[1] + b) for a in (-1, 0, 1) for b in (-1, 0, 1) if (a or b)
|
| 113 |
-
and (cur[0] + a, cur[1] + b) in pts and not seen[cur[0] + a, cur[1] + b]]
|
| 114 |
-
if not nxt:
|
| 115 |
-
break
|
| 116 |
-
cur = nxt[0]; seen[cur] = True; path.append(cur)
|
| 117 |
-
if len(path) >= min_px:
|
| 118 |
-
lines.append(LineString([transform * (x + 0.5, y + 0.5) for y, x in path]).simplify(abs(transform.a)))
|
| 119 |
-
F = gpd.GeoDataFrame({"type": ["indéterminée"] * len(lines)}, geometry=lines, crs=crs)
|
| 120 |
-
F["longueur_km"] = (F.length / 1000).round(2)
|
| 121 |
-
return F
|
| 122 |
-
|
| 123 |
-
|
| 124 |
-
# ------------------------------------------------------------------ 4. OCR
|
| 125 |
-
def read_text(rgb, reader, B=1024, OV=96):
|
| 126 |
-
words = []
|
| 127 |
-
H, W = rgb.shape[:2]
|
| 128 |
-
for y in range(0, H, B - OV):
|
| 129 |
-
for x in range(0, W, B - OV):
|
| 130 |
-
for box, txt, conf in reader.readtext(rgb[y:y + B, x:x + B], mag_ratio=2.0, text_threshold=0.6, low_text=0.3):
|
| 131 |
-
b = np.array(box) + [x, y]; cx, cy = b.mean(0)
|
| 132 |
-
if (x > 0 and cx < x + OV / 2) or (y > 0 and cy < y + OV / 2):
|
| 133 |
-
continue
|
| 134 |
-
words.append(dict(text=txt, conf=conf, cx=cx, cy=cy, x0=b[:, 0].min(), x1=b[:, 0].max(), y0=b[:, 1].min(), y1=b[:, 1].max()))
|
| 135 |
-
return pd.DataFrame(words, columns=["text", "conf", "cx", "cy", "x0", "x1", "y0", "y1"])
|
| 136 |
-
|
| 137 |
-
|
| 138 |
-
# ------------------------------------------------------------------ 5. nommage
|
| 139 |
-
def swatch_features(lab_pixels):
|
| 140 |
-
"""Signature d'une case de légende / d'un polygone : couleur dominante, couleurs claire et foncée du motif, texture."""
|
| 141 |
-
px = lab_pixels[lab_pixels[:, 0] > 45]
|
| 142 |
-
if len(px) < 20:
|
| 143 |
-
px = lab_pixels
|
| 144 |
-
L = px[:, 0]; med = np.percentile(L, 50)
|
| 145 |
-
return np.concatenate([np.median(px, 0), np.median(px[L >= med], 0), np.median(px[L <= med], 0), px.std(0) / 4])
|
| 146 |
-
|
| 147 |
-
|
| 148 |
-
def ocr_candidates(rgb, lab, words, vocab, naming_pkl, flt=None):
|
| 149 |
-
"""Toutes les paires (code lu, polygone candidat) avec leur score appris ; et l'affectation un code -> un polygone."""
|
| 150 |
-
import naming_model as NM
|
| 151 |
-
NM.VOCAB = sorted(vocab); NM.VSET = set(NM.VOCAB)
|
| 152 |
-
mdl = pickle.load(open(naming_pkl, "rb"))
|
| 153 |
-
reg = dict(inst=lab, rgb=rgb, lum=rgb.astype(np.float32) @ np.array([0.299, 0.587, 0.114], np.float32),
|
| 154 |
-
area=np.bincount(lab.ravel()), flt=flt)
|
| 155 |
-
rows = []
|
| 156 |
-
for k, wd in enumerate(words.itertuples()):
|
| 157 |
-
code, q = NM.to_code(wd.text)
|
| 158 |
-
if code is None:
|
| 159 |
-
continue
|
| 160 |
-
class _W: pass
|
| 161 |
-
w = _W(); w.__dict__.update(wd._asdict()); w.q = q
|
| 162 |
-
for r in NM.candidates(reg, w):
|
| 163 |
-
r.update(region="carte", word=k, code=code, lu=wd.text); rows.append(r)
|
| 164 |
-
if not rows:
|
| 165 |
-
return pd.DataFrame(columns=["word", "code", "s"]), {}
|
| 166 |
-
D = pd.DataFrame(rows)
|
| 167 |
-
D["s"] = mdl["model"].predict_proba(D[mdl["features"]])[:, 1]
|
| 168 |
-
return D, NM.assign(D, D["s"].values)
|
| 169 |
-
|
| 170 |
-
RANK_FEATS = ["d_norm", "d_rel", "rang", "gap12", "d_ratio", "ocr", "ocr_assigne", "ocr_rel", "lu", "n_lu", "dist_lu", "log_aire"]
|
| 171 |
-
|
| 172 |
-
def rank_pairs(pids, feats, areas, cents, codes, Fm, D, ch, words, K=12):
|
| 173 |
-
"""Paires (polygone, code candidat) et leurs indices. Candidats = K cases de légende les plus proches + codes lus par l'OCR.
|
| 174 |
-
Indices : uniquement le texte lu sur la carte et la légende (pas de propagation d'apparence entre polygones)."""
|
| 175 |
-
cidx = {c: i for i, c in enumerate(codes)}
|
| 176 |
-
d = np.linalg.norm(feats[:, None, :] - Fm[None], axis=2)
|
| 177 |
-
E, lu_pos, asg = {}, {}, {}
|
| 178 |
-
if len(D):
|
| 179 |
-
for (p, c), s in D.groupby(["poly", "code"]).s.max().items():
|
| 180 |
-
E[(p, c)] = s
|
| 181 |
-
for w, c in D[["word", "code"]].drop_duplicates().values:
|
| 182 |
-
lu_pos.setdefault(c, []).append((words.iloc[w].cy, words.iloc[w].cx))
|
| 183 |
-
for (_, w), (_, p, c, s) in ch.items():
|
| 184 |
-
asg[(p, c)] = max(asg.get((p, c), 0), s)
|
| 185 |
-
Ep = {}
|
| 186 |
-
for (p, c), s in E.items():
|
| 187 |
-
Ep.setdefault(p, []).append(c)
|
| 188 |
-
lu_pos = {c: np.array(v) for c, v in lu_pos.items()}
|
| 189 |
-
rows = []
|
| 190 |
-
for i, p in enumerate(pids):
|
| 191 |
-
o = np.argsort(d[i]); rank = np.empty(len(codes), int); rank[o] = np.arange(len(codes))
|
| 192 |
-
cand = list(dict.fromkeys(list(codes[o[:K]]) + [c for c in Ep.get(p, []) if c in cidx]))
|
| 193 |
-
emax = max([E[(p, c)] for c in Ep.get(p, [])] + [0]); gap = d[i, o[1]] - d[i, o[0]] if len(o) > 1 else 0
|
| 194 |
-
for c in cand:
|
| 195 |
-
j = cidx[c]; pos = lu_pos.get(c)
|
| 196 |
-
rows.append(dict(poly=p, code=c, d=d[i, j], d_rel=d[i, j] - d[i, o[0]], rang=rank[j], gap12=gap,
|
| 197 |
-
ocr=E.get((p, c), 0), ocr_assigne=asg.get((p, c), 0), ocr_max_poly=emax,
|
| 198 |
-
lu=int(pos is not None), n_lu=0 if pos is None else len(pos),
|
| 199 |
-
dist_lu=5000 if pos is None else np.hypot(pos[:, 0] - cents[i][0], pos[:, 1] - cents[i][1]).min(),
|
| 200 |
-
log_aire=np.log(areas[i])))
|
| 201 |
-
X = pd.DataFrame(rows)
|
| 202 |
-
X["d_ratio"] = X.d_rel / (X.gap12 + 1); X["d_norm"] = X.d / X.d.median(); X["ocr_rel"] = X.ocr - X.ocr_max_poly
|
| 203 |
-
return X
|
| 204 |
-
|
| 205 |
-
def name_polygons(rgb, lab, words, legend, naming_pkl, flt=None, ranker_pkl="legend_ranker.pkl", sure=0.8, check=0.5):
|
| 206 |
-
"""legend : DataFrame index=code, colonnes nom, age, lithologie, couleur_hex, feat (signature de la case).
|
| 207 |
-
Retourne par polygone : code, source_nom, confiance, texte_lu, suggestions.
|
| 208 |
-
source_nom : 'code lu sur la carte' / 'légende' (confiance >= sure), '… – à vérifier' (check <= confiance < sure), 'non identifiée'."""
|
| 209 |
-
D, ch = ocr_candidates(rgb, lab, words, legend.index, naming_pkl, flt)
|
| 210 |
-
R = pickle.load(open(ranker_pkl, "rb"))
|
| 211 |
-
labc = sk_rgb2lab(rgb).reshape(-1, 3)
|
| 212 |
-
ids = np.unique(lab[lab > 0])
|
| 213 |
-
order = np.argsort(lab.ravel(), kind="stable"); sl = lab.ravel()[order]
|
| 214 |
-
starts = np.searchsorted(sl, ids); ends = np.searchsorted(sl, ids, side="right")
|
| 215 |
-
feats = np.stack([swatch_features(labc[order[a:b]]) for a, b in zip(starts, ends)])
|
| 216 |
-
cents = ndi.center_of_mass(np.ones_like(lab), lab, ids)
|
| 217 |
-
codes = np.array(legend.index); Fm = np.stack(legend.feat.values)
|
| 218 |
-
X = rank_pairs(ids, feats, ends - starts, cents, codes, Fm, D, ch, words, K=R.get("K", 12))
|
| 219 |
-
X["p"] = R["model"].predict_proba(X[R["features"]])[:, 1]
|
| 220 |
-
X = X.sort_values(["poly", "p"], ascending=[True, False])
|
| 221 |
-
lu_txt = {}
|
| 222 |
-
for (_, w), (_, p, c, s) in ch.items():
|
| 223 |
-
lu_txt[(p, c)] = words.iloc[w].text
|
| 224 |
-
out = {}
|
| 225 |
-
for p, g in X.groupby("poly", sort=False):
|
| 226 |
-
top = g.head(3); c, pr = top.code.iloc[0], float(top.p.iloc[0])
|
| 227 |
-
sug = ", ".join(f"{a} ({b:.0%})" for a, b in zip(top.code, top.p))
|
| 228 |
-
ocr = top.ocr.iloc[0] > 0
|
| 229 |
-
src = "code lu sur la carte" if ocr else "couleur de la légende"
|
| 230 |
-
if pr >= sure:
|
| 231 |
-
out[p] = dict(code=c, source_nom=src, confiance=round(pr, 2), texte_lu=lu_txt.get((p, c), ""), suggestions=sug)
|
| 232 |
-
elif pr >= check: # trop peu fiable sur cartes réelles (31-64 % juste) : pas de nom, mais suggestion classée en tête
|
| 233 |
-
out[p] = dict(code="", source_nom="non identifiée – suggestion probable", confiance=round(pr, 2), texte_lu=lu_txt.get((p, c), ""), suggestions=sug)
|
| 234 |
-
else:
|
| 235 |
-
out[p] = dict(code="", source_nom="non identifiée", confiance=round(pr, 2), texte_lu="", suggestions=sug)
|
| 236 |
-
return pd.DataFrame.from_dict(out, orient="index")
|
| 237 |
-
|
| 238 |
-
# ------------------------------------------------------------------ 6. export
|
| 239 |
-
QML_UNITS = open(os.path.join(HERE, "qml_units.xml")).read() if os.path.exists(os.path.join(HERE, "qml_units.xml")) else None
|
| 240 |
-
QML_FAULTS = open(os.path.join(HERE, "qml_faults.xml")).read() if os.path.exists(os.path.join(HERE, "qml_faults.xml")) else None
|
| 241 |
-
|
| 242 |
-
|
| 243 |
-
def export_gpkg(path, P, F, legend):
|
| 244 |
-
if os.path.exists(path):
|
| 245 |
-
os.remove(path)
|
| 246 |
-
P.to_file(path, layer="unites_geologiques", driver="GPKG")
|
| 247 |
-
if len(F):
|
| 248 |
-
F.to_file(path, layer="failles", driver="GPKG")
|
| 249 |
-
con = sqlite3.connect(path)
|
| 250 |
-
legend.drop(columns=["feat"]).reset_index().rename(columns={"index": "code"}).to_sql("legende", con, index=False, if_exists="replace")
|
| 251 |
-
con.execute("""CREATE TABLE IF NOT EXISTS layer_styles (id INTEGER PRIMARY KEY AUTOINCREMENT, f_table_catalog TEXT(256),
|
| 252 |
-
f_table_schema TEXT(256), f_table_name TEXT(256), f_geometry_column TEXT(256), styleName TEXT(30), styleQML TEXT,
|
| 253 |
-
styleSLD TEXT, useAsDefault BOOLEAN, description TEXT, owner TEXT(30), ui TEXT(30), update_time DATETIME DEFAULT CURRENT_TIMESTAMP)""")
|
| 254 |
-
for tbl, qml in (("unites_geologiques", QML_UNITS), ("failles", QML_FAULTS)):
|
| 255 |
-
if qml and (tbl != "failles" or len(F)):
|
| 256 |
-
con.execute("INSERT INTO layer_styles (f_table_catalog,f_table_schema,f_table_name,f_geometry_column,styleName,styleQML,styleSLD,useAsDefault,description,owner,update_time) "
|
| 257 |
-
"VALUES ('','',?,'geom',?,?,'',1,'style GeoCarto','',strftime('%Y-%m-%dT%H:%M:%fZ','now'))", (tbl, tbl, qml))
|
| 258 |
-
con.execute("INSERT INTO gpkg_contents (table_name, data_type, identifier) SELECT 'layer_styles','attributes','layer_styles' "
|
| 259 |
-
"WHERE NOT EXISTS (SELECT 1 FROM gpkg_contents WHERE table_name='layer_styles')")
|
| 260 |
-
con.execute("INSERT INTO gpkg_contents (table_name, data_type, identifier) SELECT 'legende','attributes','legende' "
|
| 261 |
-
"WHERE NOT EXISTS (SELECT 1 FROM gpkg_contents WHERE table_name='legende')")
|
| 262 |
-
con.commit(); con.close()
|
| 263 |
-
|
| 264 |
-
|
| 265 |
-
def run(rgb, transform, crs, model, legend, reader, naming_pkl, out_gpkg, tta=False, log=print, ranker_pkl="legend_ranker.pkl"):
|
| 266 |
-
t = time.time()
|
| 267 |
-
aff, pf = infer(model, rgb, tta=tta); log(f"modèle : {time.time() - t:.0f} s")
|
| 268 |
-
lab = M.mutex_watershed(aff); log(f"segmentation : {lab.max()} polygones ({time.time() - t:.0f} s)")
|
| 269 |
-
words = read_text(rgb, reader); log(f"OCR : {len(words)} textes ({time.time() - t:.0f} s)")
|
| 270 |
-
N = name_polygons(rgb, lab, words, legend, naming_pkl, flt=(pf > 0.5).astype(np.uint8), ranker_pkl=ranker_pkl); log(f"nommage ({time.time() - t:.0f} s)")
|
| 271 |
-
P = vectorize_units(lab, transform, crs)
|
| 272 |
-
P = P.join(N, on="pid")
|
| 273 |
-
P["code"] = P.code.fillna(""); P["source_nom"] = P.source_nom.fillna("non identifiée")
|
| 274 |
-
lg = legend.reindex(P.code)
|
| 275 |
-
P["nom"] = np.where(P.code != "", (P.code + " – " + lg.lithologie.fillna("").values).str.strip(" –"), "Unité non identifiée")
|
| 276 |
-
P["age"] = lg.age.fillna("").values; P["lithologie"] = lg.lithologie.fillna("").values
|
| 277 |
-
med = [ndi.median(rgb[..., c], lab, P.pid.values) for c in range(3)]
|
| 278 |
-
P["couleur_carte"] = ["#%02x%02x%02x" % tuple(int(v) for v in x) for x in zip(*med)]
|
| 279 |
-
P["couleur_hex"] = np.where(P.code != "", lg.couleur_hex.fillna("").values, P.couleur_carte)
|
| 280 |
-
P["couleur_hex"] = np.where(P.couleur_hex == "", P.couleur_carte, P.couleur_hex)
|
| 281 |
-
P["surface_km2"] = (P.area / 1e6).round(3)
|
| 282 |
-
P["id"] = np.arange(1, len(P) + 1)
|
| 283 |
-
P = P[["id", "code", "nom", "age", "lithologie", "source_nom", "confiance", "texte_lu", "suggestions",
|
| 284 |
-
"couleur_hex", "couleur_carte", "surface_km2", "geometry"]]
|
| 285 |
-
F = vectorize_faults(pf, transform, crs)
|
| 286 |
-
export_gpkg(out_gpkg, P, F, legend.loc[legend.index.isin(P.code)])
|
| 287 |
-
log(f"export : {out_gpkg} | {len(P)} unités, {len(F)} failles ({time.time() - t:.0f} s)")
|
| 288 |
-
return P, F, lab, pf
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
prop_model.pkl
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ca118ed5248650dd98bc93403c6d57c84806d0f8bb7c0cdd9cf3b51f580df5f6
|
| 3 |
+
size 554843
|
qgis_test.py
DELETED
|
@@ -1,55 +0,0 @@
|
|
| 1 |
-
# Vrai test QGIS (PyQGIS 3.34, sans interface) d'un GeoPackage produit par pipeline.py
|
| 2 |
-
# QT_QPA_PLATFORM=offscreen python3.12 qgis_test.py <carte.gpkg> <rendu.png>
|
| 3 |
-
import sys
|
| 4 |
-
from qgis.core import (QgsApplication, QgsVectorLayer, QgsProject, QgsMapSettings, QgsMapRendererParallelJob,
|
| 5 |
-
QgsGeometry, QgsWkbTypes)
|
| 6 |
-
from qgis.PyQt.QtCore import QSize
|
| 7 |
-
from qgis.PyQt.QtGui import QColor
|
| 8 |
-
|
| 9 |
-
gpkg, png = sys.argv[1], sys.argv[2]
|
| 10 |
-
app = QgsApplication([], False); app.initQgis()
|
| 11 |
-
ok = True
|
| 12 |
-
def check(cond, msg):
|
| 13 |
-
global ok
|
| 14 |
-
print(("OK " if cond else "ÉCHEC"), msg); ok &= bool(cond)
|
| 15 |
-
|
| 16 |
-
layers = []
|
| 17 |
-
for name, geom in (("unites_geologiques", QgsWkbTypes.PolygonGeometry), ("failles", QgsWkbTypes.LineGeometry)):
|
| 18 |
-
lyr = QgsVectorLayer(f"{gpkg}|layername={name}", name, "ogr")
|
| 19 |
-
check(lyr.isValid(), f"couche {name} chargée")
|
| 20 |
-
if not lyr.isValid():
|
| 21 |
-
continue
|
| 22 |
-
msg, styled = lyr.loadDefaultStyle() # style QML stocké dans la table layer_styles du GeoPackage
|
| 23 |
-
check(styled, f"{name} : style intégré chargé automatiquement ({msg})")
|
| 24 |
-
check(lyr.geometryType() == geom, f"{name} : type de géométrie {QgsWkbTypes.geometryDisplayString(lyr.geometryType())}")
|
| 25 |
-
check(lyr.crs().isValid(), f"{name} : SCR {lyr.crs().authid() or lyr.crs().description()}")
|
| 26 |
-
n = lyr.featureCount(); check(n > 0, f"{name} : {n} entités")
|
| 27 |
-
bad = sum(1 for f in lyr.getFeatures() if not f.geometry().isGeosValid())
|
| 28 |
-
check(bad == 0, f"{name} : géométries invalides = {bad}")
|
| 29 |
-
print(" champs :", [f.name() for f in lyr.fields()])
|
| 30 |
-
r = lyr.renderer()
|
| 31 |
-
print(" rendu :", r.type(), (f"{len(r.categories())} catégories" if hasattr(r, "categories") else ""))
|
| 32 |
-
if name == "unites_geologiques":
|
| 33 |
-
need = {"code", "nom", "age", "lithologie", "source_nom", "confiance", "suggestions"}
|
| 34 |
-
check(need <= {f.name() for f in lyr.fields()}, "colonnes de nommage présentes")
|
| 35 |
-
# filtre attributaire comme le ferait un utilisateur QGIS
|
| 36 |
-
lyr.setSubsetString("\"source_nom\" = 'code lu sur la carte'")
|
| 37 |
-
print(" filtre source_nom='code lu sur la carte' :", lyr.featureCount(), "entités"); lyr.setSubsetString("")
|
| 38 |
-
ex = next(lyr.getFeatures())
|
| 39 |
-
print(" exemple :", {k: ex[k] for k in ("code", "nom", "age", "source_nom", "confiance")})
|
| 40 |
-
layers.append(lyr)
|
| 41 |
-
|
| 42 |
-
leg = QgsVectorLayer(f"{gpkg}|layername=legende", "legende", "ogr")
|
| 43 |
-
check(leg.isValid() and leg.featureCount() > 0, f"table legende : {leg.featureCount() if leg.isValid() else 0} lignes")
|
| 44 |
-
|
| 45 |
-
if layers:
|
| 46 |
-
QgsProject.instance().addMapLayers(layers)
|
| 47 |
-
ms = QgsMapSettings(); ms.setLayers(layers[::-1]); ms.setDestinationCrs(layers[0].crs())
|
| 48 |
-
ext = layers[0].extent(); ms.setExtent(ext); ms.setOutputSize(QSize(1400, int(1400 * ext.height() / ext.width())))
|
| 49 |
-
ms.setBackgroundColor(QColor("white"))
|
| 50 |
-
job = QgsMapRendererParallelJob(ms); job.start(); job.waitForFinished()
|
| 51 |
-
img = job.renderedImage(); check(img.save(png), f"rendu carte -> {png}")
|
| 52 |
-
QgsProject.instance().write(png.rsplit(".", 1)[0] + ".qgz")
|
| 53 |
-
print(" projet QGIS écrit :", png.rsplit(".", 1)[0] + ".qgz")
|
| 54 |
-
print("\nRÉSULTAT QGIS :", "TOUT OK" if ok else "PROBLÈMES")
|
| 55 |
-
app.exitQgis()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
qml_faults.xml
DELETED
|
@@ -1,17 +0,0 @@
|
|
| 1 |
-
<!DOCTYPE qgis PUBLIC 'http://mrcc.com/qgis.dtd' 'SYSTEM'>
|
| 2 |
-
<qgis version="3.34.0" styleCategories="Symbology">
|
| 3 |
-
<renderer-v2 type="singleSymbol" symbollevels="0" enableorderby="0" forceraster="0">
|
| 4 |
-
<symbols>
|
| 5 |
-
<symbol type="line" name="0" alpha="1" clip_to_extent="1" force_rhr="0">
|
| 6 |
-
<layer class="SimpleLine" enabled="1" locked="0" pass="0">
|
| 7 |
-
<Option type="Map">
|
| 8 |
-
<Option name="line_color" type="QString" value="0,0,0,255"/>
|
| 9 |
-
<Option name="line_width" type="QString" value="0.6"/>
|
| 10 |
-
<Option name="line_width_unit" type="QString" value="MM"/>
|
| 11 |
-
<Option name="line_style" type="QString" value="solid"/>
|
| 12 |
-
</Option>
|
| 13 |
-
</layer>
|
| 14 |
-
</symbol>
|
| 15 |
-
</symbols>
|
| 16 |
-
</renderer-v2>
|
| 17 |
-
</qgis>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
qml_units.xml
DELETED
|
@@ -1,41 +0,0 @@
|
|
| 1 |
-
<!DOCTYPE qgis PUBLIC 'http://mrcc.com/qgis.dtd' 'SYSTEM'>
|
| 2 |
-
<qgis version="3.34.0" styleCategories="Symbology|Labeling">
|
| 3 |
-
<renderer-v2 type="singleSymbol" symbollevels="0" enableorderby="0" forceraster="0">
|
| 4 |
-
<symbols>
|
| 5 |
-
<symbol type="fill" name="0" alpha="1" clip_to_extent="1" force_rhr="0">
|
| 6 |
-
<layer class="SimpleFill" enabled="1" locked="0" pass="0">
|
| 7 |
-
<Option type="Map">
|
| 8 |
-
<Option name="color" type="QString" value="200,200,200,255"/>
|
| 9 |
-
<Option name="outline_color" type="QString" value="40,40,40,255"/>
|
| 10 |
-
<Option name="outline_style" type="QString" value="solid"/>
|
| 11 |
-
<Option name="outline_width" type="QString" value="0.15"/>
|
| 12 |
-
<Option name="outline_width_unit" type="QString" value="MM"/>
|
| 13 |
-
<Option name="style" type="QString" value="solid"/>
|
| 14 |
-
</Option>
|
| 15 |
-
<data_defined_properties>
|
| 16 |
-
<Option type="Map">
|
| 17 |
-
<Option name="name" type="QString" value=""/>
|
| 18 |
-
<Option name="properties" type="Map">
|
| 19 |
-
<Option name="fillColor" type="Map">
|
| 20 |
-
<Option name="active" type="bool" value="true"/>
|
| 21 |
-
<Option name="expression" type="QString" value=""couleur_hex""/>
|
| 22 |
-
<Option name="type" type="int" value="3"/>
|
| 23 |
-
</Option>
|
| 24 |
-
</Option>
|
| 25 |
-
<Option name="type" type="QString" value="collection"/>
|
| 26 |
-
</Option>
|
| 27 |
-
</data_defined_properties>
|
| 28 |
-
</layer>
|
| 29 |
-
</symbol>
|
| 30 |
-
</symbols>
|
| 31 |
-
</renderer-v2>
|
| 32 |
-
<labeling type="simple">
|
| 33 |
-
<settings calloutType="simple">
|
| 34 |
-
<text-style fieldName="code" isExpression="0" fontSize="8" fontSizeUnit="Point" fontFamily="Arial" textColor="20,20,20,255" fontWeight="50">
|
| 35 |
-
<text-buffer bufferDraw="1" bufferSize="0.6" bufferSizeUnits="MM" bufferColor="255,255,255,200"/>
|
| 36 |
-
</text-style>
|
| 37 |
-
<placement placement="1" placementFlags="10" centroidInside="1" fitInPolygonOnly="1"/>
|
| 38 |
-
<rendering scaleVisibility="0" obstacle="1" limitNumLabels="0" displayAll="0"/>
|
| 39 |
-
</settings>
|
| 40 |
-
</labeling>
|
| 41 |
-
</qgis>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
requirements.txt
DELETED
|
@@ -1,13 +0,0 @@
|
|
| 1 |
-
torch>=2.2
|
| 2 |
-
segmentation-models-pytorch==0.6.0
|
| 3 |
-
timm==1.0.31
|
| 4 |
-
numba>=0.60
|
| 5 |
-
numpy
|
| 6 |
-
scipy
|
| 7 |
-
pandas
|
| 8 |
-
scikit-learn==1.9.1
|
| 9 |
-
scikit-image>=0.24
|
| 10 |
-
rasterio>=1.4
|
| 11 |
-
geopandas>=1.0
|
| 12 |
-
shapely>=2.0
|
| 13 |
-
easyocr==1.7.2
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
training/train_final.py
DELETED
|
@@ -1,471 +0,0 @@
|
|
| 1 |
-
# =====================================================================================
|
| 2 |
-
# GEOCARTO – ENTRAÎNEMENT FINAL
|
| 3 |
-
# Modèle retenu par le labo (rounds 1-3) : U-Net++ / EfficientNet-B4, entrée RGB+Lab,
|
| 4 |
-
# augmentations de couleur, affinités (10 décalages) + failles, mutex watershed.
|
| 5 |
-
# Données : dataset GeoCarto complet (6 600 tuiles, 4 continents), split spatial.
|
| 6 |
-
# Sorties (/kaggle/working/geocarto_model) :
|
| 7 |
-
# best.pt (poids + config), best_fp16.pt, history.csv, test_metrics.csv, metrics.json, figures
|
| 8 |
-
# =====================================================================================
|
| 9 |
-
import os, io, sys, json, time, copy
|
| 10 |
-
from pathlib import Path
|
| 11 |
-
import numpy as np, pandas as pd
|
| 12 |
-
import pyarrow.parquet as pq
|
| 13 |
-
from PIL import Image
|
| 14 |
-
from scipy import ndimage as ndi
|
| 15 |
-
from skimage.metrics import adapted_rand_error, variation_of_information
|
| 16 |
-
import torch, torch.nn as nn, torch.nn.functional as F
|
| 17 |
-
from torch.utils.data import Dataset, DataLoader
|
| 18 |
-
import segmentation_models_pytorch as smp
|
| 19 |
-
from numba import njit
|
| 20 |
-
|
| 21 |
-
E = os.environ.get
|
| 22 |
-
STEPS = int(E("GC_STEPS", 30000)) # pas d'entraînement
|
| 23 |
-
BATCH, CROP, LR = int(E("GC_BATCH", 8)), int(E("GC_CROP", 448)), float(E("GC_LR", 3e-4))
|
| 24 |
-
EVAL_EVERY = int(E("GC_EVAL_EVERY", 2500))
|
| 25 |
-
N_VAL = int(E("GC_NVAL", 12)) # tuiles de validation par source (suivi pendant l'entraînement)
|
| 26 |
-
P_REAL = float(E("GC_PREAL", 0.35)) # part de tuiles RÉELLES dans chaque lot (elles ne font que 21 % du dataset)
|
| 27 |
-
MAX_HOURS = float(E("GC_MAX_HOURS", 8.0)) # garde-fou : on arrête l'entraînement avant la limite Kaggle (12 h)
|
| 28 |
-
N_TEST = int(E("GC_NTEST", 0)) # 0 = tout le jeu de test
|
| 29 |
-
OUT = Path(E("GC_OUT", "/kaggle/working/geocarto_model")); OUT.mkdir(parents=True, exist_ok=True)
|
| 30 |
-
DEV = "cuda" if torch.cuda.is_available() else "cpu"
|
| 31 |
-
T_START = time.time()
|
| 32 |
-
print("Matériel :", DEV, torch.cuda.get_device_name(0) if DEV == "cuda" else "", flush=True)
|
| 33 |
-
|
| 34 |
-
CFG = dict(arch="unetpp", encoder=E("GC_ENCODER", "efficientnet-b4"), input="rgb+lab",
|
| 35 |
-
offsets=[(1, 0), (0, 1), (3, 0), (0, 3), (9, 0), (0, 9), (9, 9), (9, -9), (27, 0), (0, 27)],
|
| 36 |
-
n_short=2, mws_stride=int(E("GC_MWS_STRIDE", 1)), mws_min_size=int(E("GC_MINSIZE", 40)), tile=512,
|
| 37 |
-
small_w=E("GC_SMALLW", "0") == "1", ohem=E("GC_OHEM", "0") == "1", tta=E("GC_TTA", "0") == "1")
|
| 38 |
-
OFFSETS, N_SHORT, K = CFG["offsets"], CFG["n_short"], len(CFG["offsets"])
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
# ------------------------------------------------------------------ données
|
| 42 |
-
def find_dir(marker):
|
| 43 |
-
root = Path(E("GC_INPUT", "/kaggle/input"))
|
| 44 |
-
hits = sorted(root.rglob(marker))
|
| 45 |
-
return hits[0].parent if hits else None
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
DATA = Path(E("GC_DATA")) if E("GC_DATA") else find_dir("test-00000.parquet") # dataset complet
|
| 49 |
-
HARD = Path(E("GC_HARD")) if E("GC_HARD") else find_dir("hard_mono-00000.parquet") # jeux pièges (geocarto-mini)
|
| 50 |
-
assert DATA is not None, "dataset GeoCarto introuvable (ajoute la sortie du notebook geocarto-dataset en entrée)"
|
| 51 |
-
print("Données :", DATA, "| pièges :", HARD, flush=True)
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
def read_split(folder, split):
|
| 55 |
-
"""Garde les PNG compressés en mémoire (2 Go au lieu de 11 Go décodés) ; décodage à la volée."""
|
| 56 |
-
rows = []
|
| 57 |
-
for f in sorted(Path(folder).glob(f"{split}-*.parquet")):
|
| 58 |
-
t = pq.read_table(f, columns=["id", "source", "continent", "kind", "image", "instances", "faults"]).to_pylist()
|
| 59 |
-
rows += [dict(id=r["id"], source=r["source"], continent=r["continent"], kind=r["kind"],
|
| 60 |
-
image=r["image"]["bytes"], inst=r["instances"]["bytes"], faults=r["faults"]["bytes"]) for r in t]
|
| 61 |
-
return rows
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
def decode(r):
|
| 65 |
-
rgb = np.asarray(Image.open(io.BytesIO(r["image"])).convert("RGB"))
|
| 66 |
-
inst = np.asarray(Image.open(io.BytesIO(r["inst"]))).astype(np.int32)
|
| 67 |
-
flt = (np.asarray(Image.open(io.BytesIO(r["faults"]))) > 0).astype(np.uint8)
|
| 68 |
-
return rgb, inst, flt
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
TRAIN = read_split(DATA, "train")
|
| 72 |
-
VAL_ALL = read_split(DATA, "validation")
|
| 73 |
-
TEST = read_split(DATA, "test")
|
| 74 |
-
HARDSETS = {k: read_split(HARD, k) for k in ("hard_mono", "hard_pattern")} if HARD else {}
|
| 75 |
-
REAL_IDX = [i for i, r in enumerate(TRAIN) if r["kind"] == "real"]
|
| 76 |
-
SYN_IDX = [i for i, r in enumerate(TRAIN) if r["kind"] != "real"]
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
def spread(rows, n):
|
| 80 |
-
return [rows[i] for i in np.linspace(0, len(rows) - 1, min(n, len(rows))).astype(int)] if rows else []
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
VAL = {} # suivi : n tuiles par (source, type), réparties régulièrement
|
| 84 |
-
for key in sorted({(r["source"], r["kind"]) for r in VAL_ALL}):
|
| 85 |
-
VAL[key] = spread([r for r in VAL_ALL if (r["source"], r["kind"]) == key], N_VAL * (3 if key[1] == "real" else 1))
|
| 86 |
-
print(f"train {len(TRAIN)} ({len(REAL_IDX)} réelles) | validation suivie {sum(map(len, VAL.values()))} | test {len(TEST)} | "
|
| 87 |
-
+ " | ".join(f"{k} {len(v)}" for k, v in HARDSETS.items()), flush=True)
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
# ------------------------------------------------------------------ affinités + mutex watershed
|
| 91 |
-
def fill_holes(inst):
|
| 92 |
-
"""Chaque zone sans polygone (mer, lac, halo) devient sa propre r��gion."""
|
| 93 |
-
holes, n = ndi.label(inst == 0)
|
| 94 |
-
if n == 0:
|
| 95 |
-
return inst
|
| 96 |
-
out = inst.copy()
|
| 97 |
-
out[holes > 0] = holes[holes > 0] + inst.max()
|
| 98 |
-
return out
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
def affinity_targets(inst):
|
| 102 |
-
H, W = inst.shape
|
| 103 |
-
aff, mask = np.zeros((K, H, W), np.float32), np.zeros((K, H, W), np.float32)
|
| 104 |
-
for k, (dy, dx) in enumerate(OFFSETS):
|
| 105 |
-
ys, ye, xs, xe = max(0, -dy), H - max(0, dy), max(0, -dx), W - max(0, dx)
|
| 106 |
-
a, b = inst[ys:ye, xs:xe], inst[ys + dy:ye + dy, xs + dx:xe + dx]
|
| 107 |
-
aff[k, ys:ye, xs:xe] = a == b
|
| 108 |
-
mask[k, ys:ye, xs:xe] = 1 # après fill_holes, tous les pixels ont une région
|
| 109 |
-
return aff, mask
|
| 110 |
-
|
| 111 |
-
|
| 112 |
-
@njit
|
| 113 |
-
def _find(parent, x):
|
| 114 |
-
while parent[x] != x:
|
| 115 |
-
parent[x] = parent[parent[x]]
|
| 116 |
-
x = parent[x]
|
| 117 |
-
return x
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
@njit
|
| 121 |
-
def _mws_core(n, u, v, rep):
|
| 122 |
-
parent = np.arange(n); size = np.ones(n, np.int64); m = len(u)
|
| 123 |
-
mutex_a = np.empty(m, np.int64); mutex_b = np.empty(m, np.int64); nm = 0
|
| 124 |
-
head = -np.ones(n, np.int64); nxt = np.empty(2 * m, np.int64); val = np.empty(2 * m, np.int64); nl = 0
|
| 125 |
-
for e in range(m):
|
| 126 |
-
ra, rb = _find(parent, u[e]), _find(parent, v[e])
|
| 127 |
-
if ra == rb:
|
| 128 |
-
continue
|
| 129 |
-
if rep[e]:
|
| 130 |
-
val[nl] = nm; nxt[nl] = head[ra]; head[ra] = nl; nl += 1
|
| 131 |
-
val[nl] = nm; nxt[nl] = head[rb]; head[rb] = nl; nl += 1
|
| 132 |
-
mutex_a[nm] = ra; mutex_b[nm] = rb; nm += 1
|
| 133 |
-
else:
|
| 134 |
-
small, other = (ra, rb) if size[ra] <= size[rb] else (rb, ra)
|
| 135 |
-
blocked = False; p = head[small]
|
| 136 |
-
while p != -1:
|
| 137 |
-
k = val[p]; x = _find(parent, mutex_a[k]); y = _find(parent, mutex_b[k])
|
| 138 |
-
if (x == small and y == other) or (x == other and y == small):
|
| 139 |
-
blocked = True; break
|
| 140 |
-
p = nxt[p]
|
| 141 |
-
if blocked:
|
| 142 |
-
continue
|
| 143 |
-
parent[small] = other; size[other] += size[small]; p = head[small]
|
| 144 |
-
if p != -1:
|
| 145 |
-
while nxt[p] != -1:
|
| 146 |
-
p = nxt[p]
|
| 147 |
-
nxt[p] = head[other]; head[other] = head[small]
|
| 148 |
-
out = np.empty(n, np.int64)
|
| 149 |
-
for i in range(n):
|
| 150 |
-
out[i] = _find(parent, i)
|
| 151 |
-
return out
|
| 152 |
-
|
| 153 |
-
|
| 154 |
-
def mutex_watershed(aff, stride=CFG["mws_stride"], min_size=CFG["mws_min_size"]):
|
| 155 |
-
_, H, W = aff.shape
|
| 156 |
-
idx = np.arange(H * W).reshape(H, W)
|
| 157 |
-
us, vs, ws, rep = [], [], [], []
|
| 158 |
-
for k, (dy, dx) in enumerate(OFFSETS):
|
| 159 |
-
a = idx[max(0, -dy):H - max(0, dy), max(0, -dx):W - max(0, dx)]
|
| 160 |
-
b = idx[max(0, dy):H - max(0, -dy), max(0, dx):W - max(0, -dx)]
|
| 161 |
-
w = aff[k, max(0, -dy):H - max(0, dy), max(0, -dx):W - max(0, dx)].copy()
|
| 162 |
-
if k >= N_SHORT:
|
| 163 |
-
a, b, w = a[::stride, ::stride], b[::stride, ::stride], 1 - w[::stride, ::stride]
|
| 164 |
-
us.append(a.ravel()); vs.append(b.ravel()); ws.append(w.ravel()); rep.append(np.full(a.size, k >= N_SHORT))
|
| 165 |
-
u, v, w, rep = map(np.concatenate, (us, vs, ws, rep))
|
| 166 |
-
order = np.argsort(-w, kind="stable")
|
| 167 |
-
lab = _mws_core(H * W, u[order], v[order], rep[order]).reshape(H, W)
|
| 168 |
-
_, lab = np.unique(lab, return_inverse=True); lab = lab.reshape(H, W) + 1
|
| 169 |
-
sizes = np.bincount(lab.ravel()); tiny = sizes < min_size; tiny[0] = False
|
| 170 |
-
if tiny[lab].any():
|
| 171 |
-
_, (iy, ix) = ndi.distance_transform_edt(tiny[lab], return_indices=True)
|
| 172 |
-
lab = lab[iy, ix]; _, lab = np.unique(lab, return_inverse=True); lab = lab.reshape(H, W) + 1
|
| 173 |
-
return lab.astype(np.int32)
|
| 174 |
-
|
| 175 |
-
|
| 176 |
-
# ------------------------------------------------------------------ augmentations
|
| 177 |
-
def color_aug(rgb, rng):
|
| 178 |
-
"""Casse le lien 'couleur = unité' : rotation de teinte, saturation, gains, permutation, gris, contraste, bruit."""
|
| 179 |
-
x = rgb.astype(np.float32) / 255
|
| 180 |
-
if rng.random() < 0.8:
|
| 181 |
-
T = np.array([[0.299, 0.587, 0.114], [0.596, -0.274, -0.322], [0.211, -0.523, 0.312]], np.float32)
|
| 182 |
-
a = rng.uniform(-np.pi, np.pi)
|
| 183 |
-
Rm = np.array([[1, 0, 0], [0, np.cos(a), -np.sin(a)], [0, np.sin(a), np.cos(a)]], np.float32)
|
| 184 |
-
Rm[1:, 1:] *= rng.uniform(0.6, 1.3)
|
| 185 |
-
x = x @ (np.linalg.inv(T) @ Rm @ T).T
|
| 186 |
-
if rng.random() < 0.3:
|
| 187 |
-
x = x[..., rng.permutation(3)]
|
| 188 |
-
x = x * rng.uniform(0.85, 1.15, 3) + rng.uniform(-0.06, 0.06, 3)
|
| 189 |
-
if rng.random() < 0.1:
|
| 190 |
-
x = np.repeat(x.mean(-1, keepdims=True), 3, -1)
|
| 191 |
-
m = x.mean()
|
| 192 |
-
x = (x - m) * rng.uniform(0.75, 1.2) + m
|
| 193 |
-
if rng.random() < 0.3:
|
| 194 |
-
x = x + rng.normal(0, rng.uniform(0.005, 0.03), x.shape)
|
| 195 |
-
return np.clip(x * 255, 0, 255).astype(np.uint8)
|
| 196 |
-
|
| 197 |
-
|
| 198 |
-
class TrainSet(Dataset):
|
| 199 |
-
def __len__(self):
|
| 200 |
-
return STEPS * BATCH
|
| 201 |
-
|
| 202 |
-
def __getitem__(self, i):
|
| 203 |
-
rng = np.random.default_rng([i, 20261112])
|
| 204 |
-
pool = REAL_IDX if (REAL_IDX and rng.random() < P_REAL) else SYN_IDX
|
| 205 |
-
rgb, inst, flt = decode(TRAIN[pool[rng.integers(len(pool))]])
|
| 206 |
-
inst = fill_holes(inst) # sur la tuile entière, avant recadrage
|
| 207 |
-
y, x = rng.integers(0, rgb.shape[0] - CROP + 1), rng.integers(0, rgb.shape[1] - CROP + 1)
|
| 208 |
-
rgb, inst, flt = (a[y:y + CROP, x:x + CROP] for a in (rgb, inst, flt))
|
| 209 |
-
k = int(rng.integers(4))
|
| 210 |
-
rgb, inst, flt = (np.rot90(a, k) for a in (rgb, inst, flt))
|
| 211 |
-
if rng.random() < 0.5:
|
| 212 |
-
rgb, inst, flt = rgb[:, ::-1], inst[:, ::-1], flt[:, ::-1]
|
| 213 |
-
rgb = color_aug(rgb, rng)
|
| 214 |
-
inst = np.ascontiguousarray(inst)
|
| 215 |
-
aff, mask = affinity_targets(inst)
|
| 216 |
-
if CFG["small_w"]: # les arêtes touchant un petit polygone pèsent plus (jusqu'à x8)
|
| 217 |
-
area = np.bincount(inst.ravel())
|
| 218 |
-
w = np.clip(np.sqrt(2000.0 / np.maximum(area[inst], 1)), 1.0, 8.0).astype(np.float32)
|
| 219 |
-
H, W = inst.shape
|
| 220 |
-
for k, (dy, dx) in enumerate(OFFSETS):
|
| 221 |
-
ys, ye, xs, xe = max(0, -dy), H - max(0, dy), max(0, -dx), W - max(0, dx)
|
| 222 |
-
mask[k, ys:ye, xs:xe] = np.maximum(w[ys:ye, xs:xe], w[ys + dy:ye + dy, xs + dx:xe + dx])
|
| 223 |
-
return (torch.from_numpy(np.ascontiguousarray(rgb).transpose(2, 0, 1).copy()), torch.from_numpy(aff),
|
| 224 |
-
torch.from_numpy(mask), torch.from_numpy(np.ascontiguousarray(flt, np.float32))[None])
|
| 225 |
-
|
| 226 |
-
|
| 227 |
-
# ------------------------------------------------------------------ modèle
|
| 228 |
-
def rgb2lab(x):
|
| 229 |
-
"""x : B x 3 x H x W dans [0,1] -> Lab normalisé (L/50-1, a/64, b/64)."""
|
| 230 |
-
lin = torch.where(x <= 0.04045, x / 12.92, ((x + 0.055) / 1.055) ** 2.4)
|
| 231 |
-
M = torch.tensor([[0.4124, 0.3576, 0.1805], [0.2126, 0.7152, 0.0722], [0.0193, 0.1192, 0.9505]], device=x.device)
|
| 232 |
-
xyz = torch.einsum("ij,bjhw->bihw", M, lin) / torch.tensor([0.95047, 1.0, 1.08883], device=x.device).view(1, 3, 1, 1)
|
| 233 |
-
f = torch.where(xyz > 0.008856, xyz.clamp(min=1e-6) ** (1 / 3), 7.787 * xyz + 16 / 116)
|
| 234 |
-
L, a, b = 116 * f[:, 1] - 16, 500 * (f[:, 0] - f[:, 1]), 200 * (f[:, 1] - f[:, 2])
|
| 235 |
-
return torch.stack([L / 50 - 1, a / 64, b / 64], 1)
|
| 236 |
-
|
| 237 |
-
|
| 238 |
-
class GeoCartoNet(nn.Module):
|
| 239 |
-
"""Entrée : image uint8 B x 3 x H x W (H, W multiples de 32). Sortie : K affinités + 1 carte de failles (logits)."""
|
| 240 |
-
|
| 241 |
-
def __init__(self, cfg, pretrained=True):
|
| 242 |
-
super().__init__()
|
| 243 |
-
self.cfg = cfg
|
| 244 |
-
self.register_buffer("mean", torch.tensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1))
|
| 245 |
-
self.register_buffer("std", torch.tensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1))
|
| 246 |
-
Arch = {"unet": smp.Unet, "unetpp": smp.UnetPlusPlus}[cfg["arch"]]
|
| 247 |
-
self.net = Arch(cfg["encoder"], encoder_weights="imagenet" if pretrained else None, in_channels=6,
|
| 248 |
-
classes=len(cfg["offsets"]) + 1)
|
| 249 |
-
|
| 250 |
-
def forward(self, x8):
|
| 251 |
-
with torch.autocast(x8.device.type, enabled=False): # conversion de couleur en précision complète
|
| 252 |
-
x = x8.float() / 255
|
| 253 |
-
x = torch.cat([(x - self.mean) / self.std, rgb2lab(x)], 1)
|
| 254 |
-
return self.net(x)
|
| 255 |
-
|
| 256 |
-
|
| 257 |
-
def dice(p, t, eps=1.0, dims=None):
|
| 258 |
-
return 1 - (2 * (p * t).sum(dims) + eps) / (p.sum(dims) + t.sum(dims) + eps)
|
| 259 |
-
|
| 260 |
-
|
| 261 |
-
def loss_fn(out, aff, mask, flt):
|
| 262 |
-
bce = F.binary_cross_entropy_with_logits(out[:, :K], aff, reduction="none") * mask
|
| 263 |
-
la = bce.sum() / mask.sum().clamp(min=1)
|
| 264 |
-
if CFG["ohem"]: # + moyenne des 5 % d'arêtes les plus mal prédites
|
| 265 |
-
v = bce.flatten()
|
| 266 |
-
la = la + v.topk(max(1, v.numel() // 20)).values.mean() / mask.mean().clamp(min=1e-6)
|
| 267 |
-
p, t = (1 - torch.sigmoid(out[:, :K])) * mask, (1 - aff) * mask # Dice sur les COUPURES (classe rare)
|
| 268 |
-
la = la + dice(p, t, dims=(0, 2, 3)).mean()
|
| 269 |
-
lf = F.binary_cross_entropy_with_logits(out[:, K:], flt, pos_weight=torch.tensor(10.0, device=out.device)) \
|
| 270 |
-
+ dice(torch.sigmoid(out[:, K:]), flt)
|
| 271 |
-
return la + 0.5 * lf
|
| 272 |
-
|
| 273 |
-
|
| 274 |
-
# ------------------------------------------------------------------ évaluation
|
| 275 |
-
@torch.no_grad()
|
| 276 |
-
def predict(model, rgb):
|
| 277 |
-
x = torch.from_numpy(rgb.transpose(2, 0, 1).copy())[None].to(DEV)
|
| 278 |
-
with torch.autocast(DEV, enabled=DEV == "cuda"):
|
| 279 |
-
o = model(x)
|
| 280 |
-
o = torch.sigmoid(o.float())[0].cpu().numpy()
|
| 281 |
-
return o[:K], o[K]
|
| 282 |
-
|
| 283 |
-
|
| 284 |
-
def predict_tta(model, rgb):
|
| 285 |
-
"""Moyenne sur 4 miroirs ; chaque décalage ramené dans le repère d'origine (testé : exact)."""
|
| 286 |
-
H, W = rgb.shape[:2]
|
| 287 |
-
Y, X = np.mgrid[:H, :W]
|
| 288 |
-
idx = {tuple(o): k for k, o in enumerate(OFFSETS)}
|
| 289 |
-
acc, accf = 0, 0
|
| 290 |
-
for fy in (0, 1):
|
| 291 |
-
for fx in (0, 1):
|
| 292 |
-
im = rgb[::-1] if fy else rgb
|
| 293 |
-
im = im[:, ::-1] if fx else im
|
| 294 |
-
A, pf = predict(model, np.ascontiguousarray(im))
|
| 295 |
-
py, px = (H - 1 - Y) if fy else Y, (W - 1 - X) if fx else X
|
| 296 |
-
out = np.empty_like(A)
|
| 297 |
-
for k, (dy, dx) in enumerate(OFFSETS):
|
| 298 |
-
t = (-dy if fy else dy, -dx if fx else dx)
|
| 299 |
-
if t in idx:
|
| 300 |
-
out[k] = A[idx[t]][py, px]
|
| 301 |
-
else:
|
| 302 |
-
out[k] = A[idx[(-t[0], -t[1])]][np.clip(py + t[0], 0, H - 1), np.clip(px + t[1], 0, W - 1)]
|
| 303 |
-
acc = acc + out; accf = accf + pf[py, px]
|
| 304 |
-
return acc / 4, accf / 4
|
| 305 |
-
|
| 306 |
-
|
| 307 |
-
def tile_metrics(pred, pf, inst, flt):
|
| 308 |
-
gt = fill_holes(inst)
|
| 309 |
-
are = adapted_rand_error(gt, pred)[0]
|
| 310 |
-
split, merge = variation_of_information(gt, pred) # H(pred|vérité) = sur-découpage ; H(vérité|pred) = fusion
|
| 311 |
-
small = [0, 0]; big = [0, 0]
|
| 312 |
-
ids, area = np.unique(inst[inst > 0], return_counts=True)
|
| 313 |
-
if len(ids):
|
| 314 |
-
# IoU de chaque instance avec la région prédite qui la recouvre le plus
|
| 315 |
-
pair = inst.astype(np.int64) * (pred.max() + 1) + pred
|
| 316 |
-
pv, pc = np.unique(pair[inst > 0], return_counts=True)
|
| 317 |
-
gi, pi = pv // (pred.max() + 1), pv % (pred.max() + 1)
|
| 318 |
-
parea = np.bincount(pred.ravel())
|
| 319 |
-
order = np.lexsort((-pc, gi))
|
| 320 |
-
first = np.r_[True, gi[order][1:] != gi[order][:-1]]
|
| 321 |
-
g, inter, pa = gi[order][first], pc[order][first], parea[pi[order][first]]
|
| 322 |
-
a = area[np.searchsorted(ids, g)]
|
| 323 |
-
ok = inter / (a + pa - inter) >= 0.5
|
| 324 |
-
small = [int(ok[a < 1000].sum()), int((a < 1000).sum())]; big = [int(ok[a >= 1000].sum()), int((a >= 1000).sum())]
|
| 325 |
-
mid = [int(ok[(a >= 200) & (a < 1000)].sum()), int(((a >= 200) & (a < 1000)).sum())]
|
| 326 |
-
tiny = [int(ok[a < 200].sum()), int((a < 200).sum())]
|
| 327 |
-
surf = [float((a * ok).sum()), float(a.sum())]
|
| 328 |
-
else:
|
| 329 |
-
mid = tiny = [0, 0]; surf = [0.0, 0.0]
|
| 330 |
-
pb, gb = pf > 0.5, flt > 0
|
| 331 |
-
gd, pd_ = ndi.binary_dilation(gb, iterations=2), ndi.binary_dilation(pb, iterations=2)
|
| 332 |
-
return dict(ARAND=are, split=split, merge=merge, s_ok=small[0], s_n=small[1], b_ok=big[0], b_n=big[1],
|
| 333 |
-
t_ok=tiny[0], t_n=tiny[1], m_ok=mid[0], m_n=mid[1], surf_ok=surf[0], surf_n=surf[1],
|
| 334 |
-
tp=int((pb & gd).sum()), fp=int((pb & ~gd).sum()), fn=int((gb & ~pd_).sum()))
|
| 335 |
-
|
| 336 |
-
|
| 337 |
-
def summarize(rows):
|
| 338 |
-
d = pd.DataFrame(rows)
|
| 339 |
-
f1 = 2 * d.tp.sum() / max(2 * d.tp.sum() + d.fp.sum() + d.fn.sum(), 1) if (d.tp.sum() + d.fn.sum()) > 0 else np.nan
|
| 340 |
-
return dict(tuiles=len(d), ARAND=d.ARAND.mean(), sur_decoupage=d.split.mean(), fusion=d["merge"].mean(),
|
| 341 |
-
rappel_petites=d.s_ok.sum() / max(d.s_n.sum(), 1), rappel_grandes=d.b_ok.sum() / max(d.b_n.sum(), 1),
|
| 342 |
-
rappel_40_200=d.t_ok.sum() / max(d.t_n.sum(), 1), rappel_200_1000=d.m_ok.sum() / max(d.m_n.sum(), 1),
|
| 343 |
-
rappel_tous=(d.s_ok.sum() + d.b_ok.sum()) / max(d.s_n.sum() + d.b_n.sum(), 1),
|
| 344 |
-
surface_correcte=d.surf_ok.sum() / max(d.surf_n.sum(), 1), F1_failles=f1)
|
| 345 |
-
|
| 346 |
-
|
| 347 |
-
def evaluate(model, rows, tta=False):
|
| 348 |
-
model.eval()
|
| 349 |
-
out = []
|
| 350 |
-
for r in rows:
|
| 351 |
-
rgb, inst, flt = decode(r)
|
| 352 |
-
aff, pf = predict_tta(model, rgb) if tta else predict(model, rgb)
|
| 353 |
-
out.append(tile_metrics(mutex_watershed(aff), pf, inst, flt))
|
| 354 |
-
return out
|
| 355 |
-
|
| 356 |
-
|
| 357 |
-
def val_score(model):
|
| 358 |
-
"""Score de suivi : moyenne de l'ARAND réel et de l'ARAND synthétique (toutes sources à poids égal)."""
|
| 359 |
-
res = {k: summarize(evaluate(model, v)) for k, v in VAL.items()}
|
| 360 |
-
real = np.mean([v["ARAND"] for k, v in res.items() if k[1] == "real"])
|
| 361 |
-
syn = np.mean([v["ARAND"] for k, v in res.items() if k[1] != "real"])
|
| 362 |
-
return 0.5 * (real + syn), real, syn, res
|
| 363 |
-
|
| 364 |
-
|
| 365 |
-
# ------------------------------------------------------------------ entraînement
|
| 366 |
-
torch.manual_seed(0)
|
| 367 |
-
model = GeoCartoNet(CFG).to(DEV)
|
| 368 |
-
ema = copy.deepcopy(model).eval() # moyenne glissante des poids (souvent plus stable)
|
| 369 |
-
for p in ema.parameters():
|
| 370 |
-
p.requires_grad_(False)
|
| 371 |
-
EMA_DECAY = 0.999
|
| 372 |
-
dl = DataLoader(TrainSet(), BATCH, shuffle=False, num_workers=min(4, os.cpu_count() or 1), pin_memory=DEV == "cuda",
|
| 373 |
-
drop_last=True)
|
| 374 |
-
opt = torch.optim.AdamW(model.parameters(), LR, weight_decay=1e-4)
|
| 375 |
-
sched = torch.optim.lr_scheduler.OneCycleLR(opt, LR, total_steps=STEPS, pct_start=0.05)
|
| 376 |
-
scaler = torch.amp.GradScaler("cuda", enabled=DEV == "cuda")
|
| 377 |
-
mutex_watershed(np.random.rand(K, 64, 64).astype(np.float32)) # compilation numba
|
| 378 |
-
print(f"Paramètres : {sum(p.numel() for p in model.parameters()) / 1e6:.1f} M | {STEPS} pas, lot {BATCH}, recadrage {CROP}", flush=True)
|
| 379 |
-
|
| 380 |
-
HIST, best, t0, run_loss, n_loss = [], 1e9, time.time(), 0.0, 0
|
| 381 |
-
model.train()
|
| 382 |
-
step = 0
|
| 383 |
-
for step, batch in enumerate(dl, 1):
|
| 384 |
-
x, aff, mask, flt = (t.to(DEV, non_blocking=True) for t in batch)
|
| 385 |
-
with torch.autocast(DEV, enabled=DEV == "cuda"):
|
| 386 |
-
out = model(x)
|
| 387 |
-
loss = loss_fn(out.float(), aff, mask, flt)
|
| 388 |
-
if not torch.isfinite(loss):
|
| 389 |
-
print(f" pas {step} : perte non finie, lot ignoré", flush=True)
|
| 390 |
-
opt.zero_grad(set_to_none=True); sched.step()
|
| 391 |
-
continue
|
| 392 |
-
opt.zero_grad(set_to_none=True)
|
| 393 |
-
scaler.scale(loss).backward()
|
| 394 |
-
scaler.unscale_(opt); torch.nn.utils.clip_grad_norm_(model.parameters(), 5.0)
|
| 395 |
-
scaler.step(opt); scaler.update(); sched.step()
|
| 396 |
-
with torch.no_grad():
|
| 397 |
-
d = min(EMA_DECAY, (1 + step) / (10 + step))
|
| 398 |
-
for pe, pm in zip(ema.parameters(), model.parameters()):
|
| 399 |
-
pe.mul_(d).add_(pm.detach(), alpha=1 - d)
|
| 400 |
-
for be, bm in zip(ema.buffers(), model.buffers()):
|
| 401 |
-
be.copy_(bm)
|
| 402 |
-
run_loss += loss.item(); n_loss += 1
|
| 403 |
-
if step % 250 == 0:
|
| 404 |
-
print(f"pas {step}/{STEPS} | perte {run_loss / n_loss:.4f} | {(time.time() - t0) / 60:.0f} min", flush=True)
|
| 405 |
-
hours = (time.time() - T_START) / 3600
|
| 406 |
-
last = step == STEPS or hours > MAX_HOURS
|
| 407 |
-
if step % EVAL_EVERY == 0 or last:
|
| 408 |
-
row = dict(step=step, loss=run_loss / max(n_loss, 1), heures=round(hours, 2))
|
| 409 |
-
for tag, m in (("modele", model), ("ema", ema)):
|
| 410 |
-
sc, real, syn, _ = val_score(m)
|
| 411 |
-
row.update({f"{tag}_score": sc, f"{tag}_reel": real, f"{tag}_synth": syn})
|
| 412 |
-
if sc < best:
|
| 413 |
-
best = sc
|
| 414 |
-
torch.save({"state_dict": m.state_dict(), "cfg": CFG, "step": step, "val_score": float(sc), "poids": tag}, OUT / "best.pt")
|
| 415 |
-
row["meilleur"] = tag
|
| 416 |
-
model.train()
|
| 417 |
-
HIST.append(row); pd.DataFrame(HIST).to_csv(OUT / "history.csv", index=False)
|
| 418 |
-
print(" VALIDATION", json.dumps({k: (round(v, 4) if isinstance(v, float) else v) for k, v in row.items()}), flush=True)
|
| 419 |
-
run_loss, n_loss = 0.0, 0
|
| 420 |
-
if last:
|
| 421 |
-
if step < STEPS:
|
| 422 |
-
print(f"Arrêt au pas {step} : limite de {MAX_HOURS} h atteinte.", flush=True)
|
| 423 |
-
break
|
| 424 |
-
del dl
|
| 425 |
-
|
| 426 |
-
# ------------------------------------------------------------------ test final (meilleur point de contrôle)
|
| 427 |
-
ck = torch.load(OUT / "best.pt", map_location=DEV)
|
| 428 |
-
model = GeoCartoNet(CFG, pretrained=False).to(DEV)
|
| 429 |
-
model.load_state_dict(ck["state_dict"]); model.eval()
|
| 430 |
-
torch.save({"state_dict": {k: (v.half() if v.is_floating_point() else v) for k, v in ck["state_dict"].items()},
|
| 431 |
-
"cfg": CFG, "step": ck["step"]}, OUT / "best_fp16.pt")
|
| 432 |
-
print(f"\nMeilleur modèle : pas {ck['step']} ({ck['poids']}), score de validation {ck['val_score']:.4f}", flush=True)
|
| 433 |
-
|
| 434 |
-
test_rows = TEST if not N_TEST else spread(TEST, N_TEST)
|
| 435 |
-
res = evaluate(model, test_rows, tta=CFG["tta"])
|
| 436 |
-
T = pd.DataFrame(res).assign(continent=[r["continent"] for r in test_rows], source=[r["source"] for r in test_rows],
|
| 437 |
-
type=[r["kind"] for r in test_rows])
|
| 438 |
-
tables = []
|
| 439 |
-
for keys in (["continent"], ["source", "type"]):
|
| 440 |
-
for name, g in T.groupby(keys):
|
| 441 |
-
name = name if isinstance(name, tuple) else (name,)
|
| 442 |
-
tables.append({"groupe": " / ".join(name), **summarize(g.to_dict("records"))})
|
| 443 |
-
tables.append({"groupe": "TEST COMPLET", **summarize(res)})
|
| 444 |
-
for k, rows in HARDSETS.items():
|
| 445 |
-
rows = rows if not N_TEST else spread(rows, N_TEST)
|
| 446 |
-
tables.append({"groupe": {"hard_mono": "PIÈGE unités de même couleur", "hard_pattern": "PIÈGE unités à motifs"}[k],
|
| 447 |
-
**summarize(evaluate(model, rows, tta=CFG["tta"]))})
|
| 448 |
-
TM = pd.DataFrame(tables).round(4)
|
| 449 |
-
TM.to_csv(OUT / "test_metrics.csv", index=False)
|
| 450 |
-
pd.set_option("display.width", 220); pd.set_option("display.max_columns", 20)
|
| 451 |
-
print("\n=== RÉSULTATS SUR LE JEU DE TEST (jamais vu à l'entraînement) ==="); print(TM.to_string(index=False), flush=True)
|
| 452 |
-
json.dump({"cfg": CFG, "steps": int(ck["step"]), "val_score": float(ck["val_score"]), "test": TM.to_dict("records"),
|
| 453 |
-
"train_hours": round((time.time() - T_START) / 3600, 2)}, open(OUT / "metrics.json", "w"), indent=1, ensure_ascii=False)
|
| 454 |
-
|
| 455 |
-
# ------------------------------------------------------------------ figures
|
| 456 |
-
import matplotlib; matplotlib.use("Agg"); import matplotlib.pyplot as plt
|
| 457 |
-
picks = [("réel", spread([r for r in TEST if r["kind"] == "real"], 4))]
|
| 458 |
-
picks += [(s, spread([r for r in TEST if r["source"] == s and r["kind"] != "real"], 2)) for s in sorted({r["source"] for r in TEST})]
|
| 459 |
-
picks += [(k, spread(v, 3)) for k, v in HARDSETS.items()]
|
| 460 |
-
rows_ = [(n, r) for n, rs in picks for r in rs]
|
| 461 |
-
fig, ax = plt.subplots(3, len(rows_), figsize=(4 * len(rows_), 12))
|
| 462 |
-
for j, (n, r) in enumerate(rows_):
|
| 463 |
-
rgb, inst, flt = decode(r); aff, pf = predict(model, rgb); pred = mutex_watershed(aff)
|
| 464 |
-
ax[0, j].imshow(rgb); ax[0, j].set_title(f"{n}\n{r['source']}", fontsize=9)
|
| 465 |
-
ax[1, j].imshow(fill_holes(inst) % 20, cmap="tab20", interpolation="nearest"); ax[1, j].set_title("vérité", fontsize=9)
|
| 466 |
-
ax[2, j].imshow(pred % 20, cmap="tab20", interpolation="nearest")
|
| 467 |
-
ax[2, j].imshow(np.ma.masked_less(pf, 0.5), cmap="gray_r", interpolation="nearest"); ax[2, j].set_title("modèle", fontsize=9)
|
| 468 |
-
for a in ax.ravel():
|
| 469 |
-
a.axis("off")
|
| 470 |
-
plt.tight_layout(); plt.savefig(OUT / "test_examples.png", dpi=55); plt.close()
|
| 471 |
-
print("Fichiers :", sorted(p.name for p in OUT.iterdir()), f"| durée totale {(time.time() - T_START) / 3600:.2f} h", flush=True)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
vectorize.py
DELETED
|
@@ -1,56 +0,0 @@
|
|
| 1 |
-
"""GeoCarto – vectorise a georeferenced geological map raster into a GeoPackage (QGIS / ArcGIS).
|
| 2 |
-
|
| 3 |
-
python vectorize.py carte.tif legende.csv sortie.gpkg [--model best_fp16.pt] [--tta]
|
| 4 |
-
|
| 5 |
-
legende.csv : one row per legend entry, columns
|
| 6 |
-
code (required, as printed on the map), nom, age, lithologie (optional),
|
| 7 |
-
and EITHER x0,y0,x1,y1 = pixel box of the legend swatch in carte.tif (best),
|
| 8 |
-
OR couleur_hex = colour of the swatch (e.g. #f2c38a).
|
| 9 |
-
Unit names come only from text read on the map and from this legend.
|
| 10 |
-
"""
|
| 11 |
-
import argparse, os, sys
|
| 12 |
-
import numpy as np, pandas as pd, rasterio
|
| 13 |
-
from skimage.color import rgb2lab
|
| 14 |
-
HERE = os.path.dirname(os.path.abspath(__file__)); sys.path.insert(0, HERE)
|
| 15 |
-
import pipeline as PL
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
def load_legend(csv, rgb):
|
| 19 |
-
L = pd.read_csv(csv, dtype={"code": str}).fillna("")
|
| 20 |
-
lab = None
|
| 21 |
-
feats, hexes = [], []
|
| 22 |
-
for r in L.itertuples():
|
| 23 |
-
if all(hasattr(r, k) and str(getattr(r, k)) != "" for k in ("x0", "y0", "x1", "y1")):
|
| 24 |
-
x0, y0, x1, y1 = (int(float(getattr(r, k))) for k in ("x0", "y0", "x1", "y1"))
|
| 25 |
-
px = rgb[y0:y1, x0:x1].reshape(-1, 3)
|
| 26 |
-
feats.append(PL.swatch_features(rgb2lab(px[None])[0])); hexes.append("#%02x%02x%02x" % tuple(int(v) for v in np.median(px, 0)))
|
| 27 |
-
elif getattr(r, "couleur_hex", ""):
|
| 28 |
-
h = r.couleur_hex.lstrip("#"); c = np.array([[int(h[i:i + 2], 16) for i in (0, 2, 4)]], np.uint8)
|
| 29 |
-
feats.append(PL.swatch_features(np.repeat(rgb2lab(c[None])[0], 50, 0))); hexes.append("#" + h)
|
| 30 |
-
else:
|
| 31 |
-
raise SystemExit(f"légende : la ligne {r.code} n'a ni case (x0,y0,x1,y1) ni couleur_hex")
|
| 32 |
-
out = pd.DataFrame(index=L.code.values)
|
| 33 |
-
for c in ("nom", "age", "lithologie"):
|
| 34 |
-
out[c] = L[c].values if c in L else ""
|
| 35 |
-
out["nom"] = np.where(out.nom == "", out.index, out.nom)
|
| 36 |
-
out["couleur_hex"] = hexes; out["feat"] = feats
|
| 37 |
-
return out
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
if __name__ == "__main__":
|
| 41 |
-
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
| 42 |
-
ap.add_argument("raster"); ap.add_argument("legend"); ap.add_argument("out")
|
| 43 |
-
ap.add_argument("--model", default=os.path.join(HERE, "best_fp16.pt"))
|
| 44 |
-
ap.add_argument("--tta", action="store_true", help="moyenne sur 4 symétries (plus lent, pas meilleur en moyenne)")
|
| 45 |
-
a = ap.parse_args()
|
| 46 |
-
import easyocr
|
| 47 |
-
with rasterio.open(a.raster) as s:
|
| 48 |
-
rgb = np.moveaxis(s.read([1, 2, 3]), 0, -1).astype(np.uint8); tr, crs = s.transform, s.crs
|
| 49 |
-
legend = load_legend(a.legend, rgb)
|
| 50 |
-
print(f"carte {rgb.shape[1]}×{rgb.shape[0]} px | légende : {len(legend)} unités")
|
| 51 |
-
model = PL.load_model(a.model)
|
| 52 |
-
reader = easyocr.Reader(["en"], gpu=False, verbose=False)
|
| 53 |
-
P, F, lab, pf = PL.run(rgb, tr, crs, model, legend, reader, os.path.join(HERE, "naming_model.pkl"), a.out,
|
| 54 |
-
tta=a.tta, ranker_pkl=os.path.join(HERE, "legend_ranker.pkl"))
|
| 55 |
-
n = P.code.ne("").sum()
|
| 56 |
-
print(f"\n{len(P)} unités ({n} nommées, {len(P) - n} 'Unité non identifiée' avec suggestions), {len(F)} failles -> {a.out}")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
water_model.pkl
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a1a1cb40dd1f3ea2de836e854ac6fcfd7365623954f581126d12b388ed8861fd
|
| 3 |
+
size 569282
|