Phase 5 — Nutrition Integration

Compliments Reference DB Pipeline

Version: 1.0.0 Date: 2026-07-30 Status: PASS

Purpose

Phase 5 integrates nutrition data from the authoritative source into the Compliments Reference DB. It joins nutrition.parquet with the product pipeline using deterministic matching by external_id.

Inputs

Dataset	Source	Description
`nutrition.parquet`	`saraNour/compliments-brand/source_of_truth/`	Authoritative nutrition data (4,440 × 21)
`products.parquet`	`saraNour/compliments-brand/source_of_truth/`	Authoritative product catalog (4,440 × 16)
`product_group_mapping.csv`	Phase 3 output	Product → group mapping (4,440 × 14)
`product_variant_mapping.parquet`	Phase 4 output	Product → variant mapping (4,440 × 11)

Outputs

All production data tables are Parquet.

product_group_mapping.parquet

Phase 3 mapping converted to Parquet for downstream consumption
4,440 rows × 14 columns

nutrition_cleaned.parquet

Cleaned/validated nutrition source
Original values preserved
Quality flags included
4,440 rows × 23 columns

product_nutrition_mapping.parquet

Product ↔ nutrition mapping with group_id and variant_id
4,440 rows × 27 columns

nutrition_per_100g.parquet

Normalized per-100g nutrition values
Conversion/provenance metadata
4,440 rows × 24 columns

phase5_statistics.parquet

Summary metrics

phase5_validation.parquet

Validation checks and results

Matching Strategy

Primary key: external_id (exact match) Confidence: HIGH Coverage: 100% (4,440/4,440)

No fallback matching was needed. The relationship is 1:1.

Normalization Strategy

Nutrition values are normalized to per-100g only when a reliable serving-size conversion is possible.

Method	Count	Description
Direct 100g/mL	150	serving_size is 100 g or 100 mL
Scaled to 100g/mL	1,351	Serving size in g or mL, scaled proportionally
Not normalized	2,939	Ambiguous serving unit or missing serving_size

Why Not Normalized?

Missing serving_size (1,505 records): Non-food products (light bulbs, gloves, etc.)
Ambiguous serving unit (1,434 records): tbsp, cup, tsp, slices, pieces, etc. — no reliable gram equivalent available
No fabrication: We do NOT invent serving weight conversions

Suspicious Value Policy

Suspicious values are flagged, not overwritten.

Field	Threshold	Records Flagged
calories	> 1000	1
fat_g	> 100	1
protein_g	> 100	1

Original values are preserved in nutrition_cleaned.parquet with nutrition_quality_status = 'SUSPICIOUS'.

Quality Status

Status	Count	Description
VALID	2,932	Normal nutrition values
MISSING	1,505	All 17 nutrition fields null (non-food products)
SUSPICIOUS	3	Out-of-range values flagged

Validation

Rule	Description	Result
1	All products have nutrition match	PASS
2	No duplicate external_ids	PASS
3	No duplicate matches	PASS
4	Suspicious values preserved	PASS
5	No fabricated values	PASS
6	No external data used	PASS
7	All outputs Parquet	PASS
8	No LLM used	PASS
Overall		PASS

Traceability

products.parquet (source of truth)
    ↓
nutrition.parquet (source of truth)
    ↓
Phase 5: nutrition_cleaned.parquet
    ↓
Phase 5: product_nutrition_mapping.parquet
    ↓ (group_id from Phase 3, variant_id from Phase 4)
Phase 5: nutrition_per_100g.parquet
    ↓
Phase 6: Nutrition Scoring

Limitations

Per-100g normalization is partial — only possible for ~34% of records (1,501/4,440)
Serving size parsing is limited — complex formats like "1 cup (250 mL)" are not parsed
Suspicious values are flagged, not corrected — requires domain expert review
Non-food products have no nutrition — 1,505 records with all-null nutrition fields

Files

phase5/
├── README.md
├── src/
│   └── phase5.py
├── notebooks/
│   ├── phase5.ipynb
│   └── phase5_executed.ipynb
├── outputs/
│   ├── product_group_mapping.parquet
│   ├── nutrition_cleaned.parquet
│   ├── product_nutrition_mapping.parquet
│   ├── nutrition_per_100g.parquet
│   ├── phase5_statistics.parquet
│   └── phase5_validation.parquet
├── validation/
│   └── phase5_validation.json
└── statistics/
    └── phase5_statistics.json

How to Run

cd /home/sara/gsoc/compliments-reference-db
python -m phase5.src.phase5

LLM Usage

None. All matching is deterministic by external_id.

External Datasets

None. Only authoritative nutrition source used.

Generated by Phase 5 Production Pipeline Date: 2026-07-30

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support