tahamajs/Sysmem2_in_AI/ComputerAssignments/CA10_knowledge_graphs/models - Storage Bucket

Saved Models

Overview

This directory stores trained models and their checkpoints.

Model Storage Structure

models/
├── embeddings/
│   ├── transe_model_20250108.pt
│   ├── distmult_model_20250108.pt
│   └── complex_model_20250108.pt
├── reasoning/
│   └── reasoner_checkpoint.pt
└── checkpoints/
    └── training_checkpoint_epoch_50.pt

Saving Models

Save Embedding Model

from src.models.embeddings import TransE
import torch

# Train model
model = TransE(embedding_dim=100)
# ... training code ...

# Save model
torch.save(model.state_dict(), 'models/embeddings/transe_model.pt')

# Save full model with metadata
torch.save({
    'model_state_dict': model.state_dict(),
    'embedding_dim': 100,
    'num_entities': 1000,
    'num_relations': 50,
    'training_config': {...}
}, 'models/embeddings/transe_full.pt')

Loading Models

Load Embedding Model

import torch
from src.models.embeddings import TransE

# Load model
model = TransE(embedding_dim=100)
model.load_state_dict(torch.load('models/embeddings/transe_model.pt'))
model.eval()

# Load full model with metadata
checkpoint = torch.load('models/embeddings/transe_full.pt')
model = TransE(
    embedding_dim=checkpoint['embedding_dim'],
    num_entities=checkpoint['num_entities'],
    num_relations=checkpoint['num_relations']
)
model.load_state_dict(checkpoint['model_state_dict'])

Model Versioning

Use semantic versioning for models:

model_name_v1.0.0.pt
model_name_v1.1.0.pt
model_name_v2.0.0.pt

Model Metadata

Each model should have an accompanying JSON file with metadata:

{
  "model_name": "TransE",
  "version": "1.0.0",
  "created_at": "2025-01-08T12:00:00Z",
  "training_config": {
    "embedding_dim": 100,
    "learning_rate": 0.001,
    "num_epochs": 100,
    "batch_size": 32
  },
  "performance": {
    "hits@1": 0.45,
    "hits@3": 0.67,
    "hits@10": 0.82,
    "mean_rank": 25.3
  },
  "dataset": {
    "num_entities": 1000,
    "num_relations": 50,
    "num_triples": 5000
  }
}

Best Practices

1. Version Control: Always version your models
2. Metadata: Include comprehensive metadata
3. Checkpoints: Save checkpoints during training
4. Compression: Compress large models
5. Documentation: Document model architecture and usage

Model Registry

Keep a registry of all models:

# models/registry.json
{
  "models": [
    {
      "id": "transe_v1",
      "path": "embeddings/transe_model_v1.0.0.pt",
      "description": "TransE model trained on scientific KG",
      "status": "production"
    },
    {
      "id": "distmult_v1",
      "path": "embeddings/distmult_model_v1.0.0.pt",
      "description": "DistMult model for link prediction",
      "status": "experimental"
    }
  ]
}

Cleanup

Remove old or unused models:

# Remove models older than 30 days
find models/ -name "*.pt" -mtime +30 -delete

---

Note: Large models (>100MB) should not be committed to version control. Use Git LFS or cloud storage instead.