README.md · BoostedJonP/powell-phi3-mini at main

powell-phi3-mini / README.md

BoostedJonP

Update README.md

5edf221 verified 11 months ago

preview code

Raw

History Blame Contribute Delete

6.71 kB

	---
	license: mit
	tags:
	- Jerome Powell AI model
	- Federal Reserve chatbot
	- fine-tuned Phi-3
	- financial language model
	- LLM fine-tuning
	- machine learning engineering
	- LoRA training
	- NLP
	datasets:
	- BoostedJonP/JeromePowell-SFT
	language:
	- en
	base_model:
	- microsoft/Phi-3-mini-4k-instruct
	pipeline_tag: text-generation
	---

	# Powell-Phi3-Mini — Jerome Powell Style Language Model

	[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-yellow)](https://huggingface.co/BoostedJonP/powell-phi3-mini)
	[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
	[![GPU Training](https://img.shields.io/badge/Trained%20on-TESLA%20P100-green)](https://images.nvidia.com/content/tesla/pdf/nvidia-tesla-p100-PCIe-datasheet.pdf)
	[![Fine-tuning](https://img.shields.io/badge/Method-LoRA%2FQLoRA-orange)](https://arxiv.org/abs/2106.09685)

	## 🎯Summary

	Powell-Phi3-Mini is an fine-tuned language model that replicates Federal Reserve Chair Jerome Powell's distinctive communication style, tone, and strategic hedging patterns. This project showcases expertise in modern LLM fine-tuning techniques, parameter-efficient training methods, and responsible AI development — demonstrating industry-ready machine learning engineering skills.

	---

	## 🚀 Key Features & Capabilities

	### Style Mimicry & Linguistic Analysis
	- ✅ Authentic Communication Style: Replicates Powell's cautious, data-dependent rhetoric
	- ✅ Strategic Hedging Patterns: Maintains appropriate uncertainty in speculative scenarios
	- ✅ Domain-Specific Responses: Handles economic and monetary policy discussions contextually
	- ✅ Refusal Training: Appropriately declines to provide financial advice or policy predictions (to an extent)

	### Technical Implementation
	- ✅ Efficient Architecture: Built on Microsoft Phi-3-mini-4k-instruct (3.8B parameters)
	- ✅ Scalable Training: LoRA r=16, alpha=32 configuration optimized for consumer GPUs
	- ✅ Deployment Flexibility: Available as lightweight adapter or full merged model
	- ✅ Integration Ready: One-line inference with Hugging Face Transformers

	---

	## 💻 Implementation Examples

	### Production Ready - Merged Model
	```python
	from transformers import AutoTokenizer, AutoModelForCausalLM

	# One-line model loading
	tokenizer = AutoTokenizer.from_pretrained("BoostedJonP/powell-phi3-mini")
	model = AutoModelForCausalLM.from_pretrained("BoostedJonP/powell-phi3-mini", device_map="auto")

	# Economic analysis prompt
	prompt = "How is the current labor market affecting your inflation outlook?"
	inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
	response = model.generate(**inputs, max_new_tokens=200, do_sample=True)
	print(tokenizer.decode(response[0], skip_special_tokens=True))
	```

	---

	## 📊 Technical Specifications & Training Pipeline

	### Model Architecture
	\| Component \| Specification \|
	\|-----------\|---------------\|
	\| Base Model \| microsoft/Phi-3-mini-4k-instruct (3.8B parameters) \|
	\| License \| MIT License (Commercial Use Approved) \|
	\| Fine-tuning Method \| QLoRA with PEFT integration \|
	\| Context Length \| 4,096 tokens \|
	\| Training Hardware \| NVIDIA TESLA P100 (16GB VRAM) \|

	### Training Configuration
	\| Hyperparameter \| Value \| Rationale \|
	\|----------------\|-------\|-----------\|
	\| LoRA Rank (r) \| 16 \| Optimal parameter/performance balance \|
	\| LoRA Alpha \| 32 \| 2x rank for stable training \|
	\| Dropout Rate \| 0.05 \| Regularization without overfitting \|
	\| Learning Rate \| 1.5e-4 \| Conservative rate for stable convergence \|
	\| Scheduler \| Cosine decay \| Smooth learning rate reduction \|
	\| Training Epochs \| 3 \| Prevents overfitting on specialized domain \|
	\| Sequence Length \| 1,536 tokens \| Optimized for dataset \|
	\| Precision \| Mixed fp16 \| 2x memory efficiency, maintained accuracy \|

	### Dataset & Methodology
	- Data Source: Public domain FOMC transcripts and Federal Reserve speeches -> [Jerome Powell Press Release Q&A](https://www.kaggle.com/datasets/jonathanpaserman/fed-press-release-text)
	- Data Processing: Instruction-response pairs extracted from press conferences -> [Jerome Powell Press Release SFT data processing](https://www.kaggle.com/code/jonathanpaserman/jerome-powell-press-release-sft-data-processing)
	- Available on [HuggingFace](https://huggingface.co/datasets/BoostedJonP/JeromePowell-SFT)
	- Quality Control: Manual review and filtering for authentic Powell communication patterns

	---

	## 📈 Performance Metrics & Evaluation

	### Quantitative Results
	\| Metric \| Baseline (Phi-3) \| Powell-Phi3-Mini \| Improvement \|
	\|--------\|------------------\|------------------\|-------------\|
	\| Powell-style Classification \|NA \| NA \| NA \|
	\| Economic Domain Accuracy \|NA \| NA \| NA \|
	\| Response Coherence (BLEU)\|NA \| NA \| NA \|

	### Qualitative Assessment
	- NA

	---

	## 🌐 Deployment & Access

	### 🚀 Live Demo
	[Try Powell-Phi3-Mini Interactive Demo →](https://huggingface.co/spaces/BoostedJonP/powell-assistant)

	### 📦 Model Downloads
	- Adapter Version: `BoostedJonP/powell-phi3-mini-adapter`
	- Merged Model: `BoostedJonP/powell-phi3-mini` (Full Model - 7.4GB)

	### 🔗 Resources
	- [GitHub Repository](https://github.com/BigJonP/powell-phi3-sft): Complete training code and evaluation scripts
	- [Technical Blog Post](https://medium.com/@jonathanpaserman): Detailed implementation walkthrough
	- [Hugging Face Collection](https://huggingface.co/collections/BoostedJonP/jerome-powell-68b9e7843f64507481d24ce9): All model variants and datasets

	---

	## ⚖️ Responsible AI & Legal Compliance

	### Ethical Considerations
	- ⚠️ No Official Affiliation: Not endorsed by or affiliated with the Federal Reserve System
	- ⚠️ Educational Purpose Only: Designed for research, education, and demonstration purposes
	- ⚠️ No Financial Advice: Model responses should not be interpreted as investment guidance
	- ⚠️ Transparency: All training data sourced from public domain government transcripts

	### Licensing & Usage Rights
	- Base Model License: MIT License (Microsoft Phi-3)
	- Fine-tuned Weights: MIT License (Commercial use permitted)
	- Training Data: Public domain (U.S. government works)
	- Usage: Unrestricted for research, education, and commercial applications

	---

	### 👨‍💻 Connect & Collaborate
	- GitHub: [Jonathan Paserman](https://github.com/BigJonP)
	- Kaggle: [Jonathan Paserman](https://www.kaggle.com/jonathanpaserman)
	- HuggingFace: [Jonathan Paserman](https://huggingface.co/BoostedJonP)

	---
	license: mit
	tags:
	- Jerome Powell AI model
	- Federal Reserve chatbot
	- fine-tuned Phi-3
	- financial language model
	- LLM fine-tuning
	- machine learning engineering
	- LoRA training
	- NLP
	datasets:
	- BoostedJonP/JeromePowell-SFT
	language:
	- en
	base_model:
	- microsoft/Phi-3-mini-4k-instruct
	pipeline_tag: text-generation
	---

	# Powell-Phi3-Mini — Jerome Powell Style Language Model

	[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-yellow)](https://huggingface.co/BoostedJonP/powell-phi3-mini)
	[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
	[![GPU Training](https://img.shields.io/badge/Trained%20on-TESLA%20P100-green)](https://images.nvidia.com/content/tesla/pdf/nvidia-tesla-p100-PCIe-datasheet.pdf)
	[![Fine-tuning](https://img.shields.io/badge/Method-LoRA%2FQLoRA-orange)](https://arxiv.org/abs/2106.09685)

	## 🎯Summary

	Powell-Phi3-Mini is an fine-tuned language model that replicates Federal Reserve Chair Jerome Powell's distinctive communication style, tone, and strategic hedging patterns. This project showcases expertise in modern LLM fine-tuning techniques, parameter-efficient training methods, and responsible AI development — demonstrating industry-ready machine learning engineering skills.

	---

	## 🚀 Key Features & Capabilities

	### Style Mimicry & Linguistic Analysis
	- ✅ Authentic Communication Style: Replicates Powell's cautious, data-dependent rhetoric
	- ✅ Strategic Hedging Patterns: Maintains appropriate uncertainty in speculative scenarios
	- ✅ Domain-Specific Responses: Handles economic and monetary policy discussions contextually
	- ✅ Refusal Training: Appropriately declines to provide financial advice or policy predictions (to an extent)

	### Technical Implementation
	- ✅ Efficient Architecture: Built on Microsoft Phi-3-mini-4k-instruct (3.8B parameters)
	- ✅ Scalable Training: LoRA r=16, alpha=32 configuration optimized for consumer GPUs
	- ✅ Deployment Flexibility: Available as lightweight adapter or full merged model
	- ✅ Integration Ready: One-line inference with Hugging Face Transformers

	---

	## 💻 Implementation Examples

	### Production Ready - Merged Model
	```python
	from transformers import AutoTokenizer, AutoModelForCausalLM

	# One-line model loading
	tokenizer = AutoTokenizer.from_pretrained("BoostedJonP/powell-phi3-mini")
	model = AutoModelForCausalLM.from_pretrained("BoostedJonP/powell-phi3-mini", device_map="auto")

	# Economic analysis prompt
	prompt = "How is the current labor market affecting your inflation outlook?"
	inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
	response = model.generate(**inputs, max_new_tokens=200, do_sample=True)
	print(tokenizer.decode(response[0], skip_special_tokens=True))
	```

	---

	## 📊 Technical Specifications & Training Pipeline

	### Model Architecture
	\| Component \| Specification \|
	\|-----------\|---------------\|
	\| Base Model \| microsoft/Phi-3-mini-4k-instruct (3.8B parameters) \|
	\| License \| MIT License (Commercial Use Approved) \|
	\| Fine-tuning Method \| QLoRA with PEFT integration \|
	\| Context Length \| 4,096 tokens \|
	\| Training Hardware \| NVIDIA TESLA P100 (16GB VRAM) \|

	### Training Configuration
	\| Hyperparameter \| Value \| Rationale \|
	\|----------------\|-------\|-----------\|
	\| LoRA Rank (r) \| 16 \| Optimal parameter/performance balance \|
	\| LoRA Alpha \| 32 \| 2x rank for stable training \|
	\| Dropout Rate \| 0.05 \| Regularization without overfitting \|
	\| Learning Rate \| 1.5e-4 \| Conservative rate for stable convergence \|
	\| Scheduler \| Cosine decay \| Smooth learning rate reduction \|
	\| Training Epochs \| 3 \| Prevents overfitting on specialized domain \|
	\| Sequence Length \| 1,536 tokens \| Optimized for dataset \|
	\| Precision \| Mixed fp16 \| 2x memory efficiency, maintained accuracy \|

	### Dataset & Methodology
	- Data Source: Public domain FOMC transcripts and Federal Reserve speeches -> [Jerome Powell Press Release Q&A](https://www.kaggle.com/datasets/jonathanpaserman/fed-press-release-text)
	- Data Processing: Instruction-response pairs extracted from press conferences -> [Jerome Powell Press Release SFT data processing](https://www.kaggle.com/code/jonathanpaserman/jerome-powell-press-release-sft-data-processing)
	- Available on [HuggingFace](https://huggingface.co/datasets/BoostedJonP/JeromePowell-SFT)
	- Quality Control: Manual review and filtering for authentic Powell communication patterns

	---

	## 📈 Performance Metrics & Evaluation

	### Quantitative Results
	\| Metric \| Baseline (Phi-3) \| Powell-Phi3-Mini \| Improvement \|
	\|--------\|------------------\|------------------\|-------------\|
	\| Powell-style Classification \|NA \| NA \| NA \|
	\| Economic Domain Accuracy \|NA \| NA \| NA \|
	\| Response Coherence (BLEU)\|NA \| NA \| NA \|

	### Qualitative Assessment
	- NA

	---

	## 🌐 Deployment & Access

	### 🚀 Live Demo
	[Try Powell-Phi3-Mini Interactive Demo →](https://huggingface.co/spaces/BoostedJonP/powell-assistant)

	### 📦 Model Downloads
	- Adapter Version: `BoostedJonP/powell-phi3-mini-adapter`
	- Merged Model: `BoostedJonP/powell-phi3-mini` (Full Model - 7.4GB)

	### 🔗 Resources
	- [GitHub Repository](https://github.com/BigJonP/powell-phi3-sft): Complete training code and evaluation scripts
	- [Technical Blog Post](https://medium.com/@jonathanpaserman): Detailed implementation walkthrough
	- [Hugging Face Collection](https://huggingface.co/collections/BoostedJonP/jerome-powell-68b9e7843f64507481d24ce9): All model variants and datasets

	---

	## ⚖️ Responsible AI & Legal Compliance

	### Ethical Considerations
	- ⚠️ No Official Affiliation: Not endorsed by or affiliated with the Federal Reserve System
	- ⚠️ Educational Purpose Only: Designed for research, education, and demonstration purposes
	- ⚠️ No Financial Advice: Model responses should not be interpreted as investment guidance
	- ⚠️ Transparency: All training data sourced from public domain government transcripts

	### Licensing & Usage Rights
	- Base Model License: MIT License (Microsoft Phi-3)
	- Fine-tuned Weights: MIT License (Commercial use permitted)
	- Training Data: Public domain (U.S. government works)
	- Usage: Unrestricted for research, education, and commercial applications

	---

	### 👨‍💻 Connect & Collaborate
	- GitHub: [Jonathan Paserman](https://github.com/BigJonP)
	- Kaggle: [Jonathan Paserman](https://www.kaggle.com/jonathanpaserman)
	- HuggingFace: [Jonathan Paserman](https://huggingface.co/BoostedJonP)