| --- |
| license: agpl-3.0 |
| base_model: unsloth/DeepSeek-R1-0528-Qwen3-8B |
| tags: |
| - marxism-leninism |
| - grpo |
| - llama-cpp |
| - ollama |
| - political-education |
| - marxism |
| - communism |
| - political-extremism |
| language: |
| - en |
| pipeline_tag: text-generation |
| --- |
| |
| # MLMLML - Machine Learning Marxist-Leninist Models of Language |
|
|
| A GRPO fine-tuned language model for Marxist-Leninist political education and analysis. |
|
|
| ## Model Description |
|
|
| This model is fine-tuned from `unsloth/DeepSeek-R1-0528-Qwen3-8B` using Group Relative Policy Optimization (GRPO) |
| on a curated dataset of Marxist-Leninist Q&A pairs from [ProleWiki](https://en.prolewiki.org/). |
|
|
| The training rewards: |
| - **Ideological firmness**: Clear positions grounded in material analysis |
| - **Coherence**: Self-consistent, well-structured responses |
| - **Accuracy**: Faithful to Marxist-Leninist theory and historical evidence |
|
|
| The training penalizes: |
| - "Both-sidesing" and false balance |
| - Hedging and evasive language |
| - Bourgeois framing and ahistorical claims |
|
|
| ## Writing Style |
|
|
| Following Chairman Mao's guidance in "Oppose Stereotyped Party Writing": |
| - **Vigorous, lively, fresh and forceful** - never drab or stereotyped |
| - **Audience-aware** - "When shooting an arrow, one must aim at the target" |
| - **Investigation-based** - "No investigation, no right to speak" |
| - **Clear positions** - FOR and AGAINST, using scientific argument |
|
|
| ## Usage |
|
|
| ### Download and Convert to GGUF |
|
|
| ```bash |
| # Clone the repo |
| git lfs install |
| git clone https://huggingface.co/percyraskova/MLMLML |
| cd MLMLML |
| |
| # Convert to GGUF (requires llama.cpp) |
| python ~/llama.cpp/convert_hf_to_gguf.py . --outfile MLMLML-F16.gguf --outtype f16 |
| |
| # Quantize to Q4_K_M |
| ~/llama.cpp/build/bin/llama-quantize MLMLML-F16.gguf MLMLML-Q4_K_M.gguf Q4_K_M |
| |
| # Create Ollama model |
| ollama create mlmlml -f Modelfile |
| ollama run mlmlml |
| ``` |
|
|
| ### Direct with Transformers |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model = AutoModelForCausalLM.from_pretrained("percyraskova/MLMLML") |
| tokenizer = AutoTokenizer.from_pretrained("percyraskova/MLMLML") |
| |
| inputs = tokenizer("What is imperialism?", return_tensors="pt") |
| outputs = model.generate(**inputs, max_new_tokens=512) |
| print(tokenizer.decode(outputs[0])) |
| ``` |
|
|
| ## Training Details |
|
|
| - **Base model**: unsloth/DeepSeek-R1-0528-Qwen3-8B |
| - **Method**: GRPO (Group Relative Policy Optimization) |
| - **Dataset**: ProleWiki Q&A pairs (~4500 samples) |
| - **Epochs**: 2 |
| - **Hardware**: NVIDIA A100 80GB |
|
|
| ## Limitations |
|
|
| This model is designed for educational purposes about Marxist-Leninist theory and analysis. |
| It takes clear ideological positions and is not intended to be "neutral" on class struggle, |
| imperialism, or other questions where Marxism-Leninism has definite answers. |
|
|
| ## License |
|
|
| Apache 2.0 |
|
|
| ## Citation |
|
|
| If you use this model, please cite ProleWiki as the source of training data. |
