| 🧠 Joint fMRI-Text Model |
| This model jointly predicts cognitive response type, trial type, and generates a 3D fMRI-like brain activation tensor based on natural language input and user-level metadata. |
|
|
| 🧩 Inputs |
| Text: a belief or statement in natural language (e.g., "Handwashing reduces disease risk."), processed using distilbert-base-uncased. |
|
|
| Metadata: a vector of 14 user features including: |
|
|
| Scaled continuous inputs: Age_scaled, Openness_scaled, Conscientiousness_scaled, Extraversion_scaled, Agreeableness_scaled, Neuroticism_scaled, ICAR_Total_scaled, MOCA_scaled, VMN_Sum_scaled |
| |
| Encoded categorical features: Gender_encoded, Education_encoded, Ethnicity_fused_encoded, Income_level_encoded, VCBS_cat_encoded |
| |
| 🎯 Outputs |
| Response Type: probabilities over 4 possible response categories |
| |
| Trial Type: probabilities over 3 trial categories |
| |
| fMRI Tensor: synthetic output of shape (74 × 74 × 52) representing brain activity across four timepoints |
| |
| 🚀 Example Usage |
| from transformers import AutoTokenizer |
| from joint_fmri_model import JointFMRIModelWithHub |
| import torch |
| |
| model = JointFMRIModelWithHub.from_pretrained("kenchenxingyu/joint-fmri-text-model") |
| tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased") |
| |
| text = "I think regular hand washing reduces disease risk." |
| inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True) |
|
|
| with torch.no_grad(): |
| text_vec = model.text_encoder(inputs["input_ids"]).squeeze(0) |
|
|
| meta_input = torch.rand(1, 14) # Replace with realistic metadata |
| |
| output = model(text_vec.unsqueeze(0), meta_input) |
| |
| print("Response:", output["response_probs"]) |
| print("Trial:", output["trial_probs"]) |
| print("fMRI shape:", output["fmri"].shape) |
| 📂 Files |
| pytorch_model.bin: trained model weights |
|
|
| config.json: model configuration |
|
|
| README.md: this file |
|
|
| 🏷 License |
| This model is released under an open academic research license. For other use cases, please contact the author. |
|
|
