Upload 8 files

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+SO-NMF-main/sample_data_for_SONMF.txt filter=lfs diff=lfs merge=lfs -text

SO-NMF-main/H.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9261f02573309b66218051515118b3e452e92232fc924464543cf403d471a35a
+size 936128

SO-NMF-main/License.txt

@@ -0,0 +1,31 @@
+Copyright (C) 2025, Yokohama National University, all rights reserved
+
+ACADEMIC OR NON-PROFIT ORGANIZATION NONCOMMERCIAL RESEARCH USE ONLY
+
+Permission is hereby granted, free of charge, to any academic institution or non-profit organization (hereinafter referred to as "Licensee") to use, copy, modify, and distribute this software and its associated documentation files (the "Software") for noncommercial research purposes, subject to the following conditions:
+
+1. Purpose
+The Software is provided exclusively for academic or non-profit organization noncommercial research use. Commercial use, or use by any for-profit entity, is strictly prohibited without prior written permission from the copyright holder.
+
+2. Attribution
+Any use of the Software must include proper attribution to the original author(s) and this license. Publications that result from using the Software must cite the following reference:
+M. Imamura, A. Tashiro, S. Kumano and K. Otsuka, "Synergistic Functional Spectrum Analysis: A Framework for Exploring the Multifunctional Interplay Among Multimodal Nonverbal Behaviours in Conversations," IEEE Transactions on Affective Computing, , 2024, doi: 10.1109/TAFFC.2024.3491097.
+
+3. Redistribution
+Redistributions of the Software, whether in source code or binary form, must retain this license and the above copyright notice, as well as this list of conditions.
+
+4. Modifications
+Modifications to the Software are permitted under this license, provided that the modified version is also made available under this license and appropriately attributed.
+
+5. Prohibited Uses
+The following uses are explicitly prohibited:
+*Commercial applications or deployment, including integration into for-profit services or products.
+*Use by any for-profit entity, including, but not limited to, corporations and private companies.
+*Distribution or sublicensing of the Software for financial gain or commercial benefit.
+
+6. Disclaimer of Warranty
+THE SOFTWARE IS PROVIDED "AS IS," WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT. IN NO EVENT SHALL THE COPYRIGHT HOLDER(S) BE LIABLE FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT, OR OTHERWISE, ARISING FROM, OUT OF, OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+7. Termination
+The rights granted under this license will automatically terminate if the Licensee fails to comply with the terms and conditions of this license.
+

SO-NMF-main/README.md

@@ -0,0 +1,36 @@
+SO-NMF Python Implementation
+
+This repository contains the Python implementation of SO-NMF, as described in the paper:
+“Synergistic Functional Spectrum Analysis: A Framework for Exploring the Multifunctional Interplay Among Multimodal Nonverbal Behaviours in Conversations”
+Published in IEEE Transactions on Affective Computing (doi: 10.1109/TAFFC.2024.3491097).
+This code has been verified with Python 3.8.16.
+
+1. Overview
+    SO-NMF (Semi Orthogonal Non-negative Matrix Factorization) is the core computational module employed in the framework of synergistic functional spectrum analysis (sFSA), dedicated to analyzing the communicative functions of multimodal nonverbal behaviors that appear in human conversations.
+
+2. Preliminary preparation
+    Dependency: onmf.py
+        This project uses the onmf.py module from the torch-onmf repository to implement Orthogonal Non-negative Matrix Factorization (O-NMF).
+    Steps:
+        1. Download the onmf.py file from the following link: https://github.com/signofyouth/torch-onmf/blob/main/onmf.py
+        2. Import onmf.py into your project directory.
+
+3. Example of use
+    Sample data: 
+        ・File: sample_data_for_SONMF.txt
+        ・Description: The file contains 24 columns of waveform data with a length of 19500 frames, each representing different amplitudes and frequencies. This waveform data is just dummy data. As we mentioned in our paper, the actual data used in the paper is not publicly available. We plan to create another dataset and make it publicly available in 2026 (limited to academic research in universities or public research institutes. The detailed licensing is TBA). 
+    Output: 
+        Running SO-NMF on the sample data will generate the following output files:
+            WIN.npy: The orthonormal basis matrix (the initial basis matrix for the NMF).
+            W.npy: The basis matrix resulting from SO-NMF.
+            H.npy: The coefficient matrix resulting from SO-NMF.
+            WH.npy: The product of the basis and coefficient matrices.
+    Note:
+        The default number of bases in SO-NMF.py is set to 6, as per the paper. However, this parameter can be adjusted based on user requirements.
+
+5. Citation
+    If you use any of the resources provided on this page in any of your publications, we ask you to cite the following work.
+    M. Imamura, A. Tashiro, S. Kumano and K. Otsuka, "Synergistic Functional Spectrum Analysis: A Framework for Exploring the Multifunctional Interplay Among Multimodal Nonverbal Behaviours in Conversations," IEEE Transactions on Affective Computing, 2024, doi: 10.1109/TAFFC.2024.3491097.
+
+4. Article Link
+    For more information, please refer to the original paper: https://ieeexplore.ieee.org/document/10742505

SO-NMF-main/SO-NMF.py

@@ -0,0 +1,145 @@
+#--------------------------------------------------------------------------------
+# Copyright (C) 2025, Yokohama National University, all rights reserved.
+# See License.txt in the project root for license information.
+#--------------------------------------------------------------------------------
+
+import torch
+import numpy as np
+import numpy as np
+from sklearn.metrics import mean_squared_error,mean_absolute_error
+from sklearn.decomposition import NMF
+from sklearn.metrics import mean_squared_error,mean_absolute_error
+# Import Asteris's public code (onmf.py) for ONMF. https://github.com/signofyouth/torch-onmf/blob/main/onmf.py
+from onmf import onmf, low_rank_nnpca, _local_opt_w, local_opt_w_single, local_opt_w_batch, binarize, sphere_sample_cartesian
+
+def fix_seed_ONMF(seed): 
+    np.random.seed(seed)
+    torch.manual_seed(seed)
+    torch.cuda.manual_seed_all(seed)
+    torch.backends.cudnn.deterministic = True
+
+# Extract an orthogonal basis
+def orthogonal_check(basis,new_basis): 
+    for i in range(basis.shape[0]):
+        if np.dot(basis[i],new_basis)==0: 
+            if i+1==basis.shape[0]:
+                basis=np.vstack((basis,new_basis)) 
+        else: 
+            break
+    return basis
+
+def fix_seed_NMF(seed): 
+    np.random.seed(seed)
+
+if __name__=='__main__':
+
+    input_file = "sample_data_for_SONMF.txt"
+
+    basis = 6
+    W_dict = {}  # dictionary to store W for each SEED
+
+    for SEED in range(0,100):
+        fix_seed_ONMF(SEED)
+        bs = 2
+        m  = 3
+        k  = 2
+        A = torch.randn(bs, m, k)
+        sgns = binarize(torch.Tensor([i for i in range(2**k)]).int(), k)
+
+        data_load = np.loadtxt(input_file)
+        # Convert a NumPy array to a PyTorch tensor
+        data_t = torch.tensor(data_load, dtype=torch.float32)
+        data = data_t.T
+        data_numpy = data.to('cpu').detach().numpy().copy()
+
+        W, H = onmf(data, basis, rank=10, max_iter=1000)
+        WH = W @ H
+        w_numpy = W.to('cpu').detach().numpy().copy()
+        h_numpy = H.to('cpu').detach().numpy().copy()
+        wh_numpy = WH.to('cpu').detach().numpy().copy()
+
+        # Binarize using a threshold
+        threshold = 0.1
+        W_dict[SEED] = np.where(w_numpy >= threshold,1,0)
+
+
+    w_all = None
+
+    for SEED in range(0,100):
+        if SEED == 0:
+            # Process 1st SEED (set first w_0)
+            w_0 = W_dict[SEED]
+            w_0_t = w_0.T  
+            w_all = w_0_t  
+        else:
+            w = W_dict[SEED]  
+            w_t = w.T  
+            w_all = np.append(w_all, w_t, axis=0)
+
+
+    count_list = np.empty((0, 2), dtype=object)  # array (elements and number of occurrences)
+
+    while len(w_all) > 0:
+        index = [0]
+        criterion = w_all[0]
+        
+        for i in range(1, len(w_all)):
+            if all(w_all[i] == criterion):
+                index.append(i)
+        
+        count_index = len(index)
+        
+        # Store only if count_index is non-zero
+        if count_index != 0:
+            count_list = np.append(count_list, np.array([[criterion, count_index]]), axis=0)
+        
+        # Remove elements once stored from w_all
+        for idx in sorted(index, reverse=True):
+            w_all = np.delete(w_all, idx, 0)
+
+    # Sort in descending order according to count_index
+    sorted_list = count_list[count_list[:, 1].argsort()[::-1]]
+
+    w_candi = sorted_list[:, 0]  
+
+    # Create basis matrix
+    w = np.empty((0, len(w_candi[0])), dtype=np.float64)  
+    for i in range(w_candi.shape[0]):
+        if not all(w_candi[i]==0):
+            if len(w)==0:
+                w=w_candi[i][np.newaxis,:].astype(np.float64)  
+            elif w.shape[0]<basis:
+                w=orthogonal_check(w,w_candi[i].astype(np.float64))  
+            else:
+                break
+
+    W = np.ascontiguousarray(w.T)  # orthogonal basis matrix (WIN)  
+    if W.shape[1]<basis:
+        print(f'Only {W.shape[1]} bases were obtained.')
+    else:
+        np.save('WIN.npy',W)
+
+    # NMF
+    nmf = NMF(n_components=basis , init = 'custom')
+
+    SEED = 0
+    fix_seed_NMF(SEED)
+
+    # initial basis matrix (WIN)
+    w_init = W 
+
+    # random initial coefficient matrix
+    h_init = np.random.rand(basis,data.shape[1])
+
+    W = nmf.fit_transform(data, W = w_init , H = h_init)
+    H = nmf.components_
+    WH = W @ H
+
+    # Results output
+    np.save("W", W)
+    np.save("H", H)
+    np.save("WH", WH)
+
+    # MSE evaluation
+    mse = mean_squared_error(data, WH) 
+    print("mse",mse)

SO-NMF-main/W.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8baec6ae901236b3a1ee8be3e3328e52e5fb9d0f5a0963ddbfda4fa1108558fd
+size 1280

SO-NMF-main/WH.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d4e011a54a53c4d8b4e508c90d98414459182b62962d76474f5b955c38222348
+size 3744128

SO-NMF-main/WIN.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a3659c4266e1b34c6d9b378858098970a92f3bba3848e25c6ea3bd6b87aabb9
+size 1280

SO-NMF-main/sample_data_for_SONMF.txt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cfdd01705f49b54eaa388cba0cca19d43c49ff7455c77b99c021b9208526f195
+size 11719500