| 2024-11-21 14:22:37,025 INFO MainThread:2298044 [wandb_setup.py:_flush():79] Current SDK version is 0.18.7 |
| 2024-11-21 14:22:37,025 INFO MainThread:2298044 [wandb_setup.py:_flush():79] Configure stats pid to 2298044 |
| 2024-11-21 14:22:37,026 INFO MainThread:2298044 [wandb_setup.py:_flush():79] Loading settings from /home/pawanprakash/.config/wandb/settings |
| 2024-11-21 14:22:37,026 INFO MainThread:2298044 [wandb_setup.py:_flush():79] Loading settings from /blue/hennig/pawanprakash/diffusion/Diffscp_guide/hydra/singlerun/2024-11-21/alex_20/wandb/settings |
| 2024-11-21 14:22:37,026 INFO MainThread:2298044 [wandb_setup.py:_flush():79] Loading settings from environment variables: {'start_method': 'thread', 'root_dir': '/blue/hennig/pawanprakash/diffusion/Diffscp_guide/wandb'} |
| 2024-11-21 14:22:37,026 INFO MainThread:2298044 [wandb_setup.py:_flush():79] Applying setup settings: {'mode': 'online', '_disable_service': None} |
| 2024-11-21 14:22:37,026 WARNING MainThread:2298044 [wandb_setup.py:_flush():79] Could not save program above cwd: /blue/hennig/pawanprakash/diffusion/Diffscp_guide/runjobs/alex_20/./../../MaterialsDiffusion_unguided/diffcsp/run.py |
| 2024-11-21 14:22:37,026 INFO MainThread:2298044 [wandb_setup.py:_flush():79] Inferring run settings from compute environment: {'program_relpath': None, 'program_abspath': '/blue/hennig/pawanprakash/diffusion/Diffscp_guide/MaterialsDiffusion_unguided/diffcsp/run.py', 'program': '/blue/hennig/pawanprakash/diffusion/Diffscp_guide/runjobs/alex_20/./../../MaterialsDiffusion_unguided/diffcsp/run.py'} |
| 2024-11-21 14:22:37,026 INFO MainThread:2298044 [wandb_setup.py:_flush():79] Applying login settings: {} |
| 2024-11-21 14:22:37,026 INFO MainThread:2298044 [wandb_init.py:_log_setup():533] Logging user logs to ./wandb/run-20241121_142236-scq8411d/logs/debug.log |
| 2024-11-21 14:22:37,027 INFO MainThread:2298044 [wandb_init.py:_log_setup():534] Logging internal logs to ./wandb/run-20241121_142236-scq8411d/logs/debug-internal.log |
| 2024-11-21 14:22:37,027 INFO MainThread:2298044 [wandb_init.py:init():619] calling init triggers |
| 2024-11-21 14:22:37,027 INFO MainThread:2298044 [wandb_init.py:init():626] wandb.init called with sweep_config: {} |
| config: {} |
| 2024-11-21 14:22:37,027 INFO MainThread:2298044 [wandb_init.py:init():669] starting backend |
| 2024-11-21 14:22:37,027 INFO MainThread:2298044 [wandb_init.py:init():673] sending inform_init request |
| 2024-11-21 14:22:37,040 INFO MainThread:2298044 [backend.py:_multiprocessing_setup():104] multiprocessing start_methods=fork,spawn,forkserver, using: fork |
| 2024-11-21 14:22:37,040 INFO MainThread:2298044 [wandb_init.py:init():686] backend started and connected |
| 2024-11-21 14:22:37,055 INFO MainThread:2298044 [wandb_init.py:init():781] updated telemetry |
| 2024-11-21 14:22:37,058 INFO MainThread:2298044 [wandb_init.py:init():814] communicating run to backend with 90.0 second timeout |
| 2024-11-21 14:22:37,703 INFO MainThread:2298044 [wandb_init.py:init():867] starting run threads in backend |
| 2024-11-21 14:22:38,014 INFO MainThread:2298044 [wandb_run.py:_console_start():2456] atexit reg |
| 2024-11-21 14:22:38,015 INFO MainThread:2298044 [wandb_run.py:_redirect():2305] redirect: wrap_raw |
| 2024-11-21 14:22:38,015 INFO MainThread:2298044 [wandb_run.py:_redirect():2370] Wrapping output streams. |
| 2024-11-21 14:22:38,015 INFO MainThread:2298044 [wandb_run.py:_redirect():2395] Redirects installed. |
| 2024-11-21 14:22:38,019 INFO MainThread:2298044 [wandb_init.py:init():911] run started, returning control to user process |
| 2024-11-21 14:22:38,019 INFO MainThread:2298044 [wandb_watch.py:_watch():71] Watching |
| 2024-11-21 14:22:38,079 INFO MainThread:2298044 [wandb_run.py:_config_callback():1387] config_cb None None {'data': {'root_path': '/blue/hennig/pawanprakash/diffusion/Diffscp_guide/MaterialsDiffusion_unguided/data/alex_20', 'prop': 'eah', 'num_targets': 1, 'niggli': True, 'primitive': False, 'graph_method': 'crystalnn', 'lattice_scale_method': 'scale_length', 'preprocess_workers': 30, 'readout': 'mean', 'max_atoms': 20, 'otf_graph': False, 'eval_model_name': 'alex20', 'tolerance': 0.1, 'use_space_group': False, 'use_pos_index': False, 'train_max_epochs': 6000, 'early_stopping_patience': 60, 'teacher_forcing_max_epoch': 500, 'datamodule': {'_target_': 'diffcsp.pl_data.datamodule.CrystDataModule', 'datasets': {'train': {'_target_': 'diffcsp.pl_data.dataset.CrystDataset', 'name': 'Formation energy train', 'path': '/blue/hennig/pawanprakash/diffusion/Diffscp_guide/MaterialsDiffusion_unguided/data/alex_20/train.csv', 'save_path': '/blue/hennig/pawanprakash/diffusion/Diffscp_guide/MaterialsDiffusion_unguided/data/alex_20/train_ori.pt', 'prop': 'eah', 'niggli': True, 'primitive': False, 'graph_method': 'crystalnn', 'tolerance': 0.1, 'use_space_group': False, 'use_pos_index': False, 'lattice_scale_method': 'scale_length', 'preprocess_workers': 30}, 'val': [{'_target_': 'diffcsp.pl_data.dataset.CrystDataset', 'name': 'Formation energy val', 'path': '/blue/hennig/pawanprakash/diffusion/Diffscp_guide/MaterialsDiffusion_unguided/data/alex_20/val.csv', 'save_path': '/blue/hennig/pawanprakash/diffusion/Diffscp_guide/MaterialsDiffusion_unguided/data/alex_20/val_ori.pt', 'prop': 'eah', 'niggli': True, 'primitive': False, 'graph_method': 'crystalnn', 'tolerance': 0.1, 'use_space_group': False, 'use_pos_index': False, 'lattice_scale_method': 'scale_length', 'preprocess_workers': 30}], 'test': [{'_target_': 'diffcsp.pl_data.dataset.CrystDataset', 'name': 'Formation energy test', 'path': '/blue/hennig/pawanprakash/diffusion/Diffscp_guide/MaterialsDiffusion_unguided/data/alex_20/test.csv', 'save_path': '/blue/hennig/pawanprakash/diffusion/Diffscp_guide/MaterialsDiffusion_unguided/data/alex_20/test_ori.pt', 'prop': 'eah', 'niggli': True, 'primitive': False, 'graph_method': 'crystalnn', 'tolerance': 0.1, 'use_space_group': False, 'use_pos_index': False, 'lattice_scale_method': 'scale_length', 'preprocess_workers': 30}]}, 'num_workers': {'train': 0, 'val': 0, 'test': 0}, 'batch_size': {'train': 4096, 'val': 4096, 'test': 1024}}}, 'logging': {'val_check_interval': 1, 'wandb': {'name': 'alex_20', 'project': 'diffcsp', 'entity': None, 'log_model': True, 'mode': 'online', 'group': 'alex_20'}, 'wandb_watch': {'log': 'all', 'log_freq': 500}, 'lr_monitor': {'logging_interval': 'step', 'log_momentum': False}}, 'model': {'decoder': {'_target_': 'diffcsp.pl_modules.cspnet.CSPNet', 'hidden_dim': 512, 'latent_dim': 0, 'max_atoms': 100, 'num_layers': 6, 'act_fn': 'silu', 'dis_emb': 'sin', 'num_freqs': 128, 'edge_style': 'fc', 'max_neighbors': 20, 'cutoff': 7.0, 'ln': True, 'ip': True}, 'beta_scheduler': {'_target_': 'diffcsp.pl_modules.diff_utils.BetaScheduler', 'timesteps': 1000, 'scheduler_mode': 'cosine'}, 'sigma_scheduler': {'_target_': 'diffcsp.pl_modules.diff_utils.SigmaScheduler', 'timesteps': 1000, 'sigma_begin': 0.005, 'sigma_end': 0.5}, '_target_': 'diffcsp.pl_modules.diffusion_w_type.CSPDiffusion', 'time_dim': 256, 'latent_dim': 0, 'cost_coord': 1.0, 'cost_lattice': 1.0, 'cost_type': 20.0, 'max_neighbors': 20, 'radius': 7.0, 'timesteps': 1000}, 'optim': {'optimizer': {'_target_': 'torch.optim.Adam', 'lr': 0.001, 'betas': [0.9, 0.999], 'eps': 1e-08, 'weight_decay': 0}, 'use_lr_scheduler': True, 'lr_scheduler': {'_target_': 'torch.optim.lr_scheduler.ReduceLROnPlateau', 'factor': 0.6, 'patience': 30, 'min_lr': 0.0001}}, 'train': {'deterministic': True, 'random_seed': 42, 'pl_trainer': {'fast_dev_run': False, 'accelerator': 'auto', 'precision': 32, 'max_epochs': 6000, 'accumulate_grad_batches': 1, 'num_sanity_val_steps': 2, 'gradient_clip_val': 0.5, 'gradient_clip_algorithm': 'value', 'profiler': 'simple'}, 'monitor_metric': 'val_loss', 'monitor_metric_mode': 'min', 'early_stopping': {'patience': 60, 'verbose': True}, 'model_checkpoints': {'save_top_k': 1, 'verbose': False, 'save_last': False}, 'ckpt_path': None}, 'expname': 'alex_20', 'core': {'version': '0.0.1', 'tags': ['2024-11-21']}, 'stats/params_total': 12346468, 'stats/params_trainable': 12346468, 'stats/params_not_trainable': 0} |
| 2024-11-24 04:29:47,970 INFO MainThread:2298044 [wandb_run.py:_finish():2155] finishing run uf_hennig/diffcsp/scq8411d |
| 2024-11-24 04:29:48,052 INFO MainThread:2298044 [wandb_run.py:_atexit_cleanup():2420] got exitcode: 0 |
| 2024-11-24 04:29:48,054 INFO MainThread:2298044 [wandb_run.py:_restore():2402] restore |
| 2024-11-24 04:29:48,054 INFO MainThread:2298044 [wandb_run.py:_restore():2408] restore done |
| 2024-11-24 04:29:53,834 INFO MainThread:2298044 [wandb_run.py:_footer_history_summary_info():3960] rendering history |
| 2024-11-24 04:29:53,836 INFO MainThread:2298044 [wandb_run.py:_footer_history_summary_info():3992] rendering summary |
| 2024-11-24 04:29:53,864 INFO MainThread:2298044 [wandb_run.py:_footer_sync_info():3921] logging synced files |
|
|
