Spaces:

AwakeningOS
/

vistarc-research-notes

Running

vistarc-research-notes / data /evidence /width_lr_identifiability.json

Publish bilingual VISTARC research notebook and evidence-linked model explorer

3897c38 verified 11 days ago

600 Bytes

{
  "n_width_settings": 4,
  "widths": [
    256,
    272,
    320,
    384
  ],
  "initial_qk_lrs": [
    0.0008,
    0.0007529411764705883,
    0.00064,
    0.0005333333333333334
  ],
  "lr_times_width": [
    0.2048,
    0.2048,
    0.2048,
    0.2048
  ],
  "centered_width_LR_design_rank": 1,
  "singular_values": [
    0.4452184052010086,
    6.283435673255382e-17
  ],
  "interpretation": "Most old width series implements LR=.0008*256/width. This is an assigned recipe, not independently estimated optimal LR. Width and logLR cannot be identified as separate coefficients from these rows."
}

	{
	"n_width_settings": 4,
	"widths": [
	256,
	272,
	320,
	384
	],
	"initial_qk_lrs": [
	0.0008,
	0.0007529411764705883,
	0.00064,
	0.0005333333333333334
	],
	"lr_times_width": [
	0.2048,
	0.2048,
	0.2048,
	0.2048
	],
	"centered_width_LR_design_rank": 1,
	"singular_values": [
	0.4452184052010086,
	6.283435673255382e-17
	],
	"interpretation": "Most old width series implements LR=.0008*256/width. This is an assigned recipe, not independently estimated optimal LR. Width and logLR cannot be identified as separate coefficients from these rows."
	}