Can you reduce the kv head num of this model? "num_key_value_heads": 64, which requies a lots of kv cache
#1
by luchangli03 - opened
We will continue iterating on this going forward. Let's try training with this suggestion.
We will continue iterating on this going forward. Let's try training with this suggestion.