Hugging Face's logo

GeoV
/
GeoV-9b

Text Generation
Transformers
PyTorch
English
geov
causal-lm
custom_code
Model card Files
xet
 Community
1
vpj commited on
Commit
6213f0f
·
1 Parent(s): b8a3342

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -11,10 +11,10 @@ license: bigscience-openrail-m
11
  [GeoV](https://github.com/geov-ai/geov)-9B is a 9 billion parameter causal language model.
12
 
13
  The GeoV model was designed by Georges Harik and uses
14
- [Rotary Positional Embeddings with Relative distances (RoPER)](http://research.labml.ai/RoPER.html)
15
  by [Georges Harik](https://twitter.com/gharik) and [Varuna Jayasiri](https://twitter.com/vpj).
16
 
17
- [RoPER](http://research.labml.ai/RoPER.html),
18
  in addition to using relative positions in the attention score calculation by RoPE embeddings,
19
  adds relative positional information explicitly to value embeddings.
20
  Specifically, it incorporates the relative positions of the tokens paid attention to.
 
11
  [GeoV](https://github.com/geov-ai/geov)-9B is a 9 billion parameter causal language model.
12
 
13
  The GeoV model was designed by Georges Harik and uses
14
+ [Rotary Positional Embeddings with Relative distances (RoPER)](https://research.labml.ai/RoPER.html)
15
  by [Georges Harik](https://twitter.com/gharik) and [Varuna Jayasiri](https://twitter.com/vpj).
16
 
17
+ [RoPER](https://research.labml.ai/RoPER.html),
18
  in addition to using relative positions in the attention score calculation by RoPE embeddings,
19
  adds relative positional information explicitly to value embeddings.
20
  Specifically, it incorporates the relative positions of the tokens paid attention to.