Spaces:

codeparrot
/

code-generation-models

Running

code-generation-models / architectures /incoder.txt

Loubna ben allal

add architecture info

d97087d over 4 years ago

288 Bytes

[InCoder](https://huggingface.co/facebook/incoder-6B) uses a decoder-only Transformer with [Causal Masking objective](https://arxiv.org/abs/2201.07520), to train a left-to-right language model to fill in masked token segments. 

|Model | # parameters |
|   -   |   -  | 
| Decoder |6.7B |

	[InCoder](https://huggingface.co/facebook/incoder-6B) uses a decoder-only Transformer with [Causal Masking objective](https://arxiv.org/abs/2201.07520), to train a left-to-right language model to fill in masked token segments.

	\|Model \| # parameters \|
	\| - \| - \|
	\| Decoder \|6.7B \|