rajabmondal commited on
Commit
b4bb557
·
verified ·
1 Parent(s): dc4aef9

updated model summary

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -26,7 +26,7 @@ widget:
26
 
27
  ## Model Summary
28
 
29
- The Narrow Transformer (NT) model NT-Java-1.1B is an open-source specialized code model built on StarCoderBase, designed for code completion tasks in Java programming. The model is a decoder-only transformer with Multi-Query-Attention and learned absolute positional embeddings and was finetuned for Java subset of the training data (starcoderdata) which is ~22B tokens and with a context of 8192 tokens.
30
 
31
  - **Repository:** [bigcode/Megatron-LM](https://github.com/bigcode-project/Megatron-LM)
32
  - **Project Website:**
 
26
 
27
  ## Model Summary
28
 
29
+ The Narrow Transformer (NT) model NT-Java-1.1B is an open-source specialized code model built by extending pre-training on starcoderbase-1b, designed for code related tasks in Java programming. The model is a decoder-only transformer with Multi-Query-Attention and a context length of 8192 tokens. The model has been trained with Java subset of the starcoderdata dataset, which is ~22B tokens.
30
 
31
  - **Repository:** [bigcode/Megatron-LM](https://github.com/bigcode-project/Megatron-LM)
32
  - **Project Website:**