Spaces:

ethanlshen
/

SuperposedDecoding

Runtime error

ethanlshen commited on Jun 25, 2024

Commit

5321ffc

verified ·

1 Parent(s): 1838b97

Update superposed/llama/superposed_generation.py

Files changed (1) hide show

superposed/llama/superposed_generation.py CHANGED Viewed

@@ -68,7 +68,7 @@ class SuperposedLlama:
         )
         tokenizer = Tokenizer(model_path=tokenizer_path)
         model_args.vocab_size = tokenizer.n_words
-        torch.set_default_tensor_type(torch.cuda.HalfTensor)
         # Set up superposed decoding
         model = SuperposedTransformer(model_args)
         model.load_state_dict(checkpoint, strict=False)

         )
         tokenizer = Tokenizer(model_path=tokenizer_path)
         model_args.vocab_size = tokenizer.n_words
+        torch.set_default_dtype(torch.cuda.HalfTensor)
         # Set up superposed decoding
         model = SuperposedTransformer(model_args)
         model.load_state_dict(checkpoint, strict=False)