Issue on running the model in google colab
#2
by
Manikandan-t
- opened
/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py in _sample(self, input_ids, logits_processor, stopping_criteria, generation_config, synced_gpus, streamer, **model_kwargs)
2996 probs = nn.functional.softmax(next_token_scores, dim=-1)
2997 # TODO (joao): this OP throws "skipping cudagraphs due to ['incompatible ops']", find solution
-> 2998 next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1)
2999 else:
3000 next_tokens = torch.argmax(next_token_scores, dim=-1)
RuntimeError: probability tensor contains either inf, nan or element < 0