pad_token and eos_token are the same
#4
by
prudant
- opened
It's ok to have the pad_token same has the eos_token, how the model knows when to stop generating if padding uses the same has eos D:
this is the base model and in general you should not use it to generate stuff.
padding tokens are never seen by the model, so in theory, its id can be anything and even -1 will work, as long as the masking functions understand that id means padding.
jklj077
changed discussion status to
closed
thanks!