Spaces:

CaioXapelaum
/

GPT-4o-Tokenizer

Running

CaioXapelaum commited on 22 days ago

Commit

219340b

•

1 Parent(s): ffdc71d

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -4,7 +4,7 @@ import spaces
 tokenizer = GPT2TokenizerFast.from_pretrained('Xenova/gpt-4o')
 assert tokenizer.encode('hello world') == [24912, 2375]
-@spaces.gpu()
 def tokenize(message, history):
     return str(tokenizer.encode(message))

 tokenizer = GPT2TokenizerFast.from_pretrained('Xenova/gpt-4o')
 assert tokenizer.encode('hello world') == [24912, 2375]
+@spaces.GPU
 def tokenize(message, history):
     return str(tokenizer.encode(message))