Is this coming soon?
I would love to give 3.5 a blast on my onnx cuda setup. Is this coming soon?
Oh I am sorry I thought I uploaded it. Doing it now.
Just finished the https://huggingface.co/Maximum2000/Phi-3.5-mini-instruct-cuda-fp32-onnx/tree/main and uploading the fp16 now. Check back in an hour or so.
Thank you so much! I am in early stages with Semantic Kernel and OnnxRuntime and only know how to use onnx/cuda models.
Ben
I posted something in Semantic Kernel's issue as soon as I heard about OnnxRuntime and I do believe they have an integration let me checkout the repo and see where they are at and get back to you.
I found this and looks like they have implemented this option: https://github.com/microsoft/semantic-kernel/blob/43a36fd75fd2458e80eae5b4df426d4b5c524a87/dotnet/src/Connectors/Connectors.Onnx/OnnxRuntimeGenAIChatCompletionService.cs
Good luck