Nous Hermes 2 - Mistral 7B - DPO - AWQ

Model Description

This repo contains the AWQ quantized version of the Nous Hermes 2 - Mistral 7B - DPO model. It was quantized with AutoAWQ using the following settings:

{"zero_point": True, "q_group_size": 128, "w_bit": 4, "version": "GEMM"}

Downloads last month: 6

Safetensors

Model size

1.2B params

Tensor type

I32

FP16

Inference API

Text Generation

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for heyholetsgo/Nous-Hermes-2-Mistral-7B-DPO-AWQ

Base model

mistralai/Mistral-7B-v0.1

Finetuned

NousResearch/Nous-Hermes-2-Mistral-7B-DPO

Quantized

this model

Dataset used to train heyholetsgo/Nous-Hermes-2-Mistral-7B-DPO-AWQ

Evaluation results

Metadata error: specify a dataset to view leaderboard