Update README.md
Browse files
README.md
CHANGED
|
@@ -1,5 +1,7 @@
|
|
| 1 |
see our paper in https://arxiv.org/abs/2405.17743
|
| 2 |
|
|
|
|
|
|
|
| 3 |
## Model Details
|
| 4 |
|
| 5 |
LLaMA-3-8B-ORLM is fully fine-tuned on the OR-Instruct data and built with Meta [LLaMA-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) model.
|
|
@@ -8,14 +10,14 @@ More training details can be seen at https://arxiv.org/abs/2405.17743
|
|
| 8 |
## Model Usage
|
| 9 |
|
| 10 |
Prompting Template:
|
| 11 |
-
|
| 12 |
Below is an operations research question. Build a mathematical model and corresponding python code using `coptpy` that appropriately addresses the question.
|
| 13 |
|
| 14 |
-
|
| 15 |
{Question}
|
| 16 |
|
| 17 |
-
|
| 18 |
-
|
| 19 |
|
| 20 |
Please replace the `{Question}` with any natural language OR question.
|
| 21 |
|
|
@@ -54,7 +56,7 @@ Using this model, we can apply linear programming techniques to find the optimal
|
|
| 54 |
## Python Code Solution Using `coptpy`:
|
| 55 |
Here is a Python script using the `coptpy` library to solve the problem:
|
| 56 |
|
| 57 |
-
|
| 58 |
import coptpy as cp
|
| 59 |
from coptpy import COPT
|
| 60 |
|
|
@@ -86,7 +88,7 @@ if model.status == COPT.OPTIMAL:
|
|
| 86 |
print("Number of small pills to be made: {:.0f}".format(y.x))
|
| 87 |
else:
|
| 88 |
print("No optimal solution found.")
|
| 89 |
-
|
| 90 |
|
| 91 |
In this script, we first create a `COPT` environment and model. Then, we add two integer decision variables `x` and `y`, representing the number of large and small pills to be made, respectively.
|
| 92 |
|
|
@@ -104,21 +106,21 @@ This script provides a complete example of using the `coptpy` library to solve t
|
|
| 104 |
|
| 105 |
## Performances
|
| 106 |
|
| 107 |
-
Below is the comparison of performance on the NL4OPT, MAMO, and IndustryOR benchmarks. Values marked with a
|
| 108 |
|
| 109 |
| **Method** | **NL4OPT** | **MAMO EasyLP** | **MAMO ComplexLP** | **IndustryOR** | **Micro Avg** | **Macro Avg** |
|
| 110 |
|------------------------------------------------|-------------------------|-----------------------|----------------------|-------------------|-----------------|-----------------|
|
| 111 |
| *Methods based on PLMs* | | | | | | |
|
| 112 |
-
| `tag-BART` | 47.9
|
| 113 |
| *Methods based on GPT-3.5* | | | | | | |
|
| 114 |
-
| `Standard` | 42.4
|
| 115 |
-
| `Reflexion` | 50.7
|
| 116 |
-
| `Chain-of-Experts` | 58.9
|
| 117 |
| *Methods based on GPT-4* | | | | | | |
|
| 118 |
-
| `Standard` | 47.3
|
| 119 |
-
| `Reflexion` | 53.0
|
| 120 |
-
| `Chain-of-Experts` | 64.2
|
| 121 |
-
| `OptiMUS` | 78.8
|
| 122 |
| *ORLMs based on open-source LLMs* | | | | | | |
|
| 123 |
| `ORLM-Mistral-7B` | 84.4% | 81.4% | 32.0% | 27.0% | 68.8% | 56.2% |
|
| 124 |
| `ORLM-Deepseek-Math-7B-Base` | **86.5%** | 82.2% | **37.9%** | 33.0% | 71.2% | 59.9% |
|
|
|
|
| 1 |
see our paper in https://arxiv.org/abs/2405.17743
|
| 2 |
|
| 3 |
+
github repo: https://github.com/Cardinal-Operations/ORLM
|
| 4 |
+
|
| 5 |
## Model Details
|
| 6 |
|
| 7 |
LLaMA-3-8B-ORLM is fully fine-tuned on the OR-Instruct data and built with Meta [LLaMA-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) model.
|
|
|
|
| 10 |
## Model Usage
|
| 11 |
|
| 12 |
Prompting Template:
|
| 13 |
+
```text
|
| 14 |
Below is an operations research question. Build a mathematical model and corresponding python code using `coptpy` that appropriately addresses the question.
|
| 15 |
|
| 16 |
+
# Question:
|
| 17 |
{Question}
|
| 18 |
|
| 19 |
+
# Response:
|
| 20 |
+
```
|
| 21 |
|
| 22 |
Please replace the `{Question}` with any natural language OR question.
|
| 23 |
|
|
|
|
| 56 |
## Python Code Solution Using `coptpy`:
|
| 57 |
Here is a Python script using the `coptpy` library to solve the problem:
|
| 58 |
|
| 59 |
+
\`\`\`python
|
| 60 |
import coptpy as cp
|
| 61 |
from coptpy import COPT
|
| 62 |
|
|
|
|
| 88 |
print("Number of small pills to be made: {:.0f}".format(y.x))
|
| 89 |
else:
|
| 90 |
print("No optimal solution found.")
|
| 91 |
+
\`\`\`
|
| 92 |
|
| 93 |
In this script, we first create a `COPT` environment and model. Then, we add two integer decision variables `x` and `y`, representing the number of large and small pills to be made, respectively.
|
| 94 |
|
|
|
|
| 106 |
|
| 107 |
## Performances
|
| 108 |
|
| 109 |
+
Below is the comparison of performance on the NL4OPT, MAMO, and IndustryOR benchmarks. Values marked with a <sup>*</sup> are directly copied from original papers, with blanks where data were not reported. The highest results are highlighted in bold.
|
| 110 |
|
| 111 |
| **Method** | **NL4OPT** | **MAMO EasyLP** | **MAMO ComplexLP** | **IndustryOR** | **Micro Avg** | **Macro Avg** |
|
| 112 |
|------------------------------------------------|-------------------------|-----------------------|----------------------|-------------------|-----------------|-----------------|
|
| 113 |
| *Methods based on PLMs* | | | | | | |
|
| 114 |
+
| `tag-BART` | 47.9%<sup>*</sup> | - | - | - | - | - |
|
| 115 |
| *Methods based on GPT-3.5* | | | | | | |
|
| 116 |
+
| `Standard` | 42.4%<sup>*</sup> | - | - | - | - | - |
|
| 117 |
+
| `Reflexion` | 50.7%<sup>*</sup> | - | - | - | - | - |
|
| 118 |
+
| `Chain-of-Experts` | 58.9%<sup>*</sup> | - | - | - | - | - |
|
| 119 |
| *Methods based on GPT-4* | | | | | | |
|
| 120 |
+
| `Standard` | 47.3%<sup>*</sup> | 66.5%<sup>*</sup> | 14.6%<sup>*</sup> | 28.0% | 50.2% | 39.1% |
|
| 121 |
+
| `Reflexion` | 53.0%<sup>*</sup> | - | - | - | - | - |
|
| 122 |
+
| `Chain-of-Experts` | 64.2%<sup>*</sup> | - | - | - | - | - |
|
| 123 |
+
| `OptiMUS` | 78.8%<sup>*</sup> | - | - | - | - | - |
|
| 124 |
| *ORLMs based on open-source LLMs* | | | | | | |
|
| 125 |
| `ORLM-Mistral-7B` | 84.4% | 81.4% | 32.0% | 27.0% | 68.8% | 56.2% |
|
| 126 |
| `ORLM-Deepseek-Math-7B-Base` | **86.5%** | 82.2% | **37.9%** | 33.0% | 71.2% | 59.9% |
|