SetFit with sentence-transformers/all-MiniLM-L6-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/all-MiniLM-L6-v2 as the Sentence Transformer embedding model. A SetFitHead instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/all-MiniLM-L6-v2
Classification head: a SetFitHead instance
Maximum Sequence Length: 256 tokens
Number of Classes: 14 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
12	"My itinerary's been turned upside down, forcing me to cancel. What's the process for a refund?" 'Yesterday my flight AA 273 was canceled and I had to be rebooked on a British airways flight for today. Is there anyway for you to get some sort of refund from American Airlines?I recently cancelled a trip reservation for travel to Midland, TX on 10/3/2019. The confirmation code: WUOQZY. When can I expect to see a refund of the airfare charge, or will I see a credit that I can use for future air travel?' 'My flight got cancelled. Can you walk me through how to claim a refund?'
11	'Please create a new travel profile for: Annette Marie Marshall Annette.marshall@kpsglobal.com Department code: 4500 Install Level Manager is: Aaron W. Brandt Location: Fort Worth Office' 'Please add the below employee to our existing system. Sandy Faucett sfaucett@arlingtonroe.com ' 'Can you change the saved payment method in my profile?'
8	'I need my itinerary for my outbound flight on Feb 5, 256pm, Flt 898. Please send.' 'Al asked me to locate his return ticket to verify the confirmation number. Can you forward that to me please ?' 'Just checking in to see about the status of the below 2 members airline itineraries?? Jose Gustavo Robles Leon, Adrian Fabricio Villarreal Alfaro, CLI, FSCP. Thanks very much'
7	'Hello, Can you please send me a list my 5 latest invoices? Thanks' 'I need your assistance in getting all the travel Tev Finger has done to NY from January 2017 until December 2018. Could you please send me all the invoices you have for the two years. You could also send them to me on a spreadsheet. I also need to know the hotels where he stayed in NY.' 'Please can you provide with the invoices for my stays this month as follows: 1. Premier Splendid Inn Bayshore (07 Aug - 08 Aug) 2. Port Nolloth Beach Shack (14 Aug - 17 Aug)'
13	'Boost your credit score by 100 points in just 30 days. Click here to get started.' 'Your bank account is at risk. Click here to verify your identity and secure your funds.' 'Claim your free trial of our premium streaming service. No credit card required. Sign up today!'
3	"I booked a rental car with a GPS, but the confirmation email doesn't mention the GPS. Can you ensure it's included?" 'My flight booking was cancelled by the airline, and I need to rebook it. Can you assist me with the rebooking process?' "I received a notification that my hotel booking was cancelled, but I didn't request a cancellation. Can you reinstate my booking?"
6	'Please be informed that Ms. Laura Palmer did not check out on the scheduled date. We are holding her belongings in the lost and found. Please advise on the next steps.' "We have received a request for a baby crib in Ms. Olivia Smith's room. Please confirm if this service is required and if there are any additional charges." "Dear Travel Agency, Mr. Noah Lewis's booking includes a complimentary breakfast. Please ensure the guest is aware of this amenity."
1	"There seems to be a charge from your agency on my credit card statement that I can't match with any booking. Could you investigate and send me a detailed report?" 'Did we get charged twice for the same trip? See attached' 'we have a discrepancy between our travel booking history and our credit card. Who is the right person to talk to about this topic? We have a deduction on our US CC on 08/13/2019 but no according booking was made AIR CANADA 0147405030321 281.88'
5	'can you please cancel the Allegiant flights for Jenna Teyshak on Friday, 12/6 and Sunday, 12/8? She is no longer with the company and does not need these flights anymore. Thank you so much!' 'Please cancel any and all reservations for Joel Morris for his trip. His annual meeting was canceled because of the COVID - 19 Virus.' 'Please cancel Mike Constantinides flight on Tues, Sept 24th from Nash to Denver. If you could do this as soon as possible that would be great - I am not allowed to modify it myself without having to cancel whole trip'
4	'Please book a one-way flight from Houston (IAH) to New Orleans (MSY) on Feb 3rd, 2025, and a hotel for the duration at the Marriott, 555 Canal St.' 'Kindly assist with a quote for accommodation in Wolmaransstad check in on the 13/09/2023 for 3 people.' 'I need a return flight from Seattle (SEA) to Denver (DEN), departing on November 5th and coming back on November 10th.'
2	'Your car rental reservation with Thrifty has been confirmed. Pickup at ORD Airport, October 25th, 11:00 AM.' 'Hotel reservation confirmed at The Ritz-Carlton, San Francisco. Check-in: December 1st, Check-out: December 5th. Booking reference: #123456.' 'Your flight booking was unsuccessful due to invalid travel dates. Please check the dates and try again.'
9	'Can you please help with upgrading the below Nov. 8th PEK to LAX lag to an Economy Plus Seat Aisle. I tried to go to Air China directly and because this ticket was booked through a Third Party it will not allow me to make any changes. I have ccd Sonia (the Traveler) in case you have any questions. Please use the credit card on file for any additional charges.' "Linda Thank you for taking my call just now. Please find as discussed Pat ' s itinerary. May you please change my flight from Raleigh to Newark, so it is at the same time as Pat's flight. Our rest of the times are the same. We leave for Johannesburg at 20:55." 'I understand that it is a non-refundable booking. Somehow my customer asked for rescheduling the meeting and I wonder if it is possible to ask the hotel to change the date of this booking?'
0	'I concur with the travel arrangements. Please confirm the tickets and accommodations.' 'Given the approval from our end, please initiate the booking process for the mentioned journey.' 'there I just got a request to assist with approval to issue a flight ticket without a PO for Righardt Coetzee. Requisition 786247 was only approved at 18h40 - see below. I am going to give approval for the ticket to be issued since we do have an approved requisition, but this will be logged as an after-hour call and the project will pay an additional R290 for CWT after-hours. Below the information as provided in the requisition. The PO will be issued on 20 September.'
10	'Please see the attached statement of due invoices and arrange for payment. If already settled, kindly provide proof of payment.' 'To Whom it may concern, Mr. Steven Anderson has extended his stay, but the credit card on file is now declining. Could someone call the hotel at 815-872-5000 and provide a current card?' 'Please assist with payment for the conference room booking at Hilton last week.'

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("mann2107/BCMPIIRAB_ALL_Test")
# Run inference
preds = model("I need invoice please with Engela Petzer name")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	26.3827	136

Label	Training Sample Count
0	26
1	26
2	26
3	25
4	25
5	26
6	25
7	25
8	26
9	26
10	25
11	26
12	26
13	25

Training Hyperparameters

batch_size: (8, 8)
num_epochs: (1, 1)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 1
body_learning_rate: (0.0009623401597937572, 0.0009623401597937572)
head_learning_rate: 0.0009623401597937572
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0111	1	0.2042	-
0.5556	50	0.1917	-

Framework Versions

Python: 3.11.7
SetFit: 1.1.0.dev0
Sentence Transformers: 2.2.2
Transformers: 4.35.2
PyTorch: 2.1.1+cu121
Datasets: 2.14.5
Tokenizers: 0.15.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

mann2107
/

BCMPIIRAB_ALL_Test