license: mit
model-index:
- name: RYS-XLarge
results:
- task:
type: text-generation
name: Text Generation
dataset:
name: IFEval (0-Shot)
type: HuggingFaceH4/ifeval
args:
num_few_shot: 0
metrics:
- type: inst_level_strict_acc and prompt_level_strict_acc
value: 79.96
name: strict accuracy
source:
url: >-
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=dnhkng/RYS-XLarge
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: BBH (3-Shot)
type: BBH
args:
num_few_shot: 3
metrics:
- type: acc_norm
value: 58.77
name: normalized accuracy
source:
url: >-
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=dnhkng/RYS-XLarge
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MATH Lvl 5 (4-Shot)
type: hendrycks/competition_math
args:
num_few_shot: 4
metrics:
- type: exact_match
value: 38.97
name: exact match
source:
url: >-
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=dnhkng/RYS-XLarge
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: GPQA (0-shot)
type: Idavidrein/gpqa
args:
num_few_shot: 0
metrics:
- type: acc_norm
value: 17.9
name: acc_norm
source:
url: >-
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=dnhkng/RYS-XLarge
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MuSR (0-shot)
type: TAUR-Lab/MuSR
args:
num_few_shot: 0
metrics:
- type: acc_norm
value: 23.72
name: acc_norm
source:
url: >-
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=dnhkng/RYS-XLarge
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MMLU-PRO (5-shot)
type: TIGER-Lab/MMLU-Pro
config: main
split: test
args:
num_few_shot: 5
metrics:
- type: acc
value: 49.2
name: accuracy
source:
url: >-
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=dnhkng/RYS-XLarge
name: Open LLM Leaderboard
This is a new kind of model optimization. This model is based on MaziyarPanahi/calme-2.1-qwen2-72b, which was tuned from Qwen2-72B.
A paper is currently being written on the technique. Special thanks to my wife, for putting up with me coding in the basement for too many evenings and weekends for months!
Quickstart
Here provides a code snippet with apply_chat_template to show you how to load the tokenizer and model and how to generate contents.
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" # the device to load the model onto
model = AutoModelForCausalLM.from_pretrained(
"dnhkng/RYS-XLarge",
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("dnhkng/RYS-XLarge")
prompt = "Give me a short introduction to large language model."
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(device)
generated_ids = model.generate(
model_inputs.input_ids,
max_new_tokens=512
)
generated_ids = [
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 44.75 |
| IFEval (0-Shot) | 79.96 |
| BBH (3-Shot) | 58.77 |
| MATH Lvl 5 (4-Shot) | 38.97 |
| GPQA (0-shot) | 17.90 |
| MuSR (0-shot) | 23.72 |
| MMLU-PRO (5-shot) | 49.20 |
ADVERTISING BREAK
I’m on the hunt for new challenges and a chance to dive into some exciting research opportunities. Oh, and did I mention I just snagged a top spot on the Open LLM leaderboard? 🎉
CV - Dr David Noel Ng
Profile
Innovation enthusiast, AI-strategist, and interdisciplinary-tech nerd – that's me in a nutshell. With over a decade of experience in research and project management, my professional journey has been largely shaped by my passion for artificial intelligence and its potential to transform various industries. With a solid background in artificial intelligence and machine learning, coupled with a knack for innovation and problem-solving (and a healthy dose of curiosity), I'm excited to bring my skills to a new team.
Originally from Australia, where I earned my degrees in Organic Chemistry and Biochemistry, I moved to Germany in 2004. My academic pursuit continued with a Ph.D. in Chemistry at the Max Planck Institute of Biochemistry. Today, I leverage my robust educational background and diverse industry experience to drive AI innovations in a wide range of applications. Hobbies? Lots: I've also built the world's most powerful espresso machine and am working to bring GLaDOS to life.
PROFESSIONAL EXPERIENCE
SENIOR GLOBAL INNOVATION STRATEGIST - ARTIFICIAL INTELLIGENCE
Munich Re | Munich | 05/2023 - Now
As a Senior Global Innovation Strategist at Munich Re, my passion is in steering AI/ML strategies, maximizing project impact, and advancing the use of cutting-edge technology. I built the AI Accelerator, which drives the rapid and structured development of AI use-case Implementations.
AI CONSULTANT - LEAD AI ENGINEER
appliedAI UTUM | Munich | 04/2019 - 04/2023
In my tenure at appliedAI, I held a leadership role where I spearheaded the successful development and execution of various AI/ML proof-of-concept (POC) and minimum viable product (MVP) projects. I utilized a hands-on approach to drive ideation, planning, and delivery of these solutions for our clients.
- AI-Controlled Imaging: Directed a PoC of an AI-Controlled Electron Microscope using Reinforcement Learning for a premier imaging company.
- Anomaly Detection: Oversaw development of security systems utilizing anomaly detection, integrating diverse technologies to boost client security at the Munich Security Conference..
- Project Optimization: Implemented AlphaZero-based Graph Optimization for project management in the Nuclear Energy sector.
- Food Safety: Delivered a PoC for industrial food safety equipment, significantly improving detection sensitivity.
- NLP Consulting: Consulted on automated document analysis and risk assessment for the European Central Bank, leveraging NLP technologies.
- Aerospace Anomaly Detection: Developed a PoC for Aerospace manufacturing, using generative diffusion models to create synthetic data for training anomaly detection models.
- Retail Automation: Applied Vision and Skeletal Tracking for supermarket automation, modernizing retail operations.
- Public Speaking and Training: Regularly presented talks and training sessions on topics such as KI-Transfer Plus for the Bayerischen Staatsministeriums für Digitales, and KI in Biotech for the BioEntrepreneurship Summit, spreading AI knowledge and fostering digital transformation in the Health/Pharma sector..
PROJECT LEAD - INNOVATIVE TECHNOLOGIES
Nanotemper Technologies GmbH | Munich | 5/2016 - 3/2019
Project Lead in the Future Technologies Department, Scientist Bioanalytics and all-rounder in bioanalytics/data/optoelectronics. Contributions and successes:
- Created and applied Deep Learning models for interpreting biophysical data for pharmaceutical stability in antibody development
- Designed, built, and programmed prototype optoelectronic apparatus for the rapid analysis of biosimilar pharmaceutical molecules
- Introduced FPGA technology for high-speed data collection and analysis, now used in the key products at Nanotemper
RESEARCH SCIENTIST
Max Planck Institute Of Neurobiology | Martinsried | 02/2016 - 04/2019
Driven by an interest in Biotech, I found a role in research working on biosensors, particularly on optical probes of neural activity (Optogenetics). Contribution and success:
- Designed, built and utilized a robotic screening platform for the high-throughput engineering of biosensors.
- Utilised image-processing and machine-learning techniques to collect and analyse biosensor data.
- Automated the development of large molecules by FACS-based directed protein evolution.
- Patented new CRISPR/Cas9 technology for high-throughput protein engineering.
CONSULTANT FOR THE NETFLIX SERIES 'BIOHACKERS'
Netflix | Munich | 01/2019 - 12/2019
In this role, I advised on the scientific concepts, storylines and film set for this popular Netflix series. Contribution and success:
- Helped design and build the Laboratory and ‘Biohacking’ labs
- Modified the scripts to keep scientific accuracy
- Location scouting and liaison with the LMU to organise research labs for filming
SKILLS
- Strong interest in customer experience and Machine Learning transformations (e.g. expectation management, stakeholder alignment, team reorganization etc.)
- Ability to work autonomously in the completion of deliverables
- Ability to provide technical and analytic direction, guidance and roadmaps for ML projects
- Excellent communication and presentation skills: able to explain Analytics in non-technical terms to business users (C-level, investors, public presentations etc.)
- Deep technical expertise and strong problem-solving and data-analysis skills
AWARDS
The United Nations COVID-19 Detect & Protect Challenge
- The United Nations Development Programme Centre for Technology, Innovation and Sustainable Development · Aug 2020
AI at the Edge Challenge with NVIDIA - Artificial Intelligence of Things (AIoT)
- Issued by Nvidia · Mar 2020
Create Intelligence at the Edge - Artificial Intelligence on FPGA
- Avnet and Xilinx · Dec 2018
PATENTS
- WO2018020050A1 - Targeted in situ protein diversification by site-directed DNA cleavage and repair
EDUCATION
PhD in Organic Chemistry
- Max Planck Institute of Biochemistry
Honours Degree - Biochemistry
- Monash University Melbourne
Bachelor of Science - Double Major -
- Chemistry / Molecular Biology
- University of Tasmania
Nanodegree - Deep Reinforcement Learning
- Udacity Online
Nanodegree - Deep Learning
- Udacity Online
I'm based out of Munich, Germany, but I would be interested in working remotely for a team with more compute than my 2x 4090s 🚀