--- license: mit model-index: - name: RYS-XLarge results: - task: type: text-generation name: Text Generation dataset: name: IFEval (0-Shot) type: HuggingFaceH4/ifeval args: num_few_shot: 0 metrics: - type: inst_level_strict_acc and prompt_level_strict_acc value: 79.96 name: strict accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=dnhkng/RYS-XLarge name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: BBH (3-Shot) type: BBH args: num_few_shot: 3 metrics: - type: acc_norm value: 58.77 name: normalized accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=dnhkng/RYS-XLarge name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MATH Lvl 5 (4-Shot) type: hendrycks/competition_math args: num_few_shot: 4 metrics: - type: exact_match value: 38.97 name: exact match source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=dnhkng/RYS-XLarge name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: GPQA (0-shot) type: Idavidrein/gpqa args: num_few_shot: 0 metrics: - type: acc_norm value: 17.9 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=dnhkng/RYS-XLarge name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MuSR (0-shot) type: TAUR-Lab/MuSR args: num_few_shot: 0 metrics: - type: acc_norm value: 23.72 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=dnhkng/RYS-XLarge name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MMLU-PRO (5-shot) type: TIGER-Lab/MMLU-Pro config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 49.2 name: accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=dnhkng/RYS-XLarge name: Open LLM Leaderboard --- This is a new kind of model optimization. This model is based on MaziyarPanahi/calme-2.1-qwen2-72b, which was tuned from Qwen2-72B. A paper is currently being written on the technique. Special thanks to my wife, for putting up with me coding in the basement for too many evenings and weekends for months! ## Quickstart Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents. ```python from transformers import AutoModelForCausalLM, AutoTokenizer device = "cuda" # the device to load the model onto model = AutoModelForCausalLM.from_pretrained( "dnhkng/RYS-XLarge", torch_dtype="auto", device_map="auto" ) tokenizer = AutoTokenizer.from_pretrained("dnhkng/RYS-XLarge") prompt = "Give me a short introduction to large language model." messages = [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": prompt} ] text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ) model_inputs = tokenizer([text], return_tensors="pt").to(device) generated_ids = model.generate( model_inputs.input_ids, max_new_tokens=512 ) generated_ids = [ output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids) ] response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] ``` # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_dnhkng__RYS-XLarge) | Metric |Value| |-------------------|----:| |Avg. |44.75| |IFEval (0-Shot) |79.96| |BBH (3-Shot) |58.77| |MATH Lvl 5 (4-Shot)|38.97| |GPQA (0-shot) |17.90| |MuSR (0-shot) |23.72| |MMLU-PRO (5-shot) |49.20| ___________________________________ # *ADVERTISING BREAK* I’m on the hunt for new challenges and a chance to dive into some exciting research opportunities. Oh, and did I mention I just snagged a top spot on the Open LLM leaderboard? 🎉 ## CV - Dr David Noel Ng #### Profile Innovation enthusiast, AI-strategist, and interdisciplinary-tech nerd – that's me in a nutshell. With over a decade of experience in research and project management, my professional journey has been largely shaped by my passion for artificial intelligence and its potential to transform various industries. With a solid background in artificial intelligence and machine learning, coupled with a knack for innovation and problem-solving (and a healthy dose of curiosity), I'm excited to bring my skills to a new team. Originally from Australia, where I earned my degrees in Organic Chemistry and Biochemistry, I moved to Germany in 2004. My academic pursuit continued with a Ph.D. in Chemistry at the Max Planck Institute of Biochemistry. Today, I leverage my robust educational background and diverse industry experience to drive AI innovations in a wide range of applications. Hobbies? Lots: I've also built the world's most powerful espresso machine and am working to bring [GLaDOS to life](https://github.com/dnhkng/GlaDOS). ___________________________________ ### PROFESSIONAL EXPERIENCE #### SENIOR GLOBAL INNOVATION STRATEGIST - ARTIFICIAL INTELLIGENCE #### Munich Re | Munich | 05/2023 - Now As a Senior Global Innovation Strategist at Munich Re, my passion is in steering AI/ML strategies, maximizing project impact, and advancing the use of cutting-edge technology. I built the AI Accelerator, which drives the rapid and structured development of AI use-case Implementations. #### AI CONSULTANT - LEAD AI ENGINEER #### appliedAI UTUM | Munich | 04/2019 - 04/2023 In my tenure at appliedAI, I held a leadership role where I spearheaded the successful development and execution of various AI/ML proof-of-concept (POC) and minimum viable product (MVP) projects. I utilized a hands-on approach to drive ideation, planning, and delivery of these solutions for our clients. - AI-Controlled Imaging: Directed a PoC of an AI-Controlled Electron Microscope using Reinforcement Learning for a premier imaging company. - Anomaly Detection: Oversaw development of security systems utilizing anomaly detection, integrating diverse technologies to boost client security at the Munich Security Conference.. - Project Optimization: Implemented AlphaZero-based Graph Optimization for project management in the Nuclear Energy sector. - Food Safety: Delivered a PoC for industrial food safety equipment, significantly improving detection sensitivity. - NLP Consulting: Consulted on automated document analysis and risk assessment for the European Central Bank, leveraging NLP technologies. - Aerospace Anomaly Detection: Developed a PoC for Aerospace manufacturing, using generative diffusion models to create synthetic data for training anomaly detection models. - Retail Automation: Applied Vision and Skeletal Tracking for supermarket automation, modernizing retail operations. - Public Speaking and Training: Regularly presented talks and training sessions on topics such as KI-Transfer Plus for the Bayerischen Staatsministeriums für Digitales, and KI in Biotech for the BioEntrepreneurship Summit, spreading AI knowledge and fostering digital transformation in the Health/Pharma sector.. #### PROJECT LEAD - INNOVATIVE TECHNOLOGIES #### Nanotemper Technologies GmbH | Munich | 5/2016 - 3/2019 Project Lead in the Future Technologies Department, Scientist Bioanalytics and all-rounder in bioanalytics/data/optoelectronics. Contributions and successes: - Created and applied Deep Learning models for interpreting biophysical data for pharmaceutical stability in antibody development - Designed, built, and programmed prototype optoelectronic apparatus for the rapid analysis of biosimilar pharmaceutical molecules - Introduced FPGA technology for high-speed data collection and analysis, now used in the key products at Nanotemper #### RESEARCH SCIENTIST #### Max Planck Institute Of Neurobiology | Martinsried | 02/2016 - 04/2019 Driven by an interest in Biotech, I found a role in research working on biosensors, particularly on optical probes of neural activity (Optogenetics). Contribution and success: - Designed, built and utilized a robotic screening platform for the high-throughput engineering of biosensors. - Utilised image-processing and machine-learning techniques to collect and analyse biosensor data. - Automated the development of large molecules by FACS-based directed protein evolution. - Patented new CRISPR/Cas9 technology for high-throughput protein engineering. #### CONSULTANT FOR THE NETFLIX SERIES 'BIOHACKERS' #### Netflix | Munich | 01/2019 - 12/2019 In this role, I advised on the scientific concepts, storylines and film set for this popular Netflix series. Contribution and success: - Helped design and build the Laboratory and ‘Biohacking’ labs - Modified the scripts to keep scientific accuracy - Location scouting and liaison with the LMU to organise research labs for filming ## SKILLS - Strong interest in customer experience and Machine Learning transformations (e.g. expectation management, stakeholder alignment, team reorganization etc.) - Ability to work autonomously in the completion of deliverables - Ability to provide technical and analytic direction, guidance and roadmaps for ML projects - Excellent communication and presentation skills: able to explain Analytics in non-technical terms to business users (C-level, investors, public presentations etc.) - Deep technical expertise and strong problem-solving and data-analysis skills ## AWARDS #### The United Nations COVID-19 Detect & Protect Challenge - The United Nations Development Programme Centre for Technology, Innovation and Sustainable Development · Aug 2020 #### AI at the Edge Challenge with NVIDIA - Artificial Intelligence of Things (AIoT) - Issued by Nvidia · Mar 2020 #### Create Intelligence at the Edge - Artificial Intelligence on FPGA - Avnet and Xilinx · Dec 2018 #### PATENTS - WO2018020050A1 - Targeted in situ protein diversification by site-directed DNA cleavage and repair ## EDUCATION #### PhD in Organic Chemistry - Max Planck Institute of Biochemistry #### Honours Degree - Biochemistry - Monash University Melbourne #### Bachelor of Science - Double Major - - Chemistry / Molecular Biology - University of Tasmania #### Nanodegree - Deep Reinforcement Learning - Udacity Online #### Nanodegree - Deep Learning - Udacity Online ___________________________________ I'm based out of Munich, Germany, but I would be interested in working remotely for a team with more compute than my 2x 4090s 🚀 #### Reach out via [LinkedIn](https://www.linkedin.com/in/dnhkng)