KaggleX 2024-Showcase�ReguGuard AI

Author: Shijun Ju

Advisor: Himaja Vadaga

Google | Proprietary & Confidential

KaggleX Showcase

1

Personal Background

ReguGuard AI

• Shijun Ju
• Graduate Student (AI program) at Georgian College
• Interests:
• FinTech
• NLP, LLM
• RAG
• Educational technology
• https://www.linkedin.com/in/shijunju/
• Portfolio website: https://shijunju.com/

​

​

Google | Proprietary & Confidential

KaggleX Showcase

2

Project definition - Problem statement

ReguGuard AI

• Financial risk professionals face a high volume of complex, evolving regulatory documents - manual document review is time-intensive and prone to oversight.
• Compliance AI needed for fast, accurate, and informed answers to compliance queries.
• Supports timely, reliable decision-making in financial risk compliance.
• Goal: memorize while flexible to provide answers to different financial risk questions

​

Data Science Topic(s) Applied

• Data Preparation , LLM, Fine-tuning, LoRa (Low-Rank Adaptation), QLoRA

​

Google | Proprietary & Confidential

KaggleX Showcase

3

Project Architecture and Methods

ReguGuard AI

Google | Proprietary & Confidential

KaggleX Showcase

4

Project Details – Data Preparation

ReguGuard AI

• Sample template for training

​

​

Google | Proprietary & Confidential

KaggleX Showcase

5

Project Details – Training and Evaluation

ReguGuard AI

​

​

Google | Proprietary & Confidential

KaggleX Showcase

6

Project Details – Response Generation

ReguGuard AI

• Sample Response Generated

​

​

Google | Proprietary & Confidential

KaggleX Showcase

7

Project Details – Results

ReguGuard AI

Variables Tested

• Model sizes, number of question versions, LoRA rank, normal vs beam search

​

​

​

​

​

​

​

​

​

​

​

SCC: SparseCategoricalCrossentropy; SCA: SparseCategoricalAccuracy

*** To save evaluation time, randomly selected 900 out of 2,910.

​

​

Google | Proprietary & Confidential

KaggleX Showcase

8

Conclusion

ReguGuard AI

Summary

• Larger model, seeing more versions of similar questions, and Beam Search improves accuracy while higher LoRA rank does not help
• Limitations:
• Test data: paraphrased questions
• “Don’t know” - Guardrails
• Small number of documents
• Finetuned models only able to answer information it has seen

Future work

• Fine-tune embedding models and retrieval LLM using RAG

​

Things learned

• Fine-tuning LLM with LoRA and QLoRA
• Model training with GPU and TPU
• Project management

Google | Proprietary & Confidential

KaggleX Showcase

9

Project Links

ReguGuard AI

Data

• Notebook for data generation: https://www.kaggle.com/code/shijunju/raft-data-generation
• https://huggingface.co/datasets/shijunju/fincen_all_questions_5versions

Finetuned Gemma-2b: full model

• Finetuning Kaggle Notebook: https://www.kaggle.com/code/shijunju/finetune-gemma-2b-with-quantization-using-lora
• https://huggingface.co/shijunju/gemma_2b_finRisk

Finetuned Gemma-7b: LoRA adaptor files only

• Finetuning Kaggle Notebook: https://www.kaggle.com/code/shijunju/tpu-keras-gemma-distributed-finetuning-gemma-7b
• LoRA Rank 6, 4-version-question training: https://huggingface.co/shijunju/gemma_7b_finRisk_r6_4VersionQ
• LoRA Rank 10, 4-version-question training: https://huggingface.co/shijunju/gemma_7b_finRisk_r10_4VersionQ
• LoRA Rank 6, 3-version-question training: https://huggingface.co/shijunju/gemma_7b_finRisk_r6_3VersionQ

Training / Experiment notes

• https://docs.google.com/spreadsheets/d/1iFz_1qFsGXXOi0ooO2zJgQxMxrIqK2dxDOBZp_AmTlE/edit?usp=sharing

References: TPU Training

• https://www.kaggle.com/code/nilaychauhan/keras-gemma-distributed-finetuning-and-inference

Google | Proprietary & Confidential

KaggleX Showcase

10

Thank you!��Advisor: Himaja Vadaga�Organizer: Kaggle.com���

Google | Proprietary & Confidential

Presentation Title

11