1 of 24

New Developments and Challenges in Using Large Language Models in Academic Settings with Azure Services

​

​

Anna Alber, Senior Research Scientist

Phillip Lyle, Associate Vice President of IS&T��Chapman University

�

2 of 24

About Chapman

  • Founded in 1861 – 9,600 students

​

  • $34m in research expenditures in FY2022

​

  • 7 Doctoral programs, 57 Masters programs

​

  • R2 institution

​

  • Research Computing (centralized)
    • 1 Director, 1 Systems Administrator, 1 Research Scientist

3 of 24

Early Action and Policy

  • AI Task Force around the ChatGPT release

​

  • "AI Hub" website
    • Teaching & Learning
    • Research
    • Productivity

​

  • Initial concerns
    • Lack of enterprise contracts
    • Copyright

​

​

​

4 of 24

Azure OpenAI and Copilot selected

  • Included data and copyright protections

​

  • Existing Azure knowledge within internal teams

​

  • Copilot for Web (ChatGPT Plus equivalent) for all faculty, staff, and students

​

  • Azure OpenAI for researcher / API access needs
    • AI Studio for broader set of models or as an alternative to Copilot M365 licensing

​

​

​

​

5 of 24

Early lessons learned

  • Key / API management

​

  • Understanding Azure Quotas on LLMs

​

  • Differences between web versions of ChatGPT and Copilot

​

​

​

​

6 of 24

Quotas in Azure OpenAI

  • Default quotas are assigned based on Token Per Minute (TPM)

​

  • Quota increases are auto-approved up to certain amounts, and then reviewed by Microsoft

​

  • Quotas are per-subscription and per-region

​

​

​

​

​

7 of 24

Solution – Azure API Management

  • Abstract direct key access to the underlying models

​

  • Allows load balancing across subscriptions or regions

​

  • Error handling, centralized logging and chargeback

​

  • Quotas, rate limiting

​

​

​

​

​

​

​

​

8 of 24

Azure Cognitive Service for Language with Pre-Built Features

https://medium.com/microsoftazure/azure-cognitive-service-for-language-d1445a77530a

9 of 24

Azure Open AI with Your Data

10 of 24

Azure OpenAI with Your Data cont’d

11 of 24

Azure OpenAI for Data Analysis

what are column names?�what is range in fixed acidity?

12 of 24

Azure Open AI for Data Analysis

13 of 24

Azure Open AI for Data Analysis

Anna Alber, Phillip Lyle

14 of 24

Azure Open AI for Data Analysis

Anna Alber, Phillip Lyle

15 of 24

Azure Open AI for Data Analysis

Anna Alber, Phillip Lyle

16 of 24

Anna Alber, Phillip Lyle

Source: https://deci.ai/blog/retrieval-augmented-generation-using-langchain

17 of 24

Facebook AI Semantic Search with Azure Open AI

Anna Alber, Phillip Lyle

Image Source: Ivan Campos

18 of 24

RAG with Facebook AI Semantic Search with Azure Open AI

19 of 24

RAG with Facebook AI Semantic Search with Azure Open AI

20 of 24

AI Agents with Azure Open AI

Who won Oscar for best actor in 2024

​

Cillian Murphy

Oppenheimer

​

​

File System

Database

Bing Search

​

21 of 24

AI Agent

Anna Alber, Phillip Lyle

22 of 24

AI Agent

23 of 24

What’s next for Chapman

  • Azure Copilot Studio (formerly Power Virtual Agents)

​

  • Small Azure OpenAI operational projects
    • “Chapman Job Description Generator”

​

  • Pilot for Copilot M365

​

  • Interest in other smaller, inexpensive models

​

​

​

​

​

​

​

​

​

24 of 24

Questions ?