DB Scaling, Optimization, Cost Reduction – Enterprise AI Use Case��Vincent Granville, PhD�Chief AI Architect �BondingAI.io�vincent@BondingAI.io� �September 30th, 2025

Database Optimization, Cost Reduction. AI Enterprise Use Case (xLLM).�

1

Historical Notes & Context

• Created NoGAN as alternative to Generative Adversarial Networks in 2022. Much faster, better results, no Blackbox. Elsevier book (2024).

​

• Designed xLLM (Extreme LLM) in 2023 to retrieve old references for my research, unable to find them with OpenAI, Google and so on. First version using Wolfram corpus (15,000 webpages, 5,000 categories). Structured output instead of chat-like response. Smart crawling. Backed by decades of relevant experience.

​

• First enterprise version for AB-Inbev in 2024. Lots of similarities to Wolfram despite DB corpus (JSON, context, structure).

​

• Nvidia PDF repository and other corpuses. Text response (chat-like) available from the UI, along with structured output. UI is like a browser, not just a prompt box. Proprietary agents (CRM, predictive analytics, synthetic data, medical data, …)

​

​

​

2

Agenda�

3

1. Introduction
2. Efficient AI Framework: xLLM
3. Sample Code
4. Conclusion & References

Introduction�

4

1. What is TCO?
2. Examples of TCO reduction
3. A few challenges

Total Cost of Ownership

• Acquisition Costs: Initial purchase of hardware and software.
• Implementation Costs: Installation, configuration, and integration with existing systems.
• Infrastructure Costs: Data center space, power, cooling, and network resources.
• Operational Costs: Ongoing expenses like software subscriptions, licensing, and cloud computing resources (e.g., compute, storage, GPU, API calls).
• Maintenance & Support: Costs for updates, patches, technical support, and labor for system upkeep.
• Indirect costs: Training, downtime, performance inefficiencies.
• Intangible costs: Employee productivity loss, compliance risks.

​

​

​

5

Examples of TCO Reduction

• Job Automation:
• I was once hired to build a list of top 100k commercial keywords. Found a Google API that did just that, at a fraction of the salary I was paid.
• Trained BI analysts to use my scripts for DB queries, rather than dashboards, speeding up data extraction by factor 10. Requires very good relationship with IT team.
• Automated my BI tasks: extraction, cleaning, data analysis, emails sent weekly to stakeholders with most important info
• Hire external consultant to detect these inefficiencies?

​

6

Examples of TCO Reduction (Cont.)

• Siloed databases: Your Enterprise LLM fails to answer basic questions because it is not connected to some sources (liability, wasted time). Some sources can be external.

​

• Poor QA: See previous examples. Generate synthetic prompts for testing. Include exhaustivity in evaluation metrics. Take user feedback into account: “this answer is useless” (time and over) to discover missing parts and fix it.

​

• Common sense: Do you need a model with 40B params when your corpus has much less than 1M tokens? Also linked to change resistance.

​

• Not properly leveraging AI: Learn how to get AI to respond in layman’s terms. Rather than complaining of buggy code generated by AI, find the issues and ask AI to fix them (use AI as debugger – example with GMM)

​

​

7

A Few Challenges

• In US: Bigger is better. Throw more money at it. Need mindset shift. Get CFO involved. Moving to doing better with less (discussed later in this session). See my article “Doing better with less: LLM 2.0 for enterprise”.

​

• Career growth linked to how many employees report to you and the size of your budget. Slowly but surely, some competitors adopting lean approach (my example with startup burn rate 7x below peers)

​

• AI infancy: In 2000 Google charged by the click: garbage clicks, now charges based on performance. In 2025: AI companies charge by token usage. Lots of garbage tokens (partly due to the way DNNs work). Shifting to performance-based models.

​

​

​

8

A Few Challenges (Cont.)

• Investor Pressure: Investors and executive need to be better educated about alternatives. Changes in VC environment; some LPs want to eliminate middle-men.

​

• Marketing influence: Transformer model encroachment. Big companies offering $200k free GPU for 2 years to startups. Then you must pay, thus charge your customers accordingly. Hiring focused on people who have learned/used the exact same models. College teaching the same material, slow to respond to change. Independent research done outside academia or Google labs now getting traction.

​

​

​

9

A Few Challenges (Cont.)

• Hiring in the age of AI:
• Companies hire very expensive OpenAI engineers. Don’t. Build the future, not the past.
• Great candidates outside US. But reluctance to have overseas employees. Looking for H1-B when best candidates prefer to work from their home country (no Visa, saving time and money). How to do it? Ask me or AI. Use AI to recruit.
• Hire well-known advocate. Offer to work on sexy projects besides mundane tasks (your bread and butter). In my case: predicting heart attacks, DNN watermarking.
• Einstein was a clerk tasked to get all railway stations in Switzerland to show the exact same time. Relativity theory was born out of it.
• Kitchen chef impressed me with how he could use AI to solve problems he knew nothing about. Impressed by my 18-year-old son too. Ask candidates to use AI in job interviews.

​

​

​

​

​

10

Efficient AI Framework: xLLM

11

1. General architecture
2. Key elements to reduce TCO

General Architecture

12

General Architecture (Cont.)

13

General Architecture (Cont.)

14

General Architecture (Cont.)

15

General Architecture (Cont.)

16

General Architecture (Cont.)

17

Key Elements to Reduce TCO

• Mixed RAG/LLM: Structured output (RAG layer below response) is very compact, thanks to relevancy scores, limited to corpus (specialized language model) and distillation. Minimizes risk of hallucination. Little need for prompt engineering. Provides very accurate links or references.

​

• Small DNN with distillation using structured output as input to generate (if needed) a standard response. Full corpus is < 1 million multi-tokens. Example: reduce training set size by 80% (random deletions) without loss. Smart distillation.

​

• No GPU, no expensive training, no Blackbox. Low electricity consumption, low cost. No external API calls, increased security. On-premises implementation if required by client. Fast onboarding and learning curve, easy to plug APIs from clients or other vendors.

​

• In-memory RAG. Native Python DB (nested hashes ~JSON) for research purposes, QA & training. Lighting fast debugging and development. Separate from production environment.

​

• Workarounds to fix issues with Python libraries. Alternative to WordNet.

​

​

​

​

​

18

Key Elements to Reduce TCO (Cont.)

• Cache optimization: too much is not good, can be costly if on GPU. Cache can grow unmonitored. Minimize memory leaks that clog the system over time. Switch between CPU and GPU as needed.

​

• Documentation with index, glossary, tags, and code explanation with examples; meaningful variable names and conventions, versioning.

​

• Table redundancy to accelerate retrieval (building mappings and reverse mappings, for instance:
• Parent-to-children chunk mapping, and reverse
• Keyword (multi-token) to stemmed version, and reverse (un-stemmer)
• Chunk ID to attached tags, and reverse

​

• Reproducibility. Run same query twice in two sessions with same params, get same answer. Otherwise, hard to debug!

​

​

​

​

​

​

19

Key Elements to Reduce TCO (Cont.)

• Security & compliance: Results in liability and extra costs if poorly executed.
• Control access at the corpus and chunk level (authorized users)
• On-premises, no Blackbox, explainable AI. No call to external API, your data not exported. DNN/Data watermarking to protect against unauthorized use.
• Use your data only, and/or approved external sources. Detect issues in your data by looking at response scores; fix them. Exact retrieval possible

​

• Better Evaluation and Benchmarking. See my article “Benchmarking xLLM and Specialized Language Models: New Approach & Results”

​

• Optimized algorithms. For instance, O(n²) replaced by O(n) when possible. Example: to create keyword correlation table based on proximity within a chunk.

​

• Quantization. 4 bits rather than 32. Fast nearest neighbor search (vector DB)

​

​

​

​

​

​

​

​

20

Sample Source Code

21

1. Frontend tables
2. Distillation
3. Backend tables
4. Distance-based word pairs

Sample code – frontend tables

22

Sample code – distillation

23

Sample code – distillation (Cont.)

24

Sample code – backend tables

25

Sample code – keyword correl. table

26

Sample code – keyword correl. Table (Cont.)

27

Conclusions & References

28

Conclusion

• Reduce GPU consumption, GPU/CPU mix
• Pay by usage, not by token
• Smart distillation
• Leverage AI
• Smart Hiring
• Optimized your algorithms (identify bottlenecks)
• Good documentation
• Optimize cache, reduce memory leaks
• Extract as little as possible from DB to generate answer
• Simplicity, no Blackbox, faster learning curve and onboarding
• Security & compliance
• Revisit evaluation/benchmarking metrics

​

​

​

​

​

​

29

References

• Doing Better with Less: LLM 2.0 for Enterprise – mltblog.com/4jF3dln

​

• Benchmarking xLLM and Specialized Language Models: New Approach & Solutions – mltblog.com/4nzaKUb

​

• 10 Tips to Boost Performance of your AI Models – mltblog.com/4mYXd8V

​

• How to Design LLMs that Don't Need Prompt Engineering – mltblog.com/3GAbAQu

​

• GitHub: VincentGranville/Large-Language-Models

​

• Contact us: vincent@bondingai.io | linkedin.com/in/vincentg/

​

​

MLtechniques.com - xLLM, by Vincent Granville

30

Thank You!

​

31

32

Risks Specific to Enterprise LLMs (1/3)

• Call to external APIs (OpenAI), data leakage
• Reliance on Blackbox systems (transformers, DNN)
• Hard to train, requires billions of tokens, GPU farms, lots of electricity
• Hard to distill
• Yet training to predict the next token is outdated
• Charging by token incentivizes vendors to use more tokens
• Analogy: pay-per-click monetization in 2010
• Algorithmic bias
• Prompt injection
• Costly mistakes or hallucinations
• No relevancy scores displayed to user
• Not scoring input sources

​

​

​

MLtechniques.com - xLLM, by Vincent Granville

33

Risks Specific to Enterprise LLMs (2/3)

• Not all users should have access to the full corpus
• Solution: sub-LLMs and even chunks with restricted access
• Failure to provide precise references to source
• Highly re-worded response magnifies the problem, causes hallucinations
• Most secure solutions are on-premises with full control by client
• Faulty evaluation metrics
• LLM as a judge: circular loop
• Do not capture exhaustivity and other qualities
• Use of synthetic prompts
• Evaluation is user-dependent

​

​

​

MLtechniques.com - xLLM, by Vincent Granville

34

Risks Specific to Enterprise LLMs (3/3)

• Issues with Python libraries
• Autocorrect, stopwords, stemmers (global vs. local)
• Inability to sample outside the observation range
• Failure to provide precise references to source
• Highly re-worded response magnifies the problem, causes hallucinations
• Poor QA
• No action taken following consistent poor user ratings to responses
• Failure to connect parts of the corpus to your LLM
• Execution without consulting with the right people
• Hard to debug and fine-tune
• Much easier with our architecture

​

​

​

MLtechniques.com - xLLM, by Vincent Granville

35

A Better UI for LLMs (1/2)

• User can choose:
• sub-LLMs,
• Categories,
• Tags
• Search options:
• Exact or broad match
• Negative keywords
• Search by recency
• Offer real-time fine tune and re-training, with intuitive parameters for
• Distillation, stemmer / un-stemmer, embeddings, relevancy scores, …

​

​

​

MLtechniques.com - xLLM, by Vincent Granville

36

A Better UI for LLMs (2/2)

• Lightning-fast testing, training, debugging and in-memory LLMs, from UI
• Nested hashes (JSON-like)
• Variable-length embeddings instead of vector DB and dot products
• In addition to standard response, offer structured output (summary boxes + alternate queries) to reduce hallucinations and prompt engineering, with relevancy scores and precise references to corpus
• Chunks pre- and post-tagging
• Hierarchical chunking with option to browse corpus
• Minimize re-wording in final response (important e.g. for legal documents)

​

​

​

MLtechniques.com - xLLM, by Vincent Granville

37

Elements of our architecture (1/4)

​

​

MLtechniques.com - xLLM, by Vincent Granville

38

Elements of our architecture (2/4)

​

​

MLtechniques.com - xLLM, by Vincent Granville

39

Elements of our architecture (3/4)

​

​

MLtechniques.com - xLLM, by Vincent Granville

40

Elements of our architecture (4/4)

​

​

MLtechniques.com - xLLM, by Vincent Granville

41

Our Team

Extreme LLM (xLLM) in a Nutshell

• Mixture of experts
• Specialized sub-LLM and/or sub-LLMs for authorized users
• LLM router to manage the sub-LLMs
• User selects sun-LLM, agents, and hyperparameters
• Each sub-LLM built with its own taxonomy and contextual environment

​

• No neural network, no training
• Thus, low cost, easy to fine-tune in real-time, in-memory LLM, on-premises
• Self-tuned based on favorite hyperparameters, intuitive parameters
• No GPU, no latency, exhaustive concise results, local implementation

• Concise results
• Multiple sections: links, related content, x-embeddings based on E-PMI metric
• Output with relevancy score attached to each item in each section; User offered choices for deeper or alternate queries
• Great for professional users. Not just a “prompt box”; many options in the UI, like a mini-browser
• Case studies
• Corporate datal lake corpus
• Nvidia PDF repository
• Wolfram corpus: 15k webpages, 5k categories
• Publisher, 4000 titles: clustering, predicting article performance

MLtechniques.com - xLLM, by Vincent Granville

xLLM for Enterprises to build own LLMs faster, at lower cost, with increased accuracy, security, explainable AI

43

Prompt Results – Card Format (web API)

MLtechniques.com - xLLM, by Vincent Granville

44

Prompt Results – Card Format (web API)

MLtechniques.com - xLLM, by Vincent Granville

45

Prompt Results – Listing Format (1)

MLtechniques.com - xLLM, by Vincent Granville

46

Prompt Results – Listing Format (2)

MLtechniques.com - xLLM, by Vincent Granville

47

Prompt Results – Structured Text Format

• Text entities retrieved from corpus via contextual chunking / indexation
• Blended with images, datasets, exact URLs/references and so on (multimodal)
• Featuring categories, tags, related content, titles, timestamps, links, and so on
• Structured output based on hierarchical chunking and multi-indexing, augmented with auto-tagging, acronyms / synonyms dictionary, and un-stemming
• Multiple relevancy scores based on multiple types of multi-tokens, then normalized
• Exact vs broad search, negative keywords, variable weights attached to prompt multi-tokens, search by recency.
• Auto-correct, stemmer, stopwords specific to corpus; “real estate” or “San Francisco” are single tokens. Chunks visibility (for any given chunk) depends on user privileges.

​

​

MLtechniques.com - xLLM, by Vincent Granville

48

Prompt Results – Response Generation

• Based on structured output (see previous slides)
• Generic template prompts with premade auto-filled response matched against user prompts
• Light, fast proprietary DNN, no TensorFlow, Python or Keras, explainable AI.

​

• Proprietary non-Blackbox DNN with explainable AI (patent-pending)
• With equalizer, stabilizer and other proprietary features to accelerate convergence and increase stability
• Chaotic gradient descent with temperature decay
• Sub-epochs, sub-layers, original universal function
• Global optimization or one sub-layer at a time within an epoch
• Pre-tabulated functions, generic partial derivatives

​

MLtechniques.com - xLLM, by Vincent Granville

49

Backend Features

• Smart crawling to retrieve embedded structure
• Breadcrumbs (enterprise corpus), concept associations (related links)
• Metadata, tags, taxonomy, long contextual environment
• PDF parser (TOC, index, glossaries, synonyms, titles, tables, images)
• X-embeddings
• Variable-length embeddings stored as sparse nested hashes
• Multi-token: “data~science” on top of single tokens “data” and “science”
• Contextual token: “data^science”, both words in same paragraph but not adjacent
• PMI (pointwise mutual information) instead of dot product / cosine distance
• Parametric weights attached to tokens (no loss function to optimize)

​

MLtechniques.com - xLLM, by Vincent Granville

50

Retrieved Taxonomy: Wolfram Example

MLtechniques.com - xLLM, by Vincent Granville

51

Retrieved Context: Enterprise Example

MLtechniques.com - xLLM, by Vincent Granville

52

Backend Features (Cont.)

• Home-made libraries
• Issues with Python libraries (singularize, autocorrect, “Feller” changed to “seller”)
• Minimize stemming and text transforms; keep plural if found in corpus
• Important: accented characters, separators (punctuation), capital letters
• Ad-hoc lists: home-made stopwords, do-not-singularize, do-not-autocorrect
• Backend tables (specific to each sub-LLM)
• X-embeddings not the most important table; taxonomy more important
• Compression mechanism: sorted n-grams
• Backend parameters

MLtechniques.com - xLLM, by Vincent Granville

53

Backend Features (Cont.)

• Chunking & Indexing
• Chunks called text entities: webpage, subsection (PDF), or JSON entity
• Indexed for fast retrieval of full content, and for easy content linking
• Chunks of variable length, hierarchical chunking and multi-index, content de-duping
• Auto-tagging. Use relative font size and other elements to generate contextual fields.
• NLP
• Python with workarounds + homemade
• Weighted graph tokens: multi-tokens found in the context/taxonomy elements
• Customized pointwise mutual information (PMI), instead of cosine similarity

MLtechniques.com - xLLM, by Vincent Granville

54

Frontend Features

• User Interface
• Many options, not just a search box (see previous slide)
• User can choose agents, sub-LLM, or fine-tuning in real time
• End-user debugging with catch-all parameter set
• Relevancy multi-scores to rank response chunks
• Goal: too many results to show to user prompt, which ones to display?
• Graph tokens and multi-tokens with 2+ words: boost score
• Text entity with 2+ multi-token intersection with prompt, get higher score
• Rare multi-tokens get extra boost
• Longer text entities get extra boost

​

​

MLtechniques.com - xLLM, by Vincent Granville

55

Relevancy scores

MLtechniques.com - xLLM, by Vincent Granville

56

Frontend Features (Cont.)

• Distillation
• If multi-tokens A~B~C and A~B have same count, show results from A~B~C, not A~B
• Acronyms and synonyms
• If A and B are synonyms, A in prompt but not in corpus, and B in corpus, map A to B in the prompt to retrieve B in the corpus (Goal: trying to be exhaustive)
• Self-tuning – Most popular front-end parameters used to build default parameters
• Prompt cleanup with stopwords list / stemmer different from backend list
• Stemming and un-stemming

​

MLtechniques.com - xLLM, by Vincent Granville

57

Distillation

MLtechniques.com - xLLM, by Vincent Granville

58

Proprietary JSON-based PDF Parser

MLtechniques.com - xLLM, by Vincent Granville

59

Backend: Overview

MLtechniques.com - xLLM, by Vincent Granville

60

Frontend: Overview

MLtechniques.com - xLLM, by Vincent Granville

61

Path from Prompt to Results

MLtechniques.com - xLLM, by Vincent Granville

62

Path from Crawl to Backend Tables

MLtechniques.com - xLLM, by Vincent Granville

63

Details: Indexation

MLtechniques.com - xLLM, by Vincent Granville

64

Detail: Relevancy Algorithm

MLtechniques.com - xLLM, by Vincent Granville

65

Detail: Sorted N-Grams

MLtechniques.com - xLLM, by Vincent Granville

66

Database: Nested Hashes (like JSON)

MLtechniques.com - xLLM, by Vincent Granville

67

Evaluation

• User-based (automated)
• Collect favorite hyperparameters chosen by users
• Use smart grid search to set default hyperparameters based on user favorites
• Fine-tune on one or few sub-LLMs (like LoRA) before full optimization on (say) 200 sub-LLMs. You may fine-tune all sub-LLMs in parallel.
• Taxonomy-based (automated)
• Pretend that the taxonomy backend table comes from external sources
• Assign categories to webpages based on this “external” taxonomy
• For each webpage, compare externally assigned to native category

​

MLtechniques.com - xLLM, by Vincent Granville

68

Evaluation (Cont.)

• Evaluation challenges
• We are dealing with unsupervised learning: there is no perfect output except for trivial cases
• Quality depends on user (professional users and laymen have different criteria)
• How do you measure exhaustivity, depth, and recency?
• Output value versus grammatical capabilities
• How do you integrate xLLM relevancy scores attached to each item, to evaluate output quality? No other LLM return these scores

MLtechniques.com - xLLM, by Vincent Granville

69

Taxonomy-Based Evaluation

MLtechniques.com - xLLM, by Vincent Granville

70

xLLM for Data Synthetization (with ALF)

MLtechniques.com - xLLM, by Vincent Granville

71

NoGAN Tabular Data Synthetization

• Real data: 2 concentric circles
• Synthesized, NoGAN synthesizer: blue dots. Constrained synthetization to keep loss above some threshold
• As the loss function gets more granular, the synthesized data gets more similar to the real data (the training set)

xLLM for Predictions

• Case study – media industry
• Predicting article performance (pageviews) based on title keywords and category
• 4000 articles; pageview is normalized and time-adjusted
• Evaluation and Loss function (identical)
• Based on comparing predicted with observed quantiles, using 5 quantiles (see code)
• Good proxy to Kolmogorov-Smirnov distance

​

MLtechniques.com - xLLM, by Vincent Granville

72

xLLM for Predictions – Model

MLtechniques.com - xLLM, by Vincent Granville

73

xLLM for Predictions – Category Encoding

• Create new codes sequentially as you browse the training set.
• Aggregate codes with few observations into bundles.
• Create two key-value mappings. Ex:
• Category_to_Code[‘Blog’, ‘William’] = 5
• Code_to_category[5] = [‘Blog’, ‘William’]
• Replace the categorical features by the newly created feature, “Code”.
• Number of codes ≤ number of obs.

MLtechniques.com - NoGAN Synthesizer, by Vincent Granville

74

xLLM for Predictions – Results

• Observed vs predicted normalized pageview count

MLtechniques.com - xLLM, by Vincent Granville

75

xLLM for Clustering

• Case study – media industry
• Identifying patterns / clusters in popular articles based on title keywords
• 4000 articles; pageview is normalized and time-adjusted
• Methodology
• Group multi-tokens into clusters based on a similarity metric, with hierarchical clustering and k-medoids
• Let S(t) be the set of articles containing the multi-token t in the title
• For each multi-token group G, the list L(G) of articles belonging to G is

​

MLtechniques.com - xLLM, by Vincent Granville

76

xLLM for Clustering (Cont.)

• Similarity between two multi-tokens t₁, t₂

​

​

• Remarks
• Multi-token clusters are non-overlapping, but article clusters may overlap
• Sklearn clustering methods require a distance matrix as input; the matrix (derived from the similarity metric) is huge but extremely sparse.
• In my implementation, s(t₁, t₂) is computed and stored only if it is strictly positive. Using connected components for clustering, it is far more efficient than Sklearn.

MLtechniques.com - xLLM, by Vincent Granville

77

xLLM for Clustering – Sample Structure

MLtechniques.com - xLLM, by Vincent Granville

78

xLLM for Clustering – Sample Cluster

• Cluster of popular articles linked to multi-token cluster with 3 elements, including one contextual multi-token: “Machine^vs” (pv stands for normalized pageview)

​

MLtechniques.com - xLLM, by Vincent Granville

79

Interlude – Fast Nearest Neighbor Search

• Red dot: prompt-derived embeddings
• Blue dot: backend table embedding
• Over time, arrows link red dots to their nearest blue dots
• Alternative to vector search

MLtechniques.com - xLLM, by Vincent Granville

80

xLLM for Next Token Prediction

MLtechniques.com - xLLM, by Vincent Granville

81

​

• Next token prediction: the mother of all LLMs
• Here: predict next DNA sub-sequence to generate synthetic genomic data
• Alphabet has 4 letters
• Left: Scatterplot comparing observed vs synthetic ECDFs

Part 5 �References��

MLtechniques.com – xLLM, by Vincent Granville

82