1 of 15

DSPY IN PRODUCTION

GANARAJ PERMUNDA

ZORO UK

2 of 15

ABOUT ME

  • Work at Zoro UK as Lead Technical Architect.
  • Masters in AI with Robotics.
  • Worked in London for 14 years.
  • Started 8 years back at Zoro during its inception.
  • Zoro is a wholly owned subsidiary of Grainger, a Fortune 100 company.
  • 1m MAU ecommerce.
  • Been there from login and registration, Search, Real Time Analytics, Recommendations, Product Information Management (PIM) and now Gen AI.

3 of 15

USING DSPY IN 8 STEPS

  • Define your task
  • Define your pipeline
  • Explore a few examples.
  • Define your data.
  • Define your metric.
  • Evaluate on “zero-shot”.
  • Compile with a DSPy Optimizer.
  • Iterate.
  • https://dspy-docs.vercel.app/docs/building-blocks/solving_your_task

4 of 15

TRANSLATION

5 of 15

SMRTHI.COM

Personal project.

Provides individual verses from Indian literature and scriptures.

Quote a verse on social media or other places.

Translation for individual words and whole verse.

Search for a verse.

6 of 15

SANSKRIT TRANSLATION USING DSPY

  • Direct Prompt is error prone.
  • DSPy gets very good accuracy with a local model.
  • Cheap to run.

​

​

​

​

​

​

​

7 of 15

HOW LOCAL MODEL TRANSLATES?

8 of 15

DSPY AT ZORO

9 of 15

THE PROBLEM

  • Attributes from too many sources.
  • Some have units, some without.
  • Attribute values don’t have an order.
  • Attributes coming from various sources.

​

10 of 15

WHY DSPY?

  • No Long Prompt.
  • Prompt is optimized.
  • LLM Agnostic (to an extent).
  • Can try various optimization strategies.
  • Smaller LLM can be “trained” with larger LLM.

​

​

​

​

​

11 of 15

DSPY IN PRODUCTION

12 of 15

LEARNINGS

  • Hardest step is figuring out the metric.
  • DSPy is constantly evolving, missing documentations.
  • DSPy discord channel is a great resource.
  • It's not like Neural Networks, it's more like Genetic Algorithms. 

​

​

13 of 15

PROJECTS AND REPOS

​

14 of 15

QUESTIONS?

15 of 15

THANK YOU