Generative AI

Application Migration

2024 June

1

2

What we’ll discuss today…

• Konveyor/MTA Overview
• Konveyor AI (Kai): Goal
• Kai: Approach
• Kai: Demo Recap
• Next steps…

3

Konveyor Overview

4

“Surface insights on applications at scale to empower enterprise architects to make better-informed decisions related to modernization activities”

​

github.com/konveyor/community/Charter.md

​

Konveyor

Mission and goals

5

Language Provider

Language Server Protocol Implementation

Source Code

Binaries

Translate conditions in a Rule to LSP queries

Analysis Rules

Community Knowledge + Custom Rules

Community contributed rules included with Konveyor

konveyor/rulesets

~2400+ Rules and growing

Custom Rules for Organization specific frameworks/technologies

Report informs Architect of areas of concern

Organization Wide View of Application Information

Target technology tags

Discovered source technology tags

Analysis Incident

Analysis Info Details

Sample Analysis Report

Example: Recommendation to replace JMS Topic with an Emitter

7

Konveyor AI (Kai) Goal

8

Konveyor AI (Kai)

Goal

Goal: Improve the economics of re-platforming and refactoring applications to Kubernetes and cloud-native technologies by leveraging Generative AI

​

Approach:

• Expand Konveyor capabilities beyond surfacing information
• Generate code suggestions for discovered migration issues
• Work with LLMs using structured migration data Konveyor has collected
• Avoid fine-tuning each model by using Retrieval Augmented Generation (RAG)
• Shape LLM results with examples of how Organization has solved similar problems in the past
• Model agnostic
• Integration with commercial models
• Ability to bring your own model

​

9

Modernization Engagement running at Scale

General Pattern

10

Modernization Engagement running at Scale

General Pattern

Konveyor AI (Kai) focus area

11

Konveyor AI (Kai)

Workflow

12

Konveyor AI (Kai) Approach

13

Konveyor AI (Kai)

Generative AI to automate source code changes between technologies

• Uses data in Konveyor to generate code suggestions
• Source code analysis with Rules
• Pinpoint issues to adopting a new technology
• Changelog history from source repositories
• Before/After of previously solved issues
• Crafts a tailored LLM prompt based on:
• Knowledge of specific problems that need to be addressed
• Knowledge of prior successful code changes
• Provides suggested code changes via IDE plugin

​

​

14

Kai + Konveyor

Functional Overview

15

Large Language Model (LLM)

Concerns/Limitations

• Limited context windows
• It’s not feasible to include an entire applications full source code in a prompt
• Generally models have a limited context window
• Costs increase as more data is included in a request
• Desire to augment a LLM’s knowledge without fine-tuning
• Corporate internal frameworks are generally not part of an existing models ‘training data’
• We want to leverage a RAG approach to include ‘solved examples’ in the prompt to help shape the results
• Integrate with multiple models
• Rapid advancements on models increases the desire for flexibility to explore new models as they become available

​

​

​

16

Approach: LLM Limited Context Windows

Source Code Analysis Guides LLM interaction

• Use static code analysis to help guide interaction with LLM
• We assume limited context sizes in LLM requests so we break larger questions into smaller ones
• Generally not feasible to upload an entire repository of source code to a LLM

​

​

​

​

Source Code Repository

Konveyor Analysis

LLM Request

LLM Request

LLM Request

Aggregate Responses

17

Approach: Need to augment a LLM’s knowledge

Retrieval Augmented Generation

How can we influence a LLM to give us better info on knowledge it wasn’t ‘trained’ on?

• We assume that the models we use have NOT been exhaustively trained in the domain we want to leverage it in
• Think of custom frameworks private to an organization
• Newer libraries that were invented after the model was trained
• We will augment knowledge the model can answer by leveraging a ‘Few Shot’ example with passing data into the prompt

​

​

​

18

Retrieval Augmented Generation (RAG)

Approach

Source Code

Solved Issues (code snippets)

Analysis Issues

Prompt Instructions

+

Large Language Model (LLM)

Updated Source Code

Reasoning

Issue to fix: "Stateless EJBs can be converted to a cdi bean by importing `jakarta.enterprise.context.ApplicationScoped` and replacing the `@Stateless` annotation with a scope eg `@ApplicationScoped`"

Line number: 14

diff --git a/src/main/java/org/jboss/as/quickstarts/bmt/ManagedComponent.java b/src/main/java/org/jboss/as/quickstarts/bmt/ManagedComponent.java�…

-@Stateless

-@TransactionManagement(TransactionManagementType.BEAN)

-// tell the container not to manage transactions

+@ApplicationScoped

…

Agent workflow

​

20

Differentiator

What differentiates this approach from a CoPilot clone

• Uses structured migration data Konveyor/MTA has collected across the Organization
• Application Inventory, Migration Waves, Analysis Reports, Source code repos
• Guided workflow uses discovered analysis issues to drive LLM interaction
• Kai understands the issues that need to be addressed to adopt a new technology and uses this to focus the LLM
• Leverages a shared repository of community knowledge of migration rules
• Extracts patterns from previously solved problems in the Organization and leverages to shape LLM results
• Beyond this specific application, Konveyor looks at the history of all applications in the configured Organization
• Model Agnostic:
• Integration with models from IBM’s watsonx™ Code Assistant
• Ability to specify your own models and work with other providers and/or local models
• Example on-premise models from huggingface.co hosted by OpenShift AI

​

​

​

​

21

Large scale migration methodology

22

Migration methodology

The path to Kubernetes adoption

Assess

Prepare

Pilot

Scale

Rationalize

23

Assess

Understand the application landscape

Technologies & Runtimes

ALM Processes

Early identification of risks

Mitigation strategies

Holistic view of the application landscape

24

Rationalize

Classify and define migration approach for each application type

Rehost (lift & shift)

Replatform (lift & reshape)

Repurchase

Refactor (rewrite & decouple)

Assess, review,�prioritize.

Retire

Retain as is (for now)

?

Existing�App

25

Prepare

Mitigate risks and configure the runtime platform

Deployment Manifests

Runtime Images

Configuration Model

ALM

Implement application deployment model

Execute mitigation strategy

26

Pilot

Migrate and learn

Migrate a representative set of apps

Document everything

Test the migration process

Test the organization

Execute the first migration wave

27

Scale

Migrate the whole portfolio iteratively

Scale the adoption team

Migrate in waves

Migrate the application portfolio

28

Inserting Kai into the migration process

29

Scale

Applying Kai at Scale

Impact of Kai in the Scale stage

Pilot

Seed

Support

Automate

30

Pilot

Migrate and learn

Migrate a representative set of apps

Document everything

Test the migration process

Test the organization

Execute the first migration wave

Create custom rules

31

Pilot

Create custom rules

• Architects manually migrate the first wave of applications
• Konveyor provides some assistance for issues related to standard technologies.
• Custom technologies are migrated manually. Architects document the solution for found issues in the shape of custom rules than can be used in subsequent waves.
• It is essential to pick up a representative sample from all application archetypes to maximize the effectiveness (and learnings) of this stage.
• Kai starts collecting limited data related to issues found in standard technologies for which Konveyor has rules out of the box.
• Output:
• A comprehensive set of rules that covers common migration issues for the custom technologies in the organization.
• First wave of migrated applications.
• Before and after for issues related to standard technologies.

​

​

​

​

32

Seed

Initialize the knowledge base for Kai

• Migrators start migrating applications guided by the custom rules implemented by the architects during the pilot stage:
• Changes to applications on this stage are applied manually by the migrators based on the guidance provided by the custom rules (standard Konveyor usage).
• Kai starts getting comprehensive data on representative issues that span both custom and standard technologies.
• This stage might require several migration waves depending on how representative the applications are of the different archetypes. To maximize efficiency, the application set should be as diverse as the one chosen for the pilot.
• Output:
• Issue data associated with the custom rules implemented in the previous stage.
• Before and after for issues related to both custom and standard technologies.

​

​

​

​

33

Support

Start leveraging LLMs and provide feedback to refine the data set

• Migrators start getting suggestions from Kai on how to solve issues found by the analysis:
• IDE driven user experience.
• Fixes are applied on a per issue basis:
• Analyzer finds issues on the source code.
• Migrator requests an automated fix for a given issue.
• Kai uses the rules hints and examples of how that issue was solved in previous waves to provide additional context to the underlying LLM.
• Kai provides a diff with the suggested fix.
• The migrator either approves or rejects the fix.
• Kai learns from the user choice.
• Kai refines the example set used to generate fixes based on user feedback.
• This stage will require multiple migration waves until fixes from Kai are consistently accepted.
• Output:
• Refined example set to be used by Kai to generate fixes.
• Migrated applications leveraging an increasingly more accurate autofix mechanism.
• Instructlab knowledge to augment the used LLMs.

​

​

​

​

34

Automate

Scale out automated source code fixes

• This stage can be executed once Kai has a consistent set of fixes for the most common issues for the different technologies in the organization.
• Kai executes changes at repository level without human intervention:
• Hub driven approach:
• Architect/Migrator runs an analysis in the hub.
• Issues are identified for the analyzed application. Kai provides info on how many of those issues have a documented and accepted fix.
• Architect/Migrator chooses to autofix at repository level. Kai iterates through all the issues found in the repository and applies fixes sequentially to the codebase.
• Migrator clones the repository and applies some minor manual tweaks to ensure the application is migrated correctly.
• Most of the migration work is automatically done by Kai.
• Output:
• Migrated applications with reduced to minimal human intervention.
• High migration throughput that requires less resources.
• Instructlab knowledge to augment the used LLMs.

​

​

​

​

35

Konveyor AI (Kai)

IDE Usage Walkthrough

36

IDE Usage Walkthrough

Use case and target persona

Use case

• Assisted migration of a Java EE app to Quarkus
• Java EE app konveyor-ecosystem/coolstore
• Uses community quarkus rules and custom-rules

Target Persona

• “Migrator”: the developer who is updating the source code

​

​

​

37

1. Konveyor static code analysis is run on the app
2. Migrator views the analysis report via VSCode IDE to see a list of issues to resolve
3. Migrator picks an issue and selects ‘Generate Fix’
4. A code change for the issue is generated and display via 2-sided pane for viewing the diff
5. Migrator accepts the change
6. Code is updated
7. Next issue is selected ….

​

​

​

​

​

IDE Usage Walkthrough

Workflow

38

Run Konveyor Analysis from within IDE

39

Info from each incident is used to resolve the issue.

​

These ‘hints’ are passed to the LLM so it knows exactly what we need to do to resolve the issue

Konveyor Analysis Issue Details in the IDE

40

Step 1: Right click to generate a suggested fix

Step 3: Review suggested changes or make edits

Step 2: Consult the LLMs reasoning for changes

Step 4: Accept or Reject

Thank you!

41

Backup Slides

42

Konveyor is focused on Application Modernization to help accelerate the adoption of Kubernetes at scale

​

​

44

Konveyor Hub

Assets Generation

Reporting

Assessment

Code Transformation

Planning

Analysis

45

Konveyor Unified Experience

46

Unified Experience: Overview

​

See https://github.com/konveyor/enhancements/tree/master/enhancements/unified_experience for more information

47

Unified Experience: Personas

​

48

Step #1 - Surface Information

​

49

Step #2 - Make Decisions

​

50

Step #3 - Plan Work

​

51

Step #4 - Do Work

​

52

Konveyor Analysis

53

Source Code Analysis Engine for Multiple Languages

​

• Approach
• Leverage Language Server Protocol (LSP) implementations for each language
• LSP was developed by Microsoft to help IDEs like VSCode work with multiple languages
• ~200+ Implementations of popular languages VSCode can ‘understand’
• Analysis Engine works with multiple ‘Language Providers’
• Each Language Provider is a small piece of code that integrates a LSP implementation for the given language.
• Provider translates “Rules” to LSP Queries

​

​

54

Custom Migration Targets

Streamline knowledge sharing abstracting migration teams from the complexity

55

Example of a Rule

​

- category: mandatory� description: Remote EJBs are not supported in Quarkus� ruleID: remote-ejb-to-quarkus-00000� when:� or:� - java.referenced:� location: ANNOTATION� pattern: javax.ejb.Remote� - java.referenced:� location: ANNOTATION� pattern: jakarta.ejb.Remote� message: |-� Remote EJBs are not supported in Quarkus, and therefore its use must be removed and replaced with REST functionality. In order to do this:� 1. Replace the `@Remote` annotation on the class with a `@jakarta.ws.rs.Path("<endpoint>")` annotation. An endpoint must be added to the annotation in place of `<endpoint>` to specify the actual path to the REST service.� 2. Remove `@Stateless` annotations if present. Given that REST services are stateless by nature, it makes it unnecessary.

remote-ejb-to-quarkus-00000:� description: Remote EJBs are not supported in Quarkus� incidents:� - uri: file:///tmp/source-code/src/main/java/com/redhat/coolstore/service/ShippingService.java� message: "Remote EJBs are not supported in Quarkus,.....\n"� lineNumber: 12� variables:� file: file:///tmp/source-code/src/main/java/com/redhat/coolstore/service/ShippingService.java� kind: Class� name: Stateless� package: com.redhat.coolstore.service

​

Example of an Analysis Issue

​

56

Kai Backup Slides

57

Approach

1. Leverage data Konveyor has access to today
2. Address limitations with LLMs by breaking big problems into smaller chunks
1. Use Analysis info to pinpoint smaller focal areas
3. Form higher quality responses by using previously solved issues from the organizations app portfolio
• Look at source code commit changelogs to learn how other similar problems were solved
• Put relevant data in prompt to help LLM have more info

​

Approach to leveraging LLMs

• Focus on forming ‘few-shot’ prompt examples
• Augment prompt with relevant information from data Konveyor has access to Retrieval Augmented Generation (RAG)
• Address limited context windows by breaking larger problems into smaller problems

​

​

​

​

58

Kai + InstructLab Workflow

59

Solved Example:

• Extract common patterns of how issues have been solved in the past to influence future suggestions
• Check if a migration issue used to exist in an application that has been migrated
• If so, we can use how that issue was solved as an example for future problems

Konveyor Migration Data

​

Organization wide view of structured migration data

• History of static code analysis reports
• Source code repo commit history

60

How do we find a ‘Solved Example’ for initial prompt

Talk to LLM and ask to help us to pick out the most relevant pieces of code to help with future generations

Functional Overview

62

“Cascaded changes” - edits that impact multiple files in a repo

​

i.e. changing a method signature and all of it’s usages…

Leverage LSP for real-time parsing of code and recognition of errors to address

CodePlan: Repository-level Coding using LLMs and Planning

Example of possible UX (similar to CoPilot)

Example of GitHub Copilot Explain UX

Kai will help the MigrationFactory team by providing an experience similar to CoPilot

​

The value-add for Kai is:

• Analysis information will drive the key focal points that a Migrator needs surfaced via IDE
• Issues will have associated Suggestions the Migrator can review/tweak/apply
• Suggestions will improve in quality as the organization solves similar problems
• Configurable to use multiple backend AI systems Remote or On-Premise

​