LLaMo: Large Language Model-based �Molecular Graph Assistant

Jinyoung Park, Minseong Bae, Dohwan Ko, Hyunwoo J. Kim

​

Department of Computer Science and Engineering , Korea University

Korea University

MLV Lab

Korea University

​

​

​

​

​

​

​

​

​

​

​

​

MLV Lab

Introduction

> Large language models

Introduction

> Molecular graph assistant

Korea University

MLV Lab

Molecule description generation

Could you give me a brief overview of this molecule?

IUPAC name prediction�What is the IUPAC name of the molecule?

Chemical reaction prediction

Please suggest a potential product based on the given reactants and reagents.

​

Molecule description generation

The molecule is the hydrogenmaleate salt of O-(cyclohexanecarbonyl)lysergol…

Property prediction�Please provide the energy separation between the highest occupied and lowest unoccupied molecular orbitals (HOMO-LUMO gap) of this molecule.

IUPAC name prediction�The molecule's IUPAC name is 2-amino-1-phenylethanol.

Property prediction�0.1913

Chemical reaction prediction

O=[N+1]([O-1])C1=CC(CO)=C(F)C=C1F

LLaMo

Projector

Korea University

MLV Lab

Methods

> Large molecular graph-language model

Large Language Model

Graph Encoder

Could you provide a description of this molecule?

Input 2D Molecular Graph

Instruction

Generate

Large molecular graph-language model

​

• Graph encoder

​

• Projector

​

• Large language model

Korea University

MLV Lab

Methods

> Graph encoder

Large Language Model

GNN Layer 1

Could you provide a description of this molecule?

Graph encoder

Input 2D Molecular Graph

Instruction

Generate

Projector

Graph encoder

​

Generally, graph neural networks are used as a graph encoder.

​

Graph neural networks learn node representations via message-passing framework.

Korea University

MLV Lab

Methods

> Projector

Large Language Model

GNN Layer 1

Could you provide a description of this molecule?

Graph encoder

Input 2D Molecular Graph

Projector

Instruction

Generate

Projector

​

• MLP

Projector

​

• Resampler

Korea University

​

​

​

​

​

​

​

​

​

​

​

​

MLV Lab

Methods

> Problems in existing projectors

Over-smoothing problem

Node representations in a graph become nearly identical as the number of layers increases.

Conventional projectors relying on high-level node representations have a limited capability to preserve the detailed or local information of molecular graphs.

Korea University

MLV Lab

Methods

> Large Language model-based Molecular graph assistant (LLaMo)

Multi-level graph projector captures multi-hop graph information by leveraging node representations from all layers of a GNN.

Korea University

MLV Lab

Methods

> Large Language model-based Molecular graph assistant (LLaMo)

Multi-level graph projector also leverages the information of motifs, which are the statistically

important subgraphs in the molecular graphs.

Korea University

MLV Lab

Methods

> Large Language model-based Molecular graph assistant (LLaMo)

Finally, multi-level graph projector applies MLP to the multi-hop and motif-level representations to construct molecular graph tokens to be fed in LLM.

Korea University

MLV Lab

Methods

> Training LLaMo

Stage1: graph-language alignment

​

Focuses on the alignment between the graph encoder and a large language model by learning the multi-level graph projector with data consisting of molecule-language pairs.

​

Stage2: instruction-tuning

​

Train the LLM to enhance the instruction-following capabilities and enable a deeper understanding of molecular graphs with molecule-related instruction-following data.

Korea University

MLV Lab

Methods

> GPT-generated instruction data

​

Three steps of generation:

​

1) select exemplar conversations among machine-generated instruction-tuning data

​

2) generate multi-turn conversations via in-context learning with the exemplar conversations as prompts

​

3) filter out incomplete conversations and those with many turns.

​

Input of prompt:

​

(i) SMILES representation that describes the

chemical structures with special strings

​

(ii) captions that explain the molecule

​

(iii) IUPAC name that describes the molecule based on its chemical composition and structure.

Korea University

​

​

​

​

​

​

​

​

​

​

​

​

MLV Lab

Experiments

> Main experimental results

• LLaMo achieves the best performance compared to the baselines including GPT-3.5 and GPT-4!!

Korea University

​

​

​

​

​

​

​

​

​

​

​

​

MLV Lab

Experiments

> Main experimental results

Korea University

​

​

​

​

​

​

​

​

​

​

​

​

MLV Lab

Experiments

> Analysis

Korea University

​

​

​

​

​

​

​

​

​

​

​

​

MLV Lab

Experiments

> Analysis

Fine-grained (low-level)

Coarse-grained (high-level)

The molecule is a member of morpholines.

The molecule is a prostaglandin carboxylic acid anion that is the conjugate base of 13,14-dihydroxyprostaglandin F2alpha, obtained by deprotonation of the carboxy group. It is a conjugate base of a 13,14-dihydroxyprostaglandin F2alpha.

Layer 0

1

2

3

4

5

Layer 0

1

2

3

4

5

• Both low and high-level graph structural information is crucial in expressing the molecules.

​

• The attention matrix is adaptive to the caption types.

Korea University

MLV Lab

Experiments

> Qualitative analysis – Impact of molecular graph

GT: The molecule is an omega-hydroxy fatty acid anoin that is the conjugate base of 18-hydroxylinoleic acid, obtained by deprotonation of the carboxy group; major species at pH 7.3. It is a polyunsaturated fatty acid anion and an omega-hydroxy-long-chain fatty acid anion. It is a conjugate base of a 18-hydroxylinoleic acid.

GT: 18-hydroxylinoleic acid

LLaMo w/ graph: 13-hydroxyoctadec-0-enoic acid

LLaMo w/o graph: The molecule is a 1-hydroxy-2-oxo-4-oxocyclohexane-1,2-diol that is the 1-hydroxy-2-oxo-4-oxocyclohexane-1,2-diol in which the hydroxy group is at position 3. It is a 1-hydroxy-2-oxo-4-oxocyc.

LLaMo w/o graph: 1-hydroxy-2-oxo-4-oxocyclohexane-1,2-diol

LLaMo w/ graph: The molecule is a hydroxy-long-chain fatty acid anion that is the conjugate base of 13-hydroxyoctadec-9-enoic acid, obtained by deprotonation of the carboxy group; major species at pH 7.3. It is a hydroxy-long-chain fatty acid anion and a 13-hydroxy-omega-long-chain fatty.

- Invalid -

• LLaMo w/ graph accurately predicts the long-chain structure, resulting in that graph structural information is important to explain the given molecule.

Korea University

MLV Lab

Experiments

> Qualitative analysis – Impact of the multi-level graph projector

​

GT: The molecule is a member of pyrazines, a secondary carboxamide and a tertiary carboxamide.

GT: pyrazine, secondary carboxamide, tertiary carboxamide

LLaMo w/ MGProj: pyrazine, secondary carboxamide, tertiary carboxamide

LLaMo w/o MGProj: The molecule is a pyridinecarboxamide. It is a member of pyridines and a carboxamide.

LLaMo w/o MGProj: pyridine, carboxamide

LLaMo w/ MGProj: The molecule is a member of pyrazines, a secondary carboxamide and a tertiary carboxamide.

• LLaMo w/ MG Proj successfully generates “pyrazines”, “secondary carboxamide”, “tertiary carboxamide”, while LLaMo w/o MG Proj makes a mistake with the caption containing “pyridine” and “carboxamide”.

​

Korea University

​

​

​

​

​

​

​

​

​

​

​

​

MLV Lab

Conclusion

• We propose LLaMo: Large Language Model-based Molecular graph assistant consisting of graph encoder, language model, and multi-level graph projector equipped with a multi-level graph projector that captures rich information of the graph structure at multiple levels.

​

• We introduce GPT-4 generated molecular graph-text multi-turn conversation data to address the data scarcity problem of molecule-text datasets and improve the instruction-following capabilities of a large molecular graph-language model.

​

• Our experiments demonstrate that LLaMo achieves the best performance on various tasks such as molecular description generation, property prediction, and IUPAC name prediction.