1 of 1

[1] Learn how to customize a model for your application - OpenAI Documentation.

[2] Embeddings - OpenAI Documentation.

[3] The OpenAI Cookbook

After generating the embeddings for custom PII input and query input, an index file is created by obtaining a faiss index of the embeddings with PII. Faiss is a library that performs efficient similarity search and clustering of dense vectors. It allows the system to execute rapid similarity search of the embeddings in the index file, resulting in the retrieval of relevant embeddings according to the user's query input.

When a user queries with input, the embeddings are fetched from the index file, and a semantic search is conducted using faiss to compute the similarity score between the query embeddings and the embeddings in the index file. The results are generated based on the similarity score, with the most similar embeddings being returned first. The customized PII detection procedure is efficient and precise, utilizing advanced natural language processing methods to assess the input text and create accurate embeddings. The use of faiss for quick similarity search guarantees that the system can efficiently retrieve the relevant embeddings and provide precise outcomes to the user. Overall, the customized PII detection process offers users a powerful tool for identifying specific types of PII while safeguarding the security and confidentiality of their personal data.

The GPT-3.5 language model text-davinci-003 is specifically trained to identify and extract generic PII such as names, dates of birth, addresses, social security numbers, and other sensitive information. The model uses natural language processing techniques to analyze the input prompt and identify any instances of generic PII present in the text. If the model detects any PII, it returns the output in the format specified by the Jinja template.

PII Detection using Semi-Supervised Learning

Project Advisor: Vijay Eranti

�In today's digital world, personal information has become a valuable commodity that can be used for various nefarious purposes, including identity theft, financial fraud, and other criminal activities. Personal information that can be used to identify a person or their particular circumstances is legally known as personally identifiable information (PII). PII includes a wide range of data points, such as name, address, phone number, email address, social security number, passport number, and financial information. With just a few bits of personal information, a person's identity can be compromised, and criminals can carry out fraudulent activities.

Methodology

Summary/Conclusions

Key References

Acknowledgements

In conclusion, PII is an essential aspect of personal privacy and security in the digital age. With the increasing amount of personal data being generated and shared online, it is critical to ensure that PII is protected from unauthorized access and use. Customizable PII detection solutions offer an effective way to detect and protect sensitive personal information that may be specific to an individual's needs.

The authors are deeply indebted to Professor Vijay Eranti for his invaluable comments and assistance in the preparation of this study.

Protecting PII has become more important than ever as more personal data is generated and shared online. Many existing solutions offer the ability to detect common or generic types of PII. However, these solutions may not always be sufficient for detecting and protecting all types of PII that may be specific to an individual's needs.

Therefore, it is crucial to customize PII detection solutions to meet specific privacy and security requirements. By customizing PII detection, users can identify and protect sensitive personal information that may not be detected by standard PII detection algorithms. For example, a company may have unique requirements for detecting PII within their specific industry or geographic location. In such cases, a customizable PII detection solution would allow the company to tailor the system to meet their specific needs.

Firstly, an Optical Character Recognition (OCR) system will be utilized as a critical component of the architecture. OCR will extract text data from various types of documents, such as images, PDFs, and other file formats. OCR is essential because PII may be present in many different file formats, and extracting this information using an automated system is a crucial step in the process. The OCR system will enable the solution to extract PII data, which can then be fed into the data collection and analysis components.

�The next key component is data collection using ChatGPT for training the model. ChatGPT is a powerful tool for generating text data that contains PII. It will help to ensure that the model is trained on a wide range of data, including common and custom PII data. The data collection component is a crucial step in the process of ensuring that the model can accurately detect and analyze PII data.

The architecture will also incorporate GPT-3.5 language models, such as text-davinci-003 for generic PII detection and text-embedding-ada-002 for custom PII detection. These language models will help to detect and analyze PII within the extracted text data. The generic PII detection component will allow the solution to detect standard types of PII, such as social security numbers, addresses, and names. On the other hand, the custom PII detection component will detect PII that is specific to an individual or organization, which is not usually detected by standard PII detection algorithms.

Another key component of the architecture is the embedding of the extracted custom PII text as faiss indices in an index file. Faiss is a library for efficient similarity search and clustering of dense vectors. The embedding component will enable efficient storage and retrieval of the extracted text data. This will make it easier to process and analyze large amounts of data efficiently.

Finally, privacy and security will be a major focus of the architecture. This component will ensure that PII data is protected at all times.

Generic PII Detection: The generic PII detection process in the proposed architecture is designed to detect the most common types of PII present in the user input. The Optical Character Recognition (OCR) system is used to extract text data from the files uploaded by the user, which is then processed by Chat GPT to generate structured data in the form of a json. This json contains all the relevant information necessary for the generic PII detection process to take place.

�

The Jinja template plays a crucial role in the generic PII detection process, as it defines the format of the output that is returned by the system. This template is used to customize the output based on the specific requirements of the user. It allows the system to return the output in a standardized format, which can be easily interpreted and analyzed by the user. Overall, the generic PII detection process ensures that the system is equipped to handle the most common forms of PII, and provides a high level of security and privacy for the user's personal data.

Customized PII Detection: It is a crucial feature in the proposed architecture, as it allows users to detect specific types of PII that may not be covered by the generic PII detection process. The text-embedding-ada-002 model from OpenAI is used to generate embeddings for the custom PII input and query input. These embeddings are high-dimensional vectors that represent the semantic meaning of the text. The model is trained on a large dataset of text and is capable of generating highly accurate embeddings for a wide variety of input text.

Computer Engineering Department

KB, Jithesh (MS Software Engineering)

NS, Poojashree (MS Software Engineering)

Math, Priyanka (MS Software Engineering)

Mukka, Tharun (MS Software Engineering)

Introduction

Methodology

The architecture proposed for this project will consist of several key components, each with specific functions and roles to ensure the effective handling of PII in a secure and efficient manner.