InnoDB bulk insert

Project goal

Implement MDEV-515, which is about bulk-inserting a huge amount of data into InnoDB.

Project abstract

This project’s objective is to make bulk insertions in tables created in InnoDB databases faster.

As Marko Mäkelä noted, the work will consist of the following tasks:

• changes to table locking, to exclusively lock the table
• changes to MVCC so that the table will read as empty to transactions that started before the operation was committed
• logic changes so that ha_innobase::write_row() will provide the data that normally is done by row_merge_read_clustered_index() during table-rebuilding ALTER TABLE
• creating the indexes by using the BtrBulk code

Timeline

Before 20th of April:

• get familiar with InnoDB codebase

April 21 -- April 28:

• reach an agreement with InnoDB developers about new undo log record type to represent ‘truncate on rollback’

April 29 -- May 6:

• changes to table locking, to exclusively lock the table
• changes to MVCC so that the table will read as empty to transactions that started before the operation was committed

May 7 -- May 21:

• testing

May 22 -- June 4:

• logic changes so that ha_innobase::write_row() will provide the data that normally is done by row_merge_read_clustered_index() during table-rebuilding ALTER TABLE
• creating the indexes by using the BtrBulk code

June 5 -- June 19:

• testing

June 20 -- July 4:

• implementation of `truncation on rollback’

July 5 -- July 26:

• final testing

About me

My name is Yury, I’m a third year student (undergraduate) in Far Eastern Federal University at “Applied Mathematics and Computer Science”.

I have been programming since the age, like, 10 (if my Flash experience is counted, else the starting age is 14). My first steps in this way were in Adobe Flash and ActionScript, then I have become more fluent in other languages, such as JavaScript, Python, Object Pascal, Java, PHP, VBA and a lot of others.

Now I prefer C/C++, Python and JavaScript in my projects. I don’t have a huge open-source contribution background, but it was bug fixes and feature additions in TUIJam (https://github.com/cfangmeier/tuijam), a Google Play Music client for terminal. I was writing a database engine as my semester project too (https://github.com/t1meshift/YeltsinDB)

My nickname is t1meshift (email: sh1ftr@protonmail.ch), it’s the same in the vast majority of places.

GitHub: https://github.com/t1meshift