MySQL Bugs: #1542: When will Mysql have parallel query and partitioning

Bug #1542	When will Mysql have parallel query and partitioning
Submitted:	13 Oct 2003 11:10	Modified:	19 Oct 2005 13:18
Reporter:	Udi Karni	Email Updates:
Status:	Verified	Impact on me:	None
Category:	MySQL Server	Severity:	S4 (Feature request)
Version:	4.0.15, 5.5	OS:	Any (windows)
Assigned to:		CPU Architecture:	Any

Description:
We run a large data warehouse on Oracle and Windows. We have billions of rows so performance is everything. Two of the most compelling reasons to use Oracle were full data partitioning support (you only read the slice you need) and the parallel query feature (multiple query slaves work on your query concurrently). We use 8-cpu Compaq servers so every query slave gets a dedicated CPU so to speak and things fly.

We are very impressed with Mysql and would consider switching to it if we could have these 2 features, plus multi-cpu support (this is probably more of a windows thing really). They make all the difference in the world when it comes to run time.

Are there plans to include them in Mysql? timetable?

How to repeat:
n/a

Hi Udi,

we use MySQL with billions of records as well.

MySQL works wonderful using Linux. Parallel Queries should be possible. Usually every query is run on one processor. So multiple queries are run on multiple processors.

Even if you only run one query, mysql sometimes uses 2 processors when possible.

But you should have a look on tuning MySQL. This should be not as difficult as on most other databases. And you should test all queries with real data and do a lot of FORCE INDEX and EXPLAIN if necessary. Have a look at the wonderful documentation. In any case you should use MySQL > 4.0.15 as optimization is improved there!

I think it's running smoothly and secure.

I use Dualprocessor Servers and InnoDb table handler.

Hope this helped a bit, it's just a little experience report. Good Luck!

Thomas

MySQL currently never utilizes more than one CPU to run one query.

I really need this feature as well and I think it would be relatively easy to implement using MySQL Proxy. An outline for a solution can be seen at http://forums.mysql.com/read.php?146,161176,163523#msg-163523

For PostgreSQL there already is a tool for doing this called Pgpool-II. An explanation of how they rewrite the SQL queries in order to execute them in parallel can be seen at:
http://pgpool.projects.postgresql.org/#internal

I have a contact, who is interested in implementing this for approximately 20000$, I can come up with a fourth of it. If anyone is interested to help finance or implement this, please email me at mrdavidandersen(at)gmail.com.