Admin User     Feb,08 2017

Data Source Rejected Establishment of Connection, Message from Server: "Too Many Connections" Error with mySQL and RAILO 4.2.1

Recently updated to RAILO latest patch 4.2.1, though I was using its beta version and was working fine in development environment. Reason it was working well because I was only visiting my website so there were no connection issue with MySQL. But when I go released version so hoping to update it on our production server as well. Before going to production tried to load test using JMeter and working fine initially with low traffic but as I increased higher number of users (basically thread) start getting Data source rejected establishment of connection, message from server: "Too many connections" issue. Though production site was working fine with previous version 4.1.x without any kind of such issue so most probably RAILO wasn't releasing connection or not reusing it. 

I have performed various load test on same site with two RAILO version 4.1.x and 4.2.1 and below are result.

Environment:

OS: Window 7
Database : MySQL 5.6 with maximum connection limit 151
Load Testing Tool : JMeter
 

RAILO VersionNo. differentRequestsNo of Users ( In 20 Seconds)Comments
4.1.2.00543 1OK
  10OK
  20OK
  50OK
  75OK
  100OK
  200JMeter failed due to outofmemory but
RAILO request fulfilled
 Set "Connection Limits" property to - inf - and "Connection Timeout (In Minutes)" to 1.
4.2.1.000431OK
  10OK
  205% requests fail for 4 page out of 43. 
Error Type: 
java.lang.IllegalThreadStateException
  50Got Error: 
Data source rejected establishment of connection, message from server: "Too many connections"
Almost 50% request failed.
  30Overall .70% error.
Error Type: 
java.lang.IllegalThreadStateException
  40
Overall .06% error.
Error Type: 
java.lang.IllegalThreadStateException
 Change "Connection Timeout (In Minutes)" to 0
  50
Got Error: 
Data source rejected establishment of connection, message from server: "Too many connections"
Almost 60% request failed.
Set "Connection Limits" property to 100 and "Connection Timeout (In Minutes)" to 0.
  50
Overall .33% error.
Error Type: 
java.lang.IllegalThreadStateException
request (:3) has run into a timeout (50 seconds) and has been stopped.
Set "Connection Limits" property to 100 and "Connection Timeout (In Minutes)" to 1.
  50
Overall .33% error.
Error Type: 
java.lang.IllegalThreadStateException
Set "Connection Limits" property to 200 and "Connection Timeout (In Minutes)" to 1. (Max allowed connection in MySQL is 150)
  50
Overall 13% error.
Error Type: 
java.lang.IllegalThreadStateException
request (:3) has run into a timeout (50 seconds) and has been stopped.
Data source rejected establishment of connection, message from server: "Too many connections"
Stuck Tomcat Service at 25% Usage Level and no more request processed.
Set "Connection Limits" property to 200 and "Connection Timeout (In Minutes)" to 0. (Max allowed connection in MySQL is 150)
  50
Overall .14% error.
Error Type: 
java.lang.IllegalThreadStateException

Upto 200 users in 20 seconds of ramp up work fine with RAILO though I run out of memory for JMeter for version 4.1.x. In case of 4.2.1 I started getting connection limit error within 50 simultaneous users. I have tried with various combination of settings available in server administrator as mention in above table. Only solution that work is if we set "Connection Limits" property in datasource lesser than MySQL allowing. 

PS: I am assuming here only my website connecting to MySQL server and all test are based on this assumption. If you have multiple database hosted on your server and then you may want to increase limit variable in MySQL and change setting appropriately. 

I am going to submit bug on RAILO and hope we get some quick patch for this otherwise this is not a production ready.