httpd threads max out

[FuhuXia]

Some recent change brought in this bug. We observed that after a fresh Apache start, the httpd process count will keep increasing, and max out after a few hours. At this point any new api call such as /api/3/action/package_search will cause 500 error with apache error:
Error - <class 'thread.error'>: can't start new thread

Here is the detailed error info.

...
Module ckanext.googleanalytics.controller:65 in __init__
<<              t = AnalyticsPostThread(self.analytics_queue)
                   t.setDaemon(True)
                   t.start()

           def _post_analytics(
>>  t.start()
Module threading:474 in start
<<          _active_limbo_lock.release()
               try:
                   _start_new_thread(self.__bootstrap, ())
               except Exception:
                   with _active_limbo_lock:
>>  _start_new_thread(self.__bootstrap, ())
error: can't start new thread

[acorbi]

Run into a similar situation while runing a script that imports a massive number of datasets into ckan using /api/3/action/package_create and /api/3/action/package_update API methods. Here the error log ( convert to html for proper readability): http://pastebin.com/ZWUmCxTe

[rossjones]

Probably the code at https://github.com/ckan/ckanext-googleanalytics/blob/master/ckanext/googleanalytics/controller.py#L60-L65 creating some threads each time a controller is created.

[CarlQLange]

What's the story with this issue? Is it still a problem? Will it affect all users?

[rossjones]

I don't use this extension, t it must be biting @maxious in production?

[nigelbabu]

I just bit me in production. @maxious, I'm going to back out your pull request. I would highly prefer master actually work.

I'll move it to a branch called unstable and you can fix it in there. Does that sound okay?

[maxious]

Argh! Sorry guys! This is only the second time I've worked with threads in Python and it seemed to be working right. I can see in the debugger it's loading more and more threads every call :/

Anyway, I've moved the pool setup to the plugin and it only gets called once on CKAN startup.

I've got this in a branch but it can't merge to master because of the revert commit :/ https://github.com/datagovau/ckanext-googleanalytics/tree/api-analytics-event-tracking

[nigelbabu]

@maxious I'll try to revert the revert and merge your branch in on an unstable branch and see how it goes. Don't worry about the pull request not working. I'll take care of it to make sure it all works.

[nigelbabu]

I've setup a new pull request with @maxious fix. Please look at #13