These are chat archives for opf/openproject

25th
Aug 2015
Vladimir K
@cronacronis
Aug 25 2015 07:35
@machisuji nope :(
kgalli
@kgalli
Aug 25 2015 07:39
@cronacronis did you gave newrelic a try?
Vladimir K
@cronacronis
Aug 25 2015 07:41
@kgalli not yet, as I mendtioned below, this is my first time ruby experience, have some diffuculties to install bundler for ruby2.1 on ubuntu 14.04
kgalli
@kgalli
Aug 25 2015 07:42
@cronacronis you do not have to do any installation to get newrelic to work. Everything is installed already.
Vladimir K
@cronacronis
Aug 25 2015 07:43
@kgalli hmm
vuser@openproject:/opt/openproject-ce$ sudo openproject-ce run bundle install
You are trying to install in deployment mode after changing
your Gemfile. Run `bundle install` elsewhere and add the
updated Gemfile.lock to version control.

If this is a development machine, remove the Gemfile freeze 
by running `bundle install --no-deployment`.

You have added to the Gemfile:
* newrelic_rpm
kgalli
@kgalli
Aug 25 2015 07:43
you can just do openproject-ce run to run ruby or bundle, railsor whatever you can think of.
Vladimir K
@cronacronis
Aug 25 2015 07:44
I have tried :( see above
should i remove freeze?
kgalli
@kgalli
Aug 25 2015 07:45
right, ok wait I will do a quick test on my local machine and paste the best way to fix this.
Vladimir K
@cronacronis
Aug 25 2015 07:46
@kgalli thx, sorry to bother you
kgalli
@kgalli
Aug 25 2015 08:09
@cronacronis you can use the no-deployment flag but you will have to install git first. After that you have to do a service openproject-ce restartto apply the changes. This is always necessary even if you just change the settings. But you already know that, right?
With settings I mean the things you change via openproject-ce config command.
Vladimir K
@cronacronis
Aug 25 2015 08:28
@kgalli working on it :)
@kgalli newrelic should report to an account there right?
@kgalli worked
Your bundle is complete!
Gems in the groups development and test were not installed.
Use `bundle show [gemname]` to see where a bundled gem is installed.
Installing newrelic_rpm 3.13.0.299
kgalli
@kgalli
Aug 25 2015 08:31
yes you have to register and yes your data will be send to new relic. So if you have problems with that you have to check their data privacy statement.
Godd to hear that it worked.
Vladimir K
@cronacronis
Aug 25 2015 08:32
@kgalli thx again for your effort. Where can i setup newrelic account in the openproject settings?
kgalli
@kgalli
Aug 25 2015 08:33
you just have to create a account and then download a new relic.yml file. This file has to be placed into your config directory.
@cronacronis let me know if you have any further questions. I hope the setup hassle will be rewarded by findind the bottleneck of your setup.
Vladimir K
@cronacronis
Aug 25 2015 10:10
on my way! :)
Vladimir K
@cronacronis
Aug 25 2015 10:33
There are deadlocks!
Vladimir K wants you to check out an error collected by New Relic.

Error raised by Api::V2::PlanningElementsController#create:
ActiveRecord::StatementInvalid: Mysql2::Error: Deadlock found when trying to get lock; try restarting transaction: UPDATE `work_packages` SET root_id = 23864, `lft` = 1, `rgt` = 2 WHERE (id = 23864)

URL: /api/v2/projects/4/planning_elements.json
Time: 12:28 pm

To see the error, click this link:
https://rpm.newrelic.com/accounts/1072740/applications/8225066/traced_errors/7d8144-fe9c6d45-4b13-11e5-b82f-f8bc12425d4c
@kgalli see above
Vladimir K
@cronacronis
Aug 25 2015 10:50
Through tread profiler: most of the CPU time spent in the activerecord. Nor CPU nor disk IO was maxed during the tests
ORM is a huge overhead :(
16% /opt/openproject-ce/vendor/bundle/ruby/2.1.0/gems/activerecord-3.2.22/lib/active_record/persistence.rb.update_attributes :215
I don't get why there are so many UPDATE calls, whether it should be just INSERT
Deadlocks are also mostly on UPDATE
UPDATE `work_packages` SET root_id = 23864, `lft` = 1, `rgt` = 2 WHERE (id = 23864)
Jan Sandbrink
@NobodysNightmare
Aug 25 2015 10:54
The update calls also got my attention when you posted that error...
you are only creating work packages, right?
Probably related to awesome-nested-set... (lft and rgt look like attributes of a nested set...)
Vladimir K
@cronacronis
Aug 25 2015 10:55
Benchmarks
The spikes -> runs
@NobodysNightmare right
Jan Sandbrink
@NobodysNightmare
Aug 25 2015 10:56
@cronacronis I believe new relic also has a view that shows time/load split by ruby, database etc....
would also be interesting to see...
Vladimir K
@cronacronis
Aug 25 2015 10:56
I had similar issue with C++ ORM, when i changed object, the ORM was rewriting nested arrays and objects.
exactly with UPDATE statements
even if no changes to them was done
it was just update with exact same values, just on re-save
split
this is current state, after run already. No activity was anymore
Jan Sandbrink
@NobodysNightmare
Aug 25 2015 11:15
I had another graph in mind, but this is fine xD
At least we can clearly see that ruby takes the largest amount of exec time (probably not a surprise)
Vladimir K
@cronacronis
Aug 25 2015 15:00
@NobodysNightmare @kgalli guys, should I create issue related to deadlocks?
Jan Sandbrink
@NobodysNightmare
Aug 25 2015 15:01

If you encounter them more than once (which appears to be the case): yes...

Still I can't see how the workload of adding work packages can create those deadlocks. Please also report relevant changes to your setup in the ticket (e.g. are you using TokuDB or still vanilla InnoDB?)

Vladimir K
@cronacronis
Aug 25 2015 15:02
Vanilla still
Jan Sandbrink
@NobodysNightmare
Aug 25 2015 15:03
okay... that is a plus (otherwise I would be inclined to point fingers in other directions ^^)... minus is that I have fewer directions to point my fingers at ^^