These are chat archives for fossasia/open-event-server

26th
May 2017
Aseem Shrey
@LuD1161
May 26 2017 03:19

Even after reinstalling the whole setup the project fails to pass all the test, on the first run itself

Some of the errors are :

======================================================================
FAIL: test_add_user_to_trash (functionality.test_admin_trash.TestAdminTrash)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nonroot/open-event-orga-server/tests/unittests/functionality/test_admin_trash.py", line 43, in test_add_user_to_trash
    self.assertTrue(user.deleted_at is not None)
AssertionError: False is not true

======================================================================
FAIL: test_add_microlocation_to_db (functionality.test_microlocation.TestMicrolocationApi)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nonroot/open-event-orga-server/tests/unittests/functionality/test_microlocation.py", line 23, in test_add_microlocation_to_db
    self.assertEqual(microlocation.id, 1)
AssertionError: 3 != 1

======================================================================
FAIL: test_restore_event_from_trash (functionality.test_restore_trash.TestAdminTrash)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nonroot/open-event-orga-server/tests/unittests/functionality/test_restore_trash.py", line 28, in test_restore_event_from_trash
    self.assertEqual(event.deleted_at, None)
AssertionError: datetime.datetime(2017, 5, 26, 8, 35, 33, 350091) != None

======================================================================
FAIL: test_restore_session_from_trash (functionality.test_restore_trash.TestAdminTrash)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nonroot/open-event-orga-server/tests/unittests/functionality/test_restore_trash.py", line 54, in test_restore_session_from_trash
    self.assertEqual(session.deleted_at, None)
AssertionError: datetime.datetime(2017, 5, 26, 8, 35, 33, 680137) != None

======================================================================
FAIL: test_restore_user_from_trash (functionality.test_restore_trash.TestAdminTrash)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nonroot/open-event-orga-server/tests/unittests/functionality/test_restore_trash.py", line 38, in test_restore_user_from_trash
    self.assertEqual(user.deleted_at, None)
AssertionError: datetime.datetime(2017, 5, 26, 8, 35, 33, 980617) != None

======================================================================
FAIL: test_add_session_to_db (functionality.test_session.TestSessionApi)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nonroot/open-event-orga-server/tests/unittests/functionality/test_session.py", line 19, in test_add_session_to_db
    self.assertEqual(session.id, 1)
AssertionError: 7 != 1

----------------------------------------------------------------------
Ran 286 tests in 175.225s

FAILED (failures=15, errors=297)

OS Configuration ( output of uname -a )
Linux Dell-PC 4.6.0-kali1-amd64 #1 SMP Debian 4.6.4-1kali1 (2016-07-21) x86_64 GNU/Linux

Earlier I was running as root but now I created a new nonroot user and tried reinstalling but it fails again.

Some of the consequences of these fails are that data from events get lost at random , which leads to a lot of time being wasted on recreating them.

Any suggestions as to how to solve this problem ?

Avi Aryan
@aviaryan
May 26 2017 03:32
@enigmaeth I tried right now and it fails for me too. This used to work a few months ago. Was there some change in the related areas recently ? As @hongquan said, celery stuff runs in a different process and lacks flask global request variable. To solve this, I had implemented request_context_task but it seems now that the session is being propogated to celery in some other way too. Hence the error User .. is already present in this session
Afroz Ahamad
@enigmaeth
May 26 2017 05:30
@aviaryan There is also another error here : https://github.com/fossasia/open-event-orga-server/blob/development/app/api/helpers/import_helpers.py#L113
Traceback (most recent call last):
  File "/home/enigmaeth/FOSSASIA/open-event-orga-server/app/api/helpers/tasks.py", line 29, in import_event_task
    result = import_event_task_base(self, file, source_type, creator_id)
  File "/home/enigmaeth/FOSSASIA/open-event-orga-server/app/api/imports.py", line 59, in import_event_task_base
    new_event = import_event_json(task_handle, file_path)
  File "/home/enigmaeth/FOSSASIA/open-event-orga-server/app/api/helpers/import_helpers.py", line 347, in import_event_json
    raise make_error('event', er=e)
  File "/home/enigmaeth/FOSSASIA/open-event-orga-server/app/api/helpers/import_helpers.py", line 114, in make_error
    if er.code is None:
AttributeError: 'InvalidRequestError' object has no attribute 'code'
Afroz Ahamad
@enigmaeth
May 26 2017 06:35
This was the error that showed on the fronted as Internal Server Error. When I commented the above line of code, the user exists error occurs.
Afroz Ahamad
@enigmaeth
May 26 2017 07:01

Normally, in multithread situation, session.flush() is enough. Because 2 threads can use the same SQLAlchemy session (I'm not sure yet, need to read more). If thread 1 flush session, all change can be propagate to thread 2. Thread 2 can see User instance. But in case of Celery, the task is executed in different process. The session is not shared. We have to commit to database (session.commit()).

@hongquan The error is not that user is not found, but that the User .. is already present in this session.

Avi Aryan
@aviaryan
May 26 2017 07:01
we can add hasattr(er, 'code') there to check if that attribute exists. More importantly, did you try importing other events. What about creating a simple new event, exporting it and then importing it back again. If everything fails, we can be sure that it is some issue with zip import. It's surprising then that the tests are passing. Maybe try pentabarf import next. There is a fossasia/open-event-orga-server@b0242e5 with changes in celery and context by @shubham-padia . Maybe he will know what's happening.
Afroz Ahamad
@enigmaeth
May 26 2017 07:08
@aviaryan pentabarf import works!!
Though I can't export into zip.
Shubham Padia
@shubham-padia
May 26 2017 07:12
@aviaryan the tests are using a different test_request-context, https://github.com/fossasia/open-event-orga-server/commit/b0242e57bc4b77db49bf93b631ab68b87c7e0968#diff-1f9f743421417fb2794c88447083a031R166 makes sure that the right context is passed to celery when in testing.
Avi Aryan
@aviaryan
May 26 2017 07:49
@shubham-padia but import/export tests used to work even before that.
@enigmaeth same type of error with export to zip ?
Shubham Padia
@shubham-padia
May 26 2017 08:22
@aviaryan they still work, don't they , these are error logs from the actual import of the event, or am i missing something?
Anyways this discussion shouldn't be happening here, @enigmaeth raise issues on github for this kind of stuff, so it becomes easy for everyone to follow.
Afroz Ahamad
@enigmaeth
May 26 2017 09:15
There is an issue for the import error. I was making sure if there is something wrong with db on my system.
Will open issues.
Afroz Ahamad
@enigmaeth
May 26 2017 13:25
@shubham-padia The import/export for zip is not working. And yes, those are the logs from actual import of events(from zip file).
Afroz Ahamad
@enigmaeth
May 26 2017 15:43

@shubham-padia
The error on importing the event zip is this :

File "/home/enigmaeth/FOSSASIA/open-event-orga-server/app/api/helpers/tasks.py", line 29, in import_event_task
result = import_event_task_base(self, file, source_type, creator_id)
File "/home/enigmaeth/FOSSASIA/open-event-orga-server/app/api/imports.py", line 59, in import_event_task_base
new_event = import_event_json(task_handle, file_path)
File "/home/enigmaeth/FOSSASIA/open-event-orga-server/app/api/helpers/import_helpers.py", line 344, in import_event_json
raise make_error('event', er=e)
InvalidRequestError: Can't attach instance <User at 0x7fcbf98e66d0>; another instance with key (<class 'app.models.user.User'>, (1,)) is already present in this session.

How should we pass the context to the celery task here?

Nguyễn Hồng Quân
@hongquan
May 26 2017 15:44

The error is not that user is not found, but that the User .. is already present in this session

Ah, I read wrong. Hehe.