These are chat archives for fossasia/open-event-server

4th
Jun 2018
Saurav Tiwary
@srv-twry
Jun 04 2018 03:09
Should we also setup Circle CI ? @niranjan94 @SaptakS @maxlorenz @schedutron @iamareebjamal
Bhavesh Anand
@bhaveshAn
Jun 04 2018 03:26
@srv-twry Will see on build failure.
@mariobehling I have a project presentation in my college today, so not sure will be there in the todays meeting or not.
Kreijstal
@Kreijstal
Jun 04 2018 04:07
@srv-twry how long will that take?
Saurav Tiwary
@srv-twry
Jun 04 2018 04:08
I have done it before on android but never did it on a flask rest api. A day at max IMO
But we need to make sure it's indeed a problem with Travis, not in our codebase.
Kreijstal
@Kreijstal
Jun 04 2018 04:09
make it work on your side, first. and then sure
interestingly the pull request before that didn't error
also maybe we should also add exception handlers for when the database is down
Saurav Tiwary
@srv-twry
Jun 04 2018 04:11
I went back a few commits and tested it on Travis. It fails. https://travis-ci.org/srv-twry/open-event-server/builds/387546385
This one is for a PR that was merged 3 days ago
Kreijstal
@Kreijstal
Jun 04 2018 04:24
I think it's travis side then
hopefully it gets fixed quickly
otherwise PRs will get stuck
Kreijstal
@Kreijstal
Jun 04 2018 04:32
@srv-twry fossasia already uses circle CI
but they only give 16 slots per organization and fossasia already uses all of the slots
try using circle CI on your own repo though
Saurav Tiwary
@srv-twry
Jun 04 2018 04:33
okay, so we're stuck !
Alright
Kreijstal
@Kreijstal
Jun 04 2018 04:57
it must be travis
because I'm testing old branches and they all fail
Saurav Tiwary
@srv-twry
Jun 04 2018 04:57
same here.
Kreijstal
@Kreijstal
Jun 04 2018 04:58
can you find a build that doesn't fail?
and compare?
Saurav Tiwary
@srv-twry
Jun 04 2018 05:00
@mayank8318 @schedutron guys please help. We can't move ahead without fixing the build

can you find a build that doesn't fail?

Looking for it.

Kreijstal
@Kreijstal
Jun 04 2018 05:02
im retesting builds from 1 month agao
ago
Saurav Tiwary
@srv-twry
Jun 04 2018 05:09
Kreijstal
@Kreijstal
Jun 04 2018 05:10
traivs broke
what can we do
hmm try making it pass by force
try travis on your local repo
and just commit stuff out
I think it has to do with the server
the database server
it just doesn't connect to it now
Saurabh Chaturvedi
@schedutron
Jun 04 2018 05:15
@srv-twry Indeed. I’m looking into it.
Saurav Tiwary
@srv-twry
Jun 04 2018 05:15
@schedutron Do you have experience of setting circle ci for flask or python ?
Mayank Vaidya
@mayank8318
Jun 04 2018 05:50
@srv-twry looking into it :+1:
Niranjan Rajendran
@niranjan94
Jun 04 2018 07:51
Is it possible that we are reaching a max connection limit on travis to the db server ? (Maybe travis guys changed some psql config) .... check the config of postgres on travis.
I'm not for switching to Circle CI without even knowing a proper reason for the issue.
Also, CircleCI resources are pretty limited for open source projects as compared to travis.
Areeb Jamal
@iamareebjamal
Jun 04 2018 08:16
travis-ci/travis-ci#8049
Saurav Tiwary
@srv-twry
Jun 04 2018 18:04
This is the postgres configuration on travis.
I tried changing the max connections to 1000 and then 500 but it fails.
2018-06-04 18:00:08 UTC FATAL:  could not create shared memory segment: Invalid argument
2018-06-04 18:00:08 UTC DETAIL:  Failed system call was shmget(key=5432001, size=50454528, 03600).
2018-06-04 18:00:08 UTC HINT:  This error usually means that PostgreSQL's request for a shared memory segment exceeded your kernel's SHMMAX parameter.  You can either reduce the request size or reconfigure the kernel with larger SHMMAX.  To reduce the request size (currently 50454528 bytes), reduce PostgreSQL's shared memory usage, perhaps by reducing shared_buffers or max_connections.
    If the request size is already small, it's possible that it is less than your kernel's SHMMIN parameter, in which case raising the request size or reconfiguring SHMMIN is called for.
    The PostgreSQL documentation contains more information about shared memory configuration.
fail]
Saurav Tiwary
@srv-twry
Jun 04 2018 18:11
The default max connection is 255.
@niranjan94
Saurav Tiwary
@srv-twry
Jun 04 2018 20:04

@niranjan94 @SaptakS @Kreijstal @schedutron @maxlorenz The problem seems to be in the test for POST discount code. I skipped it using

@hooks.before("Discount Codes > Event Discount Code Collection > Create Event Discount Code")
def event_discount_code_post(transaction):
    """
    POST /discount-codes
    :param transaction:
    :return:
    """
    transaction['skip'] = True

and the build passes here: https://travis-ci.org/srv-twry/open-event-server/builds/387961083

Looking into it a bit more.
Saptak Sengupta
@SaptakS
Jun 04 2018 20:21
Well connecting this two should mean that discount code is posting some huge data, but that shouldn't be the case
Saurav Tiwary
@srv-twry
Jun 04 2018 20:42

This is the expected relationship from the response .

"relationships": {
      "events": {
        "links": {
          "self": "/v1/discount-codes/4/relationships/events",
          "related": "/v1/discount-codes/4/events"
        }
      }
    },

This is what the server sends:

"relationships": {
      "event": {
        "links": {
          "self": "/v1/discount-codes/1/relationships/event",
          "related": "/v1/discount-codes/1/event"
        }
      },
      "events": {
        "links": {
          "self": "/v1/discount-codes/1/relationships/events",
          "related": "/v1/discount-codes/1/events"
        }
      }
    },

Does it look like a case of infinite recursion ? @SaptakS

I mean a discount code is related to an event, event to the discount code, discount code again to the event .....
Saptak Sengupta
@SaptakS
Jun 04 2018 20:53
nope.
Supriyanta Poddar
@supriyanta
Jun 04 2018 21:46
I know basics of Flasks
Flask*
From where to start contribute!
I mean what are the prerequisite skills do I need to have?
Saurav Tiwary
@srv-twry
Jun 04 2018 21:48

Removing the after_create_object from the discount code post resolves the problem.

    def after_create_object(self, discount, data, view_kwargs):
        if data['used_for'] == 'event' and 'events' in data:
            for event_id in data['events']:
                event = safe_query(self, Event, 'id', event_id, 'event_id')
                event.discount_code_id = discount.id

Of course I get other errors but they are due to the fact that this method should be present.

I mean what are the prerequisite skills do I need to have?

Flask, REST API, SQLAlchemy should be enough. We use Marshmallow as well but you can pick it up as you contribute

Saurabh Chaturvedi
@schedutron
Jun 04 2018 21:49
What about just passing this function for now?
*method
Saurav Tiwary
@srv-twry
Jun 04 2018 21:49
That's what I did.
Saurabh Chaturvedi
@schedutron
Jun 04 2018 21:50
So we need an alternative :+1:
Which line within this function causes the error?
last 2?
Saurav Tiwary
@srv-twry
Jun 04 2018 21:50
One workaround right now would be to just skip the test for posting the discount code
Saurabh Chaturvedi
@schedutron
Jun 04 2018 21:50
Yeah
Or maybe implement it differently?
Saurav Tiwary
@srv-twry
Jun 04 2018 21:52

Which line within this function causes the error?

We don't get an error. The issue is that when we post a discount code, it is probably posting a huge amount of data(Can't figure out why) which leads to the database connection being closed on Travis. Hence tests after this test fail due to connection error.

Saurabh Chaturvedi
@schedutron
Jun 04 2018 21:54
Yeah indeed. This wasn't happening before.
Saurabh Chaturvedi
@schedutron
Jun 04 2018 22:04
I also tried rolling back to Python 3.5 but that didn't help. Travis still errored.
Saurav Tiwary
@srv-twry
Jun 04 2018 22:23

Removing the after_create_object from the discount code post resolves the problem.

    def after_create_object(self, discount, data, view_kwargs):
        if data['used_for'] == 'event' and 'events' in data:
            for event_id in data['events']:
                event = safe_query(self, Event, 'id', event_id, 'event_id')
                event.discount_code_id = discount.id

What are we even doing in this method ? Querying an event, changing it's parameter and then doing nothing. I mean should be saved in the db, right ? @schedutron

Saurabh Chaturvedi
@schedutron
Jun 04 2018 22:24
@srv-twry I’m not sure, but I suspect the Active Record Pattern does that behind the scenes for us.
SQLAlchemy uses active record pattern. I’ll read more about it to confirm.