These are chat archives for mirumee/saleor

22nd
Oct 2018
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 08:09
Anybody able to create issues on GitHub? I'm getting 405 (Not Allowed) errors :/
Filip Kucharczyk
@Pacu2
Oct 22 2018 08:10
Yup, not working for us as well
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 08:11
I just managed after the 5th attempt, but the webhooks aren't working as well ugh
...and the issue is now deleted and gone.
Guess we will have to wait a few hours :/
That may explain why some issues created during this night were returning 404 errors
M Abdullah Abid
@wulaaf
Oct 22 2018 08:21
Yup, my fork just disappeared.
GitHub acting funny
Anyone here using saleor in production?
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 08:22
They are having storage issues.
I'm using saleor in production
M Abdullah Abid
@wulaaf
Oct 22 2018 08:22
Are you using docker?
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 08:22
No
M Abdullah Abid
@wulaaf
Oct 22 2018 08:22
So you have all the services such as PostgreSQL, Redis, Celery etc. setup independently or on the same server?
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 08:24
They don't run in the same server instance, they are independent. I directly use RDS from AWS for Postgres and celery is running along side with nginx
M Abdullah Abid
@wulaaf
Oct 22 2018 08:25
What about elasticsearch?
and Redis*
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 08:25
Not need
M Abdullah Abid
@wulaaf
Oct 22 2018 08:25
Okay okay
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 08:25
Redis is a ElasticCache instance, but it's only used for sessions, so... It could easily be moved to a main instance
M Abdullah Abid
@wulaaf
Oct 22 2018 08:25
Can you share your prod website in private? I’d like to see its performance if thats okay with you.
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 08:26
(Redis is important to allow scaling without losing sessions)
M Abdullah Abid
@wulaaf
Oct 22 2018 08:26
Another thing, I believe this was being discussed earlier. How do you keep track of changes to saleor and your storefront in production?
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 08:26
Sure, just don't create fake orders and all of course, it's a production site https://big-light.fr/
M Abdullah Abid
@wulaaf
Oct 22 2018 08:28
Ofcourse not
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 08:31
I don't plan into updating my fork for a good while, for compatibility and time issues. I monthly update the (private) fork through rebasing and keep track of everything done in the PR and tags, and whenever there is an important patch made on Saleor, I patch the fork ASAP with it (you can generate patch files of pull requests in GitHub).
And some of the features added to the private fork are merged to Saleor from my pull requests. So most of the time there is not that much to sync.
M Abdullah Abid
@wulaaf
Oct 22 2018 08:32
Hmm. Okay. I gotta brush up my skills on forking and patching. Any good resources ont that?
Thanks for the info though
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 08:32
:+1:
Your fork should probably be coming back I guess https://blog.github.com/2018-10-21-october21-incident-report/
M Abdullah Abid
@wulaaf
Oct 22 2018 08:34
I forked it five times
And it disappeared everytime
After a couple of seconds
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 08:35
You need to wait... They estimate about 1h30 (https://status.github.com/messages)
M Abdullah Abid
@wulaaf
Oct 22 2018 10:53
Is there a guide for deploying it to production?
The deployment part on the docs section says just compile into an image. That’s it.
Stephen Moloney
@stephenmoloney
Oct 22 2018 10:54
@NyanKiyoshi That is some excellent performance on biglight... What is the spec of your host ?
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 10:55
@stephenmoloney a simple EC2 small instance (1GB and 1vCore iirc)
EC2 always has good performances, even small instances
Stephen Moloney
@stephenmoloney
Oct 22 2018 10:55
i see. no aws s3 ?
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 10:56
S3 for images
Stephen Moloney
@stephenmoloney
Oct 22 2018 10:56
ok, so static files serving from disk and memory and /app/media from s3.
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 10:56
That's right
One of the reasons to use Saleor is that :wink: It's fast.
Stephen Moloney
@stephenmoloney
Oct 22 2018 10:58
you know images on demand for versatile image
CREATE_IMAGES_ON_DEMAND
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 10:58
Yeah? Disabled on production
Otherwise it's slow and spams the S3 bucket
Stephen Moloney
@stephenmoloney
Oct 22 2018 10:59
Yea, sure, I have tested it in off mode
and I expect a big performance boost when turned off - is that fair to expect ?
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 11:00
Yeah, of course. Otherwise, even if we decide to serve from local disk, it's slow, because it has to check whether the file exists or not, generating a lot of system and IO calls
Worst when it's a S3 since it's generating TCP requests
Stephen Moloney
@stephenmoloney
Oct 22 2018 11:00
Right. But where I am slighly confused is this
what happens when 'create_images_on_demand': True vs 'create_images_on_demand': False
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 11:01
Well it's only good for development usage. When it's on, it creates the images whenever the user is requesting them
Stephen Moloney
@stephenmoloney
Oct 22 2018 11:01
When True - request for a thumbnail and thumnail does not exist so create it ?
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 11:01
If it's false, we generate them when we add them
Yes
Stephen Moloney
@stephenmoloney
Oct 22 2018 11:01
Ah, ok, so when false, generate ahead of time ...
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 11:01
Yeah
Stephen Moloney
@stephenmoloney
Oct 22 2018 11:01
Is S3 supported by a CDN - no ?
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 11:02
Cloudfront
Stephen Moloney
@stephenmoloney
Oct 22 2018 11:02
by default ?
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 11:02
Not by default, we need to set it up
Stephen Moloney
@stephenmoloney
Oct 22 2018 11:02
Ok, because I am thinking that perhaps serving from local SSD should also yield good performance
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 11:03
Yeah, with a CREATE_IMAGES_ON_DEMAND off of course. And with a custom domain, otherwise the web browser will limit the simultaneous requests (3 or 5 simultaneous request per domain, I think? I don't remember)
If you look at the network tab of big-light, it requests: static1.big-light.fr, static4.big-light.fr and big-light.fr
Stephen Moloney
@stephenmoloney
Oct 22 2018 11:04
I see, I don't think I'm going to use S3 myself but I'd like to support it for /app/media in the helm charts regardless
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 11:04
So we limit the hanging
Okay
Stephen Moloney
@stephenmoloney
Oct 22 2018 11:05
I'll try it out and see though.
Another thing, serving /app/static from S3 is a bit uneccessary - what do you think ?
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 11:07
It's better to use a S3 to avoid having too much bandwidth going out of the EC2 instance(s) and having too much file requests on the EC2 instances, it's a question of "Do I want to prioritize the Python code in EC2?" if you don't have too much requests, it's fine to not use S3
I only serve the CSS and JS from EC2
The images (e.g. placeholder images) are S3 (static1)
Stephen Moloney
@stephenmoloney
Oct 22 2018 11:09
yea, using a CDN for the media content seems to make sense. I might use S3 for that yet but I'll start with the ssd and see how it performs.
thanks
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 11:09
Yeah, you want to limit the network charge. IO charge has lower weight
Plus, also, it should be the web server's job to serve the static file (nginx, Apache), not django or uwsgi
Otherwise you create a bad amount of latency and the server will have more a hard time to handle requests quickly
Stephen Moloney
@stephenmoloney
Oct 22 2018 11:12
Plus, also, it should be the web server's job to serve the static file (nginx, Apache), not django or uwsgi - i agree with this.
IS there not some defaults in saleor to serve the static content with a webserver ?
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 11:13
Does GitHub issues seems to work again? I don't dear trying, seeing the mess and webhook spam it made.
Well, it's up to you to configure correctly the web server to serve /app/static. Django doesn't handle /app/static when DEBUG=False. But Saleor handles /app/media whatever DEBUG is
Stephen Moloney
@stephenmoloney
Oct 22 2018 11:18
I haven't tested DEBUG=False yet either... getting there now pretty much. Hmm... I'll probably have to take a closer look at uwsgi.ini and see. I just think that by default, there should be an option to have reasonable performance in production without S3.
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 11:19
I think the docker file uses uwsgi directly to serve? uwsgi can be used as a web server
M Abdullah Abid
@wulaaf
Oct 22 2018 11:19
I’m stuck in the same boat
Docker + Production = ?
Stephen Moloney
@stephenmoloney
Oct 22 2018 11:23
I think as a general statement Docker + Production is ok. Just how you set it up. Kubernetes uses containers which typically are docker containers (though not necessarily).
M Abdullah Abid
@wulaaf
Oct 22 2018 11:25
That I’m well aware of. It’s just that configuring saleor with docker in production is not really properly documented. I’ve tried various configurations and have found workarounds but they’re less than optimal.
Stephen Moloney
@stephenmoloney
Oct 22 2018 11:28
Yea, this could be a nice production alternative to static content serving. https://github.com/evansd/whitenoise
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 11:30
@stephenmoloney well your issue is returning 404 :wink:
Can't read it
Stephen Moloney
@stephenmoloney
Oct 22 2018 11:30
or alternatively, adding some caching rules for uwsgi
M Abdullah Abid
@wulaaf
Oct 22 2018 11:30
My fork just returned, finally
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 11:30
:tada:
My issues have cloned themselves
What a mess
M Abdullah Abid
@wulaaf
Oct 22 2018 11:33
Hahaha
It disappeared again lol
M Abdullah Abid
@wulaaf
Oct 22 2018 12:36
@NyanKiyoshi, are you using nginx with uwsgi in prod?
NyanKiyoshi
@NyanKiyoshi
Oct 22 2018 13:03
@wulaaf yes
AJ
@ajostergaard
Oct 22 2018 19:26

I'll probably have to take a closer look at uwsgi.ini and see. I just think that by default, there should be an option to have reasonable performance in production without S3.

@stephenmoloney from the uWSGI docs: "serving static files in uWSGI instances is possible, but not as good as using a dedicated full-featured web server." hence if you want that reasonable performance pop nginx in front of uWSGI.

victor felder
@vhf
Oct 22 2018 19:30

If anyone is interested, using nginx to serve statics and media is basically this:

upstream django {
    server saleor:8000;
}
server {
    listen      80 default_server;
    server_name _;
    location /media  {
        alias /media;
    }
    location /static {
        alias /static;
    }
    location / {
        uwsgi_pass  django;
        include     uwsgi_params;
    }
}

Here saleor is the name I use in the compose file, and I simply mount /media and /static

(Using the official nginx docker image.)
M Abdullah Abid
@wulaaf
Oct 22 2018 19:38
web_1 | bind(): Permission denied [core/socket.c line 769 has anyone come across this?
victor felder
@vhf
Oct 22 2018 19:38
Nope, but what port are you trying to use?
M Abdullah Abid
@wulaaf
Oct 22 2018 19:38
@vhf thanks victor I’m gonna try this
80:80
victor felder
@vhf
Oct 22 2018 19:39
That might be why. Only root can bind to port<1024
M Abdullah Abid
@wulaaf
Oct 22 2018 19:39
Hmm
Trying to run uwsgi with 80 is not a good idea then?
victor felder
@vhf
Oct 22 2018 19:40
Unless you're running it as root, no. And this wouldn't be a good idea anyway.
M Abdullah Abid
@wulaaf
Oct 22 2018 19:40
Then running nginx in a container and hooking up uwsgi with a different port setting to it would be the way to do it?
victor felder
@vhf
Oct 22 2018 19:41
That's one way to do it.
You could run nginx on the host system instead of a container as well.
I much prefer having everything in containers though.
AJ
@ajostergaard
Oct 22 2018 19:45
Or separate containers and sockets, as opposed to ports, between them.
M Abdullah Abid
@wulaaf
Oct 22 2018 20:16
I prefer using containers as well. So maybe like a backend and frontend application and web server one for uwsgi and one for nginx and then have them serve
Has anyone successfully implemented their own version of the docker compose file with nginx for saleor?
victor felder
@vhf
Oct 22 2018 20:20
yep
Lemme see if I can gist what I did
M Abdullah Abid
@wulaaf
Oct 22 2018 20:21
Awesome!
after quite some passwords and API cleaning
AJ
@ajostergaard
Oct 22 2018 20:35
@vhf nice setup! :)
victor felder
@vhf
Oct 22 2018 20:50
Thanks @ajostergaard ! I'm slowly getting a hang of this thing, I'm impressed by Saleor so far, I love it!
M Abdullah Abid
@wulaaf
Oct 22 2018 20:52
Appreciate it bud!
And yeah it is nice setup!