These are chat archives for getredash/redash

4th
Jan 2016
Wesley
@WesleyBatista
Jan 04 2016 18:21
This message was deleted
Hello @akariv. I am having some issues with Redash's scheduler. Hope that you can help me with this.
re dash   Passengers Info - Hotwheels Ride.png
I think the problem is that the scheduler isn't watching the outdated queries and pushing into the queue.
Wesley
@WesleyBatista
Jan 04 2016 18:32
I would like someway force the refresh_queries task to see the process of queuing task happening
Arik Fraimovich
@arikfr
Jan 04 2016 18:33
@WesleyBatista is it only this query that doesn't refresh or all of the scheduled ones?
Wesley
@WesleyBatista
Jan 04 2016 18:39
all queries which have Refresh Schedule set :/
Arik Fraimovich
@arikfr
Jan 04 2016 18:42
@WesleyBatista did anything change recently on the server? Does the celery beat process running?
(it's the celery worker started with --beat)
Wesley
@WesleyBatista
Jan 04 2016 18:46
[program:redash_celery]
command=/opt/redash/current/bin/run celery worker --app=redash.worker --beat -Qqueries,celery,scheduled_queries
process_name=redash_celery
numprocs=1
priority=999
autostart=true
autorestart=true
stdout_logfile=/opt/redash/logs/celery.log
stderr_logfile=/opt/redash/logs/celery_error.log
we didn't changed the configuration
The last modification in the code was 27 days ago.
And wasn't anything related to the core...
Arik Fraimovich
@arikfr
Jan 04 2016 19:20
Can you check if the process actually running? And if it does, check if there are related error in the log? (/opt/redash/logs /celery_error.log)
Wesley
@WesleyBatista
Jan 04 2016 19:28
oh, right, I forgot to mention we are having "Permission denied" error related to celerybeat-schedule
One thing which I didn't saw was the DBAccessError in the following traceback:
[2016-01-04 15:57:24,871: ERROR/Beat] Removing corrupted schedule file 'celerybeat-schedule': DBAccessError(13, 'Permission denied')
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/celery/beat.py", line 367, in setup_schedule
    writeback=True)
  File "/usr/lib/python2.7/shelve.py", line 239, in open
    return DbfilenameShelf(filename, flag, protocol, writeback)
  File "/usr/lib/python2.7/shelve.py", line 223, in __init__
    Shelf.__init__(self, anydbm.open(filename, flag), protocol, writeback)
  File "/usr/lib/python2.7/anydbm.py", line 85, in open
    return mod.open(file, flag, mode)
  File "/usr/lib/python2.7/dbhash.py", line 18, in open
    return bsddb.hashopen(file, flag, mode)
  File "/usr/lib/python2.7/bsddb/__init__.py", line 364, in hashopen
    d.open(file, db.DB_HASH, flags, mode)
DBAccessError: (13, 'Permission denied')
[2016-01-04 15:57:24,875: ERROR/Beat] Process Beat
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/billiard/process.py", line 292, in _bootstrap
    self.run()
  File "/usr/local/lib/python2.7/dist-packages/celery/beat.py", line 527, in run
    self.service.start(embedded_process=True)
  File "/usr/local/lib/python2.7/dist-packages/celery/beat.py", line 453, in start
    humanize_seconds(self.scheduler.max_interval))
  File "/usr/local/lib/python2.7/dist-packages/kombu/utils/__init__.py", line 322, in __get__
    value = obj.__dict__[self.__name__] = self.__get(obj)
    value = obj.__dict__[self.__name__] = self.__get(obj)
  File "/usr/local/lib/python2.7/dist-packages/celery/beat.py", line 491, in scheduler
    return self.get_scheduler()
  File "/usr/local/lib/python2.7/dist-packages/celery/beat.py", line 486, in get_scheduler
    lazy=lazy)
  File "/usr/local/lib/python2.7/dist-packages/celery/utils/imports.py", line 53, in instantiate
    return symbol_by_name(name)(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/celery/beat.py", line 357, in __init__
    Scheduler.__init__(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/celery/beat.py", line 184, in __init__
    self.setup_schedule()
  File "/usr/local/lib/python2.7/dist-packages/celery/beat.py", line 371, in setup_schedule
    self._remove_db()
  File "/usr/local/lib/python2.7/dist-packages/celery/beat.py", line 362, in _remove_db
    os.remove(self.schedule_filename + suffix)
OSError: [Errno 13] Permission denied: 'celerybeat-schedule'
[2016-01-04 15:57:25,883: WARNING/MainProcess] celery@redash ready.
Arik Fraimovich
@arikfr
Jan 04 2016 19:36
that's weird
@WesleyBatista I think the file it uses as the db for the scheduler is /opt/redash/current/celerybeat-schedule make sure it has correct permissions (should be owned by the user redash). Or just delete it and restart the workers to recreate it.
Wesley
@WesleyBatista
Jan 04 2016 20:13
Thanks @arikfr! It was, as you pointed, a permissioning error. To implement redash here we needed to change some ownerships.
I think that after we fix this issue Celery will work fine
Arik Fraimovich
@arikfr
Jan 04 2016 20:21
:+1:
Wesley
@WesleyBatista
Jan 04 2016 21:57
I found out that I can change the location of celerybeat-schedule setting the -s on the celery worker command.
The celery worker command now is /opt/redash/current/bin/run celery worker --app=redash.worker --beat -Qqueries,celery,scheduled_queries /tmp/redash-celerybeat-schedule
There is some other location which I need to change the celerybeat-schedule?
Because now I haven't errors on celery_log...
John Connell
@JohnConnell
Jan 04 2016 22:50
Hello guys! I'm wondering if it's possible to join data from tables in two different data sources in one query? Either by being able to specify the data source within the query (which would ignore the drop-down selector for data source?) or by setting them up as separate queries in re:dash and then using a 3rd query to join them (with re:dash db as the data source?)
The main database I'm working with is read-only so unfortunately I can't just add the data in the 2nd database to the 1st.