These are chat archives for getredash/redash

28th
May 2015
sunand chakradhar
@sunandchakradhar
May 28 2015 04:41
@arikfr when i click execute query with out saving it I get this "Error running query: execute_query() takes exactly 3 arguments (4 given)". But it shows up randomly and very often .
Arik Fraimovich
@arikfr
May 28 2015 05:10
@
@sunandchakradhar it seems like you have celery workers running with older version of redash. To be on the safe side kill them all (sudo pkill - 9 celery). Then run supervisorctl status to make sure it started new ones.
DBY
@Dnile
May 28 2015 09:22
hi, i'm trying to upgrade from redash.0.4.0.b589/ to the latest version
and my migrations keep getting stuck
sudo sudo -u redash PYTHONPATH=. bin/run python migrations/0001_allow_delete_query.py [2015-05-28 02:22:13,608][PID:24572][ERROR][peewee] ALTER TABLE "queries" ADD COLUMN "is_archived" BOOLEAN [] Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/playhouse/postgres_ext.py", line 351, in execute_sql cursor.execute(sql, params or ()) ProgrammingError: column "is_archived" of relation "queries" already exists
guess i have to drop some tables?
Arik Fraimovich
@arikfr
May 28 2015 10:15
@Dnile no, it seems that this migration ran already. you can skip it. are you using the fabric script?
DBY
@Dnile
May 28 2015 10:15
yup
it fails though
Arik Fraimovich
@arikfr
May 28 2015 10:16
@Dnile it runs all the migration it finds in the new version folder and not in the current one. so you can either run them manually, or just "trick" it by copying 0001_allow_delete_query.py to /opt/redash/current/migrations/
DBY
@Dnile
May 28 2015 10:16
re-running without migrations
Arik Fraimovich
@arikfr
May 28 2015 10:16
better be done w/ migrations first
DBY
@Dnile
May 28 2015 10:17
cool
Arik Fraimovich
@arikfr
May 28 2015 10:19
if others fail, let me know what's the error, will guide you through :)
DBY
@Dnile
May 28 2015 10:20
will do
trying that hack
DBY
@Dnile
May 28 2015 10:26
ugh, getting this now when trying anything
peewee.OperationalError: FATAL: remaining connection slots are reserved for non-replication superuser connections
Arik Fraimovich
@arikfr
May 28 2015 10:26
wtf
DBY
@Dnile
May 28 2015 10:27
indeed.
Arik Fraimovich
@arikfr
May 28 2015 10:27
see if no stuck celery or gunicorn processes left, and kill them if necessary
DBY
@Dnile
May 28 2015 10:27
on it
Arik Fraimovich
@arikfr
May 28 2015 10:27
if this doesn't help -> restart pg
DBY
@Dnile
May 28 2015 10:27
cool
restarted pg
same same
celery and redash are stopped
Arik Fraimovich
@arikfr
May 28 2015 10:31
hmmm
so, at what point you get this error message? when trying to run the migrations?
DBY
@Dnile
May 28 2015 10:32
yea
Arik Fraimovich
@arikfr
May 28 2015 10:32
can you check what's connected to pg then?
DBY
@Dnile
May 28 2015 10:32
checking
weird
looks like nothing is
i can just see a postgres connection to redshift
ok
there were some stuck migrations there i think
now it seems to have skipped right to run: sudo sudo -u redash PYTHONPATH=. bin/run python migrations/0002_fix_timestamp_fields.py
when running the fabfile
Arik Fraimovich
@arikfr
May 28 2015 10:38
this one might take some time, if you have a lot of query_results rows.
DBY
@Dnile
May 28 2015 10:38
well....:)
i was actually thinking of just starting with a new box
might end up doing that
Arik Fraimovich
@arikfr
May 28 2015 10:39
you don't need the existing queries?
DBY
@Dnile
May 28 2015 10:39
i do, that's the problem...
unless
i do a pg_dump
hmm.
Arik Fraimovich
@arikfr
May 28 2015 10:39
so what you can do, is to truncate the query_results table and update the references to it in queries table
DBY
@Dnile
May 28 2015 10:40
that sounds legit
Arik Fraimovich
@arikfr
May 28 2015 10:40
another , simpler option, is to comment out the part that touches query_results table. although this update is useful for correct timestamps
DBY
@Dnile
May 28 2015 10:42
hmmm
i like the first option better
i'll wait patiently though before doing anything, till this migration ends
Arik Fraimovich
@arikfr
May 28 2015 10:43
ack.
DBY
@Dnile
May 28 2015 10:43
my Hanz avatar and i thank you greatly.
Arik Fraimovich
@arikfr
May 28 2015 10:43
it will end eventually. and for the future: there is an option to enable slow cleanup of query results. it's off by default, but it seems safe (been using it for a few months now)
:)
did you bring him to SF already?
DBY
@Dnile
May 28 2015 10:44
that does sound safe
nah, he's still here, and so am i for a bit :)
Arik Fraimovich
@arikfr
May 28 2015 10:44
oh
DBY
@Dnile
May 28 2015 10:48
all done;redash is up
new logo is cool :P
Arik Fraimovich
@arikfr
May 28 2015 10:48
great ! :clap:
tell that to Shayke ;)
DBY
@Dnile
May 28 2015 10:48
will do
martin sarsale
@runa
May 28 2015 13:07
g not sure what do you want to see there
I hitted refresh manually before screenshoting
Arik Fraimovich
@arikfr
May 28 2015 14:15
@runa I meant http://time.is
martin sarsale
@runa
May 28 2015 14:39
g g
Arik Fraimovich
@arikfr
May 28 2015 14:56
@runa did you always had the "last update" in the future or it started recently?
martin sarsale
@runa
May 28 2015 14:56
@arikfr IIRC I never saw nothing coherent there ;)
@arikfr I thought it was a visual bug; but then I found the query was not refreshing :)
Arik Fraimovich
@arikfr
May 28 2015 14:57
damn . :| I was hoping that maybe it's a recent change. btw, the query not refreshing is a different thing (as you can see the "NOW" query does refresh)
try running it manually - it might have some errors, and therefore not updating
martin sarsale
@runa
May 28 2015 14:58
@arikfr where do you see that the NOW query refreshes?
Arik Fraimovich
@arikfr
May 28 2015 14:59
I assumed because it showed today's timestamp. Did you refresh it manually?
martin sarsale
@runa
May 28 2015 14:59
@arikfr yep. I hit refresh before screenshoting
Arik Fraimovich
@arikfr
May 28 2015 15:00
interesting. maybe Celery's scheduler isn't running
you can try restarting Celery (supervisorctl restart redash_celery)
martin sarsale
@runa
May 28 2015 15:01
@arikfr # supervisorctl restart redash_celery
redash_celery: stopped
redash_celery: started
Arik Fraimovich
@arikfr
May 28 2015 15:02
ok. let's see if it starts refreshing
martin sarsale
@runa
May 28 2015 15:05
@arikfr nope.
Arik Fraimovich
@arikfr
May 28 2015 15:08
@runa in /admin/status there is a count of outdated queries - what its value?
martin sarsale
@runa
May 28 2015 15:09
@arikfr 0!
Arik Fraimovich
@arikfr
May 28 2015 15:09
that's good
martin sarsale
@runa
May 28 2015 15:09
@arikfr weird..
ah
:)
Arik Fraimovich
@arikfr
May 28 2015 15:09
no, it's a problem. but it's a good as it gives me a clue :)
martin sarsale
@runa
May 28 2015 15:09
:)
Arik Fraimovich
@arikfr
May 28 2015 15:09
I think for some reason it fails to calculate the next refresh time properly
what's the result fo running date on the server?
martin sarsale
@runa
May 28 2015 15:11
@arikfr Thu May 28 10:14:05 CDT 2015
Arik Fraimovich
@arikfr
May 28 2015 15:15
@runa ok, and - python -c "import datetime; print datetime.datetime.utcnow()" ?
martin sarsale
@runa
May 28 2015 15:15
2015-05-28 15:18:41.314012
Arik Fraimovich
@arikfr
May 28 2015 15:17
super weird
can you send me the result of "\d queries" and "\d query_results" from psql? also, the result for select schedule from queries where schedule is not null order by created_at desc limit 10; and select retrieved_at from query_results order by retrieved_at desc limit 10; using this I'll try to recreate the same "env" and reproduce it on my machine
martin sarsale
@runa
May 28 2015 15:19
sure, do you want an email or here is fine?
Arik Fraimovich
@arikfr
May 28 2015 15:20
whatever both ok work for me
martin sarsale
@runa
May 28 2015 15:20

psql (9.3.6)
Type "help" for help.

redash=# \d queries
Table "public.queries"
Column | Type | Modifiers
----------------------+--------------------------+------------------------------------------------------
id | integer | not null default nextval('queries_id_seq'::regclass)
updated_at | timestamp with time zone | not null
created_at | timestamp with time zone | not null
data_source_id | integer | not null
latest_query_data_id | integer |
name | character varying(255) | not null
description | character varying(4096) |
query | text | not null
query_hash | character varying(32) | not null
api_key | character varying(40) | not null
user_email | character varying(360) |
user_id | integer | not null
last_modified_by_id | integer |
is_archived | boolean | not null
schedule | character varying(10) |
Indexes:
"queries_pkey" PRIMARY KEY, btree (id)
"queries_data_source_id" btree (data_source_id)
"queries_is_archived" btree (is_archived)
"queries_last_modified_by_id" btree (last_modified_by_id)
"queries_latest_query_data_id" btree (latest_query_data_id)
"queries_user_id" btree (user_id)
Foreign-key constraints:
"queries_data_source_id_fkey" FOREIGN KEY (data_source_id) REFERENCES data_sources(id)
"queries_last_modified_by_id_fkey" FOREIGN KEY (last_modified_by_id) REFERENCES users(id)
"queries_latest_query_data_id_fkey" FOREIGN KEY (latest_query_data_id) REFERENCES query_results(id)
"queries_user_id_fkey" FOREIGN KEY (user_id) REFERENCES users(id)
Referenced by:
TABLE "visualizations" CONSTRAINT "visualizations_query_id_fkey" FOREIGN KEY (query_id) REFERENCES queries(id)

redash=# \d query_results
Table "public.query_results"
Column | Type | Modifiers
----------------+--------------------------+------------------------------------------------------------
id | integer | not null default nextval('query_results_id_seq'::regclass)
data_source_id | integer | not null
query_hash | character varying(32) | not null
query | text | not null
data | text | not null
runtime | real | not null
retrieved_at | timestamp with time zone | not null
Indexes:
"query_results_pkey" PRIMARY KEY, btree (id)
"query_results_data_source_id" btree (data_source_id)
"query_results_query_hash" btree (query_hash)
Foreign-key constraints:
"query_results_data_source_id_fkey" FOREIGN KEY (data_source_id) REFERENCES data_sources(id)
Referenced by:
TABLE "queries" CONSTRAINT "queries_latest_query_data_id_fkey" FOREIGN KEY (latest_query_data_id) REFERENCES query_results(id)

redash=# select schedule from queries where schedule is not null order by created_at desc limit 10;

schedule

60
604800
21600
86400
86400
86400
86400
86400
86400
86400
(10 rows)

redash=# select retrieved_at from query_results order by retrieved_at desc limit 10
redash-# ;

     retrieved_at          

2015-05-28 15:02:59.64224-05
2015-05-28 14:54:21.226295-05
2015-05-28 14:49:14.668702-05
2015-05-28 14:46:04.196202-05
2015-05-28 14:45:05.081039-05
2015-05-28 14:41:50.988195-05
2015-05-28 14:29:53.881337-05
2015-05-28 14:29:50.041713-05
2015-05-28 14:22:12.900182-05
2015-05-28 13:45:52.409241-05
(10 rows)

Arik Fraimovich
@arikfr
May 28 2015 15:22
2015-05-28 15:02:59.64224-05 is in the future for you, right?
Arik Fraimovich
@arikfr
May 28 2015 15:27
@runa ^
martin sarsale
@runa
May 28 2015 15:28
both in my current timezone and the server TZ 15hs is the future ;)
Arik Fraimovich
@arikfr
May 28 2015 15:39
@runa got it, so it explains why it doesn't find outdated queries. I'll reproduce it and make a fix. Will keep you posted.
martin sarsale
@runa
May 28 2015 15:39
thanks! :)