Where communities thrive

Join over 1.5M+ people
Join over 100K+ communities
Free without limits
Create your own community

Explore more communities

harelba/q

q - Text as a Database. See Examples Below

People

Repo info

Activity

Harel Ben-Attia

@harelba

hi, all

unfortunately i'm not here all the time, but i'll try to be here as much as i can.

please write here even if i'm not here - i'll get back to you either through here

thanks @smith !!

Harel Ben-Attia

@harelba

Hi. I would appreciate any feedback you might have. I'm trying this gitter chatroom for that purpose. I'd appreciate any feedback on the following:

A few short words about who you are, so I get to know who my customers are :)
What are you using it for?
What is the best missing feature?
What is the best feature in q?
What is your dream installation method for q?
How do you install it now?
What bugs do you see which prevent you from being fully productive with q?
Do you encounter non-intuitive behavior when using q? In what way?
Anything else on your mind

Scott Weaver

@tdlm

hey there, i'm just messing around with this app. not sure if i'd be able to use it yet. is there a way or is there a plan to make a way to specify column headers as column names (assuming they conform to a standard)?

Cyrus David

@vohof

HOWDY

Harel Ben-Attia

@harelba

@tdlm Hi, i've just finished writing two features - column type inference and column name detection through the header line. I'm still testing it. Will be out during the weekend, i believe.

@vohof Hi

Scott Weaver

@tdlm

thanks @harelba

Nate Wildermuth

@wildermuthn

Interesting.

Harel Ben-Attia

@harelba

@wildermuthn thanks!

Harel Ben-Attia

@harelba

Releasing a beta version soon, with column names using the header line and column type detection

Harel Ben-Attia

@harelba

beta version is out - 1.3.0b. Contains column name and type detection, and multiple parsing modes

dima206

@dima206

Hi. Can I run q under windows?

Harel Ben-Attia

@harelba

@dima206 yes, it can run on windows. I'll add q.bat in the next release. In the mean time you can just create a q.bat file which runs q through python

akaRem

@akaRem

how good is q for querying nginx/apache access logs?

Harel Ben-Attia

@harelba

@akaRem hi. Not familiar with nginx logs, but it's been used several times for apache access logs. As long as there is a CSV-like behavior (e.g. column separators, and double-quotes for fields which contain the separator), there should be no problem at all

@akaRem Check it out, if there's any kind of problem, please contact me here or on email harelba@gmail.com and i'd be glad to help or fix if needed.

akaRem

@akaRem

@harelba thanks. Nginx logs are almost the same as apache. :)

Zack Piper

@zackp30

Hello, the REGEXP operator returns user defined function raised exception.

Nick Beeuwsaert

@NickBeeuwsaert

@zackp30: it looks that that is caused when a the column has a NULL value. doing ls -alh | q 'SELECT * FROM - WHERE c3 REGEXP “staff”;’ will cause the error whereas doing ls -alh * | q 'SELECT * FROM - WHERE c3 REGEXP “staff”;’ does not (note the * in the ls call)

Zack Piper

@zackp30

Hmm, I can't seem to find a NULL in my data. :/

Nick Beeuwsaert

@NickBeeuwsaert

what is your data set? are there any rows with fewer columns than the rest?

you can also add a check before the regexp to test if the column is null e.g. SELECT FROM - WHERE c3 IS NOT NULL AND c3 regexp '.*’

Zack Piper

@zackp30

Just an ls -la of my home dir (playing around with q ATM)

Nick Beeuwsaert

@NickBeeuwsaert

ls might be printing out a line at the top that is one column, something like “total 123"

Zack Piper

@zackp30

Oh, ofc...

Okay, thans. :)

Nick Beeuwsaert

@NickBeeuwsaert

welcome :D

Zack Piper

@zackp30

*thanks

Harel Ben-Attia

@harelba

@zackp30 @NickBeeuwsaert hi, yeah, it's a bug in my regexp implementation. Will add a fix for that in the next release. In the mean time, you can use the NOT NULL clause or filter out rows with fewer columns. btw, if you run q with -m strict, you'll get an error if there are rows with different column counts (such as the "total XXX" row)

opened an issue about this: harelba/q#47

Nick Beeuwsaert

@NickBeeuwsaert

@harelba: why is the SHOW_SQL constant set to False?

Harel Ben-Attia

@harelba

@NickBeeuwsaert This is a debugging parameter - Should not be turned on for any regular use. I'm planning on moving it to become some kind of "verbose" command line parameter which will be used for debugging and sending any issues to me

Harel Ben-Attia

@harelba

version 1.3.0 is officially out and working well.

jemarch

@jemarch

hey

harelba: you here?

Harel Ben-Attia

@harelba

trying to be here as much as possible. please email me harelba@gmail.com or tweet if needed

PreXident

@PreXident

Hi, I think you should warn users that are not very skilled in Python (like myself) that q is not compatible with Python 3 - it took me a while to discover why I was getting "SyntaxError: invalid syntax" even though I met "python >= 2.5" requirement. Otherwise great job!

PreXident

@PreXident

Also I had problems with query encoding (I am on English Windows with Czech keybord), so I had to change creating SQL statement to: sql_object = Sql('%s' % args[0].decode(locale.getpreferredencoding()))

Harel Ben-Attia

@harelba

@PreXident Thanks for the information. I'll handle the python 3 stuff. As for the encoding part, can you send me an example file so I can find the root cause?

Harel Ben-Attia

@harelba

@PreXident I see that the encoding part is related to encoding the query itself... Interesting. I'll take a look at it. If you can send an example query, that would be great (harelba@gmail.com)

Harel Ben-Attia

@harelba

opening a ticket for the query encoding

Harel Ben-Attia

@harelba

version 1.4.0 is out - ~2.5x performance improvement, output header option, and many bug fixes

Harel Ben-Attia

@harelba

If any of the recurring visitors to the site comes here, I would really appreciate to hear about the reasons for coming back to the site. That would help me to improve the tool.

qintao361

@qintao361

may I get help from any q-expert? when I use q to execute command for chinese, error occurred. e.g. command = q -t "select * from data"; error = Cannot encode data. Error:'latin-1' codec can't encode characters in position 88-89: ordinal not in range(256)

thanks a lot

for any help

Dan Van Brunt

@iDVB

This message was deleted