Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Mikael Lepistö
    @elhigu
    If loosing some data is not a problem and good write performance is the only goal I would think of using redis with persistency set to flush to disk every second or so.
    Snir Shechter
    @SnirShechter
    Yes, that's a risk, but the microservice is an open-source (https://github.com/mini-services/miniurl) so I don't really want to burden the database. I don't mind losing some of that count in extreme cases.
    I agree, but as an open-source, I can't really set those constraints.
    Mikael Lepistö
    @elhigu
    You can setup a separate db for its data to not to burden other databases
    Snir Shechter
    @SnirShechter
    Already done as part of the Helm chart
    But a user can just connect it to an existing database
    I think caching the updates just for the count operation would probably be fine.
    Mikael Lepistö
    @elhigu
    If you dont know what kind of loads users put to that service and to what kind of db it connects i suppose it should be configurable how many requests are queued at most and for how long maximum time
    If you really want to add that cache at all :j
    Snir Shechter
    @SnirShechter
    You're right. I'll make it configurable :)
    Chris Newhouse
    @newhouse
    Any idea the best way to add the MATERIALIZED keyword to a with / CTE in Knex? I need WITH "foo" AS (...) to become WITH "foo" AS MATERIALIZED (...).
    2 replies
    donaldball
    @donaldball_twitter
    I am on a quest to excise deadlock errors from a set of services I have inherited and am wondering if I'm missing something? I don't see any facility in knex to retry deadlock errors. I do see two mysql dialect libraries which mix that functionality in. Is that the recommended approach? Is there any interest in a common way to express this?
    Phong Thieu
    @pthieu

    Hi, I'm trying to query a table1 with a column that is an array of UUIDS:

    table1 {
      ...
      uuids: UUID[],
      ...
    }

    I want to take an input array of uuids and query on this column using the ANY operator, as in if any one of the UUIDs within the input array exists in any of the column's UUIDs, then the record is returned.

    I have this piece of code

    knex(TABLE_NAME).where('uuids', '@>', uuidsWanted);

    But what happens here is that the column uuid must be exactly what is listed in uuidsWanted. Is there a way to do this with knex or would

    I have to do this with a raw query?

    I've tried as well but I believe the knex.raw syntax is wrong

    return knex(TABLE_NAME).where('uuids', '@>', knex.raw(`ANY(${mmUuids})`));
    Phong Thieu
    @pthieu
    Update: I was able to do it like this
    return knex(TABLE_NAME).where(
        'uuids',
        '&&',
        knex.raw(`ARRAY[${uuidsWanted.map((id) => `'${id}'`)}]::UUID[]`),
      );
    Hanan F
    @hananf11
    Hello, I am wanting to use Knex in the browser. I discovered that knex doesn't work in the browser (there are errors), I have made a pull request to fix this issue #4464. I'm a first-time contributor so I need someone to approve the running of workflows, so I would really appreciate it if someone could take a look at my pull request. Thanks
    Oliver Escosura
    @olvresc
    Hi, how to ignore/off this error? My database is configured to kill idle or sleeping connections.
    Connection Error: Error: Connection lost: The server closed the connection.
    Mikael Lepistö
    @elhigu
    Maybe if you configure pool to kill idle connections before db
    Oliver Escosura
    @olvresc
    Got it. Just needed to adjust the kill idle time. Thank you!
    arshad-akra
    @arshad-akra

    Hi everyone, any ideas on why im getting this error once I migrated from 0.21 to 0.95.x

    migration file "20191104160706_migrate-appsflyer_customers.js" failed
    migration failed with error: The query is empty

    but the file actually has query

    async function up (knex) {
      // language=Postgres
      const { rows } = await knex.raw(`
        SELECT * FROM appsflyer_customer;
      `)
      const mappedRows = rows.map(row => ({
        user_id: row.user_id,
        advertising_id_type: 'appsflyer',
        advertising_id: row.appsflyer_device_id
      }))
      await knex('device_advertising_association')
        .insert(mappedRows)
    }
    async function down (knex) {
      // Clear the device advertising association table
      // WARNING: Causes data loss
      await knex.raw(`
        DELETE FROM device_advertising_association WHERE user_id NOTNULL;
      `)
    }
    module.exports = {
      up, down
    }
    Mikael Lepistö
    @elhigu
    So is the appsflyer_customers table empty?
    arshad-akra
    @arshad-akra
    oh will that cause the error too ?
    Martin Olsansky
    @olso

    Hey,

    Let's have query builder like this

    const q = knex("some-table")

    How would I extract some-table from the query builder? Something like q.getTableName()

    Mikael Lepistö
    @elhigu
    There is no api for query builder introspection
    Kristjan
    @KristjanTammekivi
    If you're not afraid of doing something really hacky you can do .toSQL and regex match from there
    But yeah have to consider all sorts of possible subqueries and lateral joins that can also have the word "from" in it
    carlino3
    @carlino3
    hi
    according to: http://knexjs.org/#Schema-bigInteger , MySQL bigint types are be converted to string to avoid loss of precision.
    but i'm getting a Number instead, and it is wrong. what am i missing here? is there any config parameter to enable the string conversion?
    Mikael Lepistö
    @elhigu
    Not related to knex. Knex doesn't change any data types 🙃
    Driver should have a way to select how it serializes big integer from sql -> javascript
    carlino3
    @carlino3
    @elhigu solved by adding supportBigNumbers: true when configuring mysql connection in knex. you should modify the docs then, as that statement is confusing and not always true: "Note that bigint data is returned as a string in queries because JavaScript may be unable to parse them without loss of precision."
    Mikael Lepistö
    @elhigu
    Yeah it should mention driver dependent behaviour. That was probably written psql in mind.
    seuaCoder
    @seuaCoder
    Hi
    i'm tryint to stream my sqlite database
    var sql = knex.select("*").from("data").orderBy("id", "desc").limit(3);
    sql.stream().pipe(JSONStream.stringify()).pipe(process.stdout);
    it diaplay the query but when i add another row from another process, it doesn't stream
    Kristjan
    @KristjanTammekivi
    Streams don't listen to database changes
    They just provide a way to not keep the whole result in memory at the same time
    seuaCoder
    @seuaCoder

    Streams don't listen to database changes

    oh ok, is there any way to do that ?

    my sqlite database is feeded by a Python script
    Kristjan
    @KristjanTammekivi
    Not with just knex and sqlite, we use debezium to feed database change messages into kafka in some places and in some places we publish messages to rabbitmq from the code
    seuaCoder
    @seuaCoder
    ok thank you
    Mikael Lepistö
    @elhigu
    Also iirc redis and postgresql has pub/sub messaging system but knex doesnt have any special support for that
    Cristian Razvan
    @CRGavrila
    Hy, I have a question and I don't really how to resolve it: I have a table with users and a table with posts. (users) one to many (posts) and posts has the column user_id referenced to users.id. How to query all users and return all users with a count of posts for each users. I've tried groupBy but i receive "GROUP BY clause or be used in an aggregate function". In docs I don't see how to use an aggregate function. Lots of thanks!
    Zerounodue
    @Zerounodue
    select users.* count(post.id) from users .....
    SunSun99
    @SunSun99
    I am using knex with sqlite version 5.0.0. Is there any way to suppress the message: ".returning() is not supported by sqlite3 and will not have any effect."?
    Mikael Lepistö
    @elhigu
    One good way is to not call .returning()
    Another way is to pass custom log function to configuration and ignore that message there
    SunSun99
    @SunSun99
    I am just confused. Expicitly, I am not using the .returning() function. Is it hidden in any knex function?
    SunSun99
    @SunSun99

    I can see, that Knex provides manipulating this warn message with http://knexjs.org/#Installation-log. But I am not sure, how to set this up for my project. I am using a knexconfigfile to define Knex:

    module.exports = { development: { // debug: true, acquireConnectionTimeout: 420000, client: 'sqlite3', connection: { filename: path.join( __dirname, '..', '..', '..', 'apps', 'backend', 'dev.sqlite3' ) } } }

    How can I integrate this:

    const knex = require('knex')({ log: { warn(message) { }, error(message) { }, deprecate(message) { }, debug(message) { }, } });