Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Nov 01 22:29
    Travis bedezign/yii2-audit (master) errored (755)
  • Nov 01 22:23

    Blizzke on master

    Updated for mail-renderer and m… (compare)

  • Nov 01 21:56
    Blizzke closed #269
  • Nov 01 21:56
    Blizzke commented #269
  • Nov 01 21:55
    Blizzke closed #151
  • Nov 01 21:54
    Blizzke commented #224
  • Nov 01 21:46
    Blizzke closed #245
  • Nov 01 21:46
    Blizzke closed #246
  • Nov 01 21:46
    Blizzke closed #261
  • Nov 01 21:46
    Blizzke closed #266
  • Nov 01 21:46
    Blizzke commented #266
  • Nov 01 21:36
    Blizzke closed #221
  • Nov 01 21:35
    Blizzke closed #251
  • Nov 01 21:35
    Blizzke commented #251
  • Nov 01 21:34
    Blizzke closed #238
  • Nov 01 21:33
    Blizzke closed #244
  • Nov 01 21:33
    Blizzke commented #244
  • Nov 01 21:33
    Blizzke closed #256
  • Nov 01 21:33
    Blizzke commented #256
  • Nov 01 21:30
    Blizzke closed #257
Abayomi Osamiluyi
@yomexzo
source_id is where it is currently
arrival_time is when it got to where it is
of course, we know last_update_time and date_generated
Steve Guns
@Blizzke
but thats a full select?
Abayomi Osamiluyi
@yomexzo
that query takes forever (at least 17 secs) on the table
Steve Guns
@Blizzke
of the entire table
Abayomi Osamiluyi
@yomexzo
i just pray i make a little sense
nope, i cut out some of the aggregations for simplicity
but that should give a picture
yes. scans the entire table apparently
Steve Guns
@Blizzke
yeah but its the aggregations that will cause the slowdown, not what you just pasted
hmm
Abayomi Osamiluyi
@yomexzo
that’s the aggregations
sums, and counts
only selected those 6 for simplicity
Steve Guns
@Blizzke
yeah i see now
you should probably run that stuff with an explain on and see which indexes get used and which dont
Abayomi Osamiluyi
@yomexzo
of course. straight away
Steve Guns
@Blizzke
but as i see it it just selects the entire table every time just to generate your stats
Abayomi Osamiluyi
@yomexzo
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE packages ALL NULL NULL NULL NULL 988883 NULL
1 SIMPLE currentStatus eq_ref PRIMARY PRIMARY 4 db.packages.package_status_id 1 NULL
yes. exactly. @entire table
Steve Guns
@Blizzke
yeah its not using indexes at all for the full table scan
Abayomi Osamiluyi
@yomexzo
yeah. because it’s a full table scan
right?
Steve Guns
@Blizzke
no because you're using ranges
for the timestamp
Abayomi Osamiluyi
@yomexzo
oh
honestly, a million records is very small to me
makes me wonder how people run analytics/reports e.t.c
i log in to dashboards of apps i sign up to
and voila, login is fast, report is fast and instant
Steve Guns
@Blizzke
probably by splitting the query into multiples
yeah but most of the time those are generated from aggregation tables, never the raw data
Abayomi Osamiluyi
@yomexzo
i would refer to google analytics scale, and would just stick with a product like mandrill that processes millions of records and still give you reports on demand
Steve Guns
@Blizzke
you log a bunch of data then run scripts to aggregate it into bitesized pieces of things you can output and use those
Abayomi Osamiluyi
@yomexzo
hmm
go on
Steve Guns
@Blizzke
in its simplest form you could just cache the result for a couple minutes
Abayomi Osamiluyi
@yomexzo
hmm...
Steve Guns
@Blizzke
yeah don't expect some kind of holy grail answer from me, all depends on a lot of things
Abayomi Osamiluyi
@yomexzo
lol… i was looking for more like holy grail :)
i understand you tho
i have a couple of ideas of my own. but just seeking something more standard approach
Steve Guns
@Blizzke
i'm not clairvoyant in any case, there are a lot of paths towards a working solution and much depends on how the data is used and how up to date it needs to be and so on
i'll throw a couple of my queries at you, see what you can tell about those, what do you think? :P
Abayomi Osamiluyi
@yomexzo
hit me
lol @being clairvoyant tho
Steve Guns
@Blizzke
i'm just kidding... just want to point out that its hard to jump in someone else's application just like that
Abayomi Osamiluyi
@yomexzo
i get you. had to do a quick lookup to figure that out :)
but hey. thanks!!!