Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 10 15:50
    dlebauer commented #627
  • Dec 10 15:49
    dlebauer commented #627
  • Dec 10 15:49
    dlebauer commented #627
  • Dec 10 15:48
    dlebauer commented #627
  • Dec 06 21:54

    dlebauer on BETYdb_5.2.2.3

    (compare)

  • Dec 06 21:52

    dlebauer on master

    Update zenodo.json Merge branch 'master' into deve… Merge pull request #683 from Pe… (compare)

  • Dec 06 21:52
    dlebauer closed #683
  • Dec 06 21:52
    dlebauer synchronize #683
  • Dec 06 21:52

    dlebauer on develop

    Merge pull request #682 from Pe… Merge branch 'master' into deve… (compare)

  • Dec 06 21:52
    dlebauer opened #683
  • Dec 06 21:52

    dlebauer on develop

    Update zenodo.json (compare)

  • Dec 06 21:50

    dlebauer on BETYdb_5.2.2.2

    (compare)

  • Dec 06 21:49

    dlebauer on master

    delete upload_type from zenodo.… Merge branch 'master' into deve… Merge pull request #682 from Pe… (compare)

  • Dec 06 21:49
    dlebauer closed #682
  • Dec 06 21:49
    dlebauer synchronize #682
  • Dec 06 21:49

    dlebauer on develop

    Merge pull request #681 from Pe… Merge branch 'master' into deve… (compare)

  • Dec 06 21:48
    dlebauer opened #682
  • Dec 06 21:48

    dlebauer on develop

    delete upload_type from zenodo.… (compare)

  • Dec 06 21:46

    dlebauer on betydb_5.2.2.1

    (compare)

  • Dec 06 21:30

    dlebauer on betydb_5.2.2

    (compare)

Michael Dietze
@mdietze
Just deleted on psql-pecan.bu.edu, which should 'unblock' the sync
Michael Dietze
@mdietze
Now I get a different, similar sync error
Updated  pfts_priors               :   1563
ERROR:  duplicate key value violates unique constraint "index_pfts_species_on_pft_id_and_specie_id"
DETAIL:  Key (pft_id, specie_id)=(16, 181) already exists.
CONTEXT:  COPY pfts_species, line 217
Michael Dietze
@mdietze
species 181 is Betula; Tried deleting on psql-pecan, but that didn't seem to solve the problem. Hmmm
Michael Dietze
@mdietze
Got past that one via the command line, but hit another duplicate
ERROR:  duplicate key value violates unique constraint "index_pfts_species_on_pft_id_and_specie_id"
DETAIL:  Key (pft_id, specie_id)=(16, 183) already exists.
istfer
@istfer

oops, I see, that's my fault then, I added A. rhombifolia and glutinosa to temperate.Early_Hardwood today, I also deleted their association with the temperate.Late_Hardwood locally, but I guess local deletions don't take effect, but additions do

I'll try reverting back my changes today and see if sync will work

istfer
@istfer
ok all is back, we can talk about how exactly I can update bety records later
Chris Black
@infotroph
y'all, does bety officially support an API version named 'v1' yet, and if so are the queries equivalent to those for the version named 'beta'? I thought I saw some notifications about this a while ago, but am failing to search them up tonight.
Chris Black
@infotroph
Ah, right. #527.
David LeBauer
@dlebauer
Yep v1 is a synonym for beta. beta will be around for a while so no rush but eventually v1 should be used everywhere
Chris Black
@infotroph
@dlebauer I think I've resolved the build failures in #94 -- see my PR to dlebauer/traits. Figured it'd be easier to merge it there so it enters ropensci/traits through your original PR.
Chris Black
@infotroph
@dlebauer Also added support for v1 API while I'm at it, will submit as a separate PR. Question: Should traits keep on defaulting to v0 if bety_api_version is unset, or should I switch it to v1?
Chris Black
@infotroph
...right, autolinks. #94 meaning ropensci/traits#94, not Bety 94.
David LeBauer
@dlebauer
Yes default v1 is the idea, if not too hard to fix tests
Chris Black
@infotroph
Great. I have most of that written but will wait until the paging is merged before pushing it -- it would have some annoying conflicts otherwise.
Shawn P. Serbin
@serbinsh
@dlebauer @gsrohde Could you remind me what the Bulk Upload Wizzard (BUW) is doing behind the scenes when a file is uploaded and before you get to the QA/QC page? I am noticing that for larger numbers of records (e.g. 300+ rows of data for 2-5 traits) it can take several minutes or more to get from the upload to the QA check page…..is that my install of BETYdb or normal? I assume it is running dup checks, looking for entities, etc but also checking each cell to see if it is formatted correctly as well…. Is there a way to speed it up? Or is that just using SQL upload commands directly?
David LeBauer
@dlebauer
@serbinsh that sounds about right. There are a lot of queries and checks etc. It could be sped up but that hasn't been a priority. It's faster than entering by hand. The API upload may be faster.
Shawn P. Serbin
@serbinsh
OK, thanks! @dlebauer not complaining I was just confirming in case its my backend setup slowing it down
In this case, I will probably break some larger files up so it doesnt time out
Shawn P. Serbin
@serbinsh
@dlebauer One thing I learned that significantly helped to speed up the upload was removing the citation info for each record and then just linking the citation during the upload workflow. The data uploaded much faster then
That is probably obvious but I wasnt aware
David LeBauer
@dlebauer
@serbinsh Good observation. I suspect this is true to a lesser extent for other fields that are constant b/c it reduces the number of lookups and data consistency checks (and lots of tables link to citations)
Scott Rohde
@gsrohde
@serbinsh Do you have the latest (September 6) release, 4.19? This sped up validation by around 2/3.
David LeBauer
@dlebauer

@/all I've submitted a PR to update the distributed instances of BETYdb page, The current list looks like

Institution server url id allocated primary key values
Energy Biosciences Institute / University of Illinois ebi-forecast.igb.illinois.edu https://betydb.org 0 1-1,000,000,000
Boston University psql-pecan.bu.edu https://psql-pecan.bu.edu/bety 1 1,000,000,001-2,000,000,000
Brookhaven National Lab modex.test.bnl.gov http://modex.test.bnl.gov/bety 2 2,000,000,001-3,000,000,000
Purdue bety.bio.purdue.edu http://bety.bio.purdue.edu/ 3 3,000,000,001-4,000,000,000
Virginia Tech 4 4,000,000,001-5,000,000,000
University of Wisconsin tree.aos.wisc.edu http://tree.aos.wisc.edu:6480/bety 5 5,000,000,001-6,000,000,000
TERRA Ref 141.142.209.94 http://terraref.ncsa.illinois.edu/bety 6 6,000,000,001-7,000,000,000
TERRA test 141.142.209.95 http://terraref.ncsa.illinois.edu/bety-test 7 7,000,000,001-8,000,000,000
TERRA MEPP UIUC terra-mepp.ncsa.illinois.edu terra-mepp.ncsa.illinois.edu/bety 8 8,000,000,001-9,000,000,000
TERRA TAMU 9 9,000,000,001-10,000,000,000
Ghent 10 10,000,000,001 - 11,000,000,000
TERRA Purdue 11 11,000,000,001 - 12,000,000,000
Development / Virtual Machine localhost https://localhost:6480/bety 99 99,000,000,000-a zillion

If there are any other updates that I've missed please let me know in the PR https://github.com/PecanProject/betydb-documentation/pull/4/files

Shawn P. Serbin
@serbinsh
How many are presently active with the ability to sync with them? Last cursory exam I did, not many more the previous 4/5
David LeBauer
@dlebauer
Terra ref dumps to Terra mepp; it could also sync out to others but I think it best to keep the networks separate b/c the data is very different and Terra is much larger but less diverse (millions of records for hundreds of genotypes in thousands of plots across two sites and two crops)
Shawn P. Serbin
@serbinsh
@/all has anyone yet looked at updating Ruby/Rails to avoid the BETY issues?
Just wondering if I should look at that yet or wait.
David LeBauer
@dlebauer
Which issues? The security issues on GitHub?
@serbinsh ^^
Shawn P. Serbin
@serbinsh
@dlebauer yes, it looks like from past discussions that at some point we will all need to update our Ruby etc?
Rob Kooper
@robkooper
-sameroom portal
Sameroom
@sameroom-bot
<Sameroom> Your Portal URL is https://sameroom.io/xnnZxaJo -- you can send the URL to someone on a different team to share this room. Note: you can connect more than two teams this way.
<Sameroom> I've connected 1 new room #bety (PEcAn) on Slack. See map
Sameroom
@sameroom-bot
<Sameroom> I've removed 2 rooms (PecanProject/pecan on Gitter, #pecan (PEcAn) on Slack). See map
Sameroom
@sameroom-bot
[Chris Black, PEcAn] bety=> select distinct cultivar_id, cultivar, scientificname, species_id from traits_and_yields_view where cultivar_id=10; cultivar_id | cultivar | scientificname | species_id -------------+--------------+------------------+------------ 10 | Cave-In-Rock | Acer rubrum | 30 10 | Cave-In-Rock | Panicum virgatum | 938 (2 rows)
[Chris Black, PEcAn] That doesn’t seem quite right…
[Chris Black, PEcAn] (traits_and_yields_yiew.id of the funky row is 49920)
Sameroom
@sameroom-bot
[David LeBauer, PEcAn] Thanks ... We have a bunch of constraints that aren't implemented b/c of rows that violate like this. Could you please create an issue in bety github?
Sameroom
@sameroom-bot
[Chris Black, PEcAn] done
Sameroom
@sameroom-bot
[David LeBauer, PEcAn] Thanks!
Sameroom
@sameroom-bot
[Michael Dietze, PEcAn] If you're trying to run through the web interface, the new version should just show up on the model pull-down menu. If you're trying to run via the R script, then yes you'll need to insert the id number into the pecan.xml. You should be able to get that by looking up the entry you created in the Models table. Once you click on the record the ID is in the URL
Anthony Gardella
@tonygardella
@/all Hello everyone, we are officially moving our chat communications to slack. Our gitter channels will remain open, but expect delayed responses if you post. To join our slack channels simply click the following link and enter your email: https://publicslack.com/slacks/pecanproject/invites/new
Sameroom
@sameroom-bot
[Tess McCabe, PEcAn] Hey all, I got a security warning from chrome trying to got to psql-pecan.bu.edu/bety . This seems to be a new thing. Is this happening for anyone else?
Shawn P. Serbin
@serbinsh
@tonygardella is there a BETY slack page?
Sameroom
@sameroom-bot
[David LeBauer, PEcAn] @sserbin it is a channel
Sameroom
@sameroom-bot
[Alkesh Srivastava, PEcAn] @kooper I wish to contribute to the bety project. What would be the best approach to move forward, would you point me to the related issues, or should I start with the solving the technical tasks in the application form first?
Sameroom
@sameroom-bot
[Shawn Paul Serbin, PEcAn] Thanks @dlebauer I couldn’t find channel or any info where to connect to it
Sameroom
@sameroom-bot
[Rob Kooper, PEcAn] Hi @alkeshsrivastava11 I would if you would get involved in the BETY project. Do you have experience with Ruby? Python? Docker? Trying to find out where to start. Feel free to contact me using direct message.
Sameroom
@sameroom-bot
[Shawn Paul Serbin, PEcAn] @kooper do you have a new recommended Ruby source? or install approach?