Meanwhile, I think I've answered the question about Jupyter Notebooks: they seem to be exclusively Python. I can do searches through the API, though they're more rate-limited and I have to wait longer, so I have the gathering script print out results as it goes.
For each repo that GitHub labels as "Jupyter Notebook", I do two searches: one for the word "include" and the other for the word "import". If imports outnumber includes, I label it as Python. I've manually followed up a few cases with non-zero "includes"; they've all been in markdown cells.
FredStober/sandbox 0 vs 10 Python michelif/bayesian_opt_skopt 0 vs 1 Python michelif/HHbbgg_ETH 1 vs 18 Python michelif/quickMLTests 0 vs 0 ??? terrenceedmonds/titanic 0 vs 2 Python hbakhshi/Analysis13TeV 0 vs 0 ??? clint-richardson/BU-TheBus 0 vs 1 Python clint-richardson/NBA-Data 0 vs 0 ??? clint-richardson/X53AnalysisDemo 0 vs 2 Python zhangzc11/Pi0Net 0 vs 1 Python joseph-taylor/LjLabBook 0 vs 5 Python mukundvarma/kaggle-instacart 1 vs 2 Python A-lxe/study-csc-daq-rate 0 vs 1 Python vlimant/summer15-ArashJofrehei 0 vs 61 Python vlimant/summer15-Irene 0 vs 12 Python vlimant/summer15-MarinaKolosova 1 vs 45 Python vlimant/summer15-SahandSeif 0 vs 25 Python vlimant/summer16-NikolausHowe 0 vs 24 Python vlimant/surf17-tutorial 0 vs 4 Python vlimant/surf18-tutorial 0 vs 7 Python davidlange6/gsocStudentSolutions 0 vs 3 Python davidlange6/toy_notebooks 2 vs 14 Python jbueghly/hzg_analysis 0 vs 5 Python kaylanb/skinapp 0 vs 0 ??? kaylanb/thesis_code 0 vs 2 Python lihux25/Projects 1 vs 3 Python ArnabPurohit/Machine-Learning-applications-in-HEP 0 vs 1 Python nhaubrich/biophysics 0 vs 8 Python emc5ud/rosalind-solutions 1 vs 6 Python nmehrle/echelle 0 vs 1 Python zaixingmao/retina 1 vs 2 Python fmanteca/ImageClassification 0 vs 1 Python alkaplan/jupyter-notebooks 2 vs 6 Python patrykel/multi-tracking-notebooks 0 vs 6 Python patrykel/MultitrackingMasterProject 0 vs 2 Python cfangmeier/HHC 0 vs 2 Python cfangmeier/Small 1 vs 4 Python cfangmeier/TTTT 0 vs 1 Python cfangmeier/UNL-Gantry-Encapsulation-Monitoring 0 vs 2 Python jiafulow/L1TMuonDocsNov2018 1 vs 1 ??? jiafulow/L1TMuonSimulationsMar2017 3 vs 18 Python jiafulow/UF-slurm 0 vs 0 ??? monttj/computational-physics 1 vs 12 Python NJManganelli/TaggerTest 0 vs 1 Python sciencecw/cmsjupyter 1 vs 14 Python lecriste/first-binder 0 vs 1 Python mzanetti79/LaboratoryOfComputationalPhysics 0 vs 11 Python mzanetti79/ML-INFN 2 vs 34 Python mzanetti79/MLCC18 0 vs 23 Python bpenning/jupyter_repo 0 vs 22 Python bencammett/ML_project_Comp2 6 vs 12 Python hbprosper/ENHEP 1 vs 8 Python hbprosper/eshep_tutorials 0 vs 13 Python
Actually, in my slow-moving scan of Jupyter notebooks, I've finally come across two legitimate C++ Jupyter repos: https://github.com/gudrutis/jupyter-book-tutorials/search?utf8=%E2%9C%93&q=include&type= and https://github.com/javadebadi/learning_cpp_again/search?q=include&unscoped_q=include
These are the first 2 out of 91.
It would be very interesting to find out what the other collaborations are doing. I've looked into the GitLab API—it functions on gitlab.cern.ch, but I wasn't able to repeat any of these queries without figuring out its (different) authentication mechanism. And even then, I might have to be a member of a collaboration to see its users. If there's a culture of "in-development analysis is private, even from other members of the collaboration," then there might not be anything any one user can do to get a global picture.
Does anyone have any other suggestions? (GitHub preferred; I already have the scripts.)
@HDembinski As I've been trying to figure out the issues that pyhf is having with iminuit this weekend I've run into a problem where installing iminuit in a Unbuntu 18.04 Docker image with Python 3.6.8 installed from source on it fails. I have a short Gist that describes what's going on, and if you have any thoughts on what to think about with regards to what is going wrong that would be great:
CXX_VERSION="$(which g++)"and tried to rebuild the Docker image, but this just results in CPython complaining loudly and then failing during the build. So unless I'm doing something stupid I guess that needs to be
CXXif not given, so it really seems like it should be g++. Never tried passing it explicitly either.
From the looks of it @henryiii's and @daritter 's suggestion of removing the
with-cxx-main flag seems to do the trick. I rebuilt and was able to build through
iminuit in the container. :+1: I'll need to do some experimentation with the configure options, but I found the following from the old SVN Python 2.7 trunk very helpful
--with-cxx-main=<compiler>: If you plan to use C++ extension modules, then -- on some platforms -- you need to compile python's main() function with the C++ compiler. With this option, make will use <compiler> to compile main() and to link the python executable. It is likely that the resulting executable depends on the C++ runtime library of <compiler>. (The default is --without-cxx-main.)
There are platforms that do not require you to build Python with a C++ compiler in order to use C++ extension modules. E.g., x86 Linux with ELF shared binaries and GCC 3.x, 4.x is such a platform. We recommend that you configure Python --without-cxx-main on those platforms because a mismatch between the C++ compiler version used to build Python and to build a C++ extension module is likely to cause a crash at runtime.
The Python installation also stores the variable CXX that determines, e.g., the C++ compiler distutils calls by default to build C++ extensions. If you set CXX on the configure command line to any string of non-zero length, then configure won't change CXX. If you do not preset CXX but pass --with-cxx-main=<compiler>, then configure sets CXX=<compiler>. In all other cases, configure looks for a C++ compiler by some common names (c++, g++, gcc, CC, cxx, cc++, cl) and sets CXX to the first compiler it finds. If it does not find any C++ compiler, then it sets CXX="".
Similarly, if you want to change the command used to link the python executable, then set LINKCC on the configure command line.
Kinda unfortunate that I can't find that level of detail in modern Python, but maybe I'm not searching hard enough through CPython's GitHub
@henryiii This information on https://bugs.python.org/issue23644 is also very nice. Thanks for taking the time to go find it!
I don't think that CPython can be built by g++. - STINNER Victor
This was exactly why I originally had it set to
which gcc, but it seems that not explicitly setting compiler flags is the way to go.
@daritter @henryiii Thanks to your help the problem is now resolved: matthewfeickert/Docker-Python3-Ubuntu#3
@HDembinski Please ignore my ping as the issue no longer exists.