Meanwhile, I think I've answered the question about Jupyter Notebooks: they seem to be exclusively Python. I can do searches through the API, though they're more rate-limited and I have to wait longer, so I have the gathering script print out results as it goes.
For each repo that GitHub labels as "Jupyter Notebook", I do two searches: one for the word "include" and the other for the word "import". If imports outnumber includes, I label it as Python. I've manually followed up a few cases with non-zero "includes"; they've all been in markdown cells.
FredStober/sandbox 0 vs 10 Python michelif/bayesian_opt_skopt 0 vs 1 Python michelif/HHbbgg_ETH 1 vs 18 Python michelif/quickMLTests 0 vs 0 ??? terrenceedmonds/titanic 0 vs 2 Python hbakhshi/Analysis13TeV 0 vs 0 ??? clint-richardson/BU-TheBus 0 vs 1 Python clint-richardson/NBA-Data 0 vs 0 ??? clint-richardson/X53AnalysisDemo 0 vs 2 Python zhangzc11/Pi0Net 0 vs 1 Python joseph-taylor/LjLabBook 0 vs 5 Python mukundvarma/kaggle-instacart 1 vs 2 Python A-lxe/study-csc-daq-rate 0 vs 1 Python vlimant/summer15-ArashJofrehei 0 vs 61 Python vlimant/summer15-Irene 0 vs 12 Python vlimant/summer15-MarinaKolosova 1 vs 45 Python vlimant/summer15-SahandSeif 0 vs 25 Python vlimant/summer16-NikolausHowe 0 vs 24 Python vlimant/surf17-tutorial 0 vs 4 Python vlimant/surf18-tutorial 0 vs 7 Python davidlange6/gsocStudentSolutions 0 vs 3 Python davidlange6/toy_notebooks 2 vs 14 Python jbueghly/hzg_analysis 0 vs 5 Python kaylanb/skinapp 0 vs 0 ??? kaylanb/thesis_code 0 vs 2 Python lihux25/Projects 1 vs 3 Python ArnabPurohit/Machine-Learning-applications-in-HEP 0 vs 1 Python nhaubrich/biophysics 0 vs 8 Python emc5ud/rosalind-solutions 1 vs 6 Python nmehrle/echelle 0 vs 1 Python zaixingmao/retina 1 vs 2 Python fmanteca/ImageClassification 0 vs 1 Python alkaplan/jupyter-notebooks 2 vs 6 Python patrykel/multi-tracking-notebooks 0 vs 6 Python patrykel/MultitrackingMasterProject 0 vs 2 Python cfangmeier/HHC 0 vs 2 Python cfangmeier/Small 1 vs 4 Python cfangmeier/TTTT 0 vs 1 Python cfangmeier/UNL-Gantry-Encapsulation-Monitoring 0 vs 2 Python jiafulow/L1TMuonDocsNov2018 1 vs 1 ??? jiafulow/L1TMuonSimulationsMar2017 3 vs 18 Python jiafulow/UF-slurm 0 vs 0 ??? monttj/computational-physics 1 vs 12 Python NJManganelli/TaggerTest 0 vs 1 Python sciencecw/cmsjupyter 1 vs 14 Python lecriste/first-binder 0 vs 1 Python mzanetti79/LaboratoryOfComputationalPhysics 0 vs 11 Python mzanetti79/ML-INFN 2 vs 34 Python mzanetti79/MLCC18 0 vs 23 Python bpenning/jupyter_repo 0 vs 22 Python bencammett/ML_project_Comp2 6 vs 12 Python hbprosper/ENHEP 1 vs 8 Python hbprosper/eshep_tutorials 0 vs 13 Python
Actually, in my slow-moving scan of Jupyter notebooks, I've finally come across two legitimate C++ Jupyter repos: https://github.com/gudrutis/jupyter-book-tutorials/search?utf8=%E2%9C%93&q=include&type= and https://github.com/javadebadi/learning_cpp_again/search?q=include&unscoped_q=include
These are the first 2 out of 91.
It would be very interesting to find out what the other collaborations are doing. I've looked into the GitLab API—it functions on gitlab.cern.ch, but I wasn't able to repeat any of these queries without figuring out its (different) authentication mechanism. And even then, I might have to be a member of a collaboration to see its users. If there's a culture of "in-development analysis is private, even from other members of the collaboration," then there might not be anything any one user can do to get a global picture.
Does anyone have any other suggestions? (GitHub preferred; I already have the scripts.)
@HDembinski As I've been trying to figure out the issues that pyhf is having with iminuit this weekend I've run into a problem where installing iminuit in a Unbuntu 18.04 Docker image with Python 3.6.8 installed from source on it fails. I have a short Gist that describes what's going on, and if you have any thoughts on what to think about with regards to what is going wrong that would be great:
CXX_VERSION="$(which g++)"and tried to rebuild the Docker image, but this just results in CPython complaining loudly and then failing during the build. So unless I'm doing something stupid I guess that needs to be
CXXif not given, so it really seems like it should be g++. Never tried passing it explicitly either.
From the looks of it @henryiii's and @daritter 's suggestion of removing the
with-cxx-main flag seems to do the trick. I rebuilt and was able to build through
iminuit in the container. :+1: I'll need to do some experimentation with the configure options, but I found the following from the old SVN Python 2.7 trunk very helpful
--with-cxx-main=<compiler>: If you plan to use C++ extension modules, then -- on some platforms -- you need to compile python's main() function with the C++ compiler. With this option, make will use <compiler> to compile main() and to link the python executable. It is likely that the resulting executable depends on the C++ runtime library of <compiler>. (The default is --without-cxx-main.)
There are platforms that do not require you to build Python with a C++ compiler in order to use C++ extension modules. E.g., x86 Linux with ELF shared binaries and GCC 3.x, 4.x is such a platform. We recommend that you configure Python --without-cxx-main on those platforms because a mismatch between the C++ compiler version used to build Python and to build a C++ extension module is likely to cause a crash at runtime.
The Python installation also stores the variable CXX that determines, e.g., the C++ compiler distutils calls by default to build C++ extensions. If you set CXX on the configure command line to any string of non-zero length, then configure won't change CXX. If you do not preset CXX but pass --with-cxx-main=<compiler>, then configure sets CXX=<compiler>. In all other cases, configure looks for a C++ compiler by some common names (c++, g++, gcc, CC, cxx, cc++, cl) and sets CXX to the first compiler it finds. If it does not find any C++ compiler, then it sets CXX="".
Similarly, if you want to change the command used to link the python executable, then set LINKCC on the configure command line.
Kinda unfortunate that I can't find that level of detail in modern Python, but maybe I'm not searching hard enough through CPython's GitHub
@henryiii This information on https://bugs.python.org/issue23644 is also very nice. Thanks for taking the time to go find it!
I don't think that CPython can be built by g++. - STINNER Victor
This was exactly why I originally had it set to
which gcc, but it seems that not explicitly setting compiler flags is the way to go.
@daritter @henryiii Thanks to your help the problem is now resolved: matthewfeickert/Docker-Python3-Ubuntu#3
@HDembinski Please ignore my ping as the issue no longer exists.
Contrary to what you might expect, llvmlite does not use any LLVM shared libraries that may be present on the system, or in the conda environment. The parts of LLVM required by llvmlite are statically linked at build time. As a result, installing llvmlite from a binary package does not also require the end user to install LLVM. (For more details on the reasoning behind this, see: Why Static Linking to LLVM?)