These are chat archives for SHTOOLS/SHTOOLS

Sep 2016
Mark Wieczorek
Sep 07 2016 08:58
I contacted Numerical Recipes about their licensing agreement, and here is what they say. Even though they disagree, I don't think that they can do anything about using an algorithm that they published in an open source project if it is correctly referenced. Nevertheless, given that they are extremely uncool (you technically can't use NR in a paper published in Science or Nature), I plan on:
  • removing the references to NR,
  • probably replace their random number generator with the default fortran one,
  • reference another source for how to convert uniform deviates to normal deviates (this is only necessary for the tests)
  • use a different reference for the Gauss-Legendre Quadrature algorithm (which must be pretty standard).

If you are interested, here is the correspondance:

I have a quick question concerning the use of an algorithm published in one of the Numerical Recipes books in a software package distributed under the revised-BSD (or gnu gpl) license. I looked at the info on your website, but it is not clear if my situation is covered there or not.

In summary, I am using the algorithm described in the f90 book for generating random numbers. I bought this book a long time ago, I more-or-less transcribed the code by hand that makes use of already published concepts (e.g., Marsaglia 1985), portions of my code differ from that in NR (variable names, comments, and modifications for use with OpenMP), and I clearly site in the comments that the algorithm is based on that published in the NR F90 book.

The question is, is it acceptable to include this modified random number generator in my code that is distributed with the revised-bsd/gnu-gpl license? I have heard conflicting opinions on this matter. From my perspective as a scientist, the citation to the NR algorithm should be sufficient, but I also acknowledge that there is a lot that I don't understand about copyright law.

And their response:

I am afraid that the Numerical Recipes cannot be used in the way that you are describing. You can find the license terms in the F90 book on page xviii, under the section "Types of License Offered".

If you have transcribed your Recipes from the book, then you are operating under the "Immediate License". This license allows you to type copies of the routines into your computer for your own personal and noncommercial purposes. You are not authorized to transfer or distribute machine-readable copies to any other person, or to use the routines on more than one machine, or to distribute executable programs containing the routines.

If you have purchased a copy of the Recipes, either from us or through Cambridge University Press, then your license is the "Single-Screen License". This license is a bit more generous. It still limits your use to a single computer, but if you develop and compile executable programs (with certain restrictions) on this computer, then you are allowed to distribute the compiled programs non-commercially. You cannot pass the allowance for redistribution on to others who are not licensed users of the Recipes, nor can you distribute the source codes or object codes.

We're sorry if this interferes with your plans for distribution, and we would be happy to answer and further questions you may have about the provisions of the license.

Here is a question about the Science data release policy:

Thanks for the response. It does answer my immediate question, but now I have a question related to the data release policies of scientific journals.

In particular, I am using a Numerical Recipes routine that was used to generate results that were published in the journal Science. According to Science's data release policy, I need to make my code available to anyone who asks, which would include distributing the included Numerical Recipes code. Could you let me know how the Science data release policy conforms (or not) with that of Numerical Recipes? Here is the relevant passage from their website:

Data and Materials Availability after Publication

After publication, all data and materials necessary to understand, assess, and extend the conclusions of the manuscript must be available to any reader of Science. All computer codes involved in the creation or analysis of data must also be available to any reader of Science. After publication, all reasonable requests for data or materials must be fulfilled. Any restrictions on the availability of data, codes, or materials, including fees and restrictions on original data obtained from other sources must be disclosed to the editors as must any Material Transfer Agreements (MTAs) pertaining to data or materials used or produced in this research, that place constraints on providing these data or materials. Patents (whether applications or awards to the authors or home institutions) related to the work should also be declared. Fossils or other rare specimens must be deposited in a public museum or repository and available for research. Unreasonable restrictions on data or material availability may preclude publication.

And their response:

That's a difficult question! Ultimately, I think the meaning has to be arbitrated by Science. As I read their statement, they only ask that the computer codes be available to the readers, and not that you deliver them, or even that the reader has the resources to use them. You must simply inform Science about any limitations or costs.

I am thinking, in particular, about the many people who analyze their data using Matlab or MS Excel. In these cases, they could provide their Matlab program or Excel worksheet to readers, but these would be unusable unless the reader also had a license for Matlab or MS Office, and the licenses are quite expensive. Could it be that Science is suggesting that you should not use valuable tools simply because you cannot provide everyone with a license?

In the case of using a Numerical Recipes routine, you could deliver your source code with a call to the routine, but with a reference in place of the source code. At the current time, a copy of the F77 and F90 books can be found for free access at Perhaps that is enough to satisfy their requirement, but I am thinking that this may not be a suitable reference because it is not guaranteed to exist forever. Probably a better reference would be to Cambridge University Press and the book, which is found in most university libraries. Alternatively, you could cite the CDROM from Cambridge University Press. These are both "available to the reader", and the price is much lower than the cost of a license for Excel or Matlab.

Alternatively, you could look for a public-domain replacement for the Numerical Recipes routine in question. Since we wrote many of our routines based on algorithms that have been used for years. The algorithms, themselves, are "free as air", and it is only our particular implementations that are copyrighted by us. (See section "About Copyrights on Computer Programs" on page xix.) You may find other implementations on the internet, particularly at sites like NetLib, though it may be difficult to find the needed routine specifically in F90.

So, if you have any suggestions on fixing the fortran routines
  • PreGLQ
  • RandomN
  • RandomGaussian
    let me know.
Ilya Oshchepkov
Sep 07 2016 10:04
Interesting and strange. Seems like NR can be freely used only for self-educational purpose. It's pity, because the books are actually good and often helpful.
This message was deleted
Mark Wieczorek
Sep 07 2016 10:14
Honestly, I don't take this all too seriously. However, not being open source compatible is certain death in the long run...
Ilya Oshchepkov
Sep 07 2016 10:28
Yes, I agree, but I don't quite understand about Fortran routines. Do you want to rewrite the routines or simply to change the references?
Mark Wieczorek
Sep 07 2016 10:51
If I can find a different reference that uses the same algorithm, I will just rename the variables and site that paper instead. I think that will be easy for the GLQ routine. However, for the RandomN routine, its my impression that the NR algorithm is their own, so this might not be possible. It might just be best to find a better open source Random number generator, as I am certain there are better ones that in NR.
it is python, but maybe we can transfer it to Fortran?
interesting conversation btw. Even though their policy is not great, at least they gave a clear answer ...
@MarkWieczorek if you can look through #61 and merge, I can make a clean PR for a get_spectrum routine (at least if we agree now that we call it get_spectrum
Mark Wieczorek
Sep 07 2016 14:58
Also, maybe this is the time to fix This file is so long that it is hard to work with. What we could do (in fact, it was like this at one point), would be to create a submodule. You just create a subdirectory called shclasses, cut the original file into one file per classes, and then create a very simple __init__ file to import all the classes into the module. Given that there are 11 classes and subclasses, this might be the best way to go. Also, once we start replacing fortran routines with native python routines, we will probabaly want to create a subdirectory for each module anyways.
What do you think?
Ilya Oshchepkov
Sep 07 2016 15:01
Yes, in terms of Python it is not submodule but subpackage
because module is only one file
Sep 07 2016 16:11
sounds good !
Let's cut it into 4 files?
1 for each parent class and 1 for the coeff and grid underlying subclasses
I forgot the windows :)
or just 3 for coeffs, grids and windows ?
Mark Wieczorek
Sep 07 2016 17:07
Given the length of the existing file, I'm not sure if just three files for the three classes is good enough.