pypa/setuptools

Official project repository for the Setuptools build system

Paul Ganssle

@pganssle

Did you try pinning setuptools_scm to an older version in your pyproject.toml? Seems like it is most likely to be a change in that package.

Setuptools has no dependencies.

sdraper-CS

@sdraper-CS

Yes - that was indeed the problem. Issue has been reported on setuptools_scm git. Thanks

drocsid

@drocsid

Hi yeah I'm not terribly well versed in python but inherited a broken python project. Inside the project I have a setup.py file that follows:

# -*- coding: utf-8 -*-

from setuptools import setup, find_packages

setup(
    name='int-airflow',
    version='1.0.270',
    license='MIT',
    packages=find_packages(exclude=('tests', 'tests.*', 'hql')),
    install_requires=[
        'apache-airflow>=1.7.0,<1.11.0'
    ],
    classifiers=[
        'Programming Language :: Python :: 2.7'
    ]
)

The project tells me to python setup.py build which fails fast and gives no errors

but when I try python setup.py test then it seems like it tries to grab the dependencies and fails

drocsid

@drocsid

Using /Users/colinwilliams/code/int-airflow/.eggs/pytzdata-2019.1-py2.7.egg
Traceback (most recent call last):
  File "setup.py", line 18, in <module>
    'Programming Language :: Python :: 2.7'
  File "/Users/colinwilliams/code/int-airflow/env/lib/python2.7/site-packages/setuptools/__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/Users/colinwilliams/code/int-airflow/env/lib/python2.7/site-packages/setuptools/command/test.py", line 217, in run
    installed_dists = self.install_dists(self.distribution)
  File "/Users/colinwilliams/code/int-airflow/env/lib/python2.7/site-packages/setuptools/command/test.py", line 208, in install_dists
    ir_d = dist.fetch_build_eggs(dist.install_requires)
  File "/Users/colinwilliams/code/int-airflow/env/lib/python2.7/site-packages/setuptools/dist.py", line 717, in fetch_build_eggs
    replace_conflicting=True,
  File "/Users/colinwilliams/code/int-airflow/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (tzlocal 2.0.0b2 (/Users/colinwilliams/code/int-airflow/.eggs/tzlocal-2.0.0b2-py2.7.egg), Requirement.parse('tzlocal<2.0.0.0,>=1.5.0.0'), set(['pendulum']))

So I have two issues. 1)The version conflict, and 2) python setup.py build doesn't install the deps , since I only get the install, etc with python setup.py test

drocsid

@drocsid

I'm not sure if the version conflict is caused by the classifiers line or just the airflow dependency itself...

Ian

@ionox0

Will python setup.py install install the dependencies of listed dependencies automatically?

Paul Ganssle

@pganssle

@ionox0 Do not use setup.py install, use pip install instead.

And yes, pip will install transitive dependencies automatically.

Ian

@ionox0

thank you, I also realized I was missing the extras part of the package that I was trying to install

toil vs toil[cwl]

drocsid

@drocsid

@ionox0 or channel I'm trying to use airflow[qds] from an inherited setup.py project but not sure how to specify. You mention something similar above. My setup.py setup() has

install_requires=[
        'apache-airflow>=1.7.0,<1.11.0',
        'jinja2>=2.10.0',
        'werkzeug>=0.15.0'

    ],

but I don't know how to add the qds module. Can anybody help? I've tried to search this out but am just getting more confused.

Ian

@ionox0

'apache-airflow[qds]>=1.7.0,<1.11.0’

does that work?

im no expert

drocsid

@drocsid

I didn't think it did let me try again

So I try that and the pip install -e . but then when I bring up airflow I get

line 32, in <module>
    from qds_sdk.qubole import Qubole
ImportError: No module named qds_sdk.qubole
Done.

so it allows the install but it looks like it doesn't let me load the qds module.

Ian

@ionox0

might need to use extras_require?

drocsid

@drocsid

I tried that also already

Ian

@ionox0

setup(
    name="Project-A",
    ...
    extras_require={
        'PDF':  ["ReportLab>=1.2", "RXP"],
        'reST': ["docutils>=0.3"],
    }
)

you could try with python setup.py install as well

drocsid

@drocsid

install_requires=[
        'apache-airflow>=1.7.0,<1.11.0',
        'jinja2>=2.10.0',
        'werkzeug>=0.15.0'

    ],
    extras_require=[
        'qds' : 'airflow[qds]',
        ]

above people are saying not so use python setup.py install but to use pip install instead.

Ian

@ionox0

could it be

    extras_require=[
        'apache-airflow' : ‘qds',
        ]

nah nvm

drocsid

@drocsid

From the extras_require description it sounds like that's for defining my own project modules

Ian

@ionox0

right right

it does look like its installing qds

but install is erroring

drocsid

@drocsid

https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies

Ian

@ionox0

Maybe qds_sdk.qubole is also a requirement?

Try with qds-sdk also in install_requires?

drocsid

@drocsid

That looks like it was the issue. My bad. Thanks!

Ian

@ionox0

Cool, awesome

drocsid

@drocsid

Well it was able to initalize the database without exception but then when I started the webserver I got No module named qds_sdk.qubole again...

from the webserver ui

despite adding it to install_requrires

Ian

@ionox0

what does pip show qds-sdk say?

drocsid

@drocsid

pip show qds-sdk
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Name: qds-sdk
Version: 1.12.0
Summary: Python SDK for coding to the Qubole Data Service API
Home-page: https://github.com/qubole/qds-sdk-py
Author: Qubole
Author-email: dev@qubole.com
License: Apache License 2.0
Location: /private/tmp/id-core-test/id-correlation-offline/env/lib/python2.7/site-packages
Requires: requests, boto, inflection, six, urllib3
Required-by: idc-offline

Ian

@ionox0

are you using a virtualenv? if so is your webserver installed within / using that same virtualenv?

(or using conda)?

drocsid

@drocsid

Yeah I'm not sure what I did wrong that time but it does seem to be working on another attempt. Thanks for the help!

Ian

@ionox0

Np!

Thomas Robitaille

@astrofrog

Is there a document somewhere that describes that the plan for Python packaging is to move away from setuptools commands? I've read it in a few comments but I'd like to find a more official place where this is mentioned so that I can use this to convince the developer communities I'm part of to move in that direction.

Paul Ganssle

@pganssle

No.

It's true that you should, and almost certainly there is at least documentation that says not to use specific commands.

Where communities thrive

People

Repo info

Activity