Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 19:52
    codeandfire ready_for_review #3331
  • 19:52
    codeandfire edited #3331
  • 19:52
    codeandfire synchronize #3331
  • 19:47
    codeandfire opened #3331
  • 16:44
    codeandfire commented #1698
  • 09:42
    abravalheri synchronize #3265
  • 09:42
    abravalheri commented #3260
  • 09:14
    abravalheri commented #2816
  • 08:54
    abravalheri synchronize #3265
  • 08:52
    abravalheri commented #2816
  • 08:17
    EwoutH commented #2816
  • 07:49
    vsajip commented #595
  • 06:57
    codeandfire commented #1698
  • 06:50
    abravalheri ready_for_review #3330
  • 06:50
    abravalheri edited #3330
  • 06:49
    abravalheri edited #3330
  • 02:36
    codeandfire commented #3301
  • 00:33
    jaraco commented #595
  • 00:05
    jaraco commented #2640
  • May 18 22:52
    rwarren commented #3329
sdraper-CS
@sdraper-CS
I have spent several hours following various issue threads and checking with respect to them (including detailed debug output from pip and setuptools but not gotten anywhere. Has anyone seen anything similar recently?
Paul Ganssle
@pganssle
Did you try pinning setuptools_scm to an older version in your pyproject.toml? Seems like it is most likely to be a change in that package.
Setuptools has no dependencies.
sdraper-CS
@sdraper-CS
Yes - that was indeed the problem. Issue has been reported on setuptools_scm git. Thanks
drocsid
@drocsid
Hi yeah I'm not terribly well versed in python but inherited a broken python project. Inside the project I have a setup.py file that follows:
# -*- coding: utf-8 -*-

from setuptools import setup, find_packages

setup(
    name='int-airflow',
    version='1.0.270',
    license='MIT',
    packages=find_packages(exclude=('tests', 'tests.*', 'hql')),
    install_requires=[
        'apache-airflow>=1.7.0,<1.11.0'
    ],
    classifiers=[
        'Programming Language :: Python :: 2.7'
    ]
)
The project tells me to python setup.py build which fails fast and gives no errors
but when I try python setup.py test then it seems like it tries to grab the dependencies and fails
drocsid
@drocsid
Using /Users/colinwilliams/code/int-airflow/.eggs/pytzdata-2019.1-py2.7.egg
Traceback (most recent call last):
  File "setup.py", line 18, in <module>
    'Programming Language :: Python :: 2.7'
  File "/Users/colinwilliams/code/int-airflow/env/lib/python2.7/site-packages/setuptools/__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/Users/colinwilliams/code/int-airflow/env/lib/python2.7/site-packages/setuptools/command/test.py", line 217, in run
    installed_dists = self.install_dists(self.distribution)
  File "/Users/colinwilliams/code/int-airflow/env/lib/python2.7/site-packages/setuptools/command/test.py", line 208, in install_dists
    ir_d = dist.fetch_build_eggs(dist.install_requires)
  File "/Users/colinwilliams/code/int-airflow/env/lib/python2.7/site-packages/setuptools/dist.py", line 717, in fetch_build_eggs
    replace_conflicting=True,
  File "/Users/colinwilliams/code/int-airflow/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (tzlocal 2.0.0b2 (/Users/colinwilliams/code/int-airflow/.eggs/tzlocal-2.0.0b2-py2.7.egg), Requirement.parse('tzlocal<2.0.0.0,>=1.5.0.0'), set(['pendulum']))
So I have two issues. 1)The version conflict, and 2) python setup.py build doesn't install the deps , since I only get the install, etc with python setup.py test
drocsid
@drocsid
I'm not sure if the version conflict is caused by the classifiers line or just the airflow dependency itself...
Ian
@ionox0
Will python setup.py install install the dependencies of listed dependencies automatically?
Paul Ganssle
@pganssle
@ionox0 Do not use setup.py install, use pip install instead.
And yes, pip will install transitive dependencies automatically.
Ian
@ionox0
thank you, I also realized I was missing the extras part of the package that I was trying to install
toil vs toil[cwl]
drocsid
@drocsid

@ionox0 or channel I'm trying to use airflow[qds] from an inherited setup.py project but not sure how to specify. You mention something similar above. My setup.py setup() has

install_requires=[
        'apache-airflow>=1.7.0,<1.11.0',
        'jinja2>=2.10.0',
        'werkzeug>=0.15.0'

    ],

but I don't know how to add the qds module. Can anybody help? I've tried to search this out but am just getting more confused.

Ian
@ionox0
'apache-airflow[qds]>=1.7.0,<1.11.0’
does that work?
im no expert
drocsid
@drocsid
I didn't think it did let me try again
So I try that and the pip install -e . but then when I bring up airflow I get
line 32, in <module>
    from qds_sdk.qubole import Qubole
ImportError: No module named qds_sdk.qubole
Done.
so it allows the install but it looks like it doesn't let me load the qds module.
Ian
@ionox0
might need to use extras_require?
drocsid
@drocsid
I tried that also already
Ian
@ionox0
setup(
    name="Project-A",
    ...
    extras_require={
        'PDF':  ["ReportLab>=1.2", "RXP"],
        'reST': ["docutils>=0.3"],
    }
)
you could try with python setup.py install as well
drocsid
@drocsid
install_requires=[
        'apache-airflow>=1.7.0,<1.11.0',
        'jinja2>=2.10.0',
        'werkzeug>=0.15.0'

    ],
    extras_require=[
        'qds' : 'airflow[qds]',
        ]
above people are saying not so use python setup.py install but to use pip install instead.
Ian
@ionox0
could it be
    extras_require=[
        'apache-airflow' : ‘qds',
        ]
nah nvm
drocsid
@drocsid
From the extras_require description it sounds like that's for defining my own project modules
Ian
@ionox0
right right
it does look like its installing qds
but install is erroring
Ian
@ionox0
Maybe qds_sdk.qubole is also a requirement?
Try with qds-sdk also in install_requires?
drocsid
@drocsid
That looks like it was the issue. My bad. Thanks!
Ian
@ionox0
Cool, awesome
drocsid
@drocsid
Well it was able to initalize the database without exception but then when I started the webserver I got No module named qds_sdk.qubole again...
from the webserver ui
despite adding it to install_requrires
Ian
@ionox0
what does pip show qds-sdk say?
drocsid
@drocsid
pip show qds-sdk
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Name: qds-sdk
Version: 1.12.0
Summary: Python SDK for coding to the Qubole Data Service API
Home-page: https://github.com/qubole/qds-sdk-py
Author: Qubole
Author-email: dev@qubole.com
License: Apache License 2.0
Location: /private/tmp/id-core-test/id-correlation-offline/env/lib/python2.7/site-packages
Requires: requests, boto, inflection, six, urllib3
Required-by: idc-offline
Ian
@ionox0
are you using a virtualenv? if so is your webserver installed within / using that same virtualenv?
(or using conda)?
drocsid
@drocsid
Yeah I'm not sure what I did wrong that time but it does seem to be working on another attempt. Thanks for the help!
Ian
@ionox0
Np!
Thomas Robitaille
@astrofrog
Is there a document somewhere that describes that the plan for Python packaging is to move away from setuptools commands? I've read it in a few comments but I'd like to find a more official place where this is mentioned so that I can use this to convince the developer communities I'm part of to move in that direction.
Paul Ganssle
@pganssle
No.